Category: Lab

Well it has been some time since I have had an opportunity to log in and capture some thoughts – sorry about that! The truth is, it has been a wild few months with work, kids, and the whole ‘adulting’ thing. I do hope I can start offering something of use more frequently. Something that took a lot of time – and shouldn’t have – was getting a working installation of MITRE’s Caldera open-source attack simulation tool. I have used prior versions since 2020 – I think it was version 2.7? Well, the new release (Magma or v5.0) is fantastic. If you can get it to install correctly for you.

What seemed wrong?

Keep in mind, I am an outage waiting to happen, so some of this is most certainly my self-inflicted wounds or ignorance. Call me crazy, but I think the biggest barrier to open source satisfying the masses is just how finicky it can be to set up an application and maintain it. That would seem the case here! Let’s start with the general complaints that could apply to any modern open-source solution:

Dependencies and the OS

Depending on your desired implementation, you may need to install and troubleshoot a long list of software packages:

• python & pip
• virtual environments
• vue.js/node.js and NPM
• Docker
• upx
• Go
• and numerous other dependencies of these packages

Not to mention, you might be working around any new roadblocks your chosen OS throws at you. Ubuntu is continually re-working its complicated relationship with python, as I assume many other Linux distributions are. I get it. It is both essential to apps and the OS, but can be easily broken by the other. But yeah, what a PITA. The flow I will convey here isn’t pretty, but it seems to work (thanks Mark for the help and finding the mistake!)

Changes to the base platform

Application developers have a tough job – how do they keep offering functions the customers need in ways that are appealing and keep pace with user experience trends? The Caldera team dramatically shifted gears with Caldera Magma, moving into a vue.js based UI that is quite amazing once you get to work with it. It has much more visually significant iconography, more useful tables, and a much tighter workflow for setting up and managing an operation. They also needed to bake in capabilities that allow Caldera to be extended for OT/ICS coverage and for the eventual inclusion of AI-driven adversary simulation.

These major changes and enhancements mean the inclusion of even more dependencies. Earlier versions of Caldera were largely Python focused. Now a much broader set of capabilities are being brought to bear. All of these changes make installation a much more involved task for any script. And when the script fails, well, it becomes a lot of work for the poor end user.

A seemingly broken script

One of Caldera’s awesome superpowers is its ability to install even more fun via plugins. In effect, it installs itself even more super powers! Ahh, but like any good superhero, there is always a constraint, limit, or price to be paid. In Caldera’s case, these plugins are all developed different ways and some have fallen out of support, are no longer added by default (like Mock), are coded differently (Human), or require additional dependencies (e.g. Emu). Some are an easy fix – like cloning Mock in after the main package but before build. In Emu’s case, the included script fails to pull all required files and will thus hang the server’s startup. This was resolved by installing a fresh copy directly from the Emu plugin’s profiles.

Overcoming obstacles

I am afraid to admit how much time the above took to work through, but I managed with some digging into errors, multiple tries with different flavors of Ubuntu and install types (docker containers vs. directly on the VM) and a boatload of frustrating time on StackOverflow. As much as that site has bailed me out, it is useless to someone without a good clue as to how to outline their query. I even used by good buddy Claude.ai to help me get past some really fundamental issues with a botched NPM config. It even found a way to tell me nicely that it was my fault.

The working path

So without further ado, here is the general idea for how I tackled this and finally got Caldera to cooperate!

Prerequisites

• Ubuntu 22.04 or 24.04 VM or host with sudo privileges
• >20GB of free space to allow for node.js build, expanded libraries, and room for collected (looted) intel during ops
• If you plan to use this on a real network, you might want to statically configure or reserve an IP in DHCP for the intended network interface

Installing dependencies

$ sudo apt update && sudo apt upgrade -y
$ sudo apt install python3-dev git curl python3-venv npm
$ curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
$ source ~/.bashrc
$ nvm install stable
$ sudo snap install go --classic && sudo snap install upx

Clone the main Caldera repo

git clone https://github.com/mitre/caldera.git --recursive caldera5

Install the Mock plugin

I use this no-longer-default plugin purely to help demonstrate how agents work when I am on the road and need to show a simulated environment without spinning up real targets.

cd caldera5/plugins<br>git clone https://github.com/mitre/mock.git --recursive

Install Emu payloads

This is where it gets fun. Emu’s directory structure comes through using the top-level clone, but it is missing some stuff further down the tree. Install the payloads as best you can now and we’ll clean it up shortly

cd emu
./download_payloads.sh

Fix emu plans directory

Because the plans don’t make it over completely, if you attempt to start the server the build will stall. In order to fix this, we’ll just remove the incomplete emulation plans directory and pull from source to ensure we have them all. You should see that when you re-run the download_payloads.sh script that more installs. Yippee!

cd data/
rm -rf adversary-emulation-plans/
cd ../../..
git clone --depth 1 https://github.com/center-for-threat-informed-defense/adversary_emulation_library.git plugins/emu/data/adversary-emulation-plans
cd plugins/emu
./download_payloads.sh

Finish config!

Here is where we finally get to do some customization. I know there is some sed/awk tricks that can make this easy, but I would rather spend 5 minuters making the changes than 2 hours learning how to script the change to save 5 minutes. I am not a role model for DevOps, that is for certain!

First, we’re going to modify the default.conf file so we can ensure our IP addresses are configured and the mock and emu plugins are loaded:

cd ../..
nano conf/default.yml 

Breaking the rules for science

Now that we have that all knocked out, time to install your python packages. I know this can be done in a virtual environment – and it can also be run out of Docker. At the time I was putting this together, venv wasm’t working well for me, so I ran this on a small ubuntu VM dedicated to Caldera anyway and used the –break system packages switch to force the pip install. Feel free to let me know if you have a more elegant path in the comments below!

pip3 install -r requirements.txt —break-system-packages

Now we can start the server. The first run requires more time as vue.js needs to fully inflate, build out the web application, and do its thing. Don’t freak out if this takes a while! Also – I am using the --insecure switch for testing here. Whenever I plan to run it for more than a quick run, or unattended, I create a local.yml file that looks like default.yml but includes my salt value and encryption key, and then I drop that switch for subsequent startups.

# For testing
python3 server.py --insecure --build 

# For my more persistent lab box
python3 server.py --build

You are likely to see errors related to payloads. Many of these are not allowed to be distributed to systems, and so you are left to search for them yourselves IF you have the permission and access from your organization to do so. Looking at some of those names, it makes sense that you would want to explicitly download those only if needed. Either way, your instance should build and work ok without those specific executables.

Starting up later

For subsequent start-ups, you can omit the build switch:

python3 server.py

And if all works well, you can log into your Caldera portal and get emulating! If you can get to the following portal but not log in, try a restart. More often than not that clears up any lingering issues 🙂

Conclusion

Once Caldera is up and running, be sure to try out the training paths – that is the single best way to learn the tool inside and out. As you do see it, you might notice that the newest version is much more polished and extensible. Don’t try and do it all at once – take it slow and soak it in. This tool teaches you so much not only about Caldera itself, but about ATT&CK, adversaries, and offensive security in general. And with the Blue Team features, it even can assist with enabling defenders.

Hopefully this run through helps those of you looking for a path to playing with Caldera. I love this tool, and the effort of all involved is just amazing. Knowing what the alternatives cost, this is a fine way to get some familiarity with breach attack simulation and learn what you value most. For those who don’t mind the hands-on, it can even be a great long-term platform. We can be certain that Caldera will only get better, and the new Magma platform is worth my personal growing pains. With any luck, you can avoid my errors and be emulating APTs in no time!