Back to blog
securityaillmctfprompt-injectionollamagoogle-adk

PwnGPT, 16 levels and attacking the parts of an agent people trust

2 November 20266 min read

I wrote about PwnGPT in March, and it has since grown from 11 levels to 16. PwnGPT is my open-source prompt injection Capture The Flag. Prompt injection has moved on from convincing a chatbot to repeat a secret, so the new levels go after the parts of an AI agent that people tend to trust by default. I'll go through what's new without giving away the answers, then how to run it locally and how to host it for other people.

Level 10, a decision model as the guard

On level 10 every message goes to a small decision model before the agent sees it, d1-omni:600m by default, called through Ollama's /v1/systemone endpoint. It doesn't write replies, it weighs the message and answers one question, is this person after the password or trying to change the agent's instructions? It blocks at DECISION_THRESHOLD, 0.5 by default.

The agent behind it will hand over the password if you ask nicely, so the guard is the whole defence and you only have to get one polite question past it. The block message shows the score it gave you, which means you can change a word, try again and watch the number move. Waiting for it to fall over won't help, if the guard goes down the door stays locked.

Level 11, hijacking the decision

There's nobody to talk to on level 11. You file a helpdesk ticket as a customer and a decision model reads it and picks one of four actions: reply, escalate, close or open the vault.

Only a verified administrator should ever get that last one. Your ticket says you're an unverified customer and you can't change that, all you control is the words in the ticket. Shouting "ignore all previous instructions" tends to lower your odds. Every attempt shows you the probability of each action, so small changes that follow the numbers work much better. The level is based on the Decision Hijacking paper, if you want to read how the real attack works.

Levels 12 to 14, the side doors

Each of these levels guards the obvious way in and leaves another one open, because that's how these systems tend to fail in the real world.

Level 12, Poisoned Plugin. You can't get the API key into a chat reply, but you can publish a plugin, and the assistant reads its description as trusted documentation without any of the screening your chat messages get. Agents trust the tool descriptions they're given in much the same way. Anything the assistant passes to a plugin also ends up in that plugin's server log.

Level 13, False Memories. Asking about the vault gets you nowhere, but type remember: followed by a fact and it goes straight into long-term memory. Later it comes back as something the assistant simply knows about you, and it has no reason to doubt it, even though you're the one who said it.

Level 14, Trojan Review. A code reviewer holding a deploy key won't look at a message without code in it, and it screens the note you write around that code. The comments inside the code are another matter, it reads those as part of the change.

The boss and the finale

The boss used to sit at level 10. It has moved to level 15, with every protection in the game combined, now including the decision model.

Beat your last remaining level and the finale begins. Every tower falls in a chain, the fortress goes last, the screen flashes and netrunner gremlins dance under a glitching "System breached" banner. It plays whichever level you finish on, because what counts is clearing everything, not just the boss. It only plays once, and if you've asked your system to reduce motion you go straight to the victory screen.

The quest map has been redrawn too, in a cel-shaded cyberpunk cartoon style. The new logo is a netrunner in a neon hoodie with Table Mountain and Cape Town behind them, a nod to the BSides Cape Town event where this all started. At the very top, with 6,400 XP, you earn the highest rank in the game, AGI Whisperer.

What broke along the way

Adding levels turned up some real bugs, and my favourite was also the most unfair. You could type the correct password on a guarded level and still get blocked. The model's follow-up message went back through the guards, and they stopped it from repeating the password you'd just given it. The defences were working perfectly, against a player who had already won.

In another, solving the final level without clearing the others crashed the game, because it tried to send you on to a level that doesn't exist. Hints were also offering a password search tool on levels that don't have one.

Running it locally on 2B models

You don't need an API key or a paid LLM to play any of this. By default every model runs locally on Ollama, qwen3.5:2b is a 2-billion-parameter chat model that takes about 2.7 GB of disk, and d1-omni:600m is the decision model on levels 10, 11 and 15. make run starts Ollama in Docker and pulls both, so there's nothing to configure. The Hugging Face guard models add about 1.6 GB of disk. I haven't measured memory use for this version.

The catch is that the Compose file doesn't give the Ollama container a GPU, so the models run on your CPU. If you have a GPU, install Ollama 0.40 or later on the host and point the game at it. The prod profile here just means Compose doesn't start its own Ollama

ollama pull qwen3.5:2b
ollama pull d1-omni:600m

COMPOSE_PROFILES=prod \   OLLAMA_API_BASE=http://host.docker.internal:11434 \   OPENSOURCE_LLM_MODEL=qwen3.5:2b \   docker compose up --build

On Linux, Ollama only listens on localhost by default, so the containers can't reach it. Set OLLAMA_HOST to the Docker bridge address, usually 172.17.0.1. Avoid 0.0.0.0 on a shared network, because Ollama has no login. If the GPU is in another machine, pull the same models there and run make run-remote OLLAMA_API_BASE=http://:11434.

If you have the hardware for a bigger chat model, set both variables to the same tag

OLLAMA_MODEL=qwen3.5:9b OPENSOURCE_LLM_MODEL=qwen3.5:9b docker compose --profile default up --build

Hosting it at an event

PwnGPT has been run at 3 events so far, and a CTF is an unusual thing to host, because your guests are attacking it on purpose. Read the new hosting guide first, since the default setup assumes a trusted machine. In short:

  • Keep port 8000 off the network. The agent's dev tools have no login and show passwords in the raw. Map it to 127.0.0.1:8000:8000, and don't rely on ufw alone, because Docker's published ports can bypass it
  • Set your own secrets, because the default passwords are public in the repo
  • Use a dedicated machine with restricted outbound access. Some levels run injectable SQL, player-influenced Python and arbitrary web fetches, all on purpose
  • Terminate TLS and set COOKIE_SECURE=1
  • Don't publish the bundled Traefik setup as it is. It has an unauthenticated dashboard and mounts the Docker socket

Running it

You need Docker Engine 25 or later and Docker Compose v2.20 or later

git clone https://github.com/c-goosen/ai-prompt-ctf.git
cd ai-prompt-ctf
make run

Then open http://localhost:8100. The first start downloads the models, which can take several minutes.

If you beat a level, that's the game. If you find a way around one, like a flaw in the web app that hands you a password or a leaderboard credit, that's worth reporting. If you want to add a level of your own, the level guide walks through it, and please open a PR.