A prompt injection attack game to collect data for adversarial ML research

This is the source code for the Tensor Trust web game and data cleaning pipeline. See the paper website for more details on the project. You can also use the data, or go play the game!

If you build on our code or data in an academic publication, please cite us with the following BibTeX:

@misc{toyer2023tensor,
    title={{Tensor Trust}: Interpretable Prompt Injection Attacks from an Online Game},
    author={Toyer, Sam and Watkins, Olivia and Mendes, Ethan Adrian and Svegliato, Justin and Bailey, Luke and Wang, Tiffany and Ong, Isaac and Elmaaroufi, Karim and Abbeel, Pieter and Darrell, Trevor and Ritter, Alan and Russell, Stuart},
    year={2023},
    journal={arXiv preprint arXiv:2311.01011},
    url={https://arxiv.org/pdf/2311.01011.pdf}
}

Installation

To install and run, first set up OpenAI API key if you have not already:

1. Login to OpenAI account and go to https://platform.openai.com/account/api-keys.
2. Create an API key.
3. Now open a shell: on Windows run set OPENAI_API_KEY=<your-key>, and on Unix run export OPENAI_API_KEY=<your-key>.

Now run the following:

# Install Redis on Ubuntu. For other OSes see: 
# https://redis.io/docs/getting-started/installation/
sudo apt install redis
# If this command fails, try running `redis-server` directly
sudo systemctl enable redis-server \
    && sudo systemctl restart redis-server
# Install node.js on Ubuntu. For other OSes see:
# https://nodejs.org/en/download
# If this command doesn't work, try installing using nvm. See
# https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-ubuntu-20-04#option-3-installing-node-using-the-node-version-manager
sudo snap install node --classic
# setup:
conda create -n promptgame python=3.10
conda activate promptgame
pip install -e '.[dev]'
./manage.py tailwind install  # install JS modules for Tailwind
./manage.py migrate  # set up database
# For testing, we need two commands.
# Run this first command in one terminal to update the stylesheet in response to Tailwind changes:
./manage.py tailwind start
# Now run this second command in another terminal to a Django server
./manage.py runserver  # run demo server (will auto-restart when you edit files)

Now you can visit a development copy of the website at http://localhost:8000/.

Database Management

Django handles database management with Models, which we define in src/promptgame/gameui/models.py. Whenever you edit a Model, you need the change to be reflected in the underlying database that Django is managing. To do this, run:

./manage.py makemigrations
./manage.py migrate

In git terms, makemigrations is like creating a commit recording your change to the database. This migration is actually tracked within a file in the src/promptgame/migrations directory. Running migrate is like pushing this commit, and thus actually updates the database. To find out more about this process (including how to do more complex behavior such as revert your database back to a previous migration state), click here.

Note that if you are pulling from main after someone has made a change to a model, you will also have to run ./manage.py migrate to apply the new migrations generated by the other person.

Creating an admin account

To create an admin account, run:

./manage.py createsuperuser

Follow the prompts to create a username and password.

Viewing the admin interface

Log in to the admin page at localhost:8000/private/dj-login/. On the prod site, this will be at tensortrust.ai/private/dj-login/.

Enter the username and password you created above. If you are on the prod site, you'll have to get the password by opening a terminal and running gcloud secrets versions access --secret=promptgame_prod_application_settings latest.

What's up with Tailwind?

Tailwind is a CSS framework that makes it easier to embed CSS directly in your HTML tags, as opposed to putting your HTML source and your CSS source on different places. It works by stuffing style information into a set of predefined classes, like this mix of HTML and Tailwind classes that defines a rounded purple button:

<div class="ml-8 rounded-md bg-indigo-600 px-3 py-2 text-[0.8125rem]
            font-semibold leading-5 text-white hover:bg-indigo-500">
    This is a button!
</div>

You might notice from this example that the set of possible Tailwind classes is really large. e.g. text-[0.8125rem] makes the text 0.8125 rem high, but what if the user asked for 0.31 rem or