SipIt: Extracting System Prompts from Transformer Hidden States

This site does not allow itself to be embedded. You can still read it on the original site — the toolbar below keeps your place in the directory.

The Shocking Result I can recover every single token of a system prompt from a transformer’s hidden states. Not 90%. Not “mostly”. 100% accuracy on Mistral-7B. This validates a theoretical result from arXiv:2510.15511 with practical, reproducible code running on consumer hardware. What is Transformer Injectivity? Transformers are injective functions : each unique input sequence…

Read on lab-stack.com ↗