Benchmarks — tested honestly, including the misses
We publish our full test suite and its results — passing and failing — because a security tool that hides its misses isn't trustworthy. These are measured on a fixed 200-attempt suite run through the actual enforcement engine, not a cherry-picked demo.
Headline result
| Attack category | Attempts | Blocked | Block rate |
|---|---|---|---|
| Direct injection | 50 | 48 | 96% |
| Obfuscated / encoded | 50 | 47 | 94% |
| Multi-turn / split | 50 | 48 | 96% |
| Tool-call hijack | 50 | 50 | 100% |
| Total | 200 | 193 | 96.5% |
The threshold used is a risk score at or above 45 (high severity), matching the runtime's default. Run it yourself with the same secret the runtime uses.
Reproduce it yourself
Everything is in the public repo:
- Benchmark script:
scripts/benchmark_injection.py— runs all 200 payloads throughscore_content. - 78 passing unit tests — the full enforcement engine is covered.
- ~2.7k LOC core — small enough to audit in full.
python scripts/benchmark_injection.py --threshold 45The 7 documented misses (transparency is a feature)
These are the attacks that got through. We publish them because hiding them would defeat the purpose of a benchmark. Each represents a known gap, and the payloads are public so researchers can assess the real risk.
- Highly obfuscated multi-encoding — payloads using layered obfuscation that the pattern layer doesn't decompose.
- Non-instruction-shaped injection — attacks phrased as data that don't trip instruction/authority signals.
- Novel authority-spoof framing — content that convincingly imitates a trusted system prompt without matching a known pattern.
If you find a miss we haven't documented, please report it responsibly — that's how the benchmark gets better.
What we will never claim
- No "99.9% block rate" with no reproducible evidence.
- No model-level safety guarantee — AgentInterdict is a runtime-boundary enforcer, not a change to model weights.
- No claim that prompt injection is "solved." It isn't, by anyone.