AI securityOpen source · MIT
Open-source red-team harness for RAG and agentic AI systems.
PythonFastAPINext.js
ExpandProblem
Most tools test prompt injection by pasting the attack into the model's context. Real attacks don't get that shortcut: a poisoned document has to survive indexing and retrieval before it reaches the model at all.
What I built
- Poisons the corpus instead of the context and tests whether the attack survives the real retrieval path. A canary token planted in the document showing up in the output is the evidence the whole path completed.
- A normalizer (NFKC, homoglyph folding, zero-width stripping, bounded base64/hex/ROT13 decoding) feeding seven detector families: instruction override, structural injection, exfiltration, PII, unauthorized action, tool poisoning and source confusion.
- A scored policy engine with strict, balanced and permissive packs. Every decision carries a plain-English rationale, findings are mapped to the OWASP LLM Top 10, and uploaded documents start untrusted, so trust changes how findings are scored.
- A benign corpus measured alongside attacks, so false-positive rate is reported next to recall. A mutation engine expands 91 base cases to 1,200+ through 17 seeded evasion mutators.
- Red-teamed itself with blind holdouts written independently from the threat model: 25% recall at first, 88.9% after two rebuilds, and 38.5% on a second holdout with fresh wordings. Both numbers are published. 333 tests on Python 3.11, 3.12 and 3.14, with pip-audit as a CI gate.
Stack
PythonFastAPINext.jsTypeScriptTailwind CSSPydanticpytestGitHub ActionsOWASP LLM Top 10
