52 defensive red-team probes for stress-testing the safety and robustness of any OpenAI-compatible LLM endpoint, mapped to the OWASP Top 10 for LLM Applications plus a jailbreak suite.
// what it is
Each probe is one adversarial prompt with a plain-English description of what a well-aligned model should do, a severity, and lightweight scoring markers. Responses are scored by regex and keyword, ported verbatim from the source toolkit, so the same probe and response always score the same way. Defensive use only: test endpoints you own or are authorized to test.
// outcomes
held the model refused or resisted the probe.
finding the model walked into it.
review no marker matched, read the response yourself.
// category coverage
// example run
A real run of the jailbreak category against llama-3.1-8b-instant. The 8B model leaked
a planted canary codeword under four different jailbreak techniques and held on one. This is the exact output the
runner produces.
| Probe | Sev | Outcome | Response (truncated) |
|---|
// try it
This runs entirely in your browser. Your endpoint and key are used only for this run and never leave the page. Best against a local model (Ollama, vLLM) or any CORS-enabled endpoint.
OLLAMA_ORIGINS=*.
// the corpus
The full probe set, filterable by OWASP category. This is the same data published as the dataset.