Research Note
AI Red Team Exercise Framework
An AI red-team exercise needs a named system version, owner, authorization, objective, threat model, assets, actors, access paths, exclusions, safe handling rules, stop c
In this article
AI Red Team Exercise Framework
Exercise contract
An AI red-team exercise needs a named system version, owner, authorization, objective, threat model, assets, actors, access paths, exclusions, safe handling rules, stop conditions, evidence format, triage path, disclosure path, retest plan, and residual-risk owner.
Evidence record
| Field | Required record |
|---|---|
| System | Model, prompts, retrieval, tools, identities, permissions, filters, runtime, and date |
| Test | Threat, actor, precondition, input, context, settings, action, output, and observed impact |
| Finding | Reproduction, severity, affected boundary, uncertainty, and evidence location |
| Treatment | Mitigation, control owner, expected effect, side effects, and deployment state |
| Closure | Retest result, regression case, remaining exposure, acceptance owner, and review trigger |
Safety boundary
Tests that could create harmful instructions, sensitive data exposure, unauthorized access, unsafe code, or real-world impact need controlled environments and qualified review. Public editorial material should explain method and accountability without publishing operational bypass recipes.
Completion rule
Discovery is not completion. A finding becomes operationally useful when it is reproducible, owned, treated or explicitly accepted, covered by regression evidence, and connected to monitoring and incident response.
Sources
Follow the evidence.
- ai-challenges.nist.gov: ariaai-challenges.nist.gov
- ai-challenges.nist.gov: genaiai-challenges.nist.gov
- ai.meta.com: meta llama 3 1 ai responsibilityai.meta.com
- ai.meta.com: the llama 3 herd of modelsai.meta.com
- crfm.stanford.edu: indexcrfm.stanford.edu
- csrc.nist.gov: red teamingcsrc.nist.gov
- daltonanderson.ghost.io: metas llama 3 safety scaling and simple solutionsdaltonanderson.ghost.io
- github.com: MODEL CARDgithub.com
- github.com: MODEL CARDgithub.com
- github.com: PurpleLlamagithub.com
- github.com: MODEL CARDgithub.com
- huggingface.co: concept guidehuggingface.co
- mlcommons.org: safety faqmlcommons.org
- mlcommons.org: jailbreak 0 7mlcommons.org
- mlcommons.org: safety methodologymlcommons.org
- NIST Generative AI Profilenvlpubs.nist.gov
- open.spotify.com: 44o5OPSumaZJcvRkXutorBopen.spotify.com
- owasp.org: www project top 10 for large language model applicationsowasp.org
- NIST AI Risk Management Frameworknist.gov
- youtu.be: 1KNOcY e9Tsyoutu.be