Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 37 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

**A comprehensive guide to adversarial testing and security evaluation of AI systems, helping organizations identify vulnerabilities before attackers exploit them.**

[Overview](#overview) • [Frameworks](#frameworks) • [Methodologies](#methodologies) • [Tools](#tools) • [Case Studies](#case-studies) • [Resources](#resources)
[Overview](#overview) • [Frameworks](#key-frameworks-and-standards) • [Methodologies](#ai-red-teaming-methodology) • [Tools](#red-teaming-tools) • [Case Studies](#real-world-case-studies) • [Resources](#resources-and-references)

</div>

Expand Down Expand Up @@ -42,6 +42,8 @@

---

<a id="overview"></a>

## 🎯 Overview

As artificial intelligence systems become increasingly integrated into critical business operations, healthcare, finance, and decision-making processes, ensuring their security and reliability has never been more important. AI red teaming has emerged as a fundamental security practice that helps organizations identify vulnerabilities before they can be exploited in real-world scenarios.
Expand All @@ -65,6 +67,8 @@ This comprehensive guide is designed for:

---

<a id="what-is-ai-red-teaming"></a>

## 🤖 What is AI Red Teaming?

**AI Red Teaming** is a structured, proactive security practice where expert teams simulate adversarial attacks on AI systems to uncover vulnerabilities and improve their security and resilience. Unlike traditional security testing that focuses on known attack vectors, AI red teaming embraces creative, open-ended exploration to discover novel failure modes and risks.
Expand Down Expand Up @@ -94,6 +98,8 @@ AI red teaming adapts military and cybersecurity red team concepts to the unique

---

<a id="why-ai-red-teaming-matters"></a>

## 🚨 Why AI Red Teaming Matters

### The Urgency of AI Security
Expand Down Expand Up @@ -124,6 +130,8 @@ Article 15 of the European Union AI Act obliges operators of high-risk AI system

---

<a id="key-frameworks-and-standards"></a>

## 📚 Key Frameworks and Standards

### NIST AI Risk Management Framework
Expand Down Expand Up @@ -242,6 +250,8 @@ The Cloud Security Alliance's Agentic AI Red Teaming Guide explains how to test

---

<a id="ai-red-teaming-methodology"></a>

## 🔬 AI Red Teaming Methodology

### Phase 1: Planning and Threat Modeling
Expand Down Expand Up @@ -471,6 +481,8 @@ For each vulnerability:

---

<a id="threat-landscape"></a>

## 🎯 Threat Landscape

### Adversary Types
Expand Down Expand Up @@ -525,6 +537,8 @@ For each vulnerability:

---

<a id="attack-vectors-and-techniques"></a>

## ⚔️ Attack Vectors and Techniques

### 1. Prompt-Based Attacks
Expand Down Expand Up @@ -797,6 +811,8 @@ Example: Social engineering one agent to leak data to another

---

<a id="red-teaming-tools"></a>

## 🛠️ Red Teaming Tools

### Open-Source Tools
Expand Down Expand Up @@ -1133,6 +1149,8 @@ bun start

---

<a id="real-world-case-studies"></a>

## 📊 Real-World Case Studies

### Case Study 1: Microsoft's SSRF Vulnerability (2024)
Expand Down Expand Up @@ -1314,6 +1332,8 @@ Samsung employees accidentally leaked confidential company data by entering sens

---

<a id="building-your-red-team"></a>

## 👥 Building Your Red Team

### Team Composition
Expand Down Expand Up @@ -1501,6 +1521,8 @@ Samsung employees accidentally leaked confidential company data by entering sens

---

<a id="best-practices"></a>

## ✅ Best Practices

### 1. Start Early in Development
Expand Down Expand Up @@ -1752,6 +1774,8 @@ Red team members should feel comfortable:
---


<a id="implementation-quickstart-306090"></a>

## 🚀 Implementation Quickstart (30/60/90)

Use this phased plan to turn guidance into an operating program.
Expand Down Expand Up @@ -1779,6 +1803,8 @@ Use this phased plan to turn guidance into an operating program.

---

<a id="evaluation-harness-reference-implementation"></a>

## 🧪 Evaluation Harness (Reference Implementation)

A lightweight structure for repeatable red-teaming and regression tracking:
Expand Down Expand Up @@ -1814,6 +1840,8 @@ security-evals/

---

<a id="agentic-ai-attack-trees--controls-mapping"></a>

## 🕸️ Agentic AI Attack Trees + Controls Mapping

Use attack trees to connect offensive testing paths to defensive controls.
Expand Down Expand Up @@ -1853,6 +1881,8 @@ Use attack trees to connect offensive testing paths to defensive controls.

---

<a id="ai-harm-severity-and-triage-model"></a>

## 📈 AI Harm Severity and Triage Model

Use CVSS as a base, then add AI-specific modifiers:
Expand All @@ -1873,6 +1903,8 @@ Use CVSS as a base, then add AI-specific modifiers:

---

<a id="secure-sdlc-integration-artifacts"></a>

## 🧩 Secure SDLC Integration Artifacts

To reduce "one-off" testing, integrate red-team controls into delivery workflows.
Expand Down Expand Up @@ -2071,6 +2103,8 @@ Starter artifacts in `templates/`:
- `case-study-template.md`


<a id="regulatory-compliance"></a>

## 📋 Regulatory Compliance

### United States
Expand Down Expand Up @@ -2144,6 +2178,8 @@ Recommends adversarial testing before deployment and continuous monitoring in pr

---

<a id="resources-and-references"></a>

## 📚 Resources and References

### Official Frameworks
Expand Down
Loading