RedBot v1

Run it in 5 commands

Prerequisites

• Python 3.11+
• Ollama installed and running
• An Anthropic API key

Steps

# 1. Clone and install
git clone https://github.com/Sanika7777/redbot.git
cd redbot
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 2. Set your Anthropic API key
export CLAUDE_API=sk-ant-...

# 3. Pull a local target model
ollama pull mistral:7b
ollama serve &

# 4. Run a campaign
redbot run --config configs/config.yaml --fail-below 60

# 5. Open the report
open reports/*_report.html     # macOS
xdg-open reports/*_report.html # Linux

Or with Docker (one-liner)

CLAUDE_API=sk-ant-... docker-compose up --build

The Docker setup pulls and serves Mistral 7B via Ollama automatically. RedBot starts in API server mode on port 8080.

---

RedBot is an adversarial security evaluation platform for LLM-powered applications. It automatically discovers and exploits vulnerabilities by simulating a skilled human red-teamer — a goal-directed attacker that maintains session objectives, tracks what it has tried, classifies target responses into structured signal types, and adapts its strategy in real time. Not prompt sampling. Actual adversarial reasoning.

How It Works

RedBot runs an 8-step pipeline against any OpenAI-compatible or Ollama-hosted model:

1. Snapshots the run config to disk before any attack fires (audit trail)
2. Loads and validates the attack corpus (200 templates across 5 categories)
3. Runs a 20-probe calibration phase to rank category weaknesses in the target
4. Orders the attack queue by weakness rank, difficulty, and exploration bonus
5. Runs all attack sessions concurrently with semaphore-controlled parallelism
6. Evaluates each session with a hybrid rule-filter + LLM-judge approach
7. Computes a severity-weighted security score (0–100) with per-dimension breakdown
8. Generates a self-contained offline HTML report and writes all output files

The attacker is a Claude Sonnet 4.6 agent that maintains a world model of the target's defenses, tracks which strategies it has already tried, and selects the next move based on the last refusal signal. This is the core innovation — the attacker is as adaptive as a human red-teamer. The judge evaluates success; the target is whatever LLM you point it at.

CLI Reference

redbot run      --config PATH           Run a full campaign
                [--fail-below N]        Exit 1 if overall_score < N  (CI/CD gate)
                [--calibrate-only]      Run calibration probe only and exit
                [--dry-run]             Print estimated API call count and exit

redbot serve    [--port 8080]           Start FastAPI server
                [--config PATH]

redbot validate --config PATH           Validate config + dataset; exit 0 or 1

redbot corpus   --list                  List corpus versions with per-category counts

FastAPI Endpoints

Available when running redbot serve:

Method	Path	Description
POST	/run	Start a campaign (runs as background task)
GET	/run/{run_id}/status	Poll: running / complete / failed
GET	/run/{run_id}/report	Download HTML report
GET	/run/{run_id}/topology	Download DefenseTopologyMap JSON

Interactive API docs at http://localhost:8080/docs.

Attack Categories

Category	Templates	What it tests
data_extraction	50	System prompt leakage, config disclosure
jailbreak	50	Persona override, roleplay-based safety bypass
instruction_override	40	Admin/developer mode claims, embedded overrides
role_confusion	30	Identity challenges, alternative persona injection
indirect_injection	30	Trojan questions, benign multi-part with malicious sub-query

Ethics

RedBot is a defensive security tool built for authorized testing. Use it only on systems you own or have explicit written permission to test. Never point it at production systems without authorization. See CONTRIBUTING.md for responsible disclosure guidelines.

License

MIT License — see LICENSE.