RedCore — Autonomous AI Red Team Agent

Production-grade autonomous B2B AI penetration testing platform.

RedCore takes a plain-English objective, autonomously plans a multi-stage attack using a custom fine-tuned LLM, executes 242 real security tools, parses their output into structured findings, adaptively replans when new vulnerabilities surface — and delivers a professional pentest report with MITRE ATT&CK coverage heatmap.

Architecture · 242 Tools · Custom Dataset · Fine-Tuning · Quick Start · API · MCP

---

What Makes RedCore Different

Most AI security tools are wrappers. RedCore is an autonomous agent that:

1. Reasons — given a high-level objective like "Compromise the domain controller on 10.0.0.0/24", the LLM-driven planner generates a full multi-stage attack plan following the MITRE ATT&CK kill chain
2. Selects — a fine-tuned router LLM picks the best tool from 242 options for each step, understands tool purposes, failure modes, and alternatives
3. Executes — real security tools run via Docker, WSL2, or subprocess with a 4-layer adaptive retry engine
4. Parses — dedicated parsers convert raw tool output (nmap XML, nuclei JSON, bloodhound CSV, etc.) into structured Finding objects with severity, CVE references, and MITRE mappings
5. Replans — when critical findings surface (e.g., a Log4Shell on an internal host), the LLM is re-invoked to pivot the remaining plan to exploit it immediately
6. Reports — generates professional penetration test reports in Markdown with full MITRE ATT&CK Navigator layer export

---

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                      RedCore Agent Loop                              │
│                                                                      │
│  ┌────────────┐    ┌────────────┐    ┌──────────────────────────┐   │
│  │  Planner   │───▶│   Router   │───▶│    ExecutionEngine        │   │
│  │  (LLM)     │    │  (LLM)     │    │  Docker / WSL2 / subprocess│  │
│  └─────┬──────┘    └────────────┘    └────────────┬─────────────┘   │
│        │ adaptive replan                           │ ToolResult       │
│        │◀──────────────── findings ────────────────┤                  │
│  ┌─────▼──────────────────────────────────────────▼─────────────┐   │
│  │  Memory: ChromaDB (RAG) + NetworkX (host/service graph)       │   │
│  └───────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────┘
         │                                    │
   FastAPI REST API                    WebSocket Stream
   + CLI (rich)                        (real-time events)
         │
   B2B Integrations: Slack | Jira | Teams | Webhook
         │
   MCP Server → Claude Desktop + any MCP client (all 242 tools)

Agent Modules

Module	File	What it does
Orchestrator	redcore/agent/orchestrator.py	Top-level autonomous agent loop — coordinates all phases
Planner	redcore/agent/planner.py	LLM-driven multi-stage attack plan generation + adaptive replan
Router	redcore/agent/router.py	LLM-based tool selection from the 242-tool registry
Reporter	redcore/agent/reporter.py	Pentest report generation with MITRE ATT&CK mapping
Risk Scorer	redcore/agent/risk_scorer.py	CVSS-based finding severity scoring
Deduplicator	redcore/agent/deduplicator.py	Cross-job finding deduplication
MITRE Mapper	redcore/agent/mitre_mapper.py	Keyword + tool → ATT&CK technique mapping + Navigator layer export

4-Layer Adaptive Retry Engine

When a tool fails, RedCore escalates through 4 layers before giving up:

Layer 1 — Parameter Rotation
  └─ Retry with alternative_params from the tool YAML definition

Layer 2 — Tool Substitution (Router)
  └─ LLM picks the next-best tool for the same objective

Layer 3 — Strategy Pivot (Orchestrator)
  └─ After N consecutive failures, Planner replans remaining steps

Layer 4 — LLM Escalation
  └─ Full strategic replan if Planner determines the attack vector is blocked

This is implemented across ExecutionEngine (redcore/execution/engine.py) and RedCoreOrchestrator (redcore/agent/orchestrator.py).

Execution Backends

RedCore supports three tool execution modes — switchable via config.yaml:

Mode	Description
wsl	Tools run inside Ubuntu on WSL2 — recommended for Windows hosts
docker	Tools run inside redcore/toolbox container — fully isolated
subprocess	Direct subprocess — for Linux hosts with tools installed natively

The WSLExecutor (redcore/execution/wsl_executor.py) handles Windows → WSL2 process bridging, output path normalization, and per-job output directories.

---

Tool Coverage (242 Tools)

All 242 tools are formally defined in YAML (redcore/tools/definitions/) and loaded into a singleton ToolRegistry at startup. Each definition specifies: command template, parameters with types and defaults, output format, timeout, MITRE tags, alternative tools, and alternative parameter sets for retry.

Category	Notable Tools	Count
Recon	nmap, rustscan, masscan, amass, subfinder, httpx, dnsx, naabu, fierce, shodan-cli	25
OSINT	theHarvester, maltego, recon-ng, spiderfoot, sherlock, holehe, maigret	15
Web Crawling	katana, hakrawler, gospider, waybackurls, gau, crawlergo, photon	10
Directory Fuzzing	ffuf, gobuster, feroxbuster, dirsearch, wfuzz, dirb	8
Vuln Scanning	nuclei, nessus-cli, openvas-cli, nikto, wpscan, testssl.sh	18
Exploit Frameworks	metasploit, sliver, covenant, coercer, responder	10
Web Exploitation	sqlmap, xsstrike, dalfox, commix, ghauri, arjun, jwt_tool	18
Active Directory	bloodhound, certipy, kerbrute, impacket-suite, ldapdomaindump	15
Privilege Escalation	linpeas, winpeas, wesng, pspy, suid3num, sudo_killer	12
Lateral Movement	chisel, ligolo-ng, crackmapexec, evil-winrm, nxc	10
Credential Attack	hydra, medusa, spray, smartbrute, sprayhound	12
Password Cracking	hashcat, john, rainbow tables, hcxtools	8
Cloud Security	prowler, pacu, trufflehog, cloudfox, ScoutSuite, Cartography	18
Kubernetes	kube-hunter, kubescape, trivy, kube-bench, peirates	10
Wireless	aircrack-ng, wifite, kismet, bettercap, hostapd-wpe	10
Reverse Engineering	ghidra, radare2, binwalk, angr, afl-fuzz, pwndbg	10
Fuzzing	boofuzz, libfuzzer, radamsa, restler, peach fuzzer	8
Payload Generation	msfvenom, donut, nimcrypt2, freeze, scarecrow	8
C2 Frameworks	sliver, empire, havoc, mythic, brute ratel	7
API Exploitation	graphql-cop, jwt_tool, dredd, corsy, crapi	10
Total		242

Tool Registry

The registry (redcore/tools/registry.py) is a singleton that:

• Loads all YAML definitions at startup
• Resolves aliases and name variants (e.g. nxc → crackmapexec)
• Validates and fills parameter defaults before execution
• Returns ranked alternative tools when the primary fails
• Powers the MCP server's dynamic tool schema generation

The 69KB tool_install_manifest.yaml tracks installation status, install method, verify command, and exact-vs-alias classification for all 242 tools across apt, pipx, go install, cargo, and gem.

---

Custom Dataset

RedCore was fine-tuned on a custom-built, deterministic 250K+ record dataset — generated entirely without external API calls.

Dataset Generator (dataset_generator/)

The dataset generator is a standalone pipeline with 6 generation strategies:

Phase 1: KB Step Generation
  └─ 242 tools × examples × param variants × target variants × tool alts
     Augmentation: 50 target IPs × 20 param variants × 5 tool alternates = ~360K steps

Phase 2: KB Plan Generation
  └─ 25 red team scenarios × N target IPs → full multi-step AttackPlans
     Scenarios: web app pentest, AD compromise, cloud lateral movement, etc.

Phase 3: Tool Result Simulation
  └─ Success + 14 failure modes per tool (timeout, auth-fail, scope-blocked, parse-error…)
     Output simulator generates realistic stdout/stderr for each tool

Phase 4: Reasoning / Instruction Records
  └─ Tool purpose Q&A, scenario approach descriptions, failure recovery reasoning

Phase 5: External Dataset Ingestion
  └─ Specialist handlers for: Atomic Red Team, MITRE CTI, Metasploit modules,
     CVE/NVD, ExploitDB, PCAP captures, CTF writeups
     Generic normalizer fallback for unknown formats

Phase 6: Theory / Knowledge Synthesis
  └─ Converts schema records → Q&A knowledge pairs
     Generates tool theory: what, when, why, alternatives
     Output: knowledge_base.jsonl + theory_finetune.jsonl

Dataset Schema

Four JSONL output files, validated against strict schemas before write:

File	Schema	Description
attack_steps.jsonl	AttackStep	Single tool invocations with phase, params, MITRE technique
attack_plans.jsonl	AttackPlan	Full multi-step engagement plans as DAGs
tool_results.jsonl	ToolResult	Raw tool output + parsed findings (success + failure variants)
reasoning.jsonl	Free-form	Instruction-following: tool selection reasoning, approach descriptions

Distribution

55% AttackStep     (~137,500 records)
25% AttackPlan     (~62,500  records)
15% ToolResult     (~37,500  records)
 5% Reasoning      (~12,500  records)

Augmentation System

The generator uses a cross-product augmentation engine:

• TargetVariator — 50 target IP/domain variants per step (lab, cloud, internal ranges)
• ParamVariator — 20 parameter variants per tool (port ranges, wordlists, timing profiles, output formats)
• ToolSubstitutor — 5 alternative tool variants per step (same category, different binary)
• FailureSimulator — 14 failure types: timeout, auth_fail, scope_blocked, binary_missing, parse_error, rate_limited, network_unreachable, permission_denied, and more
• OutputSimulator — generates realistic tool stdout/stderr mimicking actual binary output patterns

External Dataset Handlers

The ingestion layer includes specialist handlers for real-world datasets:

• Atomic Red Team — YAML technique files → attack steps
• MITRE CTI — STIX bundles → technique mappings
• Metasploit modules — Ruby module metadata → exploit steps
• ExploitDB — CSV index + exploit code → vulnerability records
• NVD/CVE JSON — CVE descriptions → vulnerability + exploitation steps
• PCAP files — network capture parsing → ToolResult records

All handlers emit pre-structured records typed as attack_step, attack_plan, tool_result, reasoning, or vulnerability — routed by the generator's dispatcher.

---

Fine-Tuning

Model Selection

RedCore supports fine-tuning on multiple base models:

Model	VRAM Required	Training Script
Qwen 3.5 9B (primary)	8GB (4-bit QLoRA)	train_qwen.py
Mistral-Nemo 12B	12GB (4-bit QLoRA)	fine_tuning/train_lora.py
Qwen2.5 7B	6GB (4-bit QLoRA)	fine_tuning/train_lora.py
LLaMA-3 8B	8GB (4-bit QLoRA)	fine_tuning/train_lora.py

QLoRA Configuration (Qwen 3.5 9B on RTX 4060)

r              = 8       # LoRA rank
lora_alpha     = 8
target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                  "gate_proj", "up_proj", "down_proj"]
lora_dropout   = 0
quantization   = 4-bit NF4 (bitsandbytes)
seq_length     = 512     # constrained by 8GB VRAM
batch_size     = 1 × grad_accum 16  (effective ~8K tokens/step)
optimizer      = adamw_8bit
lr_scheduler   = cosine
learning_rate  = 2e-4
epochs         = 3
packing        = True    # 4-5x throughput via Unsloth

Windows-Specific Engineering

Training was done natively on Windows (RTX 4060) — a non-trivial environment. The train_qwen.py script includes:

• TorchDynamo disabled at startup (triton-windows lacks the CUDA scheduler backend)
• triton.ops.matmul_perf_model stub — patched to prevent bitsandbytes import failure
• triton_key stub — prevents torch inductor cache crash on Windows
• Eager-mode fallback — _dynamo.config.suppress_errors = True
• All LoRA, 4-bit quantization, and gradient checkpointing still fully functional

The gguf_to_hf_qwen3_5_9B.py script handles GGUF → HuggingFace SafeTensors conversion for deployment.

Data Format

Training uses ChatML format with a consistent system prompt:

<|im_start|>system
You are RedCore, an advanced adversarial AI assistant built for authorized
penetration testing and red team operations...<|im_end|>
<|im_start|>user
Create an attack plan for: Compromise all hosts in 192.168.10.0/24<|im_end|>
<|im_start|>assistant
{"steps": [...], "rationale": "..."}<|im_end|>

The to_chatml() normalizer in train_qwen.py auto-converts from any input format (Alpaca, Q&A, raw completion, RedCore schema) to ChatML — so all 6 dataset files can be trained on together without pre-processing.

Auto-Resume Training

python train_qwen.py   # start fresh or auto-resume from last checkpoint (Ctrl+C safe)

Checkpoints every 100 steps. Re-running detects and resumes from the last checkpoint automatically.

Export to GGUF (Ollama)

Training automatically exports a Q4_K_M GGUF at the end:

# After training completes:
ollama create redcore -f data/finetune/qwen_redcore_lora/Modelfile
ollama run redcore "Generate an attack plan for 192.168.1.0/24"

---

Quick Start

Prerequisites

• Python 3.11+
• Docker & Docker Compose
• Ollama (for local LLM inference)

1. Clone & Install

git clone https://github.com/AaravMehta-07/Redcore.git
cd Redcore
make install

2. Pull the LLM Model

ollama serve
ollama pull mistral-nemo   # 12GB — or use qwen2.5:7b for smaller footprint

3. Configure

cp config.yaml config.local.yaml
# Edit config.local.yaml — set your LLM backend, execution mode, and integrations

4. Validate Setup

redcore validate
redcore tools              # list all 242 tools
redcore tools --category recon

5. Run an Engagement

redcore run "Pentest all web apps on 192.168.10.0/24" \
  --target 192.168.10.0/24 \
  --exclude 192.168.10.1

6. Docker (Recommended for Production)

make build-docker build-toolbox
make up
make pull-model

# API is live at:
curl http://localhost:8000/health
curl http://localhost:8000/docs   # Swagger UI

---

API

Start a Job

curl -X POST http://localhost:8000/jobs \
  -H "Content-Type: application/json" \
  -d '{
    "objective": "Find and exploit all web vulnerabilities on 10.0.0.5",
    "targets": ["10.0.0.5"],
    "excluded_targets": [],
    "tenant_id": "client_acme"
  }'

Response:

{"job_id": "abc123", "status": "planning", "created_at": "..."}

Stream Real-Time Events (WebSocket)

const ws = new WebSocket('ws://localhost:8000/ws/jobs/abc123');
ws.onmessage = (e) => {
  const event = JSON.parse(e.data);
  // event.event_type: plan_ready | step_start | step_done | finding | replan | complete
  console.log(event);
};

Event stream example:

{"event_type": "plan_ready",  "data": {"steps": 18, "phases": ["recon", "scanning", "exploitation"]}}
{"event_type": "step_start",  "data": {"tool": "nmap", "phase": "recon"}}
{"event_type": "finding",     "data": {"title": "SMB open (port 445)", "severity": "medium"}}
{"event_type": "replan",      "data": {"reason": "critical_findings", "findings_count": 2}}
{"event_type": "complete",    "data": {"total_findings": 23, "critical": 3, "high": 7}}

Get Findings

curl http://localhost:8000/findings/abc123?severity=critical

Download Report

curl http://localhost:8000/reports/abc123 -o report.md

MITRE ATT&CK Navigator Layer

curl http://localhost:8000/mitre/abc123 -o layer.json
# Import at: https://mitre-attack.github.io/attack-navigator/

---

MCP Integration

RedCore exposes all 242 tools via the Model Context Protocol — every tool definition is dynamically converted to an MCP JSON schema. Compatible with Claude Desktop and any MCP client.

# stdio mode (Claude Desktop)
redcore mcp

# HTTP mode (API clients)
redcore mcp --http --port 8765

Claude Desktop Config

{
  "mcpServers": {
    "redcore": {
      "command": "redcore",
      "args": ["mcp", "--scope", "192.168.1.0/24"]
    }
  }
}

Claude can now call any of the 242 security tools directly:

> Use nmap to scan 192.168.1.0/24 for open ports
> Run nuclei against http://app.local with CVE templates
> Use kerbrute to enumerate users on the domain controller at 192.168.1.10

---

B2B Integrations

Add to config.yaml:

integrations:
  slack:
    enabled: true
    webhook_url: "https://hooks.slack.com/services/..."
    channel: "#security-alerts"

  jira:
    enabled: true
    base_url: "https://yourcompany.atlassian.net"
    project_key: "SEC"
    api_token: "..."
    email: "security@yourcompany.com"

  teams:
    enabled: true
    webhook_url: "https://outlook.office.com/webhook/..."

  webhook:
    enabled: true
    url: "https://your-siem.com/redcore"
    secret: "your-hmac-secret"

---

Running the Dataset Generator

# Full run: KB + all external datasets (target: 250K records)
python -m dataset_generator.generator \
  --input data/raw \
  --output data/generated \
  --target 250000

# KB-only run (no external data needed)
python -m dataset_generator.generator --kb-only --target 50000

# Dry run: estimate output counts without writing files
python -m dataset_generator.generator --dry-run

# Generate fine-tune ready output (Alpaca format)
python -m dataset_generator.generator --finetune-format

# Include theory/knowledge synthesis phase
python -m dataset_generator.generator --theory-format

---

Fine-Tuning

# 1. Generate dataset
make prepare-data          # runs fine_tuning/prepare_dataset.py

# 2. Train LoRA adapter — Qwen 3.5 9B (Windows, RTX 4060+)
python train_qwen.py

# 2b. Or generic script (Linux/any GPU):
make fine-tune             # runs fine_tuning/train_lora.py

# 3. Evaluate
make evaluate

# 4. Set adapter path in config.yaml:
# llm:
#   backend: transformers
#   adapter_path: data/finetune/qwen_redcore_lora

---

Testing

make test              # all tests
make test-unit         # unit tests only (tests/unit/)
make lint              # ruff check
make type-check        # mypy
make format            # ruff format

Unit test coverage:

• test_agents.py — planner, router, orchestrator
• test_parsers.py — nmap, nuclei, gobuster, hydra, sqlmap, bloodhound output parsing
• test_registry.py — tool loading, alias resolution, parameter validation
• test_safety.py — scope validation, command sanitizer, injection prevention
• test_wsl_executor.py — WSL2 execution, output path normalization, timeout handling

---

Security & Legal

⚠️ IMPORTANT: RedCore is for authorized penetration testing ONLY. Unauthorized use is illegal under the CFAA and equivalent laws in all jurisdictions.

Built-in safety controls:

• Mandatory scope validation — every tool invocation validates target against the defined scope before execution
• Command sanitizer — blocks shell injection (;, &&, |, backticks, etc.) in all tool parameters
• AES-256-GCM encrypted secret vault — credentials never stored in plaintext
• JWT multi-tenant auth — strict tenant isolation across all API endpoints
• Full audit log — every action logged with timestamps, job ID, and evidence hashes

---

Project Structure

redcore/
├── agent/          # Orchestrator, Planner, Router, Reporter, MITRE Mapper
├── api/            # FastAPI REST endpoints + WebSocket streaming
├── cli/            # Rich CLI (redcore validate | run | tools | mcp)
├── core/           # Config, safety, logging, evidence collection
├── execution/      # ExecutionEngine, DockerExecutor, WSLExecutor
├── integrations/   # Slack, Jira, Teams, Webhook notifiers
├── llm/            # LLM clients: Ollama, vLLM, OpenAI, Transformers
├── memory/         # ChromaDB vector store + NetworkX graph memory
├── models/         # Pydantic models: Job, Plan, Finding, ToolResult
└── tools/
    ├── definitions/   # 20 YAML files — 242 tool definitions
    ├── parsers/       # Output parsers: nmap, nuclei, gobuster, hydra…
    ├── registry.py    # Singleton tool registry + alias resolution
    └── mcp_server.py  # MCP server exposing all 242 tools

dataset_generator/
├── generator.py       # Main orchestrator — 6 generation strategies
├── config.py          # All tunable parameters (target counts, aug factors)
├── knowledge_base/    # Tool KB, scenario KB (25 red team scenarios)
├── augmentation/      # TargetVariator, ParamVariator, ToolSubstitutor
├── simulation/        # OutputSimulator, FailureSimulator (14 failure types)
├── ingestion/         # Dataset handlers: Atomic Red Team, CVE, Metasploit…
├── templates/         # Step/Plan/Result template engines
├── validation/        # Schema validator + auto-repair
└── export/            # JSONL exporter, stats reporter, knowledge exporter

fine_tuning/
├── train_lora.py      # Generic LoRA training (Mistral, LLaMA, Qwen2.5)
└── prepare_dataset.py # Dataset preparation + train/eval split

train_qwen.py          # Qwen 3.5 9B QLoRA on RTX 4060 (Windows)
gguf_to_hf_qwen3_5_9B.py  # GGUF → HuggingFace SafeTensors conversion

---

License

Proprietary — All Rights Reserved. RedCore is a commercial B2B product.

---

Support

• Issues: github.com/AaravMehta-07/Redcore/issues