LLMShield AI

Defensive GenAI Security Evaluation & Red Teaming

Test chatbots, RAG pipelines, and AI-agent workflows against prompt injection, data leakage, unsafe tool use, excessive agency, and other LLM security risks.

About

LLMShield AI is a full-stack platform for repeatable, authorized security evaluation of generative-AI applications. It combines versioned attack packs, provider adapters, explainable detectors, risk scoring, evidence redaction, audit logging, and multi-format reports in one local-first workflow.

The default MockLLM provider is deterministic and requires no paid API key. Optional adapters support OpenAI and local Ollama deployments. Scans can run synchronously for dependable local use or through Redis and RQ workers for asynchronous execution.

Important

Use LLMShield AI only with local demos, user-owned systems, or endpoints you are explicitly authorized to test. It is a defensive evaluation tool, not an exploitation framework.

Highlights

Area	Capability
Attack coverage	15 versioned YAML attack packs with 60 safe defensive test cases
Targets	Chatbots, RAG systems, agent workflows, MockLLM, and allowlisted custom endpoints
Providers	Deterministic MockLLM, optional OpenAI, and local Ollama adapters
Detection	Explainable rules for leakage, injection, tool misuse, policy violations, and related behavior
Risk	Severity, confidence, weighted risk scoring, evidence, and mitigation guidance
Reports	JSON, Markdown, HTML, PDF, and JSONL execution artifacts
Access control	JWT authentication, role-based access, and hashed API keys
Operations	Audit logs, Prometheus-style metrics, CI thresholds, and optional Redis/RQ workers
Safety	Endpoint allowlists, SSRF checks, metadata-IP blocking, size limits, timeouts, rate limits, and redaction

Interface

Every image below was captured with Playwright Chromium from the running FastAPI and Next.js application. Capture details are recorded in the screenshot manifest.

Sign in	Risk command center
Target registry	Attack packs
Scan visibility	Findings
Reports	Providers and safety
Audit logs and metrics	In-app documentation

Architecture

flowchart LR
    UI[Next.js dashboard] --> API[FastAPI API]
    API --> AUTH[JWT / RBAC / API keys]
    API --> TARGETS[Target registry + URL safety]
    API --> SCANS[Scan orchestrator]
    SCANS --> MODE{Execution mode}
    MODE -->|sync| ENGINE[Scan engine]
    MODE -->|async| QUEUE[Redis / RQ]
    QUEUE --> WORKER[Worker]
    WORKER --> ENGINE
    ENGINE --> PACKS[Versioned YAML attack packs]
    ENGINE --> PROVIDERS[MockLLM / OpenAI / Ollama]
    ENGINE --> DETECTORS[Detector pipeline]
    DETECTORS --> RISK[Risk scoring]
    RISK --> REPORTS[JSON / MD / HTML / PDF]
    API --> DB[(SQLite or PostgreSQL)]
    API --> AUDIT[Audit logs + metrics]

Loading

The scanner keeps provider calls, detector logic, scoring, persistence, and report generation separated so each layer can be tested or replaced independently. See the technical deep dive for implementation details.

Security checks

The bundled attack packs cover:

• direct and indirect prompt injection;
• jailbreak and refusal bypass;
• system-prompt and simulated secret leakage;
• sensitive-data and data-boundary violations;
• RAG poisoning and citation fabrication;
• unsafe tool use and excessive agency;
• role confusion and multi-turn injection;
• output-policy violations; and
• token and cost abuse.

Each finding can include the triggering prompt, redacted response evidence, matched detector rules, severity, confidence, risk score, OWASP-style category, MITRE ATLAS-style tactic, and remediation guidance.

Quick start

Prerequisites

• Python 3.11 or newer
• Node.js 22 or newer
• npm

1. Clone and configure

git clone https://github.com/HUSNAIN-MUNAWAR/LLMShield-AI.git
cd LLMShield-AI
python -m venv .venv

Activate the environment:

# Linux or macOS
source .venv/bin/activate

# Windows PowerShell
.\.venv\Scripts\Activate.ps1

Install dependencies and create the local environment file:

pip install -r backend/requirements.txt
cp .env.example .env
cd frontend && npm ci && cd ..

On Windows, use Copy-Item .env.example .env instead of cp.

2. Start the API

cd backend
alembic upgrade head
uvicorn app.main:app --reload --host 127.0.0.1 --port 8000

3. Start the web interface

In a second terminal:

cd frontend
npm run dev

Open:

• Web interface: http://localhost:3000
• Interactive API docs: http://localhost:8000/docs
• Health endpoint: http://localhost:8000/health
• Metrics: http://localhost:8000/metrics

Demo roles

These credentials are local examples and must not be used in a deployed environment.

Role	Email	Password
Admin	admin@llmshield.local	admin123
Analyst	analyst@llmshield.local	analyst123
Viewer	viewer@llmshield.local	viewer123

Docker

The Compose stack runs PostgreSQL, Redis, the API, an RQ worker, and the Next.js frontend:

cp .env.example .env
docker compose up --build

Then open http://localhost:3000. Docker mode uses asynchronous scan execution; the local quick start defaults to synchronous execution.

# Linux or macOS
bash scripts/docker_smoke_test.sh

# Windows PowerShell
powershell -ExecutionPolicy Bypass -File scripts/docker_smoke_test.ps1

Providers

Provider	Default	Credentials	Intended use
MockLLM	Yes	None	Deterministic local tests, CI, and demos
OpenAI	No	OPENAI_API_KEY	Explicitly configured API evaluation
Ollama	No	None	Local model evaluation through Ollama

Run a provider smoke test:

python scripts/provider_smoke_test.py --provider mock
python scripts/provider_smoke_test.py --provider ollama --model llama3.1
python scripts/provider_smoke_test.py --provider openai --model gpt-4o-mini --dry-run

Provider secrets are never committed and should not be printed in logs.

CI mode

LLMShield AI can enforce a finding threshold in an automated pipeline:

cd backend
python -m app.cli ci-scan \
  --target mock-safe \
  --attack-packs prompt_injection,jailbreak,rag_poisoning \
  --fail-on high \
  --output ../sample_outputs/reports/ci_report.json

The included GitHub Actions workflow runs backend tests, frontend type checking and build validation, repository validation, secret scanning, and a MockLLM provider smoke test. A separate example workflow uploads the CI scan report as an artifact.

Reports and evidence

Generated reports include authorization scope, target and provider context, attack-pack versions, severity distribution, findings, redacted evidence, mitigation guidance, CI status, limitations, and a test-case appendix.

Open the checked-in sample artifacts:

• PDF report
• HTML report
• Markdown report
• JSON report
• CI result

Regenerate them with:

python scripts/generate_sample_reports.py

Validation

Run the full validation suite:

python scripts/run_all_checks.py

Or run checks individually:

python scripts/validate_project.py
python scripts/secret_scan.py
cd backend && pytest -q
cd ../frontend && npm run typecheck && npm run build
cd ..
python scripts/provider_smoke_test.py --provider mock
python scripts/verify_screenshots.py

To regenerate real UI screenshots:

pip install -r requirements-dev.txt
python -m playwright install chromium
python scripts/capture_real_screenshots.py --real
python scripts/verify_screenshots.py

Project layout

LLMShield-AI/
├── backend/
│   ├── alembic/                 # Database migrations
│   ├── app/
│   │   ├── api/                 # HTTP routes and dependencies
│   │   ├── core/                # Security, config, DB, logging
│   │   ├── data/attack_packs/   # Versioned defensive test packs
│   │   ├── models/              # SQLAlchemy models
│   │   ├── services/            # Scanner, detectors, providers, reports
│   │   └── workers/             # Redis/RQ execution
│   └── tests/
├── frontend/
│   ├── app/                     # Next.js routes
│   ├── components/              # UI and feature components
│   └── public/reports/          # Downloadable sample reports
├── docs/                        # Architecture and operations guides
├── evidence/                    # Validation outputs
├── sample_outputs/              # Reports, findings, logs, screenshots
├── scripts/                     # Validation, capture, smoke-test utilities
└── .github/workflows/           # CI and security automation

Security boundaries

Implemented safeguards include:

• allowlisted custom endpoints and scheme validation;
• SSRF checks and metadata-address blocking;
• private-network blocking unless explicitly enabled;
• request timeouts and prompt/response size limits;
• basic rate limiting;
• secret and PII redaction in evidence and reports; and
• audit events for authentication, target changes, scans, exports, providers, and API keys.

LLMShield AI does not guarantee model security, fully prevent prompt injection, replace an expert security assessment, or authorize testing of third-party systems. Review security boundaries and limitations before use.

Documentation

• Architecture
• Technical deep dive
• Threat model
• OWASP mapping
• MITRE ATLAS mapping
• LLM security testing
• CI mode
• Deployment
• Production hardening
• Audit logging
• Observability
• Contributing
• Security policy

Roadmap

• OIDC and SAML integration
• team and project authorization
• server-sent events or WebSocket scan streaming
• multi-tenant workspaces
• SIEM export
• richer detector rules and dataset versioning
• provider usage and cost dashboards
• signed report artifacts

License

Released under the MIT License. Use responsibly and only for authorized defensive testing.