SLEATH — Secure LLM Proxy

A 3-layer security proxy that sits between your application and the Gemini API. Every request flows through injection detection and semantic analysis before reaching the model, and through output filtering before the response reaches the client.

Architecture

Client → L1 Sanitization → L2 Semantic → [Gemini] → L4 Output → Client

L1 — Sanitization: Unicode normalization (NFKC), homoglyph mapping (Cyrillic/Greek lookalikes → ASCII), invisible character stripping, repeated character collapsing, and regex pattern matching against known prompt injection signatures loaded from config/patterns.yaml. Supports hot-reload and YAML-configurable block thresholds per severity level (CRITICAL / HIGH / MEDIUM / LOW).

L2 — Semantic: Two-stage classifier. First, a FAISS vector index built from a configurable seed attack corpus (BAAI/bge-small-en-v1.5 embeddings) runs cosine similarity against the incoming prompt. Scores above threshold_block are rejected immediately; scores in the review band escalate to a Gemini intent classifier that returns a safe/not_safe category with confidence. Supports fail_open mode on LLM timeout and hot-reload of both FAISS and LLM config.

L4 — Output: Four-stage response filter applied before the result reaches the client. (1) Regex PII redaction (SSN, credit card, IBAN, IP, email, phone, DOB). (2) Presidio NER-based PII anonymization using spaCy en_core_web_sm with configurable entity list and score threshold. (3) Secret/credential detection (AWS keys, RSA private keys, JWT tokens, Bearer tokens, GitHub tokens, Google API keys, Slack tokens, connection strings). (4) Content safety scan with severity-ranked rules (system prompt echo, jailbreak confirmation, harmful instructions, internal data leakage, dangerous medical overconfidence) — violations are blocked or warn-only depending on config.

Setup

Backend

cd proxy
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python -m spacy download en_core_web_sm

cp .env.example .env
# Edit .env and add your GEMINI_API_KEY

uvicorn main:app --reload --host 0.0.0.0 --port 8000

Frontend

cd frontend
npm install

cp .env.example .env
# Edit .env if your proxy is not running on localhost:8000

npm run dev

Usage

Point any OpenAI-compatible SDK at http://localhost:8000 instead of the Gemini base URL:

from openai import OpenAI

client = OpenAI(
    api_key="your-gemini-api-key",
    base_url="http://localhost:8000/v1",
)

response = client.chat.completions.create(
    model="gemini-2.0-flash",
    messages=[{"role": "user", "content": "Hello"}],
)

The dashboard runs at http://localhost:5173 (frontend dev server) and shows live request logs, block rates, and per-layer status.

Configuration

All layer behavior is controlled by YAML files in proxy/config/:

File	Controls
patterns.yaml	L1 regex patterns, severity labels, and block thresholds (block_on_flag_count, block_on_medium_count)
layer2_config.yaml	FAISS block/review thresholds, seed attack corpus, Gemini classifier model, timeout, and fail_open behavior
output_policy.yaml	L3 Presidio entity list and score threshold, regex PII toggles, secret pattern toggles, content safety rules and block threshold

All configs support hot-reload without restarting the server via the /reload admin endpoint.

Admin Endpoints

Endpoint	Description
GET /health	Layer status and Gemini API key check
GET /metrics	Request counters and latency percentiles
GET /logs	Audit log (supports ?blocked_only=true&layer=L2_Semantic)
GET /logs/stream	Server-Sent Events live log stream
GET /layers	Pipeline order and per-layer config summary
GET /docs	Swagger UI