The EU AI Act takes effect August 2, 2026. Your AI system needs compliance evidence. Your observability tools collect traces. Your GRC platform manages policies. Nothing translates traces into compliance evidence. This tool does.
Open-source CLI that audits LLM traces against EU AI Act Articles 11, 12, 13, and 25, plus NIST AI RMF and GDPR Article 30. 301 tests. Zero LLM dependencies. Runs locally.
Companies have traces from Langfuse, Arize, or OpenTelemetry, but no automated way to answer: "Do our traces satisfy the regulatory requirements?" 67% of AI teams discover quality regressions from user complaints despite having tracing infrastructure. The gap isn't data collection; it's interpretation.
Consultants charge $30K-$500K per compliance engagement. CEPS estimates EUR 29K per AI model for Annex IV documentation alone. This tool automates the gap analysis for free.
| Regulation | Scope |
|---|---|
| EU AI Act Article 11 | Technical documentation (Annex IV). Auto-generates 60-70% of required sections from code scanning |
| EU AI Act Article 12 | Record-keeping. Audits trace data against 15+ discrete requirements |
| EU AI Act Article 13 | Transparency (provider to deployer). Data flow mapping with GDPR role classification |
| EU AI Act Article 25 | Value chain accountability. Multi-agent DAG auditing with per-agent penalty propagation |
| NIST AI RMF | GOVERN, MAP, MEASURE, MANAGE subcategories (~15 requirements) |
| GDPR Article 30 | Records of Processing Activities. Auto-generated from detected data flows |
Automatically audits multi-agent systems (LangGraph, CrewAI, AutoGen, Google ADK):
- Reconstructs execution DAGs from parent-child span relationships
- Per-agent compliance scores with bottom-up penalty propagation
- Article 25 "value chain accountability" checks
- Liability shift detection (deployer becoming provider)
- Mermaid DAG visualizations
No other open-source tool does multi-agent compliance auditing.
| Format | Source |
|---|---|
| OTel OTLP JSON | OpenTelemetry GenAI semantic conventions |
| Langfuse JSON | Langfuse trace exports |
| Claude Code | ~/.claude/projects/ conversation traces |
| Raw JSONL | Any provider's API logs |
Auto-detected. Use --format to override.
pip install ai-trace-auditor# Audit traces against EU AI Act
aitrace audit traces.json -r "EU AI Act" -o report.md
# Full compliance package: Articles 11 + 12 + 13 + GDPR in one run
aitrace comply ./my-ai-project/ --traces traces.json
# Generate Annex IV technical documentation from code
aitrace docs ./my-ai-project/
# Map data flows for Article 13 + GDPR Article 30
aitrace flow ./my-ai-project/
# Multi-agent audit with DAG visualization
aitrace audit multi_agent_traces.json --show-dagReal output from auditing 1,522 Claude Code spans:
Overall Compliance Score: 79.3%
| Status | Count |
|-----------|-------|
| Satisfied | 10 |
| Partial | 5 |
| Missing | 3 |
Top gaps:
1. Not logging: Temperature parameter controlling output randomness
2. Not logging: Maximum token limit for output generation
3. Incomplete: Output responses generated by the AI model (31.9% coverage)
4. Incomplete: Input prompts/messages (4.7% coverage)
5. Not logging: Operation latency in milliseconds
- name: Audit AI traces
uses: BipinRimal314/ai-trace-auditor@v0.14.0
with:
path: traces/exported.json
regulation: "EU AI Act"
output: compliance-report.md
fail-on-gaps: "true"Exit code 0 = all requirements satisfied, 1 = gaps found. CI-friendly.
from ai_trace_auditor.ingest import ingest_file
from ai_trace_auditor.analysis.engine import ComplianceAnalyzer
from ai_trace_auditor.regulations.registry import RequirementRegistry
traces = ingest_file(Path("traces.json"))
registry = RequirementRegistry()
registry.load()
report = ComplianceAnalyzer(registry).analyze(
traces=traces,
regulations=["EU AI Act"],
)
print(f"Score: {report.overall_score:.1%}")ai-trace-auditor/
├── src/ai_trace_auditor/
│ ├── cli.py # 7 commands: audit, docs, flow, comply, ingest, requirements, health
│ ├── ingest/ # Trace ingestion (OTel, Langfuse, Claude Code, raw JSONL)
│ ├── analysis/ # Gap analysis engine + multi-agent DAG auditing
│ ├── models/ # Pydantic v2 data models
│ ├── regulations/ # YAML requirement definitions (extensible)
│ ├── docs/ # Article 11 Annex IV generator
│ ├── flow/ # Article 13 data flow mapper + GDPR RoPA
│ ├── comply/ # Full compliance package runner
│ ├── evidence/ # Auditor-ready evidence pack generator
│ ├── reports/ # Markdown, JSON, PDF report generation
│ ├── scanner/ # Code scanner (AI framework detection)
│ └── guide_linter/ # Lints compliance guides for accuracy
├── requirements/ # YAML regulatory requirement definitions
│ ├── eu_ai_act/ # Articles 12, 19
│ └── nist_ai_rmf/ # GOVERN, MAP, MEASURE, MANAGE
└── tests/ # 301 tests
No dependency on any LLM framework. Intentionally framework-agnostic.
| Tool | What It Does | How We Differ |
|---|---|---|
| Langfuse / Arize | Collect and visualize traces | We interpret traces against regulations |
| Credo AI ($41M raised) | Enterprise AI governance platform | Free, open-source, runs locally |
| OneTrust / Vanta | GRC policy management | We ingest AI-specific traces, not generic policy docs |
| Holistic AI | AI risk management (consulting) | CLI + CI integration, no sales call required |
The competition is consultants, not software.
This tool provides automated compliance analysis. It is not legal advice. Risk classification under Annex III requires legal review. Consult qualified counsel for compliance decisions.
Apache 2.0