RISK//REPLAY

AI Decision Forensics & Counterfactual Replay Engine

Reconstruct the decision. Change the evidence. See what breaks.

Live demo is a static build against a precomputed dataset, no backend required, see Static demo mode below. For the full dynamic backend, run it locally: see Quick start.

The problem

Most AI-assisted systems can tell you what a decision was. Some can show you the logs around it. Almost none can answer the question an investigator actually asks after an incident:

"Here is the exact decision state, here is the evidence that influenced it, here is the policy and model state at that moment, here is what happens when we change one variable, and here is the resulting risk and governance impact."

RISK//REPLAY is a small, serious engine built to answer exactly that question, and nothing more.

What it is not

Not a dashboard, not a chatbot, not an LLM wrapper, not a generic audit-log viewer. There is no LLM anywhere in the decision path. The core is a deterministic rule engine that is fully inspectable: every score in this system is defined by a formula in backend/app/engines/, not inferred by a model.

The core idea

DECISION -> EVIDENCE GRAPH -> LINEAGE -> DETERMINISTIC REPLAY -> MUTATION
    -> COUNTERFACTUAL REPLAY -> DIFF -> CAUSAL ANALYSIS -> RISK/GOVERNANCE -> REPORT

Every historical AI-assisted decision is stored as a frozen DecisionContext: the input, the evidence used, the model version, the policy version, the tool calls, the controls. Nothing about a replay ever reaches into current production state -- it replays against exactly the historical snapshot it is given, or against a mutated copy of that snapshot if you're running a counterfactual.

The killer demo

DEC-001 is a synthetic fraud-engine decision (all data below is fictional, generated for this repo):

ORIGINAL DECISION
  Outcome:    BLOCK
  Score:      0.834   (policy-17 block threshold: 0.82)
  Model:      fraud-v3.2
  Policy:     policy-17
  Evidence:   E1 (velocity_signal) E2 (payment_history_risk)
              E3 (identity_risk)   E4 (device_mismatch)

Remove one evidence item (E3, the identity-risk signal) and replay:

COUNTERFACTUAL
  Outcome:    ALLOW
  Score:      0.549

DIVERGENCE:        YES
CHANGED VARIABLE:  Evidence E3
CAUSAL FINDING:    DECISION_CRITICAL under the replay model
GOVERNANCE IMPACT: Control C-17 (Human Oversight) status flips --
                    the decision no longer crosses the review threshold either

Put E3 back, replay again: BLOCK returns, at exactly 0.834, because the engine is deterministic. Try it yourself:

curl -X POST localhost:8000/decisions/DEC-001/counterfactual \
  -H "Content-Type: application/json" \
  -d '{"mutations": [{"type": "REMOVE_EVIDENCE", "target": "E3", "reason": "investigate"}]}'

Second demo: policy impact replay

"What would have happened if we'd deployed policy-18 yesterday?"

curl -X POST localhost:8000/policies/policy-18/impact-replay \
  -H "Content-Type: application/json" \
  -d '{"policy_id":"policy-18","block_threshold":0.75,"review_threshold":0.45,"required_controls":["C-17","C-08"]}'

Against the 40-decision golden dataset this replays every historical decision under the new thresholds and reports exactly how many outcomes change, how many now require human review, and which specific decisions moved.

Third demo: Forensic Sweep

One click on DEC-001's Decision Forensics screen: "RUN FORENSIC SWEEP" automatically builds and runs one single-variable counterfactual per evidence item, tool result and control already in the recorded context (9 experiments for DEC-001), ranks them by a deterministic sensitivity score, and shows exactly which one is decision-critical -- without you having to build any of those counterfactuals by hand:

curl -X POST localhost:8000/decisions/DEC-001/forensic-sweep

Removing E3 alone flips BLOCK -> ALLOW (score delta -0.285, DECISION_CRITICAL); removing E1, E2 or E4 alone drops the score into REVIEW but does not clear it (also DECISION_CRITICAL for the REVIEW boundary, not the BLOCK boundary); removing any tool result or control changes nothing (DECISION_IRRELEVANT). CHANGE_POLICY, CHANGE_MODEL, CHANGE_THRESHOLD and other mutation types that would require fabricating a hypothetical value are deliberately not auto-generated -- see docs/forensic-sweep.md for exactly why, and how to run them manually instead. Full design and the TS/Python parity guarantee for the static demo: docs/forensic-sweep.md.

Architecture

flowchart TD
    UI["WEB UI
React/Vite, minimal (frontend/)"]
    API["REST API
FastAPI"]
    DB["Decision Store / Event Store
Policy & Model Registry
SQLAlchemy -> Postgres or SQLite"]

    subgraph ENGINES["backend/app/engines/*.py
pure, deterministic: zero SQL, zero HTTP, zero LLM calls"]
        DE["decision_engine.py"]
        RE["replay_engine.py"]
        DNA["dna_engine.py"]
        MU["mutation_engine.py"]
        CF["counterfactual_engine.py"]
        DI["diff_engine.py"]
        RI["risk_engine.py"]
        GO["governance_engine.py"]
        BO["boundary_engine.py"]
        SW["sweep_engine.py"]
        IN["incident_engine.py"]
        IT["integrity_engine.py"]
    end

    UI --> API --> DB --> DE --> RE --> DNA --> MU --> CF --> DI --> RI --> GO
    RI --> BO
    RI --> SW
    GO --> IN
    IN --> IT

Loading

The engines never import SQLAlchemy or FastAPI. They operate purely on the frozen dataclasses in backend/app/domain/models.py. This is deliberate: it's what makes the backend independently testable and what makes "the core does not depend on an LLM" an enforceable structural fact rather than a promise.

Failure is a feature

Not every decision can be perfectly reconstructed. RISK//REPLAY has an explicit ReplayabilityStatus: REPLAYABLE, PARTIALLY_REPLAYABLE, NON_REPLAYABLE, each with a machine-readable reason (MISSING_MODEL_VERSION, NON_DETERMINISTIC_TOOL, etc). A forensic system that confidently invents an answer is worse than one that says "this cannot be reconstructed, here's why." See docs/failure-modes.md.

Same discipline applies to causal language: the causal engine never claims real-world certainty. A single-variable flip is reported as DECISION_CRITICAL under the replay model, not "the true cause." See docs/methodology.md.

Quick start

cd backend
uv sync                     # or: pip install -r requirements.txt
python -m app.seed          # loads the 40-decision golden dataset + demo incident
uv run uvicorn app.api.main:app --reload   # http://localhost:8000

Run the tests:

cd backend
python -m pytest -q         # 102 tests: unit, integration, property-based

Static demo mode

GitHub Pages only serves static files, it can't run the Python backend, so the live demo above ships a client-side build: backend/scripts/dump_static_bundle.py runs the real engines once against the golden dataset and freezes the output to frontend/public/data/bundle.json, and frontend/src/staticEngine.ts is a line-for-line TypeScript port of the decision/mutation/diff/causal engines so the Replay Lab's counterfactual feature stays genuinely interactive (build a mutation, it actually gets evaluated, not looked up from a fixed list). frontend/src/api.ts picks this (VITE_STATIC=true) or the real backend (liveApi.ts, default in dev) at build time. Verified the TS port reproduces the exact backend numbers for the golden scenario and for multi-variable mutations before deploying.

The dynamic backend (persistence, incidents, policy-impact-replay, arbitrary decisions beyond the golden dataset) only runs locally -- see Quick start below.

What's built vs. what's deferred

Built and tested end-to-end against real persisted data:

• Domain model, deterministic decision engine, replay engine with replayability assessment, mutation engine (9 mutation types), counterfactual engine, diff engine, causal engine, risk engine, governance engine, incident reconstruction, blast radius, policy-impact replay.
• Full REST API (backend/app/api/main.py) -- every route calls a real engine, nothing hardcoded.
• Forensic Sweep engine (backend/app/engines/sweep_engine.py) -- automatic, deterministic single-variable counterfactual scan per decision, reusing every existing engine rather than duplicating their formulas. See docs/forensic-sweep.md.
• Decision Boundary Analyzer (backend/app/engines/boundary_engine.py) -- single source of truth for zone/threshold math (GET /decisions/{id}/boundary), reused by the sweep engine and Decision DNA rather than re-derived. See docs/decision-boundary.md.
• Decision DNA (backend/app/engines/dna_engine.py) -- deterministic, human-readable forensic summary per decision (GET /decisions/{id}/dna), never an opaque embedding. See docs/decision-dna.md.
• Replay Integrity (backend/app/engines/integrity_engine.py) -- recomputes evidence/ input/context content hashes and compares against what was persisted (GET /decisions/{id}/integrity), catching tampered values, changed model/policy IDs after recording, missing hashes and malformed values. Explicit about what it does and does not guarantee -- not a signature scheme, no dedicated hash for model weights or raw tool output. See docs/integrity.md.
• Incident Blast-Radius Forensics (backend/app/engines/incident_engine.py) -- GET /incidents/{id}/blast-radius now runs a real per-decision counterfactual (remove the incident's tagged evidence kind, replay, check if the outcome flips) across every decision in the incident, plus a real replayability breakdown and affected policy/model/source/control sets. Fixes a real prior bug where this endpoint always passed an empty change list and reported zero impact regardless of the incident. The "common replay dependency" it reports is explicitly labeled as a fact about the replay model, never asserted as the real-world root cause. See docs/incident-forensics.md.
• 40-decision golden dataset, event log, SQLite by default / Postgres-ready.
• 102 automated tests: unit, API integration, Hypothesis property-based (Replay(original) == original, remove-then-restore round-trips, threshold mutations never leave [0,1]), Forensic Sweep-specific determinism/isolation/adversarial cases, Boundary Analyzer / Decision DNA unit tests, Replay Integrity adversarial tests (tampered evidence, tampered input, changed model/policy, missing hash, malformed value), and Incident Blast-Radius tests (dependency detection, honest no-op path, wording rule).

Deliberately deferred (see docs/ for the honest list, not hidden):

• Full 10-screen forensic UI -- v1 ships Command Center, Decision Vault, Decision Forensics, and the Replay/Mutation Lab (the killer feature), wired to the real API. Timeline, Evidence Graph visualization, and Incident Console are backend-complete (real endpoints, real data, including a real per-decision blast-radius counterfactual) but not yet given dedicated screens.
• PII redaction / field-level masking engine.
• Full adversarial test suite (forged evidence, replay poisoning) -- covered conceptually in docs/threat-model.md, not yet as executable tests.

Docs

• docs/architecture.md -- component map and why
• docs/replay-model.md -- what determinism means here, and its limits
• docs/counterfactual-engine.md -- mutation + counterfactual mechanics
• docs/decision-lineage.md -- the lineage graph
• docs/governance-engine.md, docs/risk-engine.md -- formulas
• docs/methodology.md -- causal language rules
• docs/failure-modes.md -- non-replayability, explicitly
• docs/forensic-sweep.md -- automatic counterfactual sweep design, and why some mutation types are excluded from it
• docs/decision-boundary.md -- zone/threshold math, the single source of truth for boundary crossings
• docs/decision-dna.md -- deterministic forensic summary per decision, and the honest limits of integrity_status
• docs/integrity.md -- what content-hash verification catches, and what it genuinely cannot
• docs/incident-forensics.md -- blast-radius mechanics, the bug it fixed, and the "common replay dependency" wording rule
• docs/security.md, docs/threat-model.md -- what provenance hashing does and does not guarantee

Disclosure

All data in this repository (app/golden_dataset.py) is synthetic and fictional, generated for demonstration. No real transactions, accounts, or personal data.