diff --git a/README.md b/README.md index e5d5b7b..8b15794 100644 --- a/README.md +++ b/README.md @@ -1,95 +1,98 @@ # Cogitator -Cogitator is a Rust harness for deterministic, tamper-evident records of agent/tool execution. +**A Rust harness that turns an agent run into a tamper-evident, independently verifiable record.** [](https://github.com/noctem-o/cogitator/actions/workflows/ci.yml) [](LICENSE) [](https://www.rust-lang.org/) [](https://github.com/noctem-o/cogitator/releases) [](spec/COGITATOR_WITNESS_PROTOCOL.md) -[](#design-notes--threat-model) -**Navigation:** [Quick start](#quick-start) · [Verification model](#verification-model) · [Policy interception](#policy-interception) · [Protocol](spec/COGITATOR_WITNESS_PROTOCOL.md) · [Threat model](#design-notes--threat-model) · [Development](#development) +Cogitator records what an agent did, what it asked to do, and what a policy stopped it from doing — then seals the run under a single cryptographic fingerprint, the **witness root**. Anyone holding the run bundle can recompute that fingerprint and detect whether the record was altered after the fact. -Each run emits a witness root, a replayable run bundle, and a policy-aware tool transcript. The witness root is computed from canonical witnessed semantics (not from report files), and blocked/phantom operations are recorded explicitly. +## The idea in 60 seconds -## Why it exists +Most agent "audit trails" are ordinary log files: mutable, easy to trim, and written by the same system you are trying to audit. Cogitator treats the run itself as evidence: -Most agent audit trails are mutable logs plus summaries written after execution. That is useful for debugging, but weak as evidence. +1. Every agent step and tool call is serialized as canonical JSON (sorted keys, integer-only numbers, byte-stable output). +2. Before any tool call is dispatched, a policy gate returns a verdict: **allow**, **block**, or **phantom** (observe the request without executing it). Blocked and phantom calls are recorded as first-class entries — refusals are part of the evidence, not silently dropped. +3. Each event is folded into a domain-separated BLAKE3 hash chain. The final digest is the witness root. +4. A verifier recomputes the chain from the recorded semantics. One changed byte in any committed field produces a different root. -Cogitator commits the execution boundary before tool effects become the only source of truth. It records what the agent requested, what was executed, and what was intercepted. +Timestamps, hostnames, toolchain versions, thread counts, and other environment details are recorded as *provenance*, deliberately **outside** the commitment — so the same logical run hashes identically across machines. -The project is a verifier/audit substrate. It is not a full agent framework and it does not claim to solve runtime trust on its own. +```text +Agent step + -> tool request + -> policy gate + -> allow: executed ToolCall + -> block/phantom: PhantomEntry (recorded, not executed) + -> canonical JSON witness events + -> domain-separated BLAKE3 chain + -> witness root +``` + +## Scope: what this is, and is not + +**It is** the reference implementation of the [Cogitator Witness Protocol](spec/COGITATOR_WITNESS_PROTOCOL.md): a recorder, a pre-call policy gate, and a verifier, plus deterministic fixtures that exercise and pin the protocol. -## What you get +**It is not** an agent framework, and it does not yet ship a drop-in integration for external agents. The agents and tools it currently drives are deterministic stand-ins: -| Capability | What it means in practice | +- `clawdbot` — a small scripted demo agent. +- `ordeal` — a fixed 50-task conformance suite with stubbed tool responses. It guards the protocol against accidental change (its witness root is pinned as a golden in CI); it is **not** a benchmark and passing it says nothing about real agent quality. +- Tool calls are answered by deterministic stubs, and the `llm.generate` backend is a stub. No external model or service is contacted. + +Wiring a real agent in today means driving the Rust APIs (`ToolTranscript`, the witness encoding) from your own code. + +**What a matching root proves — and what it doesn't:** + +| | | |---|---| -| Witness root | Domain-separated BLAKE3 commitment over witnessed metadata, agent trace entries, and tool/phantom operations. | -| Strict canonical witnessed bytes | Deterministic canonical JSON subset for committed bytes (JCS-style, integer-only I-JSON-oriented subset). | -| Tool transcript | Executed calls with outcomes and faults, plus policy digest when present. | -| Phantom entries | Blocked/phantom tool requests are recorded and committed instead of silently dropped. | -| Bundle verifier | Path confinement + artifact self-consistency checks on bundle contents. | -| Replay/drift reports | Deterministic diagnostics for fixture/replay mismatch analysis (report-only artifacts). | -| Provenance split | Provenance is recorded separately and excluded from witness bytes. | -| Nix provenance (optional) | `nix_provenance.json` is diagnostic/provenance-only and does not change witness root. | +| Proves | The bundle's committed semantics (metadata, agent trace, tool/phantom operations) are exactly what the root commits to — nothing was edited, reordered, truncated, or injected after sealing. | +| Does not prove | That the run actually happened at a given time. A bundle and its root can be regenerated wholesale; to prove occurrence, anchor the root externally (publish it, timestamp it, or compare against a root you received out-of-band via `--expect`). | +| Does not prove | Hardware or runtime integrity of the machine that produced the bundle, or determinism of nondeterministic model backends. | ## Quick start -`--clean` now refuses dangerous output paths (such as filesystem root, home, repo root, `.git`, and symlinked output dirs). -`--nix-provenance=auto|on` may execute bounded local Nix commands for diagnostic provenance (closed stdin, timeout-bounded, output-capped); it does not affect witness root computation. - ```bash cargo build --release ``` +Run the drift demo (four scenarios: baseline, regressed, with and without injected faults) and verify the resulting bundle: + ```bash cargo run --release -- demo drift --seed 42 --threads 1 --fault-profile stress --out-dir demo_out --clean -``` -```bash cargo run --release -- verify --witness demo_out/drift/baseline_faults cargo run --release -- verify --witness demo_out/drift/baseline_faults --recompute-witness-root ``` -Expected result: - -- verify reports no bundle issues -- witness recompute reports `matched=true` +Expected: verification reports no bundle issues, and the recompute reports `matched=true`. -## Record and verify a fixture-grade agent run +Record and verify a fixture run: ```bash cargo run --release -- run --agent ordeal --runs 1 --out-dir out --clean -# NOTE: `ordeal` is a deterministic conformance fixture, not a scientific benchmark. -``` - -```bash -cargo run --release -- verify --witness out/run_0000 -``` - -```bash cargo run --release -- verify --witness out/run_0000 --recompute-witness-root ``` -## Policy interception +Notes: -Cogitator evaluates policy before dispatch. First matching rule wins. +- `--clean` refuses dangerous output paths (filesystem root, home, repo root, `.git`, symlinked dirs). +- `--nix-provenance=auto|on` may run bounded local Nix commands (closed stdin, 3s timeout, capped output) to capture diagnostic provenance. This never affects the witness root. -- `allow`: call is executed and recorded as a real tool entry. -- `block`: call is denied; a blocked phantom entry is recorded. -- `phantom`: call is intercepted; a phantom entry is recorded. +## Policy interception -If no policy file is provided, current behavior is allow-all. +Policy is evaluated before dispatch; the first matching rule wins. When a policy file is loaded, its SHA-256 digest is committed into the witness root, so the exact policy in force is part of the evidence. If no policy file is present, the current behavior is **allow-all** and no digest is committed. ```toml schema_version = 1 [[rules]] id = "trade-budget" -tool_pattern = "trade.*" +tool_pattern = "trade.*" # '*' stays within a dot-segment; '**' crosses segments history_tool_pattern = "trade.*" -history_max_calls = 2 +history_max_calls = 2 # allow calls 1-3, block call 4 onward verdict = "block" reason = "trade call budget exceeded" @@ -100,97 +103,45 @@ verdict = "phantom" reason = "observe only" ``` -```text -Agent step - -> tool request - -> policy gate - -> allow: executed ToolCall - -> block/phantom: PhantomEntry - -> canonical witness event stream - -> BLAKE3 witness root -``` +Verdicts: -## Verification model +- `allow` — the call executes and is recorded as a real `ToolCall`. +- `block` — the call is denied; a `PhantomEntry` with disposition `blocked` is recorded and committed. +- `phantom` — the call is intercepted; a `PhantomEntry` with disposition `phantom` is recorded and committed. -Cogitator verification has three recomputation-backed layers: +## Verifying a bundle -1. **Bundle self-consistency**: manifest paths + diagnostic artifact hashes. -2. **Semantic witness recompute**: recompute root from witnessed semantics. -3. **Anchored verification**: compare against an externally supplied expected root (`--expect`). +Three layers, in increasing strength: -`witness_root.txt` stored in the same bundle is useful for internal consistency checks, but by itself it does not prove original occurrence. +| Layer | Command | What it checks | +|---|---|---| +| Bundle self-consistency | `verify --witness