Repository navigation
Add authenticated execution evidence and prepare Lens 0.12.0 - #80
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 21b0f895c1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| with tempfile.TemporaryDirectory() as directory: | ||
| path = Path(directory) / "run.json" | ||
| path.write_text(json.dumps(artifact, allow_nan=False), encoding="utf-8") | ||
| report = validate_aios_artifact(path) |
There was a problem hiding this comment.
On a normal installation without a sibling ai-operations-spec checkout, every execution export/import fails here: validate_aios_artifact() calls _resolve_spec(), which searches only sibling repository paths and raises ValueError when none exist. Consequently, capture and replay are unusable, and even valid authenticated bundles receive a failed verification result. Repository-wide search found the required AIOS schemas only under tests/fixtures/ai-operations-spec/, while the wheel configuration includes only the top-level schemas/ directory. Package the required validation resources and resolve them from the installation rather than requiring an undeclared sibling checkout.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
🟡 Changes recommended
A critical finding remains: the execution export, import, capture, replay, and verify paths call AIOS validation without a spec root, so they fail outside a sibling ai-operations-spec checkout and report valid evidence as failed.
6 open issues
What changed in this PR
This PR adds tamper-evident evidence to Lens. Saved records are sealed in an ordered hash chain with an HMAC-authenticated manifest. Verification returns verified, failed, unknown or incomplete, and only verified allows replay. The PR also adds AIOS 0.4-draft Run export and import for execution evidence, a evidence CLI, published schemas, and offline examples. It prepares the 0.12.0 release. The PR description says the MCP 0.5.2 and Tower 0.9.1 integrations will consume this release after publication.
Changes:
- Add authenticated snapshots: a SHA-256 hash chain over ordered records, sealed with an HMAC-SHA256 manifest, plus coverage-aware verification and replay.
- Add AIOS
0.4-draftexecution export and import, with authenticated bundles and redacted summaries for downstream use. - Add the
agenticlens evidenceCLI (verify,export,replay), published JSON schemas, valid and truncated fixtures, offline examples, and the trust-model guide. Bump the version to 0.12.0.
| File | Description |
|---|---|
| uv.lock | Updates the locked agenticlens version to 0.12.0. |
| tests/test_integrity.py | Tests the snapshot seal, verify and replay paths, tampering, truncation, trust-key handling and CLI behavior. |
| tests/test_execution_evidence.py | Tests AIOS export and import, status mapping, redaction, CLI replay, fixtures and schema drift. |
| src/agenticlens/integrity/execution.py | Maps native runs and explicit execution records to AIOS 0.4-draft artifacts and verifies or replays them. |
| src/agenticlens/integrity/__init__.py | Implements the hash chain, HMAC manifest, verification statuses, and replay that returns copies. |
| src/agenticlens/cli/main.py | Registers the new evidence command group. |
| src/agenticlens/cli/integrity.py | Adds evidence verify, export and replay commands that fail closed on bad input. |
| src/agenticlens/__init__.py | Bumps the package version to 0.12.0. |
| schemas/integrity/verification.schema.json | Publishes the JSON schema for verification results. |
| schemas/integrity/manifest.schema.json | Publishes the JSON schema for authenticated manifests. |
| schemas/integrity/execution-record.schema.json | Publishes the JSON schema for explicit execution records. |
| schemas/integrity/execution-bundle.schema.json | Publishes the JSON schema for the authenticated AIOS execution bundle. |
| README.md | Adds a short section on evidence snapshots and the execution export guide. |
| pyproject.toml | Bumps the project version to 0.12.0. |
| mkdocs.yml | Adds the evidence verification guide to the docs navigation. |
| examples/evidence/fixtures/valid.json | Provides a valid synthetic execution bundle for the test-only key. |
| examples/evidence/fixtures/truncated.json | Provides a bundle with a removed event that must fail verification. |
| examples/evidence/fixtures/README.md | Explains the synthetic fixtures and their test-only key. |
| examples/evidence/execution_demo.py | Offline capture, export, import and replay example for AIOS execution evidence. |
| examples/evidence/demo.py | Offline example showing verification, replay, tampering and truncation checks. |
| docs/evidence-verification.md | Documents the trust model, key rotation, coverage limits, freshness limits and AIOS export. |
| CHANGELOG.md | Adds the 0.12.0 release entry. |
| agenticlens-roadmap.md | Updates the evidence roadmap items to reflect this release. |
Review effort: Lite
Lens can trace execution today but cannot authenticate a saved execution snapshot or verify that an AIOS projection agrees with its native evidence. Add ordered hash-chain snapshots with HMAC-authenticated manifests, explicit coverage and verified/failed/unknown/incomplete outcomes, lossless AIOS 0.4-draft execution export/import, offline replay, and redacted summaries for MCP and Tower.
The evidence CLI exports, verifies and replays saved artifacts without executing tools. Published schemas, valid/tampered fixtures, offline examples and trust-model documentation cover rotation, retention, truncation, producer-honesty and freshness limits. Execution event semantics remain namespaced Lens extensions. Pinned spec-owned AIOS contracts ship in the wheel with upstream commit/hash provenance, so validation requires no sibling checkout.
Validation: make check (812 tests), Python 3.10–3.14 CI, wheel/sdist build and twine check, strict documentation build, offline execution example, and a fresh installed-wheel capture/replay smoke test outside the workspace. Fifteen focused commits preserve implementation, schemas, tests, examples, packaging and release metadata. Prepares 0.12.0; MCP 0.5.2 and Tower 0.9.1 consume this release after publication.