Skip to content

Record Decision Evidence Protocol verification #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sneakocom merged 1 commit into from
Aug 25, 2026
Merged

Record Decision Evidence Protocol verification #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions PROGRAM_STATUS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@

**Coverage: 19/19 expected repositories; duplicates: 0.**

Last verified: `2026-08-25T05:55:58Z`. HEADs are the verified default-branch revisions, not an assumption about later changes.
Last verified: `2026-08-25T06:38:48Z`. HEADs are the verified default-branch revisions, not an assumption about later changes.

## Portfolio totals

| Lifecycle stage | Count |
|---|---:|
| `THEORY` | 12 |
| `SPECIFIED` | 0 |
| `PROTOTYPED` | 7 |
| `VERIFIED` | 0 |
| `PROTOTYPED` | 6 |
| `VERIFIED` | 1 |
| `BENCHMARK_READY` | 0 |
| `EXPERIMENTED` | 0 |
| `REPRODUCED` | 0 |
Expand All @@ -30,8 +30,8 @@ Program state totals: active 4; waiting 15; complete 0.
| 2 | [semantic-edit-protocol](https://github.com/opsle/semantic-edit-protocol) | concept | `29caad5c0382` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/semantic-edit-protocol/blob/29caad5c03827cde17aabd71c38bc25899413a33/THEORY.md); placeholder only; no automated tests | No executable semantic operation, validator, tests, or benchmark fixtures. | Specify one JavaScript symbol-replacement operation with preconditions, rollback behavior, and conformance cases. | `agent-resource-claims`, `agent-trajectory-profiler` | waiting |
| 3 | [durable-supervisor](https://github.com/opsle/durable-supervisor) | concept | `555ebedb992a` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/durable-supervisor/blob/555ebedb992ac74236bb7da8230b4d6b0489830b/THEORY.md); placeholder only; no automated tests | Portable ledger, scheduler, and wakeup fixtures do not yet exist. | After ledger and wakeup fixtures exist, specify the minimum reconstruction envelope and failure behavior. | `agent-state-ledger`, `agent-scheduler-runtime`, `event-driven-agent-wakeup`, `decision-evidence-protocol` | waiting |
| 4 | [event-driven-agent-wakeup](https://github.com/opsle/event-driven-agent-wakeup) | concept | `a6209860c215` | `PROTOTYPED` | [dependency-free JavaScript state-machine prototype](https://github.com/opsle/event-driven-agent-wakeup/blob/a6209860c2151450cc28ed648bc8c2631c8db7ef/src/runtime.js); 2 of 2 automated tests passed locally at the verified HEAD | Current prototype is in-memory and has no restart or durable-store harness. | Add a deterministic persisted-event fixture covering restart, duplicate delivery, wrong-wait, and timeout cases. | `agent-state-ledger` | waiting |
| 5 | [context-firewall](https://github.com/opsle/context-firewall) | concept | `dd34bd9f6813` | `PROTOTYPED` | [dependency-free deterministic JavaScript TAP-subset reducer, CLI, evidence receipts, payload ceilings, and synthetic conformance corpus](https://github.com/opsle/context-firewall/blob/dd34bd9f681314761f1ca87f339648bf611811f3/src/reducer.js); 33 of 33 automated tests and 30 of 30 synthetic conformance fixtures passed locally at the verified HEAD; PR #1 CI passed | Decision Evidence Protocol does not yet validate packet-v1 receipts, Agent Trajectory Profiler lacks raw/reduced payload fixtures, and EXP-001 task fixtures, correctness oracle, harness, exact model configuration, and randomized/blinded allocation remain unfrozen. | After downstream receipt validation exists, freeze content-addressed EXP-001 reducer packets and a correctness oracle without launching a provider run. | `decision-evidence-protocol`, `agent-trajectory-profiler` | active |
| 6 | [decision-evidence-protocol](https://github.com/opsle/decision-evidence-protocol) | concept | `7050f83406a7` | `PROTOTYPED` | [dependency-free JavaScript envelope validator prototype](https://github.com/opsle/decision-evidence-protocol/blob/7050f83406a709da84e4b4770556319f767bbeaf/src/validate.js); 2 of 2 automated tests passed locally at the verified HEAD | Validator covers only a minimal envelope and has no cross-tool fixture suite. | Add reducer receipt and escalation conformance cases required by EXP-001. | — | active |
| 5 | [context-firewall](https://github.com/opsle/context-firewall) | concept | `dd34bd9f6813` | `PROTOTYPED` | [dependency-free deterministic JavaScript TAP-subset reducer, CLI, evidence receipts, payload ceilings, and synthetic conformance corpus](https://github.com/opsle/context-firewall/blob/dd34bd9f681314761f1ca87f339648bf611811f3/src/reducer.js); 33 of 33 automated tests and 30 of 30 synthetic conformance fixtures passed locally at the verified HEAD; PR #1 CI passed | Agent Trajectory Profiler lacks raw/reduced payload fixtures, and EXP-001 task fixtures, correctness oracle, harness, exact model configuration, and randomized/blinded allocation remain unfrozen. | Freeze content-addressed EXP-001 reducer packets and a correctness oracle after Trajectory Profiler payload fixtures exist, without launching a provider run. | `decision-evidence-protocol`, `agent-trajectory-profiler` | active |
| 6 | [decision-evidence-protocol](https://github.com/opsle/decision-evidence-protocol) | concept | `cc220abfa27a` | `VERIFIED` | [dependency-free generic envelope validator plus independent Context Firewall packet-v1 receipt/source validator, canonical CLI, and self-contained conformance suite](https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/src/context-firewall-v1.js); 46 of 46 automated tests and 24 of 24 public-safe conformance vectors passed locally at the verified HEAD; 5 of 5 exact-revision Context Firewall interoperability cases, including tamper rejection, passed; PR #1 CI passed | Additional real tool classes, measured decision adequacy, a comparative benchmark, and independent replication are missing. | Add one second public tool-class receipt profile with independently generated conformance vectors and failure evidence before widening protocol claims. | — | active |
| 7 | [agent-state-ledger](https://github.com/opsle/agent-state-ledger) | concept | `acab03b1ff71` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/agent-state-ledger/blob/acab03b1ff7168222552050e21e7553b07d00e7c/THEORY.md); placeholder only; no automated tests | No portable schema, implementation, projection oracle, or replay fixtures. | Specify a minimal append-only event schema and deterministic projection with contradiction fixtures. | — | waiting |
| 8 | [agent-scheduler-runtime](https://github.com/opsle/agent-scheduler-runtime) | concept | `d97cd3c218b2` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/agent-scheduler-runtime/blob/d97cd3c218b20e0b2b0e09873f6a3d15c396b3a0/THEORY.md); placeholder only; no automated tests | Ledger and claim interfaces are not portable or executable in these repositories. | After ledger schema exists, specify deterministic readiness and lease transitions with a fake-clock harness. | `agent-resource-claims`, `agent-state-ledger` | waiting |
| 9 | [verifiable-agent-handoff](https://github.com/opsle/verifiable-agent-handoff) | concept | `399e5cfae943` | `PROTOTYPED` | [dependency-free JavaScript HMAC manifest prototype](https://github.com/opsle/verifiable-agent-handoff/blob/399e5cfae94345affa3f087f0f6eb9e77669d33c/src/seal.js); 3 of 3 automated tests passed locally at the verified HEAD | Prototype authenticates a manifest but does not build, transport, or reconstruct artifacts. | Build a synthetic artifact round-trip harness with tamper, missing-object, and source-destruction cases. | `decision-evidence-protocol` | waiting |
Expand All @@ -48,15 +48,15 @@ Program state totals: active 4; waiting 15; complete 0.

## Highest-priority workstream

EXP-001 prerequisites: validate Context Firewall receipts in Decision Evidence Protocol, then add Trajectory Profiler payload fixtures before freezing the experiment harness.
EXP-001 prerequisites: add Agent Trajectory Profiler raw/reduced payload and escalation fixtures, then freeze content-addressed task fixtures and a correctness oracle before any experiment.

`EXP-001` — **PLANNED** — How much context can an AI coding agent safely not see?

Blockers: Decision Evidence Protocol has not validated Context Firewall packet-v1 receipts and escalation semantics. Agent Trajectory Profiler lacks raw/reduced payload and escalation fixtures for EXP-001. Experimental fixture identities, correctness oracle, baseline/arm harness, exact model/provider configurations, and randomized/blinded task allocation are not frozen.
Blockers: Agent Trajectory Profiler lacks raw/reduced payload and escalation fixtures for EXP-001. Experimental fixture identities, correctness oracle, baseline/arm harness, exact model/provider configurations, and randomized/blinded task allocation are not frozen.

## Exact recommended next execution

Extend the dependency-free Decision Evidence Protocol validator with Context Firewall packet-v1 receipt, suppression, measurement, and NEEDS_RAW_EVIDENCE conformance cases at exact revisions; do not run a model/provider experiment.
Extend Agent Trajectory Profiler with deterministic Context Firewall raw/reduced payload, suppression, and NEEDS_RAW_EVIDENCE fixtures at exact revisions; target VERIFIED only if tests, fixture identities, and cross-project compatibility pass; do not run a model/provider experiment.

## Mechanical source of truth

Expand Down
17 changes: 14 additions & 3 deletions program/experiments.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"schema_version": 1,
"last_verified_at": "2026-08-25T05:55:58Z",
"last_verified_at": "2026-08-25T06:38:48Z",
"experiments": [
{
"id": "EXP-001",
Expand Down Expand Up @@ -63,16 +63,27 @@
"https://github.com/opsle/context-firewall/blob/dd34bd9f681314761f1ca87f339648bf611811f3/fixtures/corpus.js"
],
"limitations": "Prototype conformance does not establish model correctness, benchmark readiness, or a safe context-reduction frontier."
},
{
"repository": "decision-evidence-protocol",
"head_sha": "cc220abfa27a0bd20d80c481da09f6fe532bdabc",
"status": "COMPLETED",
"evidence": [
"https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/src/context-firewall-v1.js",
"https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/tests/context-firewall-v1.test.js",
"https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/fixtures/context-firewall-packet-v1/vectors.json",
"https://github.com/opsle/decision-evidence-protocol/pull/1"
],
"limitations": "Receipt and source conformance establish deterministic protocol validation, not model correctness, benchmark readiness, comparative benefit, or a safe context-reduction frontier."
}
],
"replication_status": "NOT_STARTED",
"verdict": "PENDING",
"blockers": [
"Decision Evidence Protocol has not validated Context Firewall packet-v1 receipts and escalation semantics.",
"Agent Trajectory Profiler lacks raw/reduced payload and escalation fixtures for EXP-001.",
"Experimental fixture identities, correctness oracle, baseline/arm harness, exact model/provider configurations, and randomized/blinded task allocation are not frozen."
],
"next_task": "Extend Decision Evidence Protocol with Context Firewall packet-v1 receipt and NEEDS_RAW_EVIDENCE conformance cases without launching a model/provider run."
"next_task": "Add deterministic Context Firewall raw/reduced payload, suppression, and NEEDS_RAW_EVIDENCE fixtures to Agent Trajectory Profiler without launching a model/provider run."
},
{
"id": "LEGACY-001",
Expand Down
38 changes: 19 additions & 19 deletions program/registry.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
"authoritative_repository_count": 19,
"lifecycle_model": "program/LIFECYCLE.md",
"experiment_registry": "program/experiments.json",
"current_highest_priority_workstream": "EXP-001 prerequisites: validate Context Firewall receipts in Decision Evidence Protocol, then add Trajectory Profiler payload fixtures before freezing the experiment harness.",
"recommended_next_execution": "Extend the dependency-free Decision Evidence Protocol validator with Context Firewall packet-v1 receipt, suppression, measurement, and NEEDS_RAW_EVIDENCE conformance cases at exact revisions; do not run a model/provider experiment.",
"last_verified_at": "2026-08-25T05:55:58Z",
"current_highest_priority_workstream": "EXP-001 prerequisites: add Agent Trajectory Profiler raw/reduced payload and escalation fixtures, then freeze content-addressed task fixtures and a correctness oracle before any experiment.",
"recommended_next_execution": "Extend Agent Trajectory Profiler with deterministic Context Firewall raw/reduced payload, suppression, and NEEDS_RAW_EVIDENCE fixtures at exact revisions; target VERIFIED only if tests, fixture identities, and cross-project compatibility pass; do not run a model/provider experiment.",
"last_verified_at": "2026-08-25T06:38:48Z",
"repositories": [
{
"name": "agent-trajectory-profiler",
Expand Down Expand Up @@ -149,44 +149,44 @@
"dependencies": ["decision-evidence-protocol", "agent-trajectory-profiler"],
"dependents": [],
"active_experiment_ids": ["EXP-001"],
"blockers": ["Decision Evidence Protocol does not yet validate packet-v1 receipts, Agent Trajectory Profiler lacks raw/reduced payload fixtures, and EXP-001 task fixtures, correctness oracle, harness, exact model configuration, and randomized/blinded allocation remain unfrozen."],
"next_task": "After downstream receipt validation exists, freeze content-addressed EXP-001 reducer packets and a correctness oracle without launching a provider run.",
"blockers": ["Agent Trajectory Profiler lacks raw/reduced payload fixtures, and EXP-001 task fixtures, correctness oracle, harness, exact model configuration, and randomized/blinded allocation remain unfrozen."],
"next_task": "Freeze content-addressed EXP-001 reducer packets and a correctness oracle after Trajectory Profiler payload fixtures exist, without launching a provider run.",
"evidence": ["https://github.com/opsle/context-firewall/blob/dd34bd9f681314761f1ca87f339648bf611811f3/src/reducer.js", "https://github.com/opsle/context-firewall/blob/dd34bd9f681314761f1ca87f339648bf611811f3/tests/reducer.test.js", "https://github.com/opsle/context-firewall/blob/dd34bd9f681314761f1ca87f339648bf611811f3/fixtures/corpus.js", "https://github.com/opsle/context-firewall/pull/1"],
"completion_criteria": ["Publish reducer policy and executable conformance validator.", "Run correctness-gated measured context-reduction experiments.", "Replicate the safe frontier and publish omission failure modes."],
"completion_evidence": [],
"completion_status": "INCOMPLETE",
"program_state": "active",
"last_verified_at": "2026-08-25T05:55:58Z"
"last_verified_at": "2026-08-25T06:38:48Z"
},
{
"name": "decision-evidence-protocol",
"github_url": "https://github.com/opsle/decision-evidence-protocol",
"default_branch": "main",
"last_verified_head_sha": "7050f83406a709da84e4b4770556319f767bbeaf",
"last_verified_head_sha": "cc220abfa27a0bd20d80c481da09f6fe532bdabc",
"project_type": "concept",
"purpose": "Represent bounded vendor-neutral decision facts with provenance and explicit raw-output escalation.",
"lifecycle_stage": "PROTOTYPED",
"implementation_status": "dependency-free JavaScript envelope validator prototype",
"lifecycle_stage": "VERIFIED",
"implementation_status": "dependency-free generic envelope validator plus independent Context Firewall packet-v1 receipt/source validator, canonical CLI, and self-contained conformance suite",
"implementation_requirement": "As a protocol, an executable validator and conformance suite can satisfy implementation; a full runtime is not required.",
"specification_status": "experimental prototype contract",
"test_status": "2 of 2 automated tests passed locally at the verified HEAD",
"benchmark_status": "prose plan only; no cross-tool harness or frozen fixtures",
"specification_status": "versioned normative experimental Context Firewall packet-v1 validation profile plus generic prototype contract",
"test_status": "46 of 46 automated tests and 24 of 24 public-safe conformance vectors passed locally at the verified HEAD; 5 of 5 exact-revision Context Firewall interoperability cases, including tamper rejection, passed; PR #1 CI passed",
"benchmark_status": "24 protocol conformance vectors and a 5-case exact-revision interoperability proof only; no frozen EXP-001 task fixtures, baseline/arm harness, correctness experiment, or model run",
"measured_experiment_status": "none",
"reproducibility_status": "prototype tests reproducible locally; cross-tool adequacy not reproduced",
"documentation_status": "public theory, specification, architecture, and benchmark plan present",
"reproducibility_status": "validator tests, self-contained vectors, deterministic CLI output, and exact-revision Context Firewall interoperability are reproducible locally and in CI where applicable; other tool classes and independent replication are missing",
"documentation_status": "public theory, versioned packet-v1 profile, enforced invariants, API/CLI, structural-versus-cryptographic verification, sufficiency/escalation semantics, conformance instructions, interoperability boundary, and limitations documented",
"site_publication_status": "GitHub documentation only; no evidence-backed site publication",
"known_limitations": ["Cross-tool adapters, versioning evidence, and protocol conformance rules are missing."],
"known_limitations": ["Only Context Firewall's strict TAP-subset packet-v1 producer is independently validated; receipt-only source claims and caller-owned raw locator existence remain unverified without external evidence, and no result establishes model correctness, comparative benefit, or independent replication."],
"dependencies": [],
"dependents": ["agent-recovery-policy", "context-firewall", "durable-supervisor", "verifiable-agent-handoff"],
"active_experiment_ids": ["EXP-001"],
"blockers": ["Validator covers only a minimal envelope and has no cross-tool fixture suite."],
"next_task": "Add reducer receipt and escalation conformance cases required by EXP-001.",
"evidence": ["https://github.com/opsle/decision-evidence-protocol/blob/7050f83406a709da84e4b4770556319f767bbeaf/src/validate.js", "https://github.com/opsle/decision-evidence-protocol/blob/7050f83406a709da84e4b4770556319f767bbeaf/tests/validate.test.js"],
"blockers": ["Additional real tool classes, measured decision adequacy, a comparative benchmark, and independent replication are missing."],
"next_task": "Add one second public tool-class receipt profile with independently generated conformance vectors and failure evidence before widening protocol claims.",
"evidence": ["https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/src/context-firewall-v1.js", "https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/tests/context-firewall-v1.test.js", "https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/fixtures/context-firewall-packet-v1/vectors.json", "https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/docs/context-firewall-packet-v1.md", "https://github.com/opsle/decision-evidence-protocol/pull/1"],
"completion_criteria": ["Publish a versioned protocol and comprehensive executable conformance suite.", "Measure adequacy across multiple real tool classes.", "Replicate interoperability and document loss/escalation failure modes."],
"completion_evidence": [],
"completion_status": "INCOMPLETE",
"program_state": "active",
"last_verified_at": "2026-08-25T05:14:23Z"
"last_verified_at": "2026-08-25T06:38:48Z"
},
{
"name": "agent-state-ledger",
Expand Down
Loading