Skip to content

Record Agent Trajectory Profiler verification #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sneakocom merged 1 commit into from
Aug 25, 2026
Merged

Record Agent Trajectory Profiler verification #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions PROGRAM_STATUS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,16 @@

**Coverage: 19/19 expected repositories; duplicates: 0.**

Last verified: `2026-08-25T06:38:48Z`. HEADs are the verified default-branch revisions, not an assumption about later changes.
Last verified: `2026-08-25T12:05:23Z`. HEADs are the verified default-branch revisions, not an assumption about later changes.

## Portfolio totals

| Lifecycle stage | Count |
|---|---:|
| `THEORY` | 12 |
| `SPECIFIED` | 0 |
| `PROTOTYPED` | 6 |
| `VERIFIED` | 1 |
| `PROTOTYPED` | 5 |
| `VERIFIED` | 2 |
| `BENCHMARK_READY` | 0 |
| `EXPERIMENTED` | 0 |
| `REPRODUCED` | 0 |
Expand All @@ -26,11 +26,11 @@ Program state totals: active 4; waiting 15; complete 0.

| # | Repository | Type | HEAD | Stage | Evidence | Blocker | Next task | Dependencies | State |
|---:|---|---|---|---|---|---|---|---|---|
| 1 | [agent-trajectory-profiler](https://github.com/opsle/agent-trajectory-profiler) | concept | `ce2fd532731d` | `PROTOTYPED` | [dependency-free JavaScript reference prototype](https://github.com/opsle/agent-trajectory-profiler/blob/ce2fd532731d9bf5a0b7a271289bfdcc404f57c1/src/metrics.js); 3 of 3 automated tests passed locally at the verified HEAD | No runnable benchmark harness, measured experiment, or independent replication. | Add synthetic payload and correctness fixtures needed by EXP-001 without running a provider experiment. | — | active |
| 1 | [agent-trajectory-profiler](https://github.com/opsle/agent-trajectory-profiler) | concept | `26f8edfb1e7d` | `VERIFIED` | [dependency-free JavaScript trajectory profiler with Context Firewall packet-v1 adapter, append-only escalation accounting, canonical CLI/API, and content-addressed conformance corpus](https://github.com/opsle/agent-trajectory-profiler/blob/26f8edfb1e7d06ee453760724f1889d1663a06f9/src/context-evidence.js); 43 of 43 automated tests and 14 of 14 content-addressed measurement fixtures passed locally at the verified HEAD; 5 of 5 exact-revision interoperability cases passed with independent arithmetic; PR #1 merged with no repository CI workflow configured | EXP-001 task fixtures, correctness oracle, baseline/arm execution harness, exact model configuration, randomized/blinded allocation, measured experiment, and independent replication remain missing. | After content-addressed EXP-001 tasks and a correctness oracle are frozen, add an offline baseline/arm ingestion harness without launching a provider run. | — | active |
| 2 | [semantic-edit-protocol](https://github.com/opsle/semantic-edit-protocol) | concept | `29caad5c0382` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/semantic-edit-protocol/blob/29caad5c03827cde17aabd71c38bc25899413a33/THEORY.md); placeholder only; no automated tests | No executable semantic operation, validator, tests, or benchmark fixtures. | Specify one JavaScript symbol-replacement operation with preconditions, rollback behavior, and conformance cases. | `agent-resource-claims`, `agent-trajectory-profiler` | waiting |
| 3 | [durable-supervisor](https://github.com/opsle/durable-supervisor) | concept | `555ebedb992a` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/durable-supervisor/blob/555ebedb992ac74236bb7da8230b4d6b0489830b/THEORY.md); placeholder only; no automated tests | Portable ledger, scheduler, and wakeup fixtures do not yet exist. | After ledger and wakeup fixtures exist, specify the minimum reconstruction envelope and failure behavior. | `agent-state-ledger`, `agent-scheduler-runtime`, `event-driven-agent-wakeup`, `decision-evidence-protocol` | waiting |
| 4 | [event-driven-agent-wakeup](https://github.com/opsle/event-driven-agent-wakeup) | concept | `a6209860c215` | `PROTOTYPED` | [dependency-free JavaScript state-machine prototype](https://github.com/opsle/event-driven-agent-wakeup/blob/a6209860c2151450cc28ed648bc8c2631c8db7ef/src/runtime.js); 2 of 2 automated tests passed locally at the verified HEAD | Current prototype is in-memory and has no restart or durable-store harness. | Add a deterministic persisted-event fixture covering restart, duplicate delivery, wrong-wait, and timeout cases. | `agent-state-ledger` | waiting |
| 5 | [context-firewall](https://github.com/opsle/context-firewall) | concept | `dd34bd9f6813` | `PROTOTYPED` | [dependency-free deterministic JavaScript TAP-subset reducer, CLI, evidence receipts, payload ceilings, and synthetic conformance corpus](https://github.com/opsle/context-firewall/blob/dd34bd9f681314761f1ca87f339648bf611811f3/src/reducer.js); 33 of 33 automated tests and 30 of 30 synthetic conformance fixtures passed locally at the verified HEAD; PR #1 CI passed | Agent Trajectory Profiler lacks raw/reduced payload fixtures, and EXP-001 task fixtures, correctness oracle, harness, exact model configuration, and randomized/blinded allocation remain unfrozen. | Freeze content-addressed EXP-001 reducer packets and a correctness oracle after Trajectory Profiler payload fixtures exist, without launching a provider run. | `decision-evidence-protocol`, `agent-trajectory-profiler` | active |
| 5 | [context-firewall](https://github.com/opsle/context-firewall) | concept | `dd34bd9f6813` | `PROTOTYPED` | [dependency-free deterministic JavaScript TAP-subset reducer, CLI, evidence receipts, payload ceilings, and synthetic conformance corpus](https://github.com/opsle/context-firewall/blob/dd34bd9f681314761f1ca87f339648bf611811f3/src/reducer.js); 33 of 33 automated tests and 30 of 30 synthetic conformance fixtures passed locally at the verified HEAD; PR #1 CI passed | EXP-001 task fixtures, correctness oracle, baseline/arm harness, exact model configuration, and randomized/blinded allocation remain unfrozen. | Freeze content-addressed EXP-001 reducer/task packets and a deterministic correctness oracle without launching a provider run. | `decision-evidence-protocol`, `agent-trajectory-profiler` | active |
| 6 | [decision-evidence-protocol](https://github.com/opsle/decision-evidence-protocol) | concept | `cc220abfa27a` | `VERIFIED` | [dependency-free generic envelope validator plus independent Context Firewall packet-v1 receipt/source validator, canonical CLI, and self-contained conformance suite](https://github.com/opsle/decision-evidence-protocol/blob/cc220abfa27a0bd20d80c481da09f6fe532bdabc/src/context-firewall-v1.js); 46 of 46 automated tests and 24 of 24 public-safe conformance vectors passed locally at the verified HEAD; 5 of 5 exact-revision Context Firewall interoperability cases, including tamper rejection, passed; PR #1 CI passed | Additional real tool classes, measured decision adequacy, a comparative benchmark, and independent replication are missing. | Add one second public tool-class receipt profile with independently generated conformance vectors and failure evidence before widening protocol claims. | — | active |
| 7 | [agent-state-ledger](https://github.com/opsle/agent-state-ledger) | concept | `acab03b1ff71` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/agent-state-ledger/blob/acab03b1ff7168222552050e21e7553b07d00e7c/THEORY.md); placeholder only; no automated tests | No portable schema, implementation, projection oracle, or replay fixtures. | Specify a minimal append-only event schema and deterministic projection with contradiction fixtures. | — | waiting |
| 8 | [agent-scheduler-runtime](https://github.com/opsle/agent-scheduler-runtime) | concept | `d97cd3c218b2` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/agent-scheduler-runtime/blob/d97cd3c218b20e0b2b0e09873f6a3d15c396b3a0/THEORY.md); placeholder only; no automated tests | Ledger and claim interfaces are not portable or executable in these repositories. | After ledger schema exists, specify deterministic readiness and lease transitions with a fake-clock harness. | `agent-resource-claims`, `agent-state-ledger` | waiting |
Expand All @@ -42,21 +42,21 @@ Program state totals: active 4; waiting 15; complete 0.
| 14 | [controlled-agent-acceptance](https://github.com/opsle/controlled-agent-acceptance) | concept | `2d652adf56e5` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/controlled-agent-acceptance/blob/2d652adf56e53953327d09b1ba9c4a9c3445f052/THEORY.md); placeholder only; no automated tests | Authorization, routing, scheduling, and handoff contracts are not yet executable together. | Wait for prerequisite contracts, then specify an offline one-shot manifest state machine before any provider run. | `agent-execution-authorization`, `agent-routing-policy`, `agent-scheduler-runtime`, `verifiable-agent-handoff` | waiting |
| 15 | [agent-recovery-policy](https://github.com/opsle/agent-recovery-policy) | concept | `1b733a111e26` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/agent-recovery-policy/blob/1b733a111e26e0a409fee3b96f627048531daefe/THEORY.md); placeholder only; no automated tests | No shared failure schema, attempt ledger, route evaluator, or comparative fixture set. | After decision evidence and route schemas stabilize, define same-failure convergence on synthetic failures. | `agent-routing-policy`, `agent-state-ledger`, `decision-evidence-protocol` | waiting |
| 16 | [ephemeral-agent-workers](https://github.com/opsle/ephemeral-agent-workers) | concept | `ad96fcfdfac0` | `THEORY` | [none; placeholder source directory only](https://github.com/opsle/ephemeral-agent-workers/blob/ad96fcfdfac06d340b5e96d369634980cee78ef4/THEORY.md); placeholder only; no automated tests | Portable authority, claim, and handoff contracts are not ready; no safe synthetic containment harness exists. | Wait for prerequisite contracts, then define a fake worker adapter and destruction receipt without infrastructure changes. | `agent-execution-authorization`, `agent-resource-claims`, `verifiable-agent-handoff` | waiting |
| 17 | [research](https://github.com/opsle/research) | program infrastructure | `f1652bf58d30` | `PROTOTYPED` | [authoritative 19-repository machine registry, experiment ledger, deterministic status renderer, validator, mutation tests, operating rules, and integrity CI on main](https://github.com/opsle/research/blob/f1652bf58d30947f7401855fe2a9a189fafe9a33/program/registry.json); 9 of 9 registry/renderer tests, including 7 mutation/failure cases, passed locally at the verified control revision; PR #1 integrity CI passed | Verified remote HEAD reconciliation is not yet mechanically fetched, and the program has no canonical measured concept experiment. | Record the next exact EXP-001 prerequisite revision when Decision Evidence Protocol accepts Context Firewall packet-v1 receipts; keep EXP-001 planned. | — | active |
| 17 | [research](https://github.com/opsle/research) | program infrastructure | `f1652bf58d30` | `PROTOTYPED` | [authoritative 19-repository machine registry, experiment ledger, deterministic status renderer, validator, mutation tests, operating rules, and integrity CI on main](https://github.com/opsle/research/blob/f1652bf58d30947f7401855fe2a9a189fafe9a33/program/registry.json); 9 of 9 registry/renderer tests, including 7 mutation/failure cases, passed locally at the verified control revision; PR #1 integrity CI passed | Verified remote HEAD reconciliation is not yet mechanically fetched, and the program has no canonical measured concept experiment. | Record content-addressed EXP-001 task/fixture identities and a deterministic correctness oracle at exact revisions; keep EXP-001 planned. | — | active |
| 18 | [site](https://github.com/opsle/site) | program infrastructure | `28ad65be4750` | `PROTOTYPED` | [React/Vinext source implementation with content routes](https://github.com/opsle/site/blob/28ad65be4750dc849976fbf5c9eae9501c6bbb25/README.md); automated build/render tests present; not rerun because this reconciliation kept other repositories read-only | Wait for validated registry data and measured research; deployment requires separate authorization. | After registry merge, add a read-only registry ingestion design without deploying the site. | `research` | waiting |
| 19 | [.github](https://github.com/opsle/.github) | program infrastructure | `01c38e726db7` | `THEORY` | [documentation-only organization profile](https://github.com/opsle/.github/blob/01c38e726db7c3e45059d25fccce55e071e35938/profile/README.md); not applicable to current single Markdown profile; consistency is unverified | No mechanical registry consistency check exists in this repository. | After registry merge, design a read-only consistency check for organization-profile repository links. | `research` | waiting |

## Highest-priority workstream

EXP-001 prerequisites: add Agent Trajectory Profiler raw/reduced payload and escalation fixtures, then freeze content-addressed task fixtures and a correctness oracle before any experiment.
EXP-001 prerequisites: freeze content-addressed experimental task fixtures and a correctness oracle, then build the offline baseline/arm harness before any model/provider run.

`EXP-001` — **PLANNED** — How much context can an AI coding agent safely not see?

Blockers: Agent Trajectory Profiler lacks raw/reduced payload and escalation fixtures for EXP-001. Experimental fixture identities, correctness oracle, baseline/arm harness, exact model/provider configurations, and randomized/blinded task allocation are not frozen.
Blockers: Experimental fixture identities, correctness oracle, baseline/arm harness, exact model/provider configurations, and randomized/blinded task allocation are not frozen.

## Exact recommended next execution

Extend Agent Trajectory Profiler with deterministic Context Firewall raw/reduced payload, suppression, and NEEDS_RAW_EVIDENCE fixtures at exact revisions; target VERIFIED only if tests, fixture identities, and cross-project compatibility pass; do not run a model/provider experiment.
Freeze content-addressed EXP-001 task fixtures and a deterministic correctness oracle across Context Firewall and Agent Trajectory Profiler at exact revisions; target no lifecycle promotion; do not run a model/provider experiment.

## Mechanical source of truth

Expand Down
17 changes: 14 additions & 3 deletions program/experiments.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@
"missing telemetry",
"interrupted run"
],
"dataset_fixture_identity": "UNVERIFIED: task-stratified experimental coding fixtures and immutable hashes have not yet been created. Context Firewall's 30 synthetic conformance fixtures are implementation evidence only, not the EXP-001 dataset.",
"dataset_fixture_identity": "UNVERIFIED: task-stratified experimental coding fixtures and immutable hashes have not yet been created. Context Firewall's 30 reducer fixtures and Agent Trajectory Profiler's 14 measurement fixtures are implementation/conformance evidence only, not the EXP-001 dataset.",
"model_provider_configuration": "UNVERIFIED: no model, provider, version, reasoning effort, or sampling configuration has been selected.",
"run_identities": [],
"result_artifacts": [],
Expand All @@ -75,15 +75,26 @@
"https://github.com/opsle/decision-evidence-protocol/pull/1"
],
"limitations": "Receipt and source conformance establish deterministic protocol validation, not model correctness, benchmark readiness, comparative benefit, or a safe context-reduction frontier."
},
{
"repository": "agent-trajectory-profiler",
"head_sha": "26f8edfb1e7d06ee453760724f1889d1663a06f9",
"status": "COMPLETED",
"evidence": [
"https://github.com/opsle/agent-trajectory-profiler/blob/26f8edfb1e7d06ee453760724f1889d1663a06f9/src/context-evidence.js",
"https://github.com/opsle/agent-trajectory-profiler/blob/26f8edfb1e7d06ee453760724f1889d1663a06f9/tests/context-evidence.test.js",
"https://github.com/opsle/agent-trajectory-profiler/blob/26f8edfb1e7d06ee453760724f1889d1663a06f9/fixtures/context-firewall-manifest.json",
"https://github.com/opsle/agent-trajectory-profiler/pull/1"
],
"limitations": "Measurement conformance establishes deterministic byte/event and escalation accounting, not preserved model correctness, benchmark readiness, comparative benefit, or a safe context-reduction frontier."
}
],
"replication_status": "NOT_STARTED",
"verdict": "PENDING",
"blockers": [
"Agent Trajectory Profiler lacks raw/reduced payload and escalation fixtures for EXP-001.",
"Experimental fixture identities, correctness oracle, baseline/arm harness, exact model/provider configurations, and randomized/blinded task allocation are not frozen."
],
"next_task": "Add deterministic Context Firewall raw/reduced payload, suppression, and NEEDS_RAW_EVIDENCE fixtures to Agent Trajectory Profiler without launching a model/provider run."
"next_task": "Freeze content-addressed EXP-001 task fixtures and a deterministic correctness oracle across Context Firewall and Agent Trajectory Profiler without launching a model/provider run."
},
{
"id": "LEGACY-001",
Expand Down
Loading
Loading