diff --git a/.agents/pm/features/pm-rl-shadow.toon b/.agents/pm/features/pm-rl-shadow.toon new file mode 100644 index 00000000..fd1971dd --- /dev/null +++ b/.agents/pm/features/pm-rl-shadow.toon @@ -0,0 +1,36 @@ +id: pm-rl-shadow +title: Observe candidate policies safely before deployment +description: "Controlled online shadow observation on explicitly consented inputs, with paired resource accounting and validated outcome receipts required by the existing promotion boundary." +type: Feature +status: in_progress +priority: 1 +tags: [] +created_at: "2026-10-23T06:41:11.687Z" +updated_at: "2026-10-23T08:22:07.469Z" +author: codex-agent +acceptance_criteria: Candidate failures never affect active actions; Budgets include both arms; Switching requires a validated outcome comparison through existing promotion and RSI code +parent: pm-rl-apvf +comments[9]{created_at,author,text}: + "2026-10-23T06:41:26.255Z",codex-agent,"Overlap checked against the pinned pm SDK and current CLI 2026.10.23, pm-context, pm-jev, and existing pm-rl evaluation, loop, meta proposer and promotion modules. Core owns tracker/history/transactions; pm-context supplies bounded context; pm-jev replays typed decisions. Existing RL loops evaluate/train/promote offline. This feature adds explicitly consented online observation with no candidate mutation authority and reuses predeclared evaluation plans plus the authoritative promotion transaction. Draft PRs 95 and 98 are context only. No dependency bump is needed unless SDK capability inspection proves otherwise." + "2026-10-23T06:56:27.540Z",codex-agent,"Implemented a bounded declarative shadow interpreter with no executable candidate tools or tracker authority. Consent pins both generation identities, both policy digests, exact minimized input hashes and separate active/candidate allowances; every frozen member reserves two calls including candidate failures. Immutable per-member SDK receipts make retries free and outcome replacement a refusal. Existing evaluation plans and the authoritative promotion transaction require the comparison receipt for online candidates; standalone offline evaluation cannot bypass it. Local linked parser test passes 1/1. Source and packed acceptance use non-git trackers; full release gates will run in CI." + "2026-10-23T07:02:21.061Z",codex-agent,"Behavioral revert proof: independently omitting enforceShadowComparison from the existing evaluation gate fails the retry-safe receipt regression (1/1); propagating a missing candidate decision fails active-isolation (1/1); disabling the two-arm allowance validator fails strict-parser regression (1/1). Restored code passed the preceding complete 7/7 shadow suite including actual generation promotion without git. Final consent-generation binding and additional corruption/concurrency regressions are now under targeted validation. The SDK/dev pin remains 2026.10.17; there is no feature dependency on newer CLI behavior." + "2026-10-23T07:07:52.857Z",codex-agent,"Focused source validation passed 11/11 with shadow.ts at 100% lines, branches and functions. Added read-only promotion revalidation and a forged-original-receipt regression; final focused verification and packed replay are running. Public MIT pm-context snapshot: 84 certified items; first 20 sorted ids selected without outcome filtering; 17 closed and 3 other; 20/20 active decisions preserved; 20 disagreements; zero candidate failures; 20 active plus 20 candidate calls charged. All 175 copied source files retain identical digest and read-only permissions. One independent tracker unit correctly refuses promotion despite observed candidate accuracy 0.85 versus active 0.15. JSON observation 548 bytes; comparison JSON 1278 bytes and TOON 1127 bytes. This measures the observation/gating mechanism, not a learned policy improvement." + "2026-10-23T07:17:22.816Z",codex-agent,"Final focused source suite passed 12/12 at 100% shadow lines/branches/functions; packed public replay passed unchanged. Review then closed an online-lineage bypass: the existing promotion gate now requires shadow evidence even for an unreserved or offline-only successor of an online-bound incumbent, so the recursive proposer cannot escape by omitting the marker. New lineage regression passes 1/1; reverting only the unreserved-lineage enforcement fails 1/1. Legacy lineages never bound online retain their offline contracts. The complete inherited-gate source suite is running; CI release gates are also running at https://github.com/unbraind/pm-rl/actions/runs/39915892410 and will be refreshed for the final head." + "2026-10-23T07:20:47.390Z",codex-agent,"Final implementation validation: node --test --experimental-test-coverage --test-coverage-include=shadow.ts test/shadow.test.ts passes 13/13, zero failures/skips/cancellations/todos, and 100% shadow lines/branches/functions. Typecheck, test typecheck, lint, zero duplication and documented declarations pass. Four independent behavioral reverts fail their intended regressions: receipt prerequisite, candidate-failure isolation, budget validation, and inherited online-lineage enforcement. Packed public validation is read-only, Git-free and reproducible with unchanged aggregate receipts. Review-ready PR https://github.com/unbraind/pm-rl/pull/99 owns the final-head npm/Bun release run URLs and exact counts; full release gates are intentionally delegated to CI on the saturated host. Scope limit: bounded declarative policy tables and valid paired outcomes; one public tracker is not an independent policy-quality win. Automated reviewer skips/unavailable access are not approval. Implementation work is complete; item stays in progress for orchestrator verification." + "2026-10-23T07:56:56.267Z",codex-agent,"Reopened for PR 99 blocking review. Confirmed two root gaps in the previous head: semantically inconsistent original receipts could pass hash/history checks, and reloaded manifests bypassed cohort and both-arm allowance checks. Fix will use shared manifest and receipt validators for observation, resume, comparison and the authoritative promotion gate. Validation will bind actions to consented input/policy tables, enforce finite bounded and same-action-consistent rewards, verify disagreement/failure/digest fields, and check actual unique receipt charges against both allowances. Four requested adversarial regressions and independent behavioral revert proofs will be recorded; no review-thread replies or mentions." + "2026-10-23T08:16:19.060Z",codex-agent,"Both blocking review findings are fixed through shared validators. validateDeployment now governs registration and every manifest reload: generation pair, exact plan cohort, consent, both reserved allowances and actual receipt inventory charges; extra slots cannot hide outside report counts. validateObservation now governs live observation, retry/resume, comparison and promotion reads: policy-produced actions for the consented input, bounded rewards, same-action reward equality, disagreement/failure flags and selected-outcome digest. Refusals use pinned SDK context.item_id and context.field and include both in the message. Focused suite passes 25/25 with 100% shadow lines/branches/functions. Four independent behavioral reverts each fail 1/1 with Missing expected rejection: identical-action rewards, zero-budget original manifest, tampered disagreement, and cohort expansion after plan registration; validators restored. Broader original-record tests also cover wrong actions, bounds, digests, slot IDs, unexpected charged receipts, both aggregate charges, edited manifests and native SDK workflow-policy refusal. No SDK mocks, history rewriting, dependency bumps, review-thread replies or mentions." + "2026-10-23T08:22:04.807Z",codex-agent,"Final follow-up verification: node --test --experimental-test-coverage --test-coverage-include=shadow.ts test/shadow.test.ts passes 25/25 with 100% lines/branches/functions for shadow.ts. Restored four requested regressions pass 4/4 with Git absent from PATH and an explicit discovery ceiling. Independent revert commands use node --test --test-name-pattern with the exact prefixes review regression: identical actions; review regression: a consented original zero-budget; review regression: an original tampered disagreement; review regression: a manifest cohort grown. Reverting only validateObservation to accept stored rows fails the reward and disagreement cases 1/1 each; reverting only validateDeployment to retain consent without cohort/budget checks fails the budget and expanded-cohort cases 1/1 each, all with Missing expected rejection. Source restored. Pinned linked-test execution passes 1/1; typecheck, test typecheck, lint, zero duplication and docstrings pass. Repacked public acceptance on the existing read-only source copy reproduces docs/shadow-evidence.json byte-for-byte. PR https://github.com/unbraind/pm-rl/pull/99 will carry the exact pushed-head npm/Bun CI run and counts; previous-head gates are historical only. Claim released for orchestrator verification; item remains in progress." +files[5]{path,scope}: + evaluation-plan.ts,project + index.ts,project + shadow.ts,project + test/shadow.acceptance.ts,project + test/shadow.test.ts,project +tests[1]{command,scope,provenance{author,created_at,source_kind,source_ref}}: + "node --test --test-name-pattern=\"strict parsers\" test/shadow.test.ts",project,codex-agent,"2026-10-23T06:53:44.350Z",local_mutation,feat/shadow-policy-deployment +docs[4]{path,scope}: + docs/shadow-evidence.json,project + docs/shadow-policy.md,project + examples/shadow-policy.json,project + README.md,project +body: "" diff --git a/.agents/pm/history/pm-rl-shadow.jsonl b/.agents/pm/history/pm-rl-shadow.jsonl new file mode 100644 index 00000000..ddb275c9 --- /dev/null +++ b/.agents/pm/history/pm-rl-shadow.jsonl @@ -0,0 +1,19 @@ +{"hash_algorithm":"sha256","ts":"2026-10-23T06:41:11.687Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":{"value":"implementer","source":"argv"},"topic":null},"op":"create","patch":[{"op":"add","path":"/metadata/id","value":"pm-rl-shadow"},{"op":"add","path":"/metadata/title","value":"Observe candidate policies safely before deployment"},{"op":"add","path":"/metadata/description","value":"Controlled online shadow observation on explicitly consented inputs, with paired resource accounting and validated outcome receipts required by the existing promotion boundary."},{"op":"add","path":"/metadata/type","value":"Feature"},{"op":"add","path":"/metadata/status","value":"open"},{"op":"add","path":"/metadata/priority","value":1},{"op":"add","path":"/metadata/tags","value":[]},{"op":"add","path":"/metadata/created_at","value":"2026-10-23T06:41:11.687Z"},{"op":"add","path":"/metadata/updated_at","value":"2026-10-23T06:41:11.687Z"},{"op":"add","path":"/metadata/author","value":"codex-agent"},{"op":"add","path":"/metadata/acceptance_criteria","value":"Candidate failures never affect active actions; Budgets include both arms; Switching requires a validated outcome comparison through existing promotion and RSI code"},{"op":"add","path":"/metadata/parent","value":"pm-rl-apvf"}],"before_hash":"3cc22dff72be7b14824654a7a64ea62b04799939b2fee54c1b5f52ca60bf6df0","after_hash":"c4942305f6856775781456fdc4e56c0222f59c956b16a20f21d7d1ba3ab82c78","item_hash_version":3,"message":"","context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"12d27a2f846851566d1eccc28bf9085c5a902064a2e075a59a50dd919c6033a9"} +{"hash_algorithm":"sha256","ts":"2026-10-23T06:41:17.444Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":{"value":"implementer","source":"argv"},"topic":null},"op":"claim","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T06:41:17.444Z"},{"op":"add","path":"/metadata/assignee","value":"codex-agent"},{"op":"add","path":"/metadata/claim_principal","value":"codex-agent"}],"before_hash":"c4942305f6856775781456fdc4e56c0222f59c956b16a20f21d7d1ba3ab82c78","after_hash":"5606a62b7f0208511db50110a43adaffd4872023de78a46d7bfb42127b2b0281","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"89331844fc3768778b4ee4b0d6b985b4ec07a8f5b32818958bb729215fcd3cf5"} +{"hash_algorithm":"sha256","ts":"2026-10-23T06:41:21.215Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":{"value":"implementer","source":"argv"},"topic":null},"op":"update","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T06:41:21.215Z"},{"op":"replace","path":"/metadata/status","value":"in_progress"}],"before_hash":"5606a62b7f0208511db50110a43adaffd4872023de78a46d7bfb42127b2b0281","after_hash":"056c2617f6283a7218efbc0ae838349d029aef9698905903bd7699587356c5ad","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"912ce60c00d893ad24fbf546df4ada2ef1498471ea38a167b3009da599a82cb9"} +{"hash_algorithm":"sha256","ts":"2026-10-23T06:41:26.255Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T06:41:26.255Z"},{"op":"add","path":"/metadata/comments","value":[{"created_at":"2026-10-23T06:41:26.255Z","author":"codex-agent","text":"Overlap checked against the pinned pm SDK and current CLI 2026.10.23, pm-context, pm-jev, and existing pm-rl evaluation, loop, meta proposer and promotion modules. Core owns tracker/history/transactions; pm-context supplies bounded context; pm-jev replays typed decisions. Existing RL loops evaluate/train/promote offline. This feature adds explicitly consented online observation with no candidate mutation authority and reuses predeclared evaluation plans plus the authoritative promotion transaction. Draft PRs 95 and 98 are context only. No dependency bump is needed unless SDK capability inspection proves otherwise."}]}],"before_hash":"056c2617f6283a7218efbc0ae838349d029aef9698905903bd7699587356c5ad","after_hash":"63906afa66a595bea6cfe53771ac74d4985f0a826380042cddd113d553aa4031","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"a95185982b33fc75cf3f946529ed8205c49105eab09753bd97f5af8155f59a97"} +{"hash_algorithm":"sha256","ts":"2026-10-23T06:52:55.385Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"files_add","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T06:52:55.385Z"},{"op":"add","path":"/metadata/files","value":[{"path":"evaluation-plan.ts","scope":"project"},{"path":"index.ts","scope":"project"},{"path":"shadow.ts","scope":"project"},{"path":"test/shadow.acceptance.ts","scope":"project"},{"path":"test/shadow.test.ts","scope":"project"}]}],"before_hash":"63906afa66a595bea6cfe53771ac74d4985f0a826380042cddd113d553aa4031","after_hash":"0fcb0ac482c42de0b6043a7163c3443bd55e6cb05952c9c943640f5c4b092edf","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"maintenance","record_hash_version":1,"record_hash":"d32bde6b4b8d1a2e1aa242f40c43bb3985b8033a49fefb2c596c08c376bbd65f"} +{"hash_algorithm":"sha256","ts":"2026-10-23T06:53:18.182Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"docs_add","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T06:53:18.182Z"},{"op":"add","path":"/metadata/docs","value":[{"path":"docs/shadow-policy.md","scope":"project"},{"path":"examples/shadow-policy.json","scope":"project"},{"path":"README.md","scope":"project"}]}],"before_hash":"0fcb0ac482c42de0b6043a7163c3443bd55e6cb05952c9c943640f5c4b092edf","after_hash":"e6c4cc3b9f6cb6ca889644f149b6151718b207a7547bfa7db26862edf828c5a4","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"maintenance","record_hash_version":1,"record_hash":"a27c908857d75464fba42e58ef4feff0e26c47ee13076bddfcf617b55757a294"} +{"hash_algorithm":"sha256","ts":"2026-10-23T06:53:45.427Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"tests_add","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T06:53:45.427Z"},{"op":"add","path":"/metadata/tests","value":[{"command":"node --test --test-name-pattern=\"strict parsers\" test/shadow.test.ts","scope":"project","provenance":{"author":"codex-agent","created_at":"2026-10-23T06:53:44.350Z","source_kind":"local_mutation","source_ref":"feat/shadow-policy-deployment"}}]}],"before_hash":"e6c4cc3b9f6cb6ca889644f149b6151718b207a7547bfa7db26862edf828c5a4","after_hash":"f49fe6a85a252d5aa85b8621c0ef1dfd87103010749241940ffa3897c70afc2f","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"maintenance","record_hash_version":1,"record_hash":"97f1eb791dd9350e8c5c42b24c62f4f7f44210e71008d7dc490d47c9058b2d43"} +{"hash_algorithm":"sha256","ts":"2026-10-23T06:56:27.540Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"add","path":"/metadata/comments/1","value":{"created_at":"2026-10-23T06:56:27.540Z","author":"codex-agent","text":"Implemented a bounded declarative shadow interpreter with no executable candidate tools or tracker authority. Consent pins both generation identities, both policy digests, exact minimized input hashes and separate active/candidate allowances; every frozen member reserves two calls including candidate failures. Immutable per-member SDK receipts make retries free and outcome replacement a refusal. Existing evaluation plans and the authoritative promotion transaction require the comparison receipt for online candidates; standalone offline evaluation cannot bypass it. Local linked parser test passes 1/1. Source and packed acceptance use non-git trackers; full release gates will run in CI."}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T06:56:27.540Z"}],"before_hash":"f49fe6a85a252d5aa85b8621c0ef1dfd87103010749241940ffa3897c70afc2f","after_hash":"ffe0931294246f427fd45da2f2e272dc7e31d1fedb4e68198086a8b5aa73d5d6","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"9176e897a2c52bc9683010743ed660631ea9547281ce1e81ac19983c00188e19"} +{"hash_algorithm":"sha256","ts":"2026-10-23T07:02:21.063Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"add","path":"/metadata/comments/2","value":{"created_at":"2026-10-23T07:02:21.061Z","author":"codex-agent","text":"Behavioral revert proof: independently omitting enforceShadowComparison from the existing evaluation gate fails the retry-safe receipt regression (1/1); propagating a missing candidate decision fails active-isolation (1/1); disabling the two-arm allowance validator fails strict-parser regression (1/1). Restored code passed the preceding complete 7/7 shadow suite including actual generation promotion without git. Final consent-generation binding and additional corruption/concurrency regressions are now under targeted validation. The SDK/dev pin remains 2026.10.17; there is no feature dependency on newer CLI behavior."}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T07:02:21.063Z"}],"before_hash":"ffe0931294246f427fd45da2f2e272dc7e31d1fedb4e68198086a8b5aa73d5d6","after_hash":"971f9185a6b78100ff8d07b5bea4b83bb31d4a402c70e7afe717a85f96b41051","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"3a9c3b7eb503f7b231a7d243fe2139797f7c1fd6c1e4dee8e64b71122da3e8d7"} +{"hash_algorithm":"sha256","ts":"2026-10-23T07:07:42.535Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"docs_add","patch":[{"op":"replace","path":"/metadata/docs/2/path","value":"examples/shadow-policy.json"},{"op":"replace","path":"/metadata/docs/1/path","value":"docs/shadow-policy.md"},{"op":"replace","path":"/metadata/docs/0/path","value":"docs/shadow-evidence.json"},{"op":"add","path":"/metadata/docs/3","value":{"path":"README.md","scope":"project"}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T07:07:42.535Z"}],"before_hash":"971f9185a6b78100ff8d07b5bea4b83bb31d4a402c70e7afe717a85f96b41051","after_hash":"f41b19eccd59a1374c0c8367ccca8007bf7ee688e0bc74f9edb1b98f49222f82","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"maintenance","record_hash_version":1,"record_hash":"b5f804dcca6a0fa0624b41e6ca3bc0547e53e949d09f02802449536aa51b4a85"} +{"hash_algorithm":"sha256","ts":"2026-10-23T07:07:52.857Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"add","path":"/metadata/comments/3","value":{"created_at":"2026-10-23T07:07:52.857Z","author":"codex-agent","text":"Focused source validation passed 11/11 with shadow.ts at 100% lines, branches and functions. Added read-only promotion revalidation and a forged-original-receipt regression; final focused verification and packed replay are running. Public MIT pm-context snapshot: 84 certified items; first 20 sorted ids selected without outcome filtering; 17 closed and 3 other; 20/20 active decisions preserved; 20 disagreements; zero candidate failures; 20 active plus 20 candidate calls charged. All 175 copied source files retain identical digest and read-only permissions. One independent tracker unit correctly refuses promotion despite observed candidate accuracy 0.85 versus active 0.15. JSON observation 548 bytes; comparison JSON 1278 bytes and TOON 1127 bytes. This measures the observation/gating mechanism, not a learned policy improvement."}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T07:07:52.857Z"}],"before_hash":"f41b19eccd59a1374c0c8367ccca8007bf7ee688e0bc74f9edb1b98f49222f82","after_hash":"d3ca4a6a6b84bf70f0463c1d51c6af2c7becace1bca9a51921d14a2a76fbe52d","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"ad96ca7d74f02fe8ddf83fb5a4ada84414071435843af235ffe704c1fdb067fe"} +{"hash_algorithm":"sha256","ts":"2026-10-23T07:17:22.816Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"add","path":"/metadata/comments/4","value":{"created_at":"2026-10-23T07:17:22.816Z","author":"codex-agent","text":"Final focused source suite passed 12/12 at 100% shadow lines/branches/functions; packed public replay passed unchanged. Review then closed an online-lineage bypass: the existing promotion gate now requires shadow evidence even for an unreserved or offline-only successor of an online-bound incumbent, so the recursive proposer cannot escape by omitting the marker. New lineage regression passes 1/1; reverting only the unreserved-lineage enforcement fails 1/1. Legacy lineages never bound online retain their offline contracts. The complete inherited-gate source suite is running; CI release gates are also running at https://github.com/unbraind/pm-rl/actions/runs/39915892410 and will be refreshed for the final head."}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T07:17:22.816Z"}],"before_hash":"d3ca4a6a6b84bf70f0463c1d51c6af2c7becace1bca9a51921d14a2a76fbe52d","after_hash":"935f6748888584186b5fb47bd62ef9ee92fbcbd8eca02c743ac8b486aee91658","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"795759fa9c8349089a3598b7e08f62b04569b9e818b365569110806d54b1d786"} +{"hash_algorithm":"sha256","ts":"2026-10-23T07:20:47.390Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"add","path":"/metadata/comments/5","value":{"created_at":"2026-10-23T07:20:47.390Z","author":"codex-agent","text":"Final implementation validation: node --test --experimental-test-coverage --test-coverage-include=shadow.ts test/shadow.test.ts passes 13/13, zero failures/skips/cancellations/todos, and 100% shadow lines/branches/functions. Typecheck, test typecheck, lint, zero duplication and documented declarations pass. Four independent behavioral reverts fail their intended regressions: receipt prerequisite, candidate-failure isolation, budget validation, and inherited online-lineage enforcement. Packed public validation is read-only, Git-free and reproducible with unchanged aggregate receipts. Review-ready PR https://github.com/unbraind/pm-rl/pull/99 owns the final-head npm/Bun release run URLs and exact counts; full release gates are intentionally delegated to CI on the saturated host. Scope limit: bounded declarative policy tables and valid paired outcomes; one public tracker is not an independent policy-quality win. Automated reviewer skips/unavailable access are not approval. Implementation work is complete; item stays in progress for orchestrator verification."}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T07:20:47.390Z"}],"before_hash":"935f6748888584186b5fb47bd62ef9ee92fbcbd8eca02c743ac8b486aee91658","after_hash":"3a821af7d3989f5be1a3870625a69368726e225e588bd7705bcb59726a61cb05","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"2205a95b92f3e4f81b7f7aa94ac52b72266fbb3dc702f9ba7bfe09f01032ed44"} +{"hash_algorithm":"sha256","ts":"2026-10-23T07:20:51.111Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":{"value":"implementer","source":"argv"},"topic":null},"op":"release","patch":[{"op":"remove","path":"/metadata/claim_principal"},{"op":"remove","path":"/metadata/assignee"},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T07:20:51.111Z"}],"before_hash":"3a821af7d3989f5be1a3870625a69368726e225e588bd7705bcb59726a61cb05","after_hash":"6779e3c4f7063649fa58db8e07953ef2c20445ce702100b318a0ce98fedab425","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"maintenance","record_hash_version":1,"record_hash":"0bcb94ffe3e4244dea135ef43e0c81a5c098773a170ec9fa47ab88dd1e44fbb5"} +{"hash_algorithm":"sha256","ts":"2026-10-23T07:50:55.313Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":{"value":"implementer","source":"argv"},"topic":null},"op":"claim","patch":[{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T07:50:55.313Z"},{"op":"add","path":"/metadata/assignee","value":"codex-agent"},{"op":"add","path":"/metadata/claim_principal","value":"codex-agent"}],"before_hash":"6779e3c4f7063649fa58db8e07953ef2c20445ce702100b318a0ce98fedab425","after_hash":"462aa3c45f6e2bb85736f827bf80030038fb41d372a30417957ad9b0a68aac1f","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"3d452c875dfad0b67e7cf738fdd5f81d40d9470a5e336dd7e03b5efa206b37b0"} +{"hash_algorithm":"sha256","ts":"2026-10-23T07:56:56.268Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"add","path":"/metadata/comments/6","value":{"created_at":"2026-10-23T07:56:56.267Z","author":"codex-agent","text":"Reopened for PR 99 blocking review. Confirmed two root gaps in the previous head: semantically inconsistent original receipts could pass hash/history checks, and reloaded manifests bypassed cohort and both-arm allowance checks. Fix will use shared manifest and receipt validators for observation, resume, comparison and the authoritative promotion gate. Validation will bind actions to consented input/policy tables, enforce finite bounded and same-action-consistent rewards, verify disagreement/failure/digest fields, and check actual unique receipt charges against both allowances. Four requested adversarial regressions and independent behavioral revert proofs will be recorded; no review-thread replies or mentions."}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T07:56:56.268Z"}],"before_hash":"462aa3c45f6e2bb85736f827bf80030038fb41d372a30417957ad9b0a68aac1f","after_hash":"2b7469ba2b5142af1ce801655013e7dfdb491f9e44a9b10f3b29820f700bcaf2","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"0c33753c5540c565473d2ed3ab774b914001fa125848a50c4859c5eba2509a59"} +{"hash_algorithm":"sha256","ts":"2026-10-23T08:16:19.060Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"add","path":"/metadata/comments/7","value":{"created_at":"2026-10-23T08:16:19.060Z","author":"codex-agent","text":"Both blocking review findings are fixed through shared validators. validateDeployment now governs registration and every manifest reload: generation pair, exact plan cohort, consent, both reserved allowances and actual receipt inventory charges; extra slots cannot hide outside report counts. validateObservation now governs live observation, retry/resume, comparison and promotion reads: policy-produced actions for the consented input, bounded rewards, same-action reward equality, disagreement/failure flags and selected-outcome digest. Refusals use pinned SDK context.item_id and context.field and include both in the message. Focused suite passes 25/25 with 100% shadow lines/branches/functions. Four independent behavioral reverts each fail 1/1 with Missing expected rejection: identical-action rewards, zero-budget original manifest, tampered disagreement, and cohort expansion after plan registration; validators restored. Broader original-record tests also cover wrong actions, bounds, digests, slot IDs, unexpected charged receipts, both aggregate charges, edited manifests and native SDK workflow-policy refusal. No SDK mocks, history rewriting, dependency bumps, review-thread replies or mentions."}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T08:16:19.060Z"}],"before_hash":"2b7469ba2b5142af1ce801655013e7dfdb491f9e44a9b10f3b29820f700bcaf2","after_hash":"8b64a4088224bf77f48234005cd043f5299746539ece25d729f187d479e600d7","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"12b72c2e53ec364db88dd023757f71c58f177c648daf53dfaf19dd9ce9d969ae"} +{"hash_algorithm":"sha256","ts":"2026-10-23T08:22:04.807Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":null,"topic":null},"op":"comment_add","patch":[{"op":"add","path":"/metadata/comments/8","value":{"created_at":"2026-10-23T08:22:04.807Z","author":"codex-agent","text":"Final follow-up verification: node --test --experimental-test-coverage --test-coverage-include=shadow.ts test/shadow.test.ts passes 25/25 with 100% lines/branches/functions for shadow.ts. Restored four requested regressions pass 4/4 with Git absent from PATH and an explicit discovery ceiling. Independent revert commands use node --test --test-name-pattern with the exact prefixes review regression: identical actions; review regression: a consented original zero-budget; review regression: an original tampered disagreement; review regression: a manifest cohort grown. Reverting only validateObservation to accept stored rows fails the reward and disagreement cases 1/1 each; reverting only validateDeployment to retain consent without cohort/budget checks fails the budget and expanded-cohort cases 1/1 each, all with Missing expected rejection. Source restored. Pinned linked-test execution passes 1/1; typecheck, test typecheck, lint, zero duplication and docstrings pass. Repacked public acceptance on the existing read-only source copy reproduces docs/shadow-evidence.json byte-for-byte. PR https://github.com/unbraind/pm-rl/pull/99 will carry the exact pushed-head npm/Bun CI run and counts; previous-head gates are historical only. Claim released for orchestrator verification; item remains in progress."}},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T08:22:04.807Z"}],"before_hash":"8b64a4088224bf77f48234005cd043f5299746539ece25d729f187d479e600d7","after_hash":"56ab5c2cf4cf84696f8d0ee227a94d75bcc8952989ec7b29c53a6030be75974c","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"substantive","record_hash_version":1,"record_hash":"2b01b84103ce0905fb4e28fc92f8e963a6653da94ae98f54b9e4243c20333e1b"} +{"hash_algorithm":"sha256","ts":"2026-10-23T08:22:07.469Z","author":"codex-agent","author_source":"asserted","agent_harness":"codex","agent_model":"gpt-6.1-sol","agent_model_source":"probe","agent_instance":"cf73d9707128820a198a2441","agent_provenance":{"model":{"value":"gpt-6.1-sol","source":"probe"},"effort":{"value":"high","source":"probe"},"role":{"value":"implementer","source":"argv"},"topic":null},"op":"release","patch":[{"op":"remove","path":"/metadata/claim_principal"},{"op":"remove","path":"/metadata/assignee"},{"op":"replace","path":"/metadata/updated_at","value":"2026-10-23T08:22:07.469Z"}],"before_hash":"56ab5c2cf4cf84696f8d0ee227a94d75bcc8952989ec7b29c53a6030be75974c","after_hash":"29bfd0ac60335f4d9223f2d1aaa023d86949df208ecd1cb3a3f6c6866a912e87","item_hash_version":3,"context":{"agent_provenance_outcomes":{"model":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"effort":{"status":"resolved","resolver":"codex_session_file","rule_version":"v1"},"version":{"status":"unavailable","reason":"harness_unavailable","resolver":"ai_agent_version","rule_version":"v1"}}},"event_class":"maintenance","record_hash_version":1,"record_hash":"2e98b942a28f79d6a912d588c1bc337d91afed451fd2b8df69fdf4aefe0b3b36"} diff --git a/README.md b/README.md index b9a9e04d..150d544f 100644 --- a/README.md +++ b/README.md @@ -673,8 +673,14 @@ CLI with Git absent from `PATH`. ## Without git -Tracker-based environments, runs, evaluations, predeclared plans, generation -promotion and lineage work in plain directories without Git. Public-history +Tracker-based environments, runs, evaluations, predeclared plans, shadow +observation/comparison, generation promotion and lineage work in plain directories without Git. Public-history dataset/improver builders and Git-tree episode replay need Git provenance. Plan registration, evaluation and promotion are tested with Git absent from `PATH` and an explicit discovery ceiling. + +## Shadow policy deployment + +Observe a candidate beside the active policy on explicitly consented, minimized inputs with `pm rl shadow register`, `observe` and `compare`, or the typed SDK equivalents. Candidate lookup failures preserve the active decision; both arms consume the reserved budget. The comparison receipt feeds the existing predeclared evaluation and generation promotion gate, so online switching requires validated outcomes and the existing approval. No candidate receives mutation or tool authority. + +See [the consent, budget and promotion contract](docs/shadow-policy.md) and [registration example](examples/shadow-policy.json). Policies currently use bounded declarative lookup tables. Registration, reload and promotion share cohort/budget validation; live and loaded receipts share action, reward and disagreement validation. Typed refusals identify the receipt and invalid field. diff --git a/docs/shadow-evidence.json b/docs/shadow-evidence.json new file mode 100644 index 00000000..1ebe8701 --- /dev/null +++ b/docs/shadow-evidence.json @@ -0,0 +1,36 @@ +{ + "format": "pm-rl/shadow-evidence/1", + "source": { + "repository": "unbraind/pm-context", + "commit": "f55cdbbcf2447dae29df789e133fdbe5521a47b6", + "license": "MIT", + "tracker_files": 175, + "digest_before": "4c7d4eb088fef2a3be91b759d58d3231d79b9bc713b7e2737c8c80906a36c993", + "digest_after": "4c7d4eb088fef2a3be91b759d58d3231d79b9bc713b7e2737c8c80906a36c993" + }, + "complete_items": 84, + "selected_items": 20, + "closed_items": 17, + "open_or_other_items": 3, + "active_actions_preserved": 20, + "observations": 20, + "disagreements": 20, + "candidate_failures": 0, + "reserved_calls": 40, + "charged": { + "active_calls": 20, + "candidate_calls": 20 + }, + "independent_units": 1, + "active_mean": 0.15000000000000002, + "candidate_mean": 0.8499999999999999, + "promotion_refusal": "eval_plan_refused", + "promotion_performed": false, + "non_git": true, + "git_absent_from_path": true, + "output_bytes": { + "observation_json": 548, + "comparison_json": 1278, + "comparison_toon": 1127 + } +} diff --git a/docs/shadow-policy.md b/docs/shadow-policy.md new file mode 100644 index 00000000..b21b358f --- /dev/null +++ b/docs/shadow-policy.md @@ -0,0 +1,54 @@ +# Shadow policy observation + +A shadow deployment observes the same consented input with an active policy and a candidate policy. It returns **only the active decision as `active_action`** for the caller to apply. The candidate has no executable code, tool interface, SDK client or mutation capability. The current interpreter is a bounded string-to-action lookup table: at most 10,000 entries per artifact and 1,024 characters per input or action. This supports finite routing and classification policies; model inference and arbitrary executable policies require a separate reviewed adapter. + +`registerShadowDeployment`, `observeShadow` and `compareShadow` are typed SDK exports. Their CLI equivalents are `pm rl shadow register --file deployment.json`, `pm rl shadow observe --file input.json` and `pm rl shadow compare `. All use the host's normal compact TOON output or `--json`. An observation is constant-size; a comparison contains counts, charges and the existing bounded evaluation verdict, never the full cohort. + +Before registration, create a Decision whose body is exactly: + +```ts +JSON.stringify({ + format: "pm-rl/shadow-consent/1", + deployment: hashJson(deployment), +}) +``` + +Consent binds the exact `generation` and `incumbent` identities, preventing one allowance from funding multiple deployments. It approves the exact policy artifacts, `shadowInputDigest(input)` for every declared member, and separate `active_calls` and `candidate_calls` allowances. It is verified against its SDK creation history. Consent to one deployment cannot fund another artifact. Register a new consent and fresh candidate for changed policies or inputs. Minimize inputs before hashing: neither raw tracker titles nor bodies need be retained in observation receipts. + +The registration JSON contains `deployment` and `plan`; see [the example](../examples/shadow-policy.json). Replace the example's incumbent provenance and input identity with actual SDK-derived hashes. The incumbent and candidate generations must declare `training_config.shadow_policy = hashJson(policy)`. Create the incumbent first, register the shadow deployment next, and create the candidate and collection runs afterwards. The existing generation registration automatically pins the reserved evaluation comparison in its provenance. + +Each input file has this shape: + +```json +{"id":"event-0","input":"event","rewards":{"hold":0,"accept":1}} +``` + +The active and candidate tables receive the same minimized input; neither sees the reward map. Rewards must be finite and within the plan's declared bounds. Only use paired observed outcomes for actions with valid known counterfactual rewards, such as routing/classification against a subsequently established label. If taking an action changes its outcome, this reward map is insufficient causal evidence; collect an appropriately designed experiment instead. This API does not infer unobserved action outcomes. + +Every planned input reserves **two calls**, including failed candidate attempts. Each member has exactly one immutable observation identity; identical retries do not add budget or replace outcomes. Missing candidate entries record `candidate_missing_action`; missing candidate outcomes record `candidate_missing_outcome` while preserving the active action. An invalid active policy or unconsented input refuses before either decision is admitted. A storage, consent or malformed-input refusal is an observation failure, not a candidate-policy failure; callers retain responsibility for their established active execution path. + +Manifest validation is shared by registration and every reload. It rechecks the exact generation pair, consent, cohort membership and both allowances against the frozen plan, then checks actual unique observation receipts from the SDK's certified inventory. Extra receipt slots are charged and refused rather than omitted from comparison counts. Failed candidate attempts still consume both arms' reserved slot; retries cannot create a second slot. + +Receipt validation is also shared by the live observer, resumed observations, comparison and promotion. The consented input digest resolves through the pinned active policy's input table; both actions must match those policy artifacts. Rewards must be finite and within the plan bounds, and identical actions must have identical rewards. Disagreement and failure flags are recomputed. `outcome_digest` commits the selected, known arm outcomes only; unselected reward entries are not comparison evidence. Existing unverifiable or incompatible receipts are refused and require a fresh candidate/plan, never history repair. + +A mismatch returns a typed conflict with `context.item_id` naming the receipt and `context.field` naming the invalid field, such as `candidate_reward`, `disagreement`, `inputs`, `budget.active_calls` or `charged.candidate_calls`. These locations are also present in the refusal message, without copying the rejected values. Promotion performs the same checks inside its existing writer transaction. + +`compareShadow` requires the complete frozen cohort. Candidate failures are sealed as a comparison with no eligible outcome. Otherwise it delegates the paired observations to the existing predeclared evaluation module, retaining its fixed stopping rule, equal-weight correlation units, confidence bound and effect threshold. Disagreements alone are never evidence of improvement. Observations from one tracker should normally share one correlation unit; repeated items do not become independent projects. + +The **existing** generation promotion transaction, also used by the recursive loop and inner meta training, verifies the shadow receipt inside its authoritative writer lock. A manually supplied offline evaluation result cannot bypass the receipt. No comparison command promotes anything. The existing contamination, score, approval and promotion-budget checks still apply. The online marker follows the lineage: a candidate with a `shadow_policy` binding, or with an online-bound parent, cannot use an unreserved or offline-only evaluation to escape the shadow requirement. This also blocks unobserved successors proposed by the existing recursive controller. Offline experiments whose lineage has never been bound online keep their existing evaluation contracts; selecting an offline recipe is not deployment authorization. Deploy an online successor by reserving its shadow plan and using the same generation promotion command, never a separate switching API. + +Typed conflict refusals include `shadow_consent`, `shadow_budget`, `shadow_missing`, `shadow_incomplete`, `shadow_failed` and `shadow_drift`; `shadow_invalid` identifies unsupported input/configuration. Every refusal includes a remedy. Complete the declared observations for incomplete evidence; preserve the active policy and create a fresh consented candidate after drift or failure. A statistically refused comparison requires a fresh predeclared experiment, not another look at the same candidate. + +All records use the pm SDK's certified complete reads, history snapshots, schema, content hashing and mutation locks. No Git subprocess is needed. Neither policy receives authority to mutate the tracker or execute an action. This is a controlled observation mechanism and a fail-closed deployment prerequisite, not evidence that a candidate improves real-world outcomes. + +For an SDK caller, `hashJson` accepts `JsonValue`; hash the validated deployment as `deployment as unknown as JsonValue` when constructing the consent body. The JSON example deliberately has one independent unit and fewer units than `minSamples`: it demonstrates observation and a safe promotion refusal, not a powered improvement claim. Copy the exact unrounded `outcome.candidate_mean` into a promotion score; do not recompute or round it from aggregate counts. + +Reproduce the public-data acceptance from the repository with: + +```bash +npm run build +npm pack --pack-destination +node test/shadow.acceptance.ts +``` + +The runner certifies a complete SDK inventory, selects the first 20 sorted item IDs without outcome filtering, observes constant classifiers on real closed-status labels, and verifies every selected source file remains read-only and byte-identical. It installs the packed package and runs JSON/TOON commands with Git absent. The sole repository counts as one correlation unit, so its outcome comparison refuses promotion regardless of its empirical mean. diff --git a/evaluation-plan.ts b/evaluation-plan.ts index 6b77c160..2121cf9d 100644 --- a/evaluation-plan.ts +++ b/evaluation-plan.ts @@ -3,6 +3,7 @@ import type { GetResult, PmClient } from "@unbrained/pm-cli/sdk/core"; import { createPmCliExpectedError, EXIT_CODE, isPmCliExpectedError, normalizeItemId } from "@unbrained/pm-cli/sdk/runtime"; import { extractGenerationSpec, generationProvenance, hashJson, readCompleteLoopItem, type JsonValue } from "./index.ts"; import { decidePromotion, type PromotionCriterion, type PromotionDecision } from "./promotion.ts"; +import { enforceShadowComparison } from "./shadow.ts"; import type { GenerationSpec, ScoreRecord } from "./lineage.ts"; /** Stable refusals available to SDK consumers and CLI agents. */ @@ -12,6 +13,8 @@ export interface EvaluationMember { readonly id: string; readonly unit: string } /** All statistical and cohort choices are frozen before the candidate and its collection runs exist. */ export interface EvaluationPlan { readonly format: "pm-rl/evaluation-plan/1"; + /** Content identity of the consented online observation manifest, when deployed in shadow. */ + readonly shadow?: string; readonly candidate: string; readonly incumbent: string; readonly incumbent_version: string; @@ -66,7 +69,8 @@ function label(value: unknown): string { } /** Validate the entire fixed design; adaptive stopping needs a different future plan format. */ export function parseEvaluationPlan(value: unknown): EvaluationPlan { - const p = object(value, ["format", "candidate", "incumbent", "incumbent_version", "population", "selection", "correlation_unit", "members", "objective", "objective_version", "evaluation_context", "direction", "reward_bounds", "criterion", "stopping"]); + const shadowKeys = value !== null && typeof value === "object" && Object.hasOwn(value, "shadow") ? ["shadow"] : []; + const p = object(value, [...shadowKeys, "format", "candidate", "incumbent", "incumbent_version", "population", "selection", "correlation_unit", "members", "objective", "objective_version", "evaluation_context", "direction", "reward_bounds", "criterion", "stopping"]); if (p.format !== "pm-rl/evaluation-plan/1" || (p.direction !== "maximize" && p.direction !== "minimize")) refuse("eval_plan_invalid"); const candidate = label(p.candidate); const incumbent = label(p.incumbent); if (candidate === incumbent) refuse("eval_plan_invalid"); @@ -80,7 +84,7 @@ export function parseEvaluationPlan(value: unknown): EvaluationPlan { if (typeof c.confidence !== "number" || !Number.isFinite(c.confidence) || c.confidence <= 0 || c.confidence >= 1 || typeof c.minSamples !== "number" || !Number.isSafeInteger(c.minSamples) || c.minSamples < 1 || typeof c.effectThreshold !== "number" || !Number.isFinite(c.effectThreshold) || c.effectThreshold < 0) refuse("eval_plan_invalid"); const b = p.reward_bounds; if (!Array.isArray(b) || b.length !== 2 || b.some((v) => typeof v !== "number" || !Number.isFinite(v)) || b[0] >= b[1] || !Number.isFinite(b[1] - b[0])) refuse("eval_plan_invalid"); - return { format: "pm-rl/evaluation-plan/1", candidate, incumbent, incumbent_version: label(p.incumbent_version), population: label(p.population), selection: label(p.selection), correlation_unit: label(p.correlation_unit), members, + return { ...(shadowKeys.length === 0 ? {} : { shadow: label(p.shadow) }), format: "pm-rl/evaluation-plan/1", candidate, incumbent, incumbent_version: label(p.incumbent_version), population: label(p.population), selection: label(p.selection), correlation_unit: label(p.correlation_unit), members, objective: label(p.objective), objective_version: label(p.objective_version), evaluation_context: label(p.evaluation_context), direction: p.direction as EvaluationPlan["direction"], reward_bounds: [b[0] as number, b[1] as number], criterion: { confidence: c.confidence, minSamples: c.minSamples, effectThreshold: c.effectThreshold }, stopping: { kind: "fixed", observations: members.length, independent_units: units.size } }; @@ -90,7 +94,7 @@ export function evaluationPlanId(candidate: string): string { return `${label(ca /** Every plan choice contributes to comparison identity; member order does not. */ export function evaluationComparison(value: unknown): string { return hashJson(parseEvaluationPlan(value) as unknown as JsonValue); } /** Return only a certified absence; malformed, denied and incomplete reads propagate. */ -async function optionalItem(client: PmClient, id: string, at?: string): Promise { +export async function optionalItem(client: PmClient, id: string, at?: string): Promise { try { return (await readCompleteLoopItem(client, id, at)).item; } catch (error) { if (isPmCliExpectedError(error) && error.exitCode === EXIT_CODE.NOT_FOUND) return null; throw error; } } @@ -227,9 +231,11 @@ export async function enforceEvaluationPlan(client: PmClient, candidate: string, if (registered === null) { const item = await optionalItem(client, candidate); if (item !== null && item.type === "Generation" && extractGenerationSpec(String(item.body), "Candidate generation").evaluation_plan !== undefined) refuse("eval_plan_missing"); + await enforceShadowComparison(client, candidate, null); return; } await verifiedCandidate(client, candidate, registered); + await enforceShadowComparison(client, candidate, registered.plan); const result = await optionalItem(client, `${registered.id}-result`); if (result === null) refuse("eval_plan_missing"); const raw = object(storedJson(result.body), ["comparison", "observations"]); if (result.type !== "Decision" || raw.comparison !== registered.comparison || result.affected_version !== hashJson(raw as JsonValue)) refuse("eval_plan_drift"); diff --git a/examples/shadow-policy.json b/examples/shadow-policy.json new file mode 100644 index 00000000..b8895902 --- /dev/null +++ b/examples/shadow-policy.json @@ -0,0 +1,54 @@ +{ + "deployment": { + "format": "pm-rl/shadow/1", + "consent": "rl-consent", + "active": { + "event": "hold" + }, + "candidate": { + "event": "accept" + }, + "inputs": { + "event-0": "8f6e29cfb2f0df76f9f90f221edf3a7b4cfc6ba2e409bb0a78a2c031afd78580" + }, + "budget": { + "active_calls": 1, + "candidate_calls": 1 + }, + "generation": "rl-candidate", + "incumbent": "rl-seed" + }, + "plan": { + "format": "pm-rl/evaluation-plan/1", + "candidate": "rl-candidate", + "incumbent": "rl-seed", + "incumbent_version": "replace-with-verified-incumbent-provenance", + "population": "Explicitly consented events", + "selection": "All declared events", + "correlation_unit": "tracker", + "members": [ + { + "id": "event-0", + "unit": "public-tracker" + } + ], + "objective": "decision_reward", + "objective_version": "v1", + "evaluation_context": "held-out", + "direction": "maximize", + "reward_bounds": [ + 0, + 1 + ], + "criterion": { + "confidence": 0.95, + "minSamples": 10, + "effectThreshold": 0.01 + }, + "stopping": { + "kind": "fixed", + "observations": 1, + "independent_units": 1 + } + } +} diff --git a/index.ts b/index.ts index a981c126..b568f4bf 100644 --- a/index.ts +++ b/index.ts @@ -151,6 +151,7 @@ export { type StoredLmGeneration, } from "./lm.ts"; +export { parseShadowDeployment, shadowInputDigest, registerShadowDeployment, observeShadow, compareShadow, type ShadowPolicy, type ShadowDeployment, type ShadowInput, type ShadowObservation, type ShadowReport, type ShadowRefusal } from "./shadow.ts"; export { parseEvaluationPlan, evaluationPlanId, evaluationComparison, registerEvaluationPlan, readEvaluationPlan, evaluatePlan, recordPlanEvaluation, enforceEvaluationPlan, type EvaluationPlan, type EvaluationMember, type EvaluationObservation, type EvaluationPlanReport, type RegisteredEvaluationPlan, type EvaluationPlanRefusal } from "./evaluation-plan.ts"; export { decidePromotion, hoeffdingEpsilon, parsePromotionCriterion, parsePromotionEvidence, type PromotionCriterion, type PromotionDecision, type PromotionEvidence, type PromotionGateInput } from "./promotion.ts"; @@ -209,6 +210,7 @@ import { PmClient, type GetResult, type ItemMetadata } from "@unbrained/pm-cli/s import { acquireLock, certifyCompleteListResult, createPmCliExpectedError, EXIT_CODE, isPmCliExpectedError, normalizeItemId, type PmCliExpectedError } from "@unbrained/pm-cli/sdk/runtime"; import { parseJsonRecord } from "./refuse.ts"; +import { registerShadowDeployment, observeShadow, compareShadow } from "./shadow.ts"; import { registerEvaluationPlan, readEvaluationPlan, recordPlanEvaluation, enforceEvaluationPlan } from "./evaluation-plan.ts"; import { buildTrackerLoopCorpus } from "./corpus.ts"; import { decodeCheckpointArtifacts, encodeCheckpointArtifacts, collectCheckpointArtifacts, inspectCheckpointArtifact } from "./checkpoint-artifacts.ts"; @@ -5019,6 +5021,25 @@ export const RL_COMMANDS = [ { long: "--file", value_name: "path", value_type: "string", required: true, description: "Benchmark JSON file." }, { long: "--contaminated-by", value_name: "environment-ids", value_type: "string", description: "Comma-separated environment versions whose training data overlaps the suite." }, ], run: registerBenchmark }), + defineCommand({ name: "rl shadow register", description: "Reserve consented online observation and both-arm budgets before candidate collection.", flags: [ + { long: "--file", value_name: "path", value_type: "string", required: true, description: "JSON with deployment and predeclared plan." }, + ], run: async (context) => { + const value = jsonObject(readJsonFile(stringOption(context, "file")!, "Shadow deployment"), "Shadow deployment"); + const report = await registerShadowDeployment(clientFor(context), value.deployment, value.plan); + return { action: "rl-shadow-register", id: report.id, details: { ...report } }; + } }), + defineCommand({ name: "rl shadow observe", description: "Record one active and candidate decision; a failed candidate never replaces the active action.", arguments: [{ name: "candidate", required: true, description: "Candidate generation id." }], flags: [ + { long: "--file", value_name: "path", value_type: "string", required: true, description: "One consented input and bounded observed rewards JSON." }, + ], run: async (context) => { + const client = clientFor(context); const item = await getTypedItem(client, requiredArgument(context, "a candidate id"), "Generation"); + const report = await observeShadow(client, String(item.item.id), readJsonFile(stringOption(context, "file")!, "Shadow input")); + return { action: "rl-shadow-observe", id: report.id, details: { ...report } }; + } }), + defineCommand({ name: "rl shadow compare", description: "Seal the validated shadow outcome comparison required by existing generation promotion.", arguments: [{ name: "candidate", required: true, description: "Candidate generation id." }], run: async (context) => { + const client = clientFor(context); const item = await getTypedItem(client, requiredArgument(context, "a candidate id"), "Generation"); + const report = await compareShadow(client, String(item.item.id)); + return { action: "rl-shadow-compare", id: report.receipt, details: { ...report } }; + } }), defineCommand({ name: "rl eval plan register", description: "Predeclare a fixed cohort and correlation units before candidate collection.", flags: [ { long: "--file", value_name: "path", value_type: "string", required: true, description: "Evaluation plan JSON." }, ], run: async (context) => { diff --git a/manifest.json b/manifest.json index 397b03cf..95c3a104 100644 --- a/manifest.json +++ b/manifest.json @@ -1,7 +1,7 @@ { "name": "pm-rl", "version": "2026.7.31", - "description": "Reinforcement-learning programme management on the pm SDK. Content-addressed environments and benchmarks keep runs and evaluations attributable; fail-closed leaderboards refuse mixed environments and contaminated suites. Commands: `pm rl env register/list/show`, `pm rl benchmark register`, `pm rl eval record`, `pm rl leaderboard`, `pm rl run start/log/show/finish`, `pm rl generation register/promote/show`, `pm rl lineage`, `pm rl invalidate`, and `pm rl compare`. Bounded recipe search: `pm rl dataset build` and `pm rl meta run`. Predeclared paired comparisons: `pm rl eval plan register/show/evaluate` freeze populations, correlation units and fixed stopping rules before experiments and gate planned promotions.", + "description": "Reinforcement-learning programme management on the pm SDK. Content-addressed environments and benchmarks keep runs and evaluations attributable; fail-closed leaderboards refuse mixed environments and contaminated suites. Commands: `pm rl env register/list/show`, `pm rl benchmark register`, `pm rl eval record`, `pm rl leaderboard`, `pm rl run start/log/show/finish`, `pm rl generation register/promote/show`, `pm rl lineage`, `pm rl invalidate`, and `pm rl compare`. Bounded recipe search: `pm rl dataset build` and `pm rl meta run`. Predeclared paired comparisons: `pm rl eval plan register/show/evaluate` freeze populations, correlation units and fixed stopping rules before experiments and gate planned promotions. Online shadow observation: `pm rl shadow register/observe/compare` pin consent, account both arms and gate online promotion on a validated comparison receipt.", "author": "@unbraind", "entry": "./dist/index.js", "priority": 50, diff --git a/package.json b/package.json index 87f5e149..4a0ce950 100644 --- a/package.json +++ b/package.json @@ -66,7 +66,10 @@ "docs/tracker-loop.md", "docs/tracker-loop-evidence.json", "dist/evaluation-plan.*", - "docs/evaluation-plan-evidence.json" + "docs/evaluation-plan-evidence.json", + "dist/shadow.*", + "docs/shadow-policy.md", + "docs/shadow-evidence.json" ], "engines": { "node": ">=22.18.0" @@ -141,7 +144,8 @@ "test/prefix.test.ts", "test/improver.test.ts", "test/corpus.test.ts", - "test/evaluation-plan.test.ts" + "test/evaluation-plan.test.ts", + "test/shadow.test.ts" ], "thresholds": { "statements": 100, @@ -235,7 +239,9 @@ "docs/improver-calibration-evidence.json", "docs/tracker-loop.md", "docs/tracker-loop-evidence.json", - "docs/evaluation-plan-evidence.json" + "docs/evaluation-plan-evidence.json", + "docs/shadow-policy.md", + "docs/shadow-evidence.json" ], "examples": [ "README.md", @@ -249,7 +255,8 @@ "examples/meta-approval.md", "examples/loop-corpus.json", "examples/evaluation-plan.json", - "examples/evaluation-observations.json" + "examples/evaluation-observations.json", + "examples/shadow-policy.json" ] } } diff --git a/shadow.ts b/shadow.ts new file mode 100644 index 00000000..5745b668 --- /dev/null +++ b/shadow.ts @@ -0,0 +1,280 @@ +/** Bounded online observation of declarative policies, without candidate execution authority. */ +import type { GetResult, PmClient } from "@unbrained/pm-cli/sdk/core"; +import { createPmCliExpectedError, EXIT_CODE, isPmCliExpectedError } from "@unbrained/pm-cli/sdk/runtime"; +import { canonicalJson, hashJson, extractGenerationSpec, readCompleteLoopItem, type JsonValue } from "./index.ts"; +import { optionalItem as optional, parseEvaluationPlan, readEvaluationPlan, registerEvaluationPlan, recordPlanEvaluation, evaluatePlan, type EvaluationObservation, type EvaluationPlan, type EvaluationPlanReport } from "./evaluation-plan.ts"; + +/** A total active decision table; an absent candidate entry is an isolated candidate failure. */ +export type ShadowPolicy = Readonly>; +/** Explicit consent pins every minimized input, both policy artifacts and the entire resource allowance. */ +export interface ShadowDeployment { + readonly format: "pm-rl/shadow/1"; + readonly consent: string; + readonly generation: string; + readonly incumbent: string; + readonly active: ShadowPolicy; + readonly candidate: ShadowPolicy; + readonly inputs: Readonly>; + readonly budget: { readonly active_calls: number; readonly candidate_calls: number }; +} +/** One consented input and observed rewards for the available decisions, never commands to execute. */ +export interface ShadowInput { readonly id: string; readonly input: string; readonly rewards: Readonly> } +/** Candidate diagnostics cannot substitute for the active action. */ +export interface ShadowObservation { + readonly id: string; + readonly input_digest: string; + readonly outcome_digest: string; + readonly active_action: string; + readonly candidate_action: string | null; + readonly active_reward: number; + readonly candidate_reward: number | null; + readonly disagreement: boolean; + readonly failure: "candidate_missing_action" | "candidate_missing_outcome" | null; +} +/** Compact receipt; raw outcomes stay in independently verified per-input tracker records. */ +export interface ShadowReport { + readonly receipt: string; + readonly comparison: string; + readonly observations: number; + readonly disagreements: number; + readonly candidate_failures: number; + readonly charged: { readonly active_calls: number; readonly candidate_calls: number }; + readonly outcome: EvaluationPlanReport | null; + readonly remedy: string | null; +} +/** Machine-readable refusal with a recovery action and no private input content. */ +export type ShadowRefusal = "shadow_invalid" | "shadow_consent" | "shadow_budget" | "shadow_drift" | "shadow_missing" | "shadow_incomplete" | "shadow_failed"; +/** Emit consistent typed refusals for CLI and SDK callers. */ +function refuse(code: ShadowRefusal, receipt = "shadow-deployment", field = "manifest"): never { + throw createPmCliExpectedError(`Shadow observation refused (${code}); receipt ${receipt}, field ${field}. Supply explicit Decision consent for the exact deployment digest, both-arm budgets and inputs; use a fresh candidate and plan after drift or failure. Complete and compare all observations before promotion.`, { exitCode: EXIT_CODE.CONFLICT, context: { code, item_id: receipt, field } }); +} +/** Validate structured records without executing caller-owned policy code. */ +function record(value: unknown): Record { + if (value === null || typeof value !== "object" || Array.isArray(value)) refuse("shadow_invalid"); + return value as Record; +} +/** Bound labels to keep policy evaluation and receipt sizes predictable. */ +function label(value: unknown): string { + if (typeof value !== "string" || value.length === 0 || value.length > 1024) refuse("shadow_invalid"); + return value; +} +/** Reject ignored configuration keys rather than implying unsupported behaviour. */ +function keys(value: Record, expected: readonly string[]): void { + if (Object.keys(value).length !== expected.length || expected.some((key) => !Object.hasOwn(value, key))) refuse("shadow_invalid"); +} +/** Normalize inert lookup artifacts and reject unbounded or executable configurations. */ +function table(value: unknown): ShadowPolicy { + const entries = Object.entries(record(value)); + if (entries.length > 10_000) refuse("shadow_invalid"); + return Object.fromEntries(entries.map(([key, action]) => [label(key), label(action)])); +} +/** Parse the sole supported policy interpreter and two explicit per-arm allowances. */ +export function parseShadowDeployment(value: unknown): ShadowDeployment { + const p = record(value); keys(p, ["format", "consent", "generation", "incumbent", "active", "candidate", "inputs", "budget"]); + if (p.format !== "pm-rl/shadow/1") refuse("shadow_invalid"); + const budget = record(p.budget); keys(budget, ["active_calls", "candidate_calls"]); + if ([budget.active_calls, budget.candidate_calls].some((n) => typeof n !== "number" || !Number.isSafeInteger(n) || n < 0 || n > 10_000)) refuse("shadow_budget"); + const inputs = table(p.inputs); + if (Object.values(inputs).some((digest) => !/^[a-f0-9]{64}$/.test(digest))) refuse("shadow_invalid"); + return { format: "pm-rl/shadow/1", consent: label(p.consent), generation: label(p.generation), incumbent: label(p.incumbent), active: table(p.active), candidate: table(p.candidate), inputs, + budget: { active_calls: budget.active_calls as number, candidate_calls: budget.candidate_calls as number } }; +} +/** Stable input identity allows consent without retaining source tracker content. */ +export function shadowInputDigest(input: string): string { return hashJson(label(input)); } +/** Attach an exact receipt location to structural parser refusals without disclosing record values. */ +function located(receipt: string, field: string, parse: () => T): T { + try { return parse(); } + catch (error) { refuse(isPmCliExpectedError(error) && error.context.code === "shadow_budget" ? "shadow_budget" : "shadow_drift", receipt, field); } +} +/** Verify sealed package records against SDK-certified creation history. */ +async function sealed(client: PmClient, id: string): Promise { + const item = await optional(client, id); const original = await optional(client, id, "1"); + if (item === null && original === null) return null; + if (item === null || original === null) refuse("shadow_drift", id, "history"); + if (item.type !== "Decision") refuse("shadow_drift", id, "type"); + for (const field of ["type", "body", "affected_version", "created_at"] as const) { + if (item[field] !== original[field]) refuse("shadow_drift", id, field); + } + return item; +} +/** Decode only package JSON and suppress raw parser diagnostics. */ +function json(body: unknown, receipt: string): unknown { try { return JSON.parse(String(body)); } catch { return refuse("shadow_drift", receipt, "body"); } } +/** Content-addressed writes are retry-safe under the SDK's mutation lock. */ +async function seal(client: PmClient, id: string, value: JsonValue): Promise { + const body = canonicalJson(value); const digest = hashJson(value); + const old = await sealed(client, id); + if (old !== null) { if (old.body !== body || old.affected_version !== digest) refuse("shadow_drift", id, "body"); return; } + try { await client.create({ id, type: "Decision", title: "Bounded shadow observation receipt", body, affectedVersion: digest }); } + catch (error) { const winner = await sealed(client, id); if (winner === null) throw error; if (winner.body !== body || winner.affected_version !== digest) refuse("shadow_drift", id, "body"); } +} +/** Consent is a pre-existing Decision whose entire immutable body approves one deployment digest. */ +async function consent(client: PmClient, deployment: ShadowDeployment): Promise { + const item = await sealed(client, deployment.consent); + if (item === null || item.body !== JSON.stringify({ format: "pm-rl/shadow-consent/1", deployment: hashJson(deployment as unknown as JsonValue) })) refuse("shadow_consent", deployment.consent, "body"); +} +/** Enumerate all observation slots through the SDK's certified inventory, including unexpected extra charges. */ +async function observationIds(client: PmClient, id: string): Promise { + return (await client.listAllComplete({ includeBody: false })).items.map((item) => String(item.id)) + .filter((key) => key.startsWith(`${id}-`) && key !== `${id}-comparison`).sort(); +} +/** One manifest contract governs registration, reload, resume and actual charged calls. */ +async function validateDeployment(client: PmClient, deployment: ShadowDeployment, plan: EvaluationPlan, id: string, receipts: readonly string[]): Promise { + await consent(client, deployment); + if (plan.candidate !== deployment.generation) refuse("shadow_consent", id, "generation"); + if (plan.incumbent !== deployment.incumbent) refuse("shadow_consent", id, "incumbent"); + const members = plan.members.map((member) => member.id); + if (members.length === 0 || hashJson(Object.keys(deployment.inputs).sort()) !== hashJson(members)) refuse("shadow_consent", id, "inputs"); + for (const arm of ["active_calls", "candidate_calls"] as const) { + if (deployment.budget[arm] < members.length) refuse("shadow_budget", id, `budget.${arm}`); + if (receipts.length > deployment.budget[arm]) refuse("shadow_budget", id, `charged.${arm}`); + } + const allowed = new Set(members.map((member) => `${id}-${hashJson(member)}`)); + for (const receipt of receipts) if (!allowed.has(receipt)) refuse("shadow_consent", receipt, "id"); +} +/** Register online observation before candidate collection through the existing evaluation plan reservation. */ +export async function registerShadowDeployment(client: PmClient, value: unknown, input: unknown): Promise<{ id: string; comparison: string; reserved_calls: number }> { + const deployment = parseShadowDeployment(value); const plan = parseEvaluationPlan(input); + const id = `${plan.candidate}-evaluation-plan-shadow`; + await validateDeployment(client, deployment, plan, id, await observationIds(client, id)); + const digest = hashJson(deployment as unknown as JsonValue); + const registered = await registerEvaluationPlan(client, { ...plan, shadow: digest }); + await seal(client, id, deployment as unknown as JsonValue); + return { id, comparison: registered.comparison, reserved_calls: plan.members.length * 2 }; +} +/** Read and revalidate the exact sealed online manifest and its actual observation inventory. */ +async function deploymentFor(client: PmClient, candidate: string): Promise<{ deployment: ShadowDeployment; plan: EvaluationPlan; id: string; policyInputs: ReadonlyMap }> { + const registered = await readEvaluationPlan(client, candidate); + if (registered === null || registered.plan.shadow === undefined) refuse("shadow_missing"); + const id = `${registered.id}-shadow`; const item = await sealed(client, id); + if (item === null) refuse("shadow_missing", id, "manifest"); + const deployment = located(id, "manifest", () => parseShadowDeployment(json(item.body, id))); + if (hashJson(deployment as unknown as JsonValue) !== registered.plan.shadow) refuse("shadow_drift", id, "comparison"); + if (item.affected_version !== registered.plan.shadow) refuse("shadow_drift", id, "affected_version"); + const receipts = await observationIds(client, id); + await validateDeployment(client, deployment, registered.plan, id, receipts); + return { deployment, plan: registered.plan, id, policyInputs: new Map(Object.keys(deployment.active).map((input) => [shadowInputDigest(input), input])) }; +} +/** Bind the inert artifact to provenance already verified by the existing evaluation gate. */ +async function policyBinding(client: PmClient, generation: string, policy: ShadowPolicy, receipt: string): Promise { + const item = await readCompleteLoopItem(client, generation); + const spec = extractGenerationSpec(String(item.item.body), "Shadow generation"); + if (record(spec.training_config).shadow_policy !== hashJson(policy as JsonValue)) refuse("shadow_drift", receipt, "training_config.shadow_policy"); +} +/** Validate a selected arm reward independently of caller-provided aggregates. */ +function reward(value: unknown, plan: EvaluationPlan, receipt: string, field: string): number { + if (typeof value !== "number" || !Number.isFinite(value) || value < plan.reward_bounds[0] || value > plan.reward_bounds[1]) refuse("shadow_drift", receipt, field); + return value; +} +/** Compare named fields without copying their potentially sensitive values into diagnostics. */ +function sameFields(value: Record, expected: Record, receipt: string): void { + for (const [field, wanted] of Object.entries(expected)) { + if (value[field] !== wanted) refuse("shadow_drift", receipt, field); + } +} +/** Canonically commit only observed arm outcomes, preserving unusual action names as own properties. */ +function selectedRewards(active: string, activeReward: number, candidate: string | null, candidateReward: number | null): Record { + return Object.fromEntries([[active, activeReward], ...(candidate !== null && candidateReward !== null ? [[candidate, candidateReward]] : [])]); +} +/** One semantic receipt validator is used before persistence, on retries and whenever evidence is loaded. */ +function validateObservation(value: unknown, deployment: ShadowDeployment, plan: EvaluationPlan, receipt: string, policyInputs: ReadonlyMap): ShadowObservation { + const row = located(receipt, "fields", () => { const row = record(value); keys(row, ["id", "input_digest", "outcome_digest", "active_action", "candidate_action", "active_reward", "candidate_reward", "disagreement", "failure"]); return row; }); + if (typeof row.id !== "string" || !Object.hasOwn(deployment.inputs, row.id) || !plan.members.some((member) => member.id === row.id)) refuse("shadow_consent", receipt, "id"); + if (receipt !== `${plan.candidate}-evaluation-plan-shadow-${hashJson(row.id)}`) refuse("shadow_drift", receipt, "id"); + if (row.input_digest !== deployment.inputs[row.id]) refuse("shadow_consent", receipt, "input_digest"); + const input = policyInputs.get(String(row.input_digest)); + if (input === undefined) refuse("shadow_drift", receipt, "active_action"); + const activeAction = deployment.active[input]; const candidateAction = Object.hasOwn(deployment.candidate, input) ? deployment.candidate[input] : null; + sameFields(row, { active_action: activeAction, candidate_action: candidateAction }, receipt); + const activeReward = reward(row.active_reward, plan, receipt, "active_reward"); + const candidateReward = row.candidate_reward === null ? null : reward(row.candidate_reward, plan, receipt, "candidate_reward"); + if (candidateAction === activeAction && candidateReward !== activeReward) refuse("shadow_drift", receipt, "candidate_reward"); + if (candidateAction === null && candidateReward !== null) refuse("shadow_drift", receipt, "candidate_reward"); + const failure = candidateAction === null ? "candidate_missing_action" : candidateReward === null ? "candidate_missing_outcome" : null; + sameFields(row, { disagreement: candidateAction !== null && candidateAction !== activeAction, failure }, receipt); + sameFields(row, { outcome_digest: hashJson(selectedRewards(activeAction, activeReward, candidateAction, candidateReward)) }, receipt); + return { id: row.id, input_digest: String(row.input_digest), outcome_digest: String(row.outcome_digest), active_action: activeAction, candidate_action: candidateAction, + active_reward: activeReward, candidate_reward: candidateReward, disagreement: candidateAction !== null && candidateAction !== activeAction, failure }; +} +/** Read one immutable observation through the same validator used by the live observer. */ +async function readObservation(client: PmClient, receipt: string, deployment: ShadowDeployment, plan: EvaluationPlan, policyInputs: ReadonlyMap): Promise { + const item = await sealed(client, receipt); if (item === null) return null; + const row = validateObservation(json(item.body, receipt), deployment, plan, receipt, policyInputs); + if (item.affected_version !== hashJson(row as unknown as JsonValue)) refuse("shadow_drift", receipt, "affected_version"); + return row; +} +/** Observe once per consented input. Missing candidate actions are data, never active-action failures. */ +export async function observeShadow(client: PmClient, candidate: string, value: unknown): Promise { + const { deployment, plan, id, policyInputs } = await deploymentFor(client, candidate); + const row = record(value); keys(row, ["id", "input", "rewards"]); + const member = label(row.id); const input = label(row.input); const inputDigest = shadowInputDigest(input); const receipt = `${id}-${hashJson(member)}`; + if (!Object.hasOwn(deployment.inputs, member) || deployment.inputs[member] !== inputDigest) refuse("shadow_consent", receipt, "input_digest"); + await policyBinding(client, plan.incumbent, deployment.active, id); await policyBinding(client, candidate, deployment.candidate, id); + const rewards = record(row.rewards); const [lo, hi] = plan.reward_bounds; + if (Object.values(rewards).some((reward) => typeof reward !== "number" || !Number.isFinite(reward) || reward < lo || reward > hi)) refuse("shadow_invalid", receipt, "rewards"); + const activeAction = Object.hasOwn(deployment.active, input) ? deployment.active[input] : null; + if (activeAction === null || !Object.hasOwn(rewards, activeAction)) refuse("shadow_invalid", receipt, "active_action"); + const candidateAction = Object.hasOwn(deployment.candidate, input) ? deployment.candidate[input] : null; + const failure = candidateAction === null ? "candidate_missing_action" : Object.hasOwn(rewards, candidateAction) ? null : "candidate_missing_outcome"; + const selected = selectedRewards(activeAction, rewards[activeAction] as number, candidateAction, failure === null ? rewards[candidateAction!] as number : null); + const observation = validateObservation({ id: member, input_digest: inputDigest, outcome_digest: hashJson(selected), active_action: activeAction, candidate_action: candidateAction, + active_reward: rewards[activeAction], candidate_reward: failure === null ? rewards[candidateAction!] : null, + disagreement: candidateAction !== null && candidateAction !== activeAction, failure }, deployment, plan, receipt, policyInputs); + const existing = await readObservation(client, receipt, deployment, plan, policyInputs); + if (existing !== null) sameFields(existing as unknown as Record, observation as unknown as Record, receipt); + await seal(client, receipt, observation as unknown as JsonValue); + return observation; +} +/** Recompute all observations from pinned artifacts; never trust mutable aggregate counts or verdicts. */ +async function shadowComparison(client: PmClient, candidate: string): Promise<{ report: ShadowReport; pairs: EvaluationObservation[] }> { + const { deployment, plan, id, policyInputs } = await deploymentFor(client, candidate); + await policyBinding(client, plan.incumbent, deployment.active, id); await policyBinding(client, candidate, deployment.candidate, id); + const observations: ShadowObservation[] = []; + for (const member of plan.members) { + const receipt = `${id}-${hashJson(member.id)}`; + const row = await readObservation(client, receipt, deployment, plan, policyInputs); + if (row === null) refuse("shadow_incomplete", receipt, "id"); + observations.push(row); + } + const failures = observations.filter((row) => row.failure !== null).length; + const pairs = observations.map((row) => ({ id: row.id, candidate: row.candidate_reward!, incumbent: row.active_reward })); + const outcome = failures === 0 ? evaluatePlan(plan, pairs) : null; + const report: ShadowReport = { receipt: `${id}-comparison`, comparison: hashJson(deployment as unknown as JsonValue), observations: observations.length, + disagreements: observations.filter((row) => row.disagreement).length, candidate_failures: failures, + charged: { active_calls: observations.length, candidate_calls: observations.length }, outcome, + remedy: failures === 0 ? outcome!.remedy : "Keep the active policy; fix the candidate and register a fresh consented plan." }; + return { report, pairs }; +} +/** Seal one online comparison using the existing immutable paired-outcome recorder. */ +export async function compareShadow(client: PmClient, candidate: string): Promise { + const { report, pairs } = await shadowComparison(client, candidate); + if (report.outcome !== null) await recordPlanEvaluation(client, candidate, pairs); + await seal(client, report.receipt, report as unknown as JsonValue); + return report; +} +/** Recognize the explicit online-policy provenance marker without restricting offline trainer config formats. */ +function onlineBound(value: JsonValue): boolean { + return value !== null && typeof value === "object" && !Array.isArray(value) && Object.hasOwn(value, "shadow_policy"); +} +/** The existing promotion transaction requires online comparisons throughout an online-bound lineage. */ +export async function enforceShadowComparison(client: PmClient, candidate: string, plan: EvaluationPlan | null): Promise { + if (plan === null || plan.shadow === undefined) { + const item = await optional(client, candidate); + if (item === null || item.type !== "Generation") return; + const spec = extractGenerationSpec(String(item.body), "Candidate generation"); + if (onlineBound(spec.training_config)) refuse("shadow_missing"); + if (spec.parent !== null) { + const parent = await readCompleteLoopItem(client, spec.parent); + if (onlineBound(extractGenerationSpec(String(parent.item.body), "Incumbent generation").training_config)) refuse("shadow_missing"); + } + return; + } + const receipt = await sealed(client, `${candidate}-evaluation-plan-shadow-comparison`); + if (receipt === null) refuse("shadow_missing", `${candidate}-evaluation-plan-shadow-comparison`, "comparison"); + const { report } = await shadowComparison(client, candidate); + const recorded = located(report.receipt, "fields", () => record(json(receipt.body, report.receipt))); + const charged = located(report.receipt, "charged", () => record(recorded.charged)); + for (const field of ["active_calls", "candidate_calls"] as const) if (charged[field] !== report.charged[field]) refuse("shadow_budget", report.receipt, `charged.${field}`); + if (receipt.body !== canonicalJson(report as unknown as JsonValue)) refuse("shadow_drift", report.receipt, "comparison"); + if (receipt.affected_version !== hashJson(report as unknown as JsonValue)) refuse("shadow_drift", report.receipt, "affected_version"); + if (report.candidate_failures !== 0) refuse("shadow_failed", report.receipt, "candidate_failures"); +} diff --git a/test/shadow.acceptance.ts b/test/shadow.acceptance.ts new file mode 100644 index 00000000..d4c764d0 --- /dev/null +++ b/test/shadow.acceptance.ts @@ -0,0 +1,73 @@ +/** Public-tracker acceptance with aggregate-only receipts and an independently read-only source. */ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import { mkdtempSync, readdirSync, readFileSync, statSync, writeFileSync, mkdirSync, symlinkSync } from "node:fs"; +import { join, resolve, dirname } from "node:path"; +import { tmpdir } from "node:os"; +import { PmClient } from "@unbrained/pm-cli/sdk/core"; +import { init, isPmCliExpectedError } from "@unbrained/pm-cli/sdk/runtime"; +import { generationProvenance, hashJson, type JsonValue } from "../index.ts"; +import { readEvaluationPlan, enforceEvaluationPlan, type EvaluationPlan } from "../evaluation-plan.ts"; +import { shadowInputDigest, observeShadow, compareShadow, type ShadowDeployment } from "../shadow.ts"; +import type { GenerationSpec, ScoreRecord } from "../lineage.ts"; + +/** Fingerprint every supplied source file and require read-only modes throughout. */ +function fingerprint(root: string): { digest: string; files: number } { + const digest = createHash("sha256"); let files = 0; + /** Traverse only this explicitly selected public tracker copy. */ + function walk(relative: string): void { + for (const entry of readdirSync(join(root, relative), { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name))) { + const key = join(relative, entry.name); const file = join(root, key); assert.equal(statSync(file).mode & 0o222, 0); + if (entry.isDirectory()) walk(key); else { assert.ok(entry.isFile()); files += 1; digest.update(JSON.stringify(key)); digest.update(readFileSync(file)); } + } + } + walk(""); return { digest: digest.digest("hex"), files }; +} +const [source, output, commit, archive] = process.argv.slice(2).map((value) => value); +assert.match(commit, /^[a-f0-9]{40}$/); const before = fingerprint(source); +const sourceClient = new PmClient({ pmRoot: source, noExtensions: true }); +const corpus = (await sourceClient.listAllComplete({ includeBody: false })).items; +const selected = [...corpus].sort((a, b) => String(a.id).localeCompare(String(b.id))).slice(0, 20); +assert.equal(selected.length, 20); +const root = mkdtempSync(join(tmpdir(), "shadow-acceptance-")); const cli = resolve("node_modules/@unbrained/pm-cli/dist/cli.js"); +const initialized = await init("rl", { defaults: true, author: "shadow-acceptance", agentGuidance: "skip" }, { cwd: root }); +const commandEnv: NodeJS.ProcessEnv = { ...process.env, PM_PATH: initialized.path, PM_AUTHOR: "shadow-acceptance", GIT_CEILING_DIRECTORIES: dirname(root) }; +execFileSync(process.execPath, [cli, "package", "install", resolve(archive), "--project", "--json"], { cwd: root, env: commandEnv, stdio: "pipe" }); +const bin = join(root, "bin"); mkdirSync(bin); symlinkSync(process.execPath, join(bin, "node")); commandEnv.PATH = bin; +process.env.GIT_CEILING_DIRECTORIES = dirname(root); process.env.PATH = bin; +const client = new PmClient({ pmRoot: initialized.path, author: "shadow-acceptance", noExtensions: true }); +for (const name of ["Generation", "Run"]) await client.schemaAddType(name, { folder: `${name.toLowerCase()}s`, defaultStatus: "open" }); +const active = { event: "open" }; const candidate = { event: "closed" }; +const seed: GenerationSpec = { base_checkpoint: "active-policy", policy: "", collection_runs: [], training_config: { shadow_policy: hashJson(active) }, environment_version: "", reward_spec_version: "", parent: null, seed: true, promoted: false, approval: null, proxy_score: null, held_out_score: null, gap: null, promotion_evidence: null }; +await client.create({ id: "rl-seed", type: "Generation", title: "Active routing baseline", body: `\`\`\`json\n${JSON.stringify(seed)}\n\`\`\``, affectedVersion: hashJson(generationProvenance(seed)) }); +const inputs = Object.fromEntries(selected.map((item) => [hashJson(String(item.id)), shadowInputDigest("event")])); +const deployment: ShadowDeployment = { format: "pm-rl/shadow/1", consent: "rl-consent", generation: "rl-candidate", incumbent: "rl-seed", active, candidate, inputs, budget: { active_calls: selected.length, candidate_calls: selected.length } }; +await client.create({ id: "rl-consent", type: "Decision", title: "Public input shadow consent", body: JSON.stringify({ format: "pm-rl/shadow-consent/1", deployment: hashJson(deployment as unknown as JsonValue) }) }); +const plan: EvaluationPlan = { format: "pm-rl/evaluation-plan/1", candidate: "rl-candidate", incumbent: "rl-seed", incumbent_version: hashJson(generationProvenance(seed)), population: `Public unbraind/pm-context tracker at ${commit}`, selection: "First 20 item ids in sorted complete snapshot; no outcome filtering", correlation_unit: "public repository", members: Object.keys(inputs).map((id) => ({ id, unit: "public-repository" })), objective: "closed_status_accuracy", objective_version: "constant-classifiers/1", evaluation_context: hashJson(selected.map((item) => ({ id: item.id, status: item.status })) as JsonValue), direction: "maximize", reward_bounds: [0, 1], criterion: { confidence: 0.95, minSamples: 10, effectThreshold: 0.01 }, stopping: { kind: "fixed", observations: selected.length, independent_units: 1 } }; +const config = join(root, "deployment.json"); writeFileSync(config, JSON.stringify({ deployment, plan })); +/** Invoke the packed command surface in a plain tracker with no Git executable. */ +function command(args: string[]): string { return execFileSync(process.execPath, [cli, "rl", "shadow", ...args], { cwd: root, env: commandEnv, encoding: "utf8" }); } +const registered = JSON.parse(command(["register", "--file", config, "--json"])) as { details: { reserved_calls: number } }; +assert.equal(registered.details.reserved_calls, 40); +await client.create({ id: "rl-collection", type: "Run", title: "Post-declaration collection" }); +const reservation = (await readEvaluationPlan(client, "rl-candidate"))!; +const successor: GenerationSpec = { ...seed, seed: false, parent: "rl-seed", policy: "collection-policy", collection_runs: ["rl-collection"], environment_version: "train-env", reward_spec_version: "reward", training_config: { shadow_policy: hashJson(candidate) }, evaluation_plan: reservation.comparison }; +await client.create({ id: "rl-candidate", type: "Generation", title: "Shadow candidate", body: `\`\`\`json\n${JSON.stringify(successor)}\n\`\`\``, affectedVersion: hashJson(generationProvenance(successor)) }); +let closed = 0; let unchangedActive = 0; let jsonBytes = 0; +for (const [i, item] of selected.entries()) { + const isClosed = item.status === "closed"; if (isClosed) closed += 1; + const value = { id: hashJson(String(item.id)), input: "event", rewards: { open: isClosed ? 0 : 1, closed: isClosed ? 1 : 0 } }; + const observed = await observeShadow(client, "rl-candidate", value); if (observed.active_action === "open") unchangedActive += 1; + if (i === 0) { + const inputFile = join(root, "input.json"); writeFileSync(inputFile, JSON.stringify(value)); + const serialized = command(["observe", "rl-candidate", "--file", inputFile, "--json"]); jsonBytes = Buffer.byteLength(serialized); assert.match(serialized, /active_action/); + } +} +const report = await compareShadow(client, "rl-candidate"); const cliJson = command(["compare", "rl-candidate", "--json"]); const toon = command(["compare", "rl-candidate"]); +assert.equal(report.candidate_failures, 0); assert.equal(report.observations, 20); assert.equal(report.outcome?.independent_units, 1); assert.equal(report.outcome?.verdict.decision, "refuse"); +const score: ScoreRecord = { objective: plan.objective, objective_version: plan.objective_version, evaluation_context: plan.evaluation_context, seed_set: reservation.comparison, direction: "maximize", scale: 1, value: report.outcome!.candidate_mean }; +let code = ""; +try { await enforceEvaluationPlan(client, "rl-candidate", score); assert.fail("Single tracker must not authorize deployment"); } catch (error) { assert.ok(isPmCliExpectedError(error)); code = String(error.context.code); assert.equal(code, "eval_plan_refused"); } +assert.deepEqual(fingerprint(source), before); assert.equal(unchangedActive, 20); +writeFileSync(output, `${JSON.stringify({ format: "pm-rl/shadow-evidence/1", source: { repository: "unbraind/pm-context", commit, license: "MIT", tracker_files: before.files, digest_before: before.digest, digest_after: before.digest }, complete_items: corpus.length, selected_items: selected.length, closed_items: closed, open_or_other_items: selected.length - closed, active_actions_preserved: unchangedActive, observations: report.observations, disagreements: report.disagreements, candidate_failures: report.candidate_failures, reserved_calls: registered.details.reserved_calls, charged: report.charged, independent_units: report.outcome?.independent_units, active_mean: report.outcome?.incumbent_mean, candidate_mean: report.outcome?.candidate_mean, promotion_refusal: code, promotion_performed: false, non_git: true, git_absent_from_path: true, output_bytes: { observation_json: jsonBytes, comparison_json: Buffer.byteLength(cliJson), comparison_toon: Buffer.byteLength(toon) } }, null, 2)}\n`); diff --git a/test/shadow.test.ts b/test/shadow.test.ts new file mode 100644 index 00000000..c13b7235 --- /dev/null +++ b/test/shadow.test.ts @@ -0,0 +1,394 @@ +/** Online deployment regressions use real SDK trackers with no Git or mocked clients. */ +import assert from "node:assert/strict"; +import { mkdtempSync, rmSync, writeFileSync, mkdirSync, symlinkSync } from "node:fs"; +import { join, dirname } from "node:path"; +import { tmpdir } from "node:os"; +import { after, test } from "node:test"; +import { PmClient } from "@unbrained/pm-cli/sdk/core"; +import { init, isPmCliExpectedError } from "@unbrained/pm-cli/sdk/runtime"; +import { createExtensionTestHarness } from "@unbrained/pm-cli/sdk/testing"; +import extension, { canonicalJson, hashJson, generationProvenance, extractGenerationSpec, readCompleteLoopItem, type JsonValue, type RlCommandResult } from "../index.ts"; +import { registerShadowDeployment, parseShadowDeployment, shadowInputDigest, observeShadow, compareShadow, enforceShadowComparison, type ShadowDeployment, type ShadowObservation, type ShadowReport } from "../shadow.ts"; +import { evaluationComparison, registerEvaluationPlan, readEvaluationPlan, recordPlanEvaluation, enforceEvaluationPlan, evaluatePlan, type EvaluationPlan } from "../evaluation-plan.ts"; +import type { GenerationSpec, ScoreRecord } from "../lineage.ts"; + +const roots: string[] = []; +after(() => { for (const root of roots) rmSync(root, { recursive: true, force: true }); }); +/** A ten-input explicitly consented lookup deployment. */ +function deployment(candidate: ShadowDeployment["candidate"] = { event: "accept" }): ShadowDeployment { + return { format: "pm-rl/shadow/1", consent: "rl-consent", generation: "rl-candidate", incumbent: "rl-seed", active: { event: "hold" }, candidate, + inputs: Object.fromEntries(Array.from({ length: 10 }, (_, i) => [`event-${i}`, shadowInputDigest("event")])), budget: { active_calls: 10, candidate_calls: 10 } }; +} +/** Checkpoint provenance binds the same inert artifacts later interpreted online. */ +function spec(policy: ShadowDeployment["active"], parent: string | null = null): GenerationSpec { + return { base_checkpoint: "checkpoint", policy: parent === null ? "" : "collection-policy", collection_runs: parent === null ? [] : ["rl-collection"], training_config: { shadow_policy: hashJson(policy as JsonValue) }, environment_version: parent === null ? "" : "train-env", reward_spec_version: parent === null ? "" : "rewards", parent, seed: parent === null, promoted: false, approval: null, proxy_score: null, held_out_score: null, gap: null, promotion_evidence: null }; +} +/** Freeze statistical choices before collection; these independent units are synthetic controls only. */ +function plan(d = deployment()): EvaluationPlan { + return { format: "pm-rl/evaluation-plan/1", candidate: "rl-candidate", incumbent: "rl-seed", incumbent_version: hashJson(generationProvenance(spec(d.active))), population: "Consented events", selection: "all events", correlation_unit: "event", members: Object.keys(d.inputs).map((id) => ({ id, unit: id })), objective: "decision_reward", objective_version: "v1", evaluation_context: "held-out", direction: "maximize", reward_bounds: [0, 1], criterion: { confidence: 0.5, minSamples: 10, effectThreshold: 0 }, stopping: { kind: "fixed", observations: 10, independent_units: 10 } }; +} +/** Initialize a non-Git tracker and install actual extension schema. */ +async function workspace(d = deployment()) { + const root = mkdtempSync(join(tmpdir(), "rl-shadow-")); roots.push(root); + const initialized = await init("rl", { defaults: true, author: "shadow-test", agentGuidance: "skip" }, { cwd: root }); + const client = new PmClient({ pmRoot: initialized.path, cwd: root, author: "shadow-test", noExtensions: true }); + for (const [type, folder] of [["Generation", "generations"], ["Run", "runs"]]) await client.schemaAddType(type, { folder, defaultStatus: "open" }); + await addGeneration(client, "rl-seed", spec(d.active)); + await client.create({ id: d.consent, title: "Explicit shadow input consent", type: "Decision", body: JSON.stringify({ format: "pm-rl/shadow-consent/1", deployment: hashJson(d as unknown as JsonValue) }) }); + return { root, pmRoot: initialized.path, client, d, p: plan(d) }; +} +/** Store the same provenance shape as real generation registration. */ +async function addGeneration(client: PmClient, id: string, value: GenerationSpec) { + await client.create({ id, title: id, type: "Generation", body: `\`\`\`json\n${JSON.stringify(value)}\n\`\`\``, affectedVersion: hashJson(generationProvenance(value)) }); +} +/** Collection follows reservation, including SDK creation timestamps. */ +async function collect(w: Awaited>) { + await w.client.create({ id: "rl-collection", title: "Training collection", type: "Run", environment: "train-env" }); + const registered = (await readEvaluationPlan(w.client, "rl-candidate"))!; + await addGeneration(w.client, "rl-candidate", { ...spec(w.d.candidate, "rl-seed"), evaluation_plan: registered.comparison }); +} +/** Check stable typed codes and concrete recovery text. */ +function refusal(code: string) { return (error: unknown): boolean => isPmCliExpectedError(error) && error.context.code === code && error.exitCode === 4 && /fresh|Complete/.test(error.message); } +/** One reward-bearing observation; these rewards are synthetic controls. */ +function input(id = "event-0") { return { id, input: "event", rewards: { accept: 1, hold: 0 } }; } +/** The held-out score is derived from the frozen plan identity. */ +function score(p: EvaluationPlan): ScoreRecord { return { objective: p.objective, objective_version: p.objective_version, evaluation_context: p.evaluation_context, seed_set: evaluationComparison(p), direction: "maximize", scale: 1, value: 1 }; } + +test("strict parsers reject executable, unbounded, ambiguous and invalid consent data", () => { + const d = deployment(); assert.deepEqual(parseShadowDeployment(d), d); + for (const value of [null, [], 1, { ...d, extra: true }, { ...d, format: "v2" }, { ...d, consent: "" }, { ...d, consent: 5 }, { ...d, consent: "x".repeat(1025) }, { ...d, inputs: { id: "not-a-hash" } }, { ...d, active: null }, { ...d, candidate: { event: 1 } }, { ...d, candidate: Object.fromEntries(Array.from({ length: 10001 }, (_, i) => [String(i), "a"])) }]) assert.throws(() => parseShadowDeployment(value), refusal("shadow_invalid")); + for (const n of [-1, 1.2, NaN, Infinity, 10001, "2"]) assert.throws(() => parseShadowDeployment({ ...d, budget: { active_calls: n, candidate_calls: 10 } }), refusal("shadow_budget")); + assert.throws(() => parseShadowDeployment({ ...d, budget: { active_calls: 10 } }), refusal("shadow_invalid")); +}); + +test("same input active action survives candidate failure; both attempts are charged and promotion refuses", async () => { + const w = await workspace(deployment({})); const r = await registerShadowDeployment(w.client, w.d, w.p); + assert.equal(r.reserved_calls, 20); await collect(w); + for (const m of w.p.members) { + const observation = await observeShadow(w.client, "rl-candidate", input(m.id)); + assert.equal(observation.active_action, "hold"); assert.equal(observation.candidate_action, null); assert.equal(observation.failure, "candidate_missing_action"); + } + const report = await compareShadow(w.client, "rl-candidate"); + assert.equal(report.candidate_failures, 10); assert.equal(report.disagreements, 0); assert.deepEqual(report.charged, { active_calls: 10, candidate_calls: 10 }); assert.equal(report.outcome, null); assert.ok(report.remedy); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score((await readEvaluationPlan(w.client, "rl-candidate"))!.plan)), refusal("shadow_failed")); + assert.equal(extractGenerationSpec(String((await readCompleteLoopItem(w.client, "rl-candidate")).item.body), "candidate").promoted, false); +}); + +test("consent and complete dual-arm reservation precede all collection", async () => { + const w = await workspace(); + await assert.rejects(registerShadowDeployment(w.client, { ...w.d, consent: "absent" }, w.p), refusal("shadow_consent")); + await assert.rejects(registerShadowDeployment(w.client, { ...w.d, active: { event: "other" } }, w.p), refusal("shadow_consent")); + await assert.rejects(registerShadowDeployment(w.client, w.d, { ...w.p, candidate: "rl-other" }), refusal("shadow_consent")); + const invalid: ShadowDeployment[] = [{ ...deployment(), inputs: {} }, { ...deployment(), inputs: { extra: shadowInputDigest("event") } }, { ...deployment(), budget: { active_calls: 9, candidate_calls: 10 } }, { ...deployment(), budget: { active_calls: 10, candidate_calls: 9 } }]; + for (const d of invalid) { + const other = await workspace(d); + await assert.rejects(registerShadowDeployment(other.client, d, w.p), refusal(Object.keys(d.inputs).length === 10 ? "shadow_budget" : "shadow_consent")); + } + await assert.rejects(compareShadow(w.client, "missing"), refusal("shadow_missing")); + await registerEvaluationPlan(w.client, w.p); + await assert.rejects(compareShadow(w.client, "rl-candidate"), refusal("shadow_missing")); + await enforceShadowComparison(w.client, "rl-candidate", w.p); +}); + +test("retry-safe observations and comparisons cannot authorize a switch without the sealed outcome", async () => { + const w = await workspace({ ...deployment(), active: { unused: "hold", event: "hold" }, candidate: { unused: "accept", event: "accept" } }); const registered = await registerShadowDeployment(w.client, w.d, w.p); + const reordered = { ...w.d, active: { event: "hold", unused: "hold" }, candidate: { event: "accept", unused: "accept" } }; + assert.deepEqual(await registerShadowDeployment(w.client, reordered, w.p), registered); + await collect(w); const p = (await readEvaluationPlan(w.client, "rl-candidate"))!.plan; + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(p)), refusal("shadow_missing")); + await assert.rejects(compareShadow(w.client, "rl-candidate"), refusal("shadow_incomplete")); + const observations = p.members.map((m) => ({ id: m.id, candidate: 1, incumbent: 0 })); + await recordPlanEvaluation(w.client, "rl-candidate", observations); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(p)), refusal("shadow_missing")); + for (const m of p.members) { + const observed = await observeShadow(w.client, "rl-candidate", input(m.id)); assert.equal(observed.active_action, "hold"); assert.equal(observed.candidate_action, "accept"); + assert.deepEqual(await observeShadow(w.client, "rl-candidate", input(m.id)), observed); + } + const report = await compareShadow(w.client, "rl-candidate"); + assert.equal(report.outcome?.verdict.decision, "promote"); assert.equal(report.candidate_failures, 0); assert.equal(report.remedy, null); assert.ok(JSON.stringify(report).length < 1600); + assert.deepEqual(await compareShadow(w.client, "rl-candidate"), report); + await enforceEvaluationPlan(w.client, "rl-candidate", score(p)); + await assert.rejects(observeShadow(w.client, "rl-candidate", { ...input(), rewards: { accept: 0, hold: 1 } }), refusal("shadow_drift")); + await w.client.delete(`${report.outcome!.plan}-result`); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(p)), refusal("eval_plan_missing")); + await assert.rejects(w.client.get(`${report.outcome!.plan}-result`)); + assert.equal(extractGenerationSpec(String((await readCompleteLoopItem(w.client, "rl-candidate")).item.body), "candidate").promoted, false); +}); + +test("unconsented, malformed and out-of-bound inputs cannot invoke either arm", async () => { + const w = await workspace(); await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + for (const patch of [{ id: "foreign" }, { input: "other" }]) await assert.rejects(observeShadow(w.client, "rl-candidate", { ...input(), ...patch }), refusal("shadow_consent")); + for (const value of [null, { ...input(), extra: 1 }, { ...input(), rewards: { hold: NaN } }, { ...input(), rewards: { hold: -1 } }, { ...input(), rewards: { hold: 2 } }, { ...input(), rewards: { hold: "0" } }, { ...input(), rewards: { accept: 1 } }]) await assert.rejects(observeShadow(w.client, "rl-candidate", value), refusal("shadow_invalid")); + const valid = await observeShadow(w.client, "rl-candidate", { ...input(), rewards: { hold: 0 } }); assert.equal(valid.failure, "candidate_missing_outcome"); + const missingActive = await workspace({ ...deployment(), active: {} }); + await registerShadowDeployment(missingActive.client, missingActive.d, missingActive.p); await collect(missingActive); + await assert.rejects(observeShadow(missingActive.client, "rl-candidate", input()), refusal("shadow_invalid")); +}); + +test("changed policy bindings and edited sealed records fail closed", async () => { + const w = await workspace(); const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const candidate = (await readCompleteLoopItem(w.client, "rl-candidate")).item; + const changed = { ...spec(w.d.candidate, "rl-seed"), training_config: {} }; + await w.client.update("rl-candidate", { body: `\`\`\`json\n${JSON.stringify(changed)}\n\`\`\`` }); + await assert.rejects(observeShadow(w.client, "rl-candidate", input()), refusal("shadow_drift")); + await w.client.update("rl-candidate", { body: String(candidate.body) }); + await w.client.update(registered.id, { body: "{}" }); + await assert.rejects(compareShadow(w.client, "rl-candidate"), refusal("shadow_drift")); +}); + +test("CLI online observation and authoritative promotion work with git absent from PATH", async () => { + const w = await workspace(); const bin = join(w.root, "bin"); mkdirSync(bin); symlinkSync(process.execPath, join(bin, "node")); + const oldPath = process.env.PATH; const oldCeiling = process.env.GIT_CEILING_DIRECTORIES; + process.env.PATH = bin; process.env.GIT_CEILING_DIRECTORIES = dirname(w.root); + try { + const harness = await createExtensionTestHarness(extension, { name: "pm-rl", capabilities: ["commands", "hooks", "schema"] }); + /** Exercise the actual typed extension handlers and persisted SDK data. */ + async function command(name: string, args: string[] = [], options: Record = {}): Promise { + const result = await harness.runCommand({ command: name, pmRoot: w.pmRoot, args, options }); assert.equal(result.handled, true); return result.result as RlCommandResult; + } + const configFile = join(w.root, "deployment.json"); writeFileSync(configFile, JSON.stringify({ deployment: w.d, plan: w.p })); + assert.equal((await command("rl shadow register", [], { file: configFile })).details?.reserved_calls, 20); + await collect(w); + const p = (await readEvaluationPlan(w.client, "rl-candidate"))!.plan; + const scoresFile = join(w.root, "scores.json"); writeFileSync(scoresFile, JSON.stringify({ proxy_score: { ...score(p), evaluation_context: "proxy" }, held_out_score: score(p) })); + await w.client.create({ id: "rl-approval", title: "One promotion", type: "Decision", body: '```json\n{"permitted_promotions":1}\n```' }); + const options = { scores: scoresFile, approval: "rl-approval", evidence: "Validated online paired outcomes" }; + await assert.rejects(command("rl generation promote", ["rl-candidate"], options), refusal("shadow_missing")); + for (const m of p.members) { const rowFile = join(w.root, "input.json"); writeFileSync(rowFile, JSON.stringify(input(m.id))); const result = await command("rl shadow observe", ["candidate"], { file: rowFile }); assert.equal(result.details?.active_action, "hold"); } + assert.equal((await command("rl shadow compare", ["candidate"])).details?.candidate_failures, 0); + assert.equal((await command("rl generation promote", ["rl-candidate"], options)).details?.budget_consumed, 1); + } finally { process.env.PATH = oldPath; if (oldCeiling === undefined) delete process.env.GIT_CEILING_DIRECTORIES; else process.env.GIT_CEILING_DIRECTORIES = oldCeiling; } +}); + + +test("agreement and tied outcomes preserve the incumbent and require a fresh experiment", async () => { + const w = await workspace(deployment({ event: "hold" })); await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + for (const m of w.p.members) { const row = await observeShadow(w.client, "rl-candidate", input(m.id)); assert.equal(row.disagreement, false); } + const report = await compareShadow(w.client, "rl-candidate"); assert.equal(report.outcome?.verdict.decision, "refuse"); assert.equal(report.disagreements, 0); assert.ok(report.remedy); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", { ...score((await readEvaluationPlan(w.client, "rl-candidate"))!.plan), value: 0 }), refusal("eval_plan_refused")); +}); + +test("deleting sealed manifests never restores a legacy promotion path", async () => { + const w = await workspace(); const registration = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + await w.client.delete(registration.id); + await assert.rejects(compareShadow(w.client, "rl-candidate"), refusal("shadow_drift")); +}); + +test("original malformed manifests, missing manifests and forged observation identities refuse", async () => { + const w = await workspace(); + for (const [name, body, expected] of [["missing", null, "shadow_missing"], ["json", "invalid-json", "shadow_drift"], ["digest", JSON.stringify(w.d), "shadow_drift"]] as const) { + const candidate = `rl-${name}`; const p = { ...w.p, candidate, shadow: "wrong-digest" }; + const registered = await registerEvaluationPlan(w.client, p); + if (body !== null) await w.client.create({ id: `${registered.id}-shadow`, title: "Invalid original manifest", type: "Decision", body, affectedVersion: "wrong-digest" }); + await assert.rejects(compareShadow(w.client, candidate), refusal(expected)); + } + const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const fake = { id: "foreign", input_digest: "foreign" }; + await w.client.create({ id: `${registered.id}-${hashJson(w.p.members[0].id)}`, type: "Decision", title: "Invalid original observation", body: JSON.stringify(fake), affectedVersion: hashJson(fake) }); + await assert.rejects(compareShadow(w.client, "rl-candidate"), refusal("shadow_drift")); +}); + +test("concurrent identical observations converge and conflicting outcomes cannot replace the winner", async () => { + const w = await workspace(); await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const identical = await Promise.all([observeShadow(w.client, "rl-candidate", input()), observeShadow(w.client, "rl-candidate", input())]); + assert.deepEqual(identical[0], identical[1]); + const races = await Promise.allSettled([observeShadow(w.client, "rl-candidate", input("event-1")), observeShadow(w.client, "rl-candidate", { ...input("event-1"), rewards: { hold: 1, accept: 0 } })]); + assert.equal(races.filter((result) => result.status === "fulfilled").length, 1); assert.equal(races.filter((result) => result.status === "rejected").length, 1); +}); + +test("a forged original aggregate receipt cannot authorize valid individual observations", async () => { + const w = await workspace(); const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + for (const m of w.p.members) await observeShadow(w.client, "rl-candidate", input(m.id)); + await w.client.create({ id: `${registered.id}-comparison`, title: "Unvalidated aggregate", type: "Decision", body: "{}", affectedVersion: hashJson({}) }); + await assert.rejects(compareShadow(w.client, "rl-candidate"), refusal("shadow_drift")); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score((await readEvaluationPlan(w.client, "rl-candidate"))!.plan)), refusal("shadow_drift")); +}); + +test("online incumbent lineage cannot bypass shadow observation through an unreserved or offline-only child", async () => { + const w = await workspace(); + await w.client.create({ id: "rl-collection", title: "Unreserved collection", type: "Run", environment: "train-env" }); + await addGeneration(w.client, "rl-unreserved", { ...spec(w.d.candidate, "rl-seed"), training_config: {} }); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-unreserved", score(w.p)), refusal("shadow_missing")); + await addGeneration(w.client, "rl-bound", spec(w.d.candidate, "rl-seed")); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-bound", score(w.p)), refusal("shadow_missing")); + const planned = await workspace(); const reserved = await registerEvaluationPlan(planned.client, planned.p); await collect(planned); + await recordPlanEvaluation(planned.client, "rl-candidate", planned.p.members.map((m) => ({ id: m.id, candidate: 1, incumbent: 0 }))); + await assert.rejects(enforceEvaluationPlan(planned.client, "rl-candidate", score(reserved.plan)), refusal("shadow_missing")); + await addGeneration(w.client, "rl-offline-seed", { ...spec(w.d.active), training_config: null }); + await addGeneration(w.client, "rl-offline-child", { ...spec(w.d.candidate, "rl-offline-seed"), training_config: [] }); + await enforceEvaluationPlan(w.client, "rl-offline-child", score(w.p)); + await enforceShadowComparison(w.client, "rl-offline-seed", null); + await w.client.create({ id: "rl-not-generation", title: "Offline task", type: "Task" }); + await enforceShadowComparison(w.client, "rl-not-generation", null); +}); + +/** Match refusal identity and field through the pinned SDK's typed diagnostic contract. */ +function fieldRefusal(code: string, receipt: string, field: string) { + return (error: unknown): boolean => refusal(code)(error) && isPmCliExpectedError(error) && error.context.item_id === receipt && error.context.field === field && error.message.includes(receipt) && error.message.includes(field); +} +/** Persist an original correctly hashed record without rewriting any SDK history. */ +async function originalDecision(client: PmClient, id: string, value: JsonValue, version = hashJson(value)) { + await client.create({ id, type: "Decision", title: "Adversarial original evidence", body: canonicalJson(value), affectedVersion: version }); +} +/** Reproduce a loader-only manifest that bypasses the registration entrypoint. */ +async function originalManifest(w: Awaited>, p = w.p) { + const registered = await registerEvaluationPlan(w.client, { ...p, shadow: hashJson(w.d as unknown as JsonValue) }); + const id = `${registered.id}-shadow`; + await originalDecision(w.client, id, w.d as unknown as JsonValue); + await collect(w); + return { id, plan: registered.plan }; +} +/** A complete original cohort and matching aggregate reproduce pre-fix admission, not just malformed hashes. */ +async function originalCohort(w: Awaited>, p: EvaluationPlan, patch: Partial = {}, reportPatch: Partial = {}, version?: string) { + const id = `${p.candidate}-evaluation-plan-shadow`; + const rows = p.members.map((m): ShadowObservation => ({ id: m.id, input_digest: shadowInputDigest("event"), outcome_digest: hashJson({ hold: 0, accept: 1 }), + active_action: "hold", candidate_action: "accept", active_reward: 0, candidate_reward: 1, disagreement: true, failure: null, ...patch })); + for (const row of rows) await originalDecision(w.client, `${id}-${hashJson(row.id)}`, row as unknown as JsonValue); + const pairs = rows.map((row) => ({ id: row.id, candidate: row.candidate_reward!, incumbent: row.active_reward })); + const outcome = evaluatePlan(p, pairs); assert.equal(outcome.verdict.decision, "promote"); + await recordPlanEvaluation(w.client, p.candidate, pairs); + const report = { receipt: `${id}-comparison`, comparison: hashJson(w.d as unknown as JsonValue), observations: rows.length, + disagreements: rows.filter((row) => row.disagreement).length, candidate_failures: 0, charged: { active_calls: rows.length, candidate_calls: rows.length }, outcome, remedy: null }; + await originalDecision(w.client, report.receipt, { ...report, ...reportPatch } as unknown as JsonValue, version); + return rows; +} + +test("review regression: identical actions with original rewards zero and one never promote", async () => { + const w = await workspace(deployment({ event: "hold" })); const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const p = (await readEvaluationPlan(w.client, "rl-candidate"))!.plan; + await originalCohort(w, p, { candidate_action: "hold", disagreement: false }); + const receipt = `${registered.id}-${hashJson(p.members[0].id)}`; + await assert.rejects(compareShadow(w.client, "rl-candidate"), (error: unknown) => isPmCliExpectedError(error) && error.context.field === "candidate_reward" && String(error.context.item_id).startsWith(`${registered.id}-`)); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(p)), (error: unknown) => isPmCliExpectedError(error) && error.context.field === "candidate_reward"); + await assert.rejects(observeShadow(w.client, "rl-candidate", input(p.members[0].id)), fieldRefusal("shadow_drift", receipt, "candidate_reward")); +}); + +test("review regression: a consented original zero-budget manifest cannot observe compare or promote", async () => { + const w = await workspace({ ...deployment(), budget: { active_calls: 0, candidate_calls: 0 } }); const registered = await originalManifest(w); + await originalCohort(w, registered.plan); + const reject = fieldRefusal("shadow_budget", registered.id, "budget.active_calls"); + await assert.rejects(observeShadow(w.client, "rl-candidate", input()), reject); + await assert.rejects(compareShadow(w.client, "rl-candidate"), reject); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(registered.plan)), reject); +}); + +test("review regression: an original tampered disagreement flag cannot authorize promotion or resume", async () => { + const w = await workspace(); const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const p = (await readEvaluationPlan(w.client, "rl-candidate"))!.plan; + await originalCohort(w, p, { disagreement: false }); + await assert.rejects(compareShadow(w.client, "rl-candidate"), (error: unknown) => isPmCliExpectedError(error) && error.context.field === "disagreement"); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(p)), (error: unknown) => isPmCliExpectedError(error) && error.context.field === "disagreement"); + await assert.rejects(observeShadow(w.client, "rl-candidate", input()), fieldRefusal("shadow_drift", `${registered.id}-${hashJson("event-0")}`, "disagreement")); +}); + +test("review regression: a manifest cohort grown after plan registration never extends consented observation", async () => { + const d = deployment(); const w = await workspace({ ...d, inputs: { ...d.inputs, "event-10": shadowInputDigest("event") }, budget: { active_calls: 11, candidate_calls: 11 } }); + const registered = await originalManifest(w, plan(d)); await originalCohort(w, registered.plan); + const reject = fieldRefusal("shadow_consent", registered.id, "inputs"); + await assert.rejects(compareShadow(w.client, "rl-candidate"), reject); + await assert.rejects(observeShadow(w.client, "rl-candidate", input("event-10")), reject); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(registered.plan)), reject); +}); + +/** A valid compact original observation for testing one broken field at a time. */ +function originalObservation(overrides: Partial = {}): ShadowObservation { + return { id: "event-0", input_digest: shadowInputDigest("event"), outcome_digest: hashJson({ hold: 0, accept: 1 }), active_action: "hold", candidate_action: "accept", active_reward: 0, candidate_reward: 1, disagreement: true, failure: null, ...overrides }; +} + +test("loaded observations reject wrong actions bounds digests failures membership and slot identity", async () => { + const cases: { value: unknown; field: string; code?: string; version?: string }[] = [ + { value: null, field: "fields" }, { value: { ...originalObservation(), extra: true }, field: "fields" }, + { value: originalObservation({ id: "foreign" }), field: "id", code: "shadow_consent" }, + { value: originalObservation({ id: "event-1" }), field: "id" }, + { value: originalObservation({ input_digest: "foreign" }), field: "input_digest", code: "shadow_consent" }, + { value: originalObservation({ active_action: "other" }), field: "active_action" }, + { value: originalObservation({ candidate_action: "other" }), field: "candidate_action" }, + { value: originalObservation({ active_reward: -1 }), field: "active_reward" }, + { value: originalObservation({ candidate_reward: 2 }), field: "candidate_reward" }, + { value: originalObservation({ failure: "candidate_missing_outcome" }), field: "failure" }, + { value: originalObservation({ outcome_digest: "wrong" }), field: "outcome_digest" }, + { value: originalObservation(), field: "affected_version", version: "wrong" }, + ]; + for (const entry of cases) { + const w = await workspace(); const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const receipt = `${registered.id}-${hashJson("event-0")}`; + await w.client.create({ id: receipt, type: "Decision", title: "Invalid original observation field", body: canonicalJson(entry.value as JsonValue), affectedVersion: entry.version ?? hashJson(entry.value as JsonValue) }); + await assert.rejects(compareShadow(w.client, "rl-candidate"), fieldRefusal(entry.code ?? "shadow_drift", receipt, entry.field)); + } +}); + +test("actual inventory charges include unexpected receipts and enforce each arm separately", async () => { + for (const [budget, field] of [[{ active_calls: 10, candidate_calls: 10 }, "charged.active_calls"], [{ active_calls: 11, candidate_calls: 10 }, "charged.candidate_calls"], [{ active_calls: 12, candidate_calls: 12 }, "id"]] as const) { + const w = await workspace({ ...deployment(), budget }); const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const p = (await readEvaluationPlan(w.client, "rl-candidate"))!.plan; await originalCohort(w, p); + const extra = `${registered.id}-${hashJson("extra")}`; await originalDecision(w.client, extra, {}); + const reject = fieldRefusal(field === "id" ? "shadow_consent" : "shadow_budget", field === "id" ? extra : registered.id, field); + await assert.rejects(compareShadow(w.client, "rl-candidate"), reject); + await assert.rejects(observeShadow(w.client, "rl-candidate", input()), reject); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(p)), reject); + } +}); + +test("candidate absence and absent policy inputs cannot be disguised as successful receipts", async () => { + for (const [d, row, field] of [ + [deployment({}), originalObservation({ candidate_action: null, failure: "candidate_missing_action", disagreement: false }), "candidate_reward"], + [{ ...deployment(), inputs: Object.fromEntries(Object.keys(deployment().inputs).map((id) => [id, shadowInputDigest("unknown")])) }, originalObservation({ input_digest: shadowInputDigest("unknown") }), "active_action"], + ] as const) { + const w = await workspace(d); const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const receipt = `${registered.id}-${hashJson("event-0")}`; await originalDecision(w.client, receipt, row as unknown as JsonValue); + await assert.rejects(compareShadow(w.client, "rl-candidate"), fieldRefusal("shadow_drift", receipt, field)); + } +}); + +test("selected outcome digests ignore unselected rewards and preserve prototype-like action names", async () => { + const d = { ...deployment(), active: { event: "__proto__" }, candidate: { event: "constructor" } }; + const w = await workspace(d); await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const rewards = Object.fromEntries([["__proto__", 0], ["constructor", 1], ["unused", 0.5]]); + const row = await observeShadow(w.client, "rl-candidate", { id: "event-0", input: "event", rewards }); + assert.equal(row.outcome_digest, hashJson(Object.fromEntries([["__proto__", 0], ["constructor", 1]]))); + assert.equal(row.active_action, "__proto__"); assert.equal(row.candidate_action, "constructor"); + assert.deepEqual(await observeShadow(w.client, "rl-candidate", { id: "event-0", input: "event", rewards }), row); +}); + +test("reloaded manifest structure and edited creation fields name their exact receipt", async () => { + const w = await workspace(); const p = await registerEvaluationPlan(w.client, { ...w.p, shadow: "wrong" }); + const receipt = `${p.id}-shadow`; + await originalDecision(w.client, receipt, { ...w.d, budget: { active_calls: -1, candidate_calls: 10 } } as unknown as JsonValue); + await assert.rejects(compareShadow(w.client, "rl-candidate"), fieldRefusal("shadow_budget", receipt, "manifest")); + const other = await workspace(); const registered = await registerShadowDeployment(other.client, other.d, other.p); await collect(other); + await other.client.update(registered.id, { type: "Task" }); + await assert.rejects(compareShadow(other.client, "rl-candidate"), fieldRefusal("shadow_drift", registered.id, "type")); +}); + +test("native metadata governance refuses an uncommitted observation without inventing a winning receipt", async () => { + const w = await workspace(); const registration = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + await w.client.workflowPolicy("policy-put", "require-shadow-risk", { definition: { id: "require-shadow-risk", effect: "refuse", subject: { types: ["Decision"], operations: ["create"] }, rule: { kind: "require_fields", fields: ["risk"] } } }); + await w.client.workflowPolicy("policy-mode", "refuse"); + await assert.rejects(observeShadow(w.client, "rl-candidate", input()), /risk|required/); + await assert.rejects(w.client.get(`${registration.id}-${hashJson("event-0")}`)); +}); + +test("aggregate charges and comparison digests cannot override validated individual evidence", async () => { + for (const [patch, version, field] of [ + [{ charged: { active_calls: 9, candidate_calls: 10 } }, undefined, "charged.active_calls"], + [{ charged: { active_calls: 10, candidate_calls: 9 } }, undefined, "charged.candidate_calls"], + [{ disagreements: 0 }, undefined, "comparison"], + [{}, "wrong", "affected_version"], + ] as const) { + const w = await workspace(); const registered = await registerShadowDeployment(w.client, w.d, w.p); await collect(w); + const p = (await readEvaluationPlan(w.client, "rl-candidate"))!.plan; + await originalCohort(w, p, {}, patch, version); + await assert.rejects(enforceEvaluationPlan(w.client, "rl-candidate", score(p)), fieldRefusal(field.startsWith("charged") ? "shadow_budget" : "shadow_drift", `${registered.id}-comparison`, field)); + } +}); + +test("manifest identity checksum and post-registration cohort edits retain field-specific refusals", async () => { + const mismatched = await workspace({ ...deployment(), incumbent: "rl-other" }); + await assert.rejects(registerShadowDeployment(mismatched.client, mismatched.d, mismatched.p), fieldRefusal("shadow_consent", "rl-candidate-evaluation-plan-shadow", "incumbent")); + const wrongVersion = await workspace(); const p = await registerEvaluationPlan(wrongVersion.client, { ...wrongVersion.p, shadow: hashJson(wrongVersion.d as unknown as JsonValue) }); + const receipt = `${p.id}-shadow`; await originalDecision(wrongVersion.client, receipt, wrongVersion.d as unknown as JsonValue, "wrong"); + await assert.rejects(compareShadow(wrongVersion.client, "rl-candidate"), fieldRefusal("shadow_drift", receipt, "affected_version")); + const edited = await workspace(); const registered = await registerShadowDeployment(edited.client, edited.d, edited.p); + await edited.client.update(registered.id, { body: canonicalJson({ ...edited.d, inputs: { ...edited.d.inputs, extra: shadowInputDigest("event") } } as unknown as JsonValue) }); + await assert.rejects(compareShadow(edited.client, "rl-candidate"), fieldRefusal("shadow_drift", registered.id, "body")); +});