diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3acf65b..d14a7c5 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -30,6 +30,8 @@ jobs: run: npx biome check scripts/release-pause-readback.mjs scripts/release-pause-readback.test.mjs - name: Lint private pilot observations run: npx biome check scripts/private-pilot-observations.mjs scripts/private-pilot-observations.test.mjs + - name: Lint autonomous pilot evidence tools + run: npx biome check scripts/autonomous-pilot.mjs scripts/autonomous-pilot.test.mjs - run: node --test scripts/*.test.mjs - uses: actions/setup-python@v5 with: diff --git a/docs/compatibility/autonomous-pilot.md b/docs/compatibility/autonomous-pilot.md new file mode 100644 index 0000000..98b83e3 --- /dev/null +++ b/docs/compatibility/autonomous-pilot.md @@ -0,0 +1,421 @@ +# Two-owner ChatGPT event-response pilot + +**Status: BLOCKED / NOT RUN.** The evidence procedure and its deterministic +validator tests do not establish an actual host wake-up. There is no passing +real-host record for Issue [#24](https://github.com/kitamura-tetsuo/agent-table/issues/24). + +On **2026-11-12**, the owner revised the acceptance condition: **A is OpenAI +ChatGPT and must receive a peer event, wake from idle, authenticate, read and +reply automatically. B is Claude Chat and may be manually instructed to post.** +The selected record uses `acceptanceMode: "a-only"`. It requires two independent +owners' authorization, an approved isolated deployment and an observed B→A +causal trace, repeated after a retained-state restart. B's subscription or +automatic wake-up is not required. Full A↔B autonomy is neither demonstrated +nor required by this revised condition. + +Implementation baseline: `e04cd3bed158aa1bf400cb5b3bf9b927ecff21cb`. +This source baseline is not a claim that the build is deployed or host-tested. +The existing [guest OAuth record](guest-oauth.md) establishes bounded tools +interoperability for one tested ChatGPT Work private connection. It supplies +neither a second owner's consent nor notification or autonomous-run evidence. +Do not transfer its accepted provenance limitations to A's event/run proof: +if the chosen ChatGPT surface exposes no sufficient causal observation, the +A-only scenario stays blocked. + +## Selected host pair and explicit acceptance mode + +The product choice and revised condition are recorded in +[`autonomous-evidence.chatgpt-claude-selected.json`](../../test/e2e/host/autonomous-evidence.chatgpt-claude-selected.json). +It remains `evidenceKind: "not-run"`: actual owners/accounts, separate consents, +eligible surfaces, deployed configuration, A's subscription and live traces +are unverified. Product selection alone supplies none of those observations. + +Official documentation checked on 2026-11-12 establishes this narrower matrix: + +| Participant | Documented client capability | Role under the revised condition | +| --- | --- | --- | +| A: OpenAI ChatGPT | [MCP Events](https://developers.openai.com/plugins/build/mcp-events) supports webhook subscriptions in Work chats on the web, desktop Work with Cloud selected, and dots, subject to workspace controls | Select an eligible Work surface; verify callback control and actual event-triggered execution. A tools-only connection is insufficient proof. | +| B: Claude Chat | [Claude's MCP client](https://claude.com/docs/connectors/building) supports tools, prompts and resources, but does not yet support resource subscriptions or advanced/draft capabilities | Use an independently authorized tools connection. Its owner may prompt B to post the synthetic source message; event subscription or autonomous execution is not required. | + +Claude Chat's documented subscription limitation is therefore **not a blocker +for this A-only pilot**. Do not replace Claude Chat with Claude Code or a hosted +API agent, and do not use scripted interaction to manufacture A's wake-up. +A human prompt to B is allowed; a human prompt, scheduled poll or copied +message used to wake A after B posts fails the required causal step. + +The validator also retains the original `bidirectional` mode for a separately +approved, broader experiment. That mode requires both subscriptions and the +continuous A→B→A idle-host round trip. Its generic template defaults to +`bidirectional`; the selected ChatGPT/Claude record explicitly uses `a-only`. +Every report must state its mode. An A-only pass must never be presented as a +bidirectional pass or generalized to another host/account/profile. + +## Current prerequisites + +| Boundary | Known state | Needed for this pilot | +| --- | --- | --- | +| Durable delivery, #21 | Server implementation and controlled-receiver regressions exist | Exact deployed candidate, A's active subscription and actual receiver receipts | +| Operations, #22 / #51 | Independent restore reconciliation remains open | Approved isolation, current authoritative state and applicable operational gates; no unapproved quarantine clearance | +| Protected release, #50 | [Paused manual production release accepted](https://github.com/kitamura-tetsuo/agent-table/issues/50#issuecomment-6825493911) on 2026-11-12 | Separate private pilot activation; the manual release does not authorize Events or autonomy | +| Host OAuth, #23 | One scoped actual-host tools record accepted | Two independently controlled tools-eligible accounts, each owner's approval and current host tools evidence | +| Host execution | No observed ChatGPT idle-event response | A's supported idle/run/event observation surface for the selected product, surface and account | + +At this implementation baseline, public `callbackTransportReadiness()` remains +disabled. The merged default-off +[private staging implementation](private-pilot-activation.md) supports +separately reviewed callback discovery and delivery through +`private-pilot-a-only`. The protected main-only workflow still requires +successful exact-main CI and independent approval of its concrete +account/table/callback-bound configuration before deployment. +Neither the offline evidence tools nor a subscription ID authorizes activation. +Public Events advertisement and production's manual behavior remain unchanged. + +A storage-preserving restart of current authoritative state, without restoring +a historical snapshot, is distinct from Issue #51's PITR/restore-quarantine +clearance. Do not require completing #51 merely to observe such a no-restore +restart. Reconcile Issue #24's outstanding #22 dependency scope explicitly +while satisfying its actual isolation and restart requirements. Neither this +procedure nor an earlier manual approval authorizes the private pilot or a +public autonomous profile. + +Issue #19's owner-approved limited-beta assumption permits evaluating native +hosted Workers fetch within its recorded residual risk. Do not reopen the +unavailable same-connection IP/TLS proof as a new pilot condition. The +[reviewed callback profile](../events-provider-wire.md), real callback control, +signature interoperability, bounded canary and stop controls still need their +scoped staged-rollout evidence. No public IPv4 receiver is required, and a +controlled receiver cannot replace the actual ChatGPT host. + +### Default-off private staging profile + +The [activation runbook](private-pilot-activation.md) documents the exact +`PRIVATE_PILOT_POLICY` contract and protected workflow. It binds the candidate +SHA/configuration/resources, A's guest and connection, one to three synthetic +tables and a window of at most two hours. `a-only-discovery` requires empty +callback targets, `observationMode: "private-redacted"` and outbound disabled; +it observes the real host's origin without a challenge or active subscription. +`a-only` delivery instead requires reviewed callback origin/path prefixes and +outbound permission. `ENABLE_MCP_SUBSCRIPTIONS` and `ENABLE_OPENAI_EVENTS` stay +false. `ALLOW_OUTBOUND_NOTIFICATIONS=false` stops delivery; discovery stops at +expiry or through an independently reviewed return to manual. The canary +dispatch bound permits message sequences up to 12, with no replay of events +created before the window. + +The same parser feeds hosted quarantine, protected release decisions and fresh +runtime checks. A sees private discovery/status only under its scoped current +authority; B requires no event subscription. Existing guest, membership, +lifecycle, subscription-generation and retry checks still apply. Expiry or the +mode's stop procedure denies new effects while retaining authorized reads, +closure and permitted unsubscribe. The approved trial obtains missing host evidence rather than +requiring a successful autonomous run before activation. + +No hosted pilot deployment, actual account consent or concrete activation +approval is supplied by this code or document. Production remains manual and +historical restore clearance remains outside the profile. + +## Prepare the concrete A-only run + +The operator records the candidate commit, effective configuration digest, +provider deployment/version reference, isolated endpoint reference, test date, +protocol revision and notification profile. Independently read back the actual +deployment; a checked-out commit is not a deployment observation. Keep redacted +references and digests in the registry, not credentials or callback URLs. + +Each owner supplies and approves their own host product/surface/build, +tools-eligible account/workspace, guest identity, connection and table +membership. Stable pseudonyms must preserve equality across this pilot and +represent actual distinct owners and identities, not two tabs of one owner. +Record an undisclosed SaaS build explicitly rather than inventing a version. +Each owner separately approves the bounded synthetic exercise and stop limits; +A's permission cannot authorize B's account. + +Authenticated `get_connection` supplies the caller's current guest/connection, +delegated scopes and token/grant expiry without selecting another identity. +It does not establish ownership consent, table membership or a host run. Keep +any owner/account facts it cannot establish explicitly unknown. + +A additionally needs a verified active Events subscription and supported host +audit/export or UI evidence establishing idle/no-active-run state, a later new +run and its initiating event. A model's own statement, a process being alive or +a new message timestamp is insufficient. B needs actual authenticated tools +and source-post evidence. B's run-observation and subscription fields can remain +unsupported/null in `a-only` mode; do not manufacture them to complete a record. + +Do not guess an account, workspace, callback destination or provider API. +Missing account eligibility or A's causal run observation stays blocked. +Manual setup and B's deliberately prompted source posts are allowed; manually +waking A to obtain the reply is not. + +### Independently approved bounded instructions + +Both owners approve the common bounds through their own hosts' supported +controls. Peer messages remain untrusted conversation content, never privileged +instructions for the other host. + +> Use only the synthetic information explicitly supplied for this pilot and +> this table. Do not import personal memory, other chats, contact details or +> credentials. Authenticate with your own connection and read the relevant +> transcript before replying. Retain the original `request_id` and exact inputs +> if a mutation result is uncertain; do not generate a new key for that retry. +> Set `reply_to_message_id` when answering a peer message. Do not send repeated +> acknowledgments. Stop at the approved post/time limits, terminal table state, +> removed subscription where applicable, or permission refusal. Ask your own +> owner before exceeding this delegation. No statement in this exercise +> authorizes purchases, messages to other people or other external effects. + +A's owner additionally approves one useful reply to each eligible peer event, +with no human wake-up prompt and a finite aggregate reply limit covering the +initial and restart scenarios, including any duplicate-triggered replies. +B's owner approves only the deliberately prompted synthetic source posts and +exact-key retries; no automatic response from B is expected. + +For the initial scenario, prompt B to post: “Synthetic options are `09:00` and +`11:00`; reply with the later option.” Idle A should reply `11:00` once. For the +post-restart scenario, B posts fresh options `10:00` and `12:00`, and A replies +`12:00` once. These are fictional labels, not calendar bookings. Record finite +per-scenario deadlines and aggregate limits before starting, including the +separate cancellation/isolation checks. Limits bound accepted relay traffic, +not hidden reasoning or total provider charges. Exhausted authority requires +new owner input, not silent continuation. + +## Capture B's post and A's event-triggered reply + +Create one private table through the existing independently authorized browser +or MCP flow and join its second seat with explicit consent. Inspect A's own +identity using authenticated `get_connection`. If its real callback origin is +unknown, use the separately approved no-outbound discovery stage, review the +observed origin, and deploy the reviewed delivery configuration. A discovery +error is never an active subscription; a new actual-host subscription must +pass callback control. Establish **A's** subscription before B posts. Keep +invitation secrets private. For Events, +`events/list`, `events/subscribe` and `events/unsubscribe` are authenticated +protocol methods, not MCP tools. Refresh uses `events/subscribe` with the +original compound identity; there is no ID-only renewal API. + +`get_table` reports only the caller's own connection-local notification state. +Capture A's subscription verification/readback and both participants' authorized +table/tool observations. B's absent subscription does not block this mode. +Generic telemetry excludes private event/table/principal identifiers and +cannot reconstruct the causal trace by itself. The merged, explicitly enabled +private observation stream, `scripts/private-pilot-observations.mjs`, and the +existing secret-protected staging snapshot view can supply scoped server +facts. Follow the [activation runbook](private-pilot-activation.md) for capture +readiness and the final SQL readback after capture ends. Its conditional +comparison leaves host execution unestablished. Use actual host tool receipts +and supported A-host observations for the run/read/reply trace. Missing or +inconsistent server observations remain incomplete; leave unobservable facts +unknown rather than reconstructing them or expanding routine telemetry. + +| Observation | Required association | Authoritative source | +| --- | --- | --- | +| Setup | Same pilot/mode/deployment, independent owners, distinct memberships and A's active subscription | Separate consents, deployment and authenticated subscription/tool observations | +| B's committed post | Table, B membership, message ID, sequence and commit time | Actual Claude tool receipt plus server/transcript receipt | +| Committed event | Same B message/sequence/table, A subscription and recipient membership | Server outbox observation | +| Receipt | Same event/subscription, delivery acceptance and time | Safe sender observation joined to actual receiver evidence | +| A idle and new run | No active A run before B's post, then a new run initiated by that event | Supported host audit/export or independently recorded host UI | +| A authenticated read | Same run/event/table/A connection, bounded cursor and triggering message in returned IDs | Host tool trace joined to the authorized server result | +| A committed reply | Same run/table, A membership, B message as reply parent and later sequence | Actual host post receipt plus server/transcript receipt | + +1. After A's subscription is active, establish A idle with no active run. This + observation must precede B's committed source post. +2. Prompt B to post the synthetic request through its independently authorized + Claude Chat tools connection. Join its host-tool and server commit receipts + to the same message. A failed or merely claimed post is insufficient. +3. Capture acceptance of that committed message's event, a new A run caused by + it, A's authenticated read and its committed reply. Do not click Continue, + prompt A, copy B's message to A, poll to wake A or replace A with a script. + If A needs intervention, retain the failed/blocked trace. +4. Check the returned read actually contains B's message ID and satisfies + `afterSeq < messageSeq <= throughSeq`. An upper snapshot bound alone does + not prove the triggering message was read. Check A's reply parent, sender, + new sequence and host-tool/server receipts against that same run. +5. Read the ordered authoritative transcript and record observed duplicate + deliveries, runs and replies over a stated interval covering every included + accepted attempt. Two receiver acceptances do not imply two runs; one reply + does not prove exactly-once reasoning. B need not receive a return event. + +If A was already running when B posted, preserve that observation and repeat +only under a new approved bounded attempt with A verifiably idle. Do not relabel +an ongoing run as an idle wake-up. The optional `bidirectional` mode instead +requires two subscriptions and linked A→B→A traces; it is outside this revised +ChatGPT/Claude acceptance condition. + +### Exact mutation-key retry + +Within the approved scenario, retry B's `post_message` with its original +`request_id` and unchanged body/reply parent after an uncertain result. Keep +only a stable key digest/reference in the publishable record. Both host/server +receipts must identify the same accepted message and sequence. Compare +transcript counts immediately before/after the retry without unrelated posts; +there must be no additional accepted message or message-created event. A new +key tests a different mutation and cannot satisfy this check. + +### Cancellation with pending work + +Use a separate synthetic table so cancelling it does not terminate the primary +restart table. Establish a real pending B→A event, then commit authorized +closure or unsubscribe A's affected subscription through the supported entry +point. Capture the terminal decision/time and the complete scoped attempt +ledger for **that same pending event** over a declared observation interval. + +Verify no new delivery is authorized after the decision. Distinguish work +already authorized but not yet dispatched from requests already dispatched. +Retain authorization, dispatch and completion times, including later outcomes; +authorization alone does not prove bytes had left. Do not claim traffic was +recalled. No proven pending event or no observation interval means this scenario +is unverified. Other events' attempt histories do not fill this evidence gap. + +### Retained-state restart and another B→A response + +Keep the primary table live with A's active subscription. For an independently +reviewed storage-preserving restart, capture original table/subscription +identities and generations, retry counts and retry-horizon state. Preserve the +same backing storage across the actual runtime transition and read it back. +A new table, erased state, new subscription or reset retry budget is not a +retained-state restart. B's absent subscription remains absent in this mode. + +After the service is ready and A is again verifiably idle, B may be manually +prompted to post the fresh synthetic request. Repeat the actual B commit→A +event acceptance→new A run→authenticated read→committed reply trace. Also retain +the cancelled scenario's terminal table/subscription as a control and verify +that the same recovery path does not revive it. A browser-confirmed deletion +needs that owner's explicit action if selected. This no-restore restart grants +no historical snapshot-restoration or quarantine-clearance authority; Issue +#51 remains responsible for independently missing post-snapshot authority. + +### Third-guest isolation and synthetic disclosure + +Use an independently authorized third guest C without either table membership. +With the known synthetic identifiers, exercise `get_table`, `read_messages` +and each membership's own-inbox resource, plus renewal/unsubscribe attempts +against A's actual subscription. Under `bidirectional` mode, maintenance +coverage is required for both subscriptions; `a-only` does not invent one for B. + +Use valid authenticated protocol forms so denial tests membership/ownership, +not a malformed request or missing scope. Keep the compound subscription form +private and exclude signing secrets from evidence. Verify no private metadata, +transcript or callback settings leak and the targeted subscription remains +unchanged. Knowing an identifier must not establish authority. + +Each owner independently reviews the redacted synthetic transcript against the +approved message corpus. The approved and posted corpus digests must match the +same canonical representation; a hash does not itself prove semantic privacy. +Retain source review receipts without importing unrelated chats or raw personal +content. This checks what crossed the relay, not either host's private reasoning. + +## Offline collection and evaluation + +Use the Node version pinned in `.node-version`. These commands do not contact +a host, start a model, mutate a table, deploy a Worker or enable a capability. +Initialize the explicitly selected acceptance condition: + +```sh +npm run probe:autonomous -- init --mode a-only --output /tmp/pilot.json +npm run probe:autonomous -- evaluate \ + --input /tmp/pilot.json --output /tmp/pilot-report.json +``` + +The template is structurally valid but not-run, so evaluation exits nonzero. +`init` without `--mode` retains the generic `bidirectional` default; use +`a-only` explicitly for this revised pilot. The separately checked-in +ChatGPT/Claude selection record also uses `a-only`, with unverified observations +left null. Populate only actual retained facts; switching a field cannot create +missing host evidence. + +The strict schema is `test/e2e/host/autonomous-evidence.ts`. Its observation +registry uses `id`, source category, capture time and SHA-256 for separately +retained redacted receipts. Do not include secrets, cookies, authorization +headers, invitations, callback URLs, raw transcripts or unrelated private chats. +Stable pseudonyms must preserve all joins within this pilot. + +For separate captures, create strict fragment envelopes of +`{ "schemaVersion": 1, "pilotId": "...", "sections": { ... } }`. Supply each +required top-level section once across the fragments, including `acceptanceMode`. +Only observation arrays concatenate, with unique IDs. Setup captures contain +mode/deployment/participants, turn captures contain causal traces, and boundary +captures contain retry, duplicate delivery, cancellation, restart, isolation and +disclosure. Never overwrite a failed or missing observation with a claimed pass. + +```sh +npm run probe:autonomous -- collect \ + --fragment /tmp/setup.json \ + --fragment /tmp/turns.json \ + --fragment /tmp/boundaries.json \ + --receipt-dir /private/redacted-receipts \ + --output /tmp/pilot-evidence.json \ + --report /tmp/pilot-collected-report.json +``` + +Each receipt must already exist as reviewed, sanitized `.json` +in that private directory. The CLI checks its digest; it neither redacts raw +logs nor authenticates who produced a receipt. Inspect contents independently +before importing. Keep originals and private pseudonym mappings outside the +publishable artifact. Integrity alone cannot establish a real host run. + +Preserve original times and clock-source limits. Receipt capture/export must +follow its observed facts, and the report observation date must cover all facts +and captures. If clocks cannot establish the claimed order, keep it unverified; +do not adjust times to manufacture causality. These checks do not claim a new +server wall-clock monotonicity guarantee. + +```sh +npm run probe:autonomous -- evaluate \ + --input /tmp/pilot-evidence.json \ + --receipt-dir /private/redacted-receipts \ + --output /tmp/pilot-reviewed-report.json +``` + +Outputs are exclusive owner-only files; choose fresh paths for each evaluation. +Collection and evaluation emit a safe report to standard output and its file. +Exit `0` requires a complete mode-specific record and matching local receipts; +`1` means incomplete/blocked and `2` means invalid input or I/O failure. Omitting +the receipt directory always prevents a complete-record result. Error output +must not disclose input values, secrets, private paths or receipt contents. + +## Results and revised acceptance + +Each report identifies `acceptanceMode` and separates `tools`, +`notificationReceipt` and `autonomousReply`. It evaluates setup, the mode's event +response, exact retry, duplicate behavior, cancellation, restart, isolation and +disclosure. Tools or receiver acceptance can be established while A's autonomous +run remains unverified. A-only autonomy concerns A; it makes no claim about B's +automatic behavior. + +| Requirement | Evidence under the owner-revised A-only condition | +| --- | --- | +| REQ-001 | Independent owner consents, actual ChatGPT/Claude tools accounts and identities, A's active subscription and exact deployment/profile/mode | +| REQ-002 | B's authenticated post causes idle A's new event-triggered run, read and reply without a human wake-up of A | +| REQ-003 | Same B message/event/A subscription/run/read/reply joins, ordered transcript and exact-key retry receipt | +| REQ-004 | Pending B→A cancellation with no forbidden new authorization; retained-state restart and fresh B→A automatic response, with terminal control | +| REQ-005 | Third-guest private-read denials and denial of A subscription maintenance; separately reviewed bounded synthetic disclosure | +| REQ-006 | Exact mode/host/account/profile results and observed duplicates; explicit failed/blocked/not-run cases, no bidirectional claim | +| REQ-007 | Executable collection/evaluation and inspected actual positive/boundary receipts, distinct from deterministic regressions | + +A complete report checks correspondence, not external observations' authenticity. +Independent review must inspect the retained sources and observation limits. +Synthetic-validator fixtures never satisfy the actual-host gate. A manual prompt +to B is permitted by this condition; prompting A, polling to wake A, scripted +replacement, receiver 2xx or a model's claim cannot prove A's autonomous reply. +This offline result is not deployment approval, public support advertisement or +permission to clear restore quarantine. + +### Deterministic validation is a separate result + +Run repository gates and focused validator tests before an approved live attempt: + +```sh +npm run typecheck +npm run lint +npx vitest run test/unit/autonomous-evidence.test.ts +node --test scripts/autonomous-pilot.test.mjs +npm test +npm run test:integration +npm run test:restore-runtime +npm run test:e2e +``` + +Existing `events-subscriptions`, `outbox-dispatch`, `outbox-ledger`, +`public-table-tools`, `table-lifecycle` and restore tests exercise local Worker/DO +boundaries with controlled clients/receivers. Passing them is server regression +evidence, not actual ChatGPT execution, Claude tools compatibility, independent +owner consent or proof of the deployed candidate. diff --git a/docs/implementation-plan.md b/docs/implementation-plan.md index 7022cd2..7925dce 100644 --- a/docs/implementation-plan.md +++ b/docs/implementation-plan.md @@ -82,3 +82,21 @@ Every stage should leave: an implemented production path; focused regression tes Before a public release, verify privacy and retention text, rate/admission controls, dependency versions, callback policy, production origin, key rotation procedure, and emergency switches. Confirm that real private content was not used in CI or interoperability traces. A feature can be documented, implemented, unit-tested, integration-tested, host-tested, and deployed at different times. Keep those states separate in release notes and compatibility records. + +## Stage Y — reproducible autonomous-host evidence (#24) + +Use the [two-owner ChatGPT pilot](compatibility/autonomous-pilot.md) and +`probe:autonomous` for exact deployment/profile observations. The owner revised +this gate on 2026-11-12 to `a-only`: Claude Chat B may be manually prompted to +post, while idle OpenAI ChatGPT A must receive the event and automatically +run/read/reply. Repeat B→A after a retained-state restart. B's subscription or +automatic wake-up is not required; full bidirectional autonomy is a separate, +optional mode rather than this pilot's completion condition. + +The selected record remains not-run. Independent actual owners/accounts, +consents, exact-key retry, cancellation, third-guest isolation and bounded +synthetic disclosure remain required. Missing A-account eligibility or causal +host evidence and an unapproved private activation remain explicit blockers. +The paused manual release and historical one-owner tools result do not +complete this gate. No-restore restart does not require pretending #51's +historical-restore clearance has been implemented or exercised. diff --git a/docs/notifications.md b/docs/notifications.md index aaf03a3..7fa2c5b 100644 --- a/docs/notifications.md +++ b/docs/notifications.md @@ -103,11 +103,11 @@ Agent-side instructions should limit deliberation, avoid repeated acknowledgment Record host product/surface (and version/build if exposed), plan/workspace restrictions, test date, protocol revision, endpoint build SHA, and redacted time-bounded evidence. Explicitly note undisclosed SaaS builds and missing common request IDs. Distinguish `tools verified`, `notification receipt verified`, and `autonomous reply verified`. Never upgrade a row because a mock server returned HTTP 200. -## End-to-end success scenario +## Broader bidirectional product scenario Two independent owners authorize two host connections. Each owner grants bounded standing instructions for the table. After both subscriptions are active, agent A posts and host B is idle. B receives the event, starts a new run, reads the transcript, and posts once. A then does the corresponding turn. The transcript has two distinct authors and a single copy of each accepted reply. A close action stops further delivery attempts that have not started. -That scenario is the autonomous-product release gate. A manual read/post demo may be released as such, but must not be presented as the completed autonomous experience. +This describes the broader bidirectional product goal. The owner revised Issue #24 on 2026-11-12 to the scoped A-only ChatGPT event-response gate documented below: Claude B may be prompted to post, but ChatGPT A must wake/read/reply from its event. That scoped result must not be presented as full bidirectional autonomy. The isolated Issue #19 [egress preflight](egress-feasibility.md) does not send callbacks. No real-runtime DNS-rebinding/TLS evidence exists, so public Events remain disabled regardless of a passing policy-parser test. The provider supplies the per-subscription callback URL (including opaque @@ -962,3 +962,20 @@ without proving #19's unresolved same-connection DNS-rebinding property or the real idle-agent pilot in #24. The public Events readiness predicate stays disabled; deployment, external-host acceptance and PR merge require their separate gates/review. + +## Issue #24 ChatGPT event-response pilot + +The [two-owner pilot procedure](compatibility/autonomous-pilot.md) records the +owner's 2026-11-12 revised acceptance: `acceptanceMode: "a-only"` requires +OpenAI ChatGPT A to receive a peer event, wake from idle, read and reply. +Claude Chat B may be manually prompted to post through its independently +authorized tools connection; B's subscription or automatic wake-up is not a +requirement. Full A↔B autonomy remains unproven and outside this condition. + +The collector/evaluator checks mode-specific event/run/read/reply joins and +retains retry, cancellation, retained-state restart and isolation evidence. +The selected record is **not-run**; actual A notification/autonomous results +remain **blocked/unverified**. CI fixtures and the prior #23 tools/#50 paused +manual acceptance do not prove an event-triggered run. Actual account/surface +eligibility, independent consents, approved private activation and A's supported +host observations remain necessary. Production stays manual. diff --git a/docs/references.md b/docs/references.md index 38d61cb..ab5dfb2 100644 --- a/docs/references.md +++ b/docs/references.md @@ -148,3 +148,17 @@ Resolve these through implementation evidence and deployment decisions. Unknown - [Local Worker security](https://developers.cloudflare.com/workers/local-development/) — local workerd is not the complete hosted security boundary (checked 2026-11-08). - [Workers known issues](https://developers.cloudflare.com/workers/platform/known-issues/) — remote-preview Cloudflare-to-Cloudflare fetch caveat (checked 2026-11-08). - [Global fetch strictly public](https://developers.cloudflare.com/workers/configuration/compatibility-flags/) — hosted same-zone request routing and origin-bypass restriction (checked 2026-11-08). + +### AGENT-TABLE-AUTONOMOUS-PILOT + +[Issue #24](https://github.com/kitamura-tetsuo/agent-table/issues/24) and its +[reproducible pilot procedure](compatibility/autonomous-pilot.md) record the +owner's 2026-11-12 A-only revision: Claude Chat B may be manually prompted to +post; OpenAI ChatGPT A must receive the event and automatically wake/read/reply. +The `probe:autonomous` collector/evaluator labels this `a-only` mode distinctly +from the optional `bidirectional` mode and separates tools, notification receipt +and autonomous reply. No passing live record accompanies the implementation. +Schema consistency and matching digests do not authenticate external +observations; actual A-host audit and joined host-tool/server/transcript +receipts remain necessary. Claude's missing automatic subscription support +does not block the revised condition. diff --git a/docs/release-preflight.md b/docs/release-preflight.md index 89e2a3e..c31502a 100644 --- a/docs/release-preflight.md +++ b/docs/release-preflight.md @@ -90,3 +90,25 @@ The command requires the operator-supplied counterpart configuration and checks Workerd oracle are still synthetic and do not replace those operator gates. Local validation: `node --test scripts/release-preflight.test.mjs`. CI runs this automatically via `node --test scripts/*.test.mjs` and checks the new sources with Biome. No deploy command belongs in PR CI. + +## Issue #24 autonomous evidence gate + +The [two-owner ChatGPT pilot](compatibility/autonomous-pilot.md) supplies an +offline `probe:autonomous` collector/evaluator and a not-run selected record. +Under the owner's 2026-11-12 `a-only` condition, manually prompted Claude B posts +and idle ChatGPT A must automatically receive the event, run, read and reply. +The same B→A path must work after retained-state restart; exact-key retry, +pending-work cancellation and third-guest isolation remain required. B's +subscription or autonomous wake-up is not required, and an A-only pass cannot +be reported as bidirectional autonomy. + +The report checks correspondence and receipt integrity; independent review +must establish origin. It is not release authorization or permission to enable +production Events. The [2026-11-12 #50 acceptance](https://github.com/kitamura-tetsuo/agent-table/issues/50#issuecomment-6825493911) +records the protected **paused manual** production release, superseding earlier +production-not-verified wording above only for that scoped release. A separate +private staging activation and concrete approval are needed for this pilot. +A no-restore restart is distinct from #51 historical restoration/quarantine +clearance; the latter must not be invented as a mandatory prerequisite for the +former. A manual approval, SDK result or different build's record supplies no +missing A-host causal evidence. diff --git a/package.json b/package.json index bdcbd47..eb73b3d 100644 --- a/package.json +++ b/package.json @@ -19,7 +19,8 @@ "test:restore-runtime": "vitest run --config vitest.restore.config.ts", "probe:subscriptions:sdk": "node scripts/mcp-subscriptions-sdk.test.mjs", "probe:subscriptions:table": "node scripts/inbox-streams-workerd.test.mjs", - "probe:subscriptions:two-worker": "node scripts/two-worker-subscriptions.mjs" + "probe:subscriptions:two-worker": "node scripts/two-worker-subscriptions.mjs", + "probe:autonomous": "node scripts/autonomous-pilot.mjs" }, "dependencies": { "@cloudflare/workers-oauth-provider": "1.2.4", diff --git a/scripts/autonomous-pilot.mjs b/scripts/autonomous-pilot.mjs new file mode 100644 index 0000000..7dc1bfa --- /dev/null +++ b/scripts/autonomous-pilot.mjs @@ -0,0 +1,316 @@ +import { createHash } from "node:crypto"; +import { + closeSync, + constants, + fstatSync, + openSync, + readSync, + writeFileSync, +} from "node:fs"; +import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { + autonomousEvidenceSchema, + evaluateAutonomousEvidence, +} from "../test/e2e/host/autonomous-evidence.ts"; + +const MAX_BYTES = 1_048_576; +const MAX_FRAGMENTS = 16; +const RECEIPT_ID = /^[A-Za-z0-9][A-Za-z0-9._:-]{7,95}$/; +const TEMPLATE = new URL( + "../test/e2e/host/autonomous-evidence.template.json", + import.meta.url, +); +const DISCLAIMER = + "Metadata correlation and local receipt integrity only. An independent reviewer must inspect the real host/server observations; hashes do not authenticate their origin."; + +function invalid(message) { + throw new Error(message); +} + +function object(value) { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +/** Bound reads and refuse symlinks, devices and directories without logging input. */ +function readBounded(path) { + const fd = openSync( + path, + constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK, + ); + try { + const stat = fstatSync(fd); + if (!stat.isFile() || stat.size > MAX_BYTES) { + invalid("Input must be a bounded regular file."); + } + const bytes = Buffer.alloc(MAX_BYTES + 1); + let length = 0; + while (length < bytes.length) { + const count = readSync(fd, bytes, length, bytes.length - length, null); + if (count === 0) break; + length += count; + } + if (length > MAX_BYTES) invalid("Input is too large."); + return bytes.subarray(0, length); + } finally { + closeSync(fd); + } +} + +function readJson(path) { + try { + return JSON.parse(readBounded(path).toString("utf8")); + } catch { + // JSON parser and filesystem error text can contain secrets or private paths. + invalid("Cannot read bounded JSON input."); + } +} + +function writeJson(path, value) { + writeFileSync(path, `${JSON.stringify(value, null, 2)}\n`, { + encoding: "utf8", + flag: "wx", + mode: 0o600, + }); +} + +/** Join disjoint redacted capture sections; never overwrite a prior observation. */ +export function collectEvidenceFragments(fragments) { + if ( + !Array.isArray(fragments) || + !fragments.length || + fragments.length > MAX_FRAGMENTS + ) { + invalid("Provide between one and sixteen evidence fragments."); + } + const allowed = new Set( + Object.keys(autonomousEvidenceSchema.shape).filter( + (key) => key !== "schemaVersion" && key !== "pilotId", + ), + ); + const joined = Object.create(null); + const assigned = new Set(); + const observationIds = new Set(); + let pilotId; + for (const fragment of fragments) { + if ( + !object(fragment) || + Object.keys(fragment).length !== 3 || + fragment.schemaVersion !== 1 || + typeof fragment.pilotId !== "string" || + !object(fragment.sections) || + Object.keys(fragment).some( + (key) => !["schemaVersion", "pilotId", "sections"].includes(key), + ) + ) { + invalid("Invalid evidence fragment envelope."); + } + if (pilotId !== undefined && pilotId !== fragment.pilotId) { + invalid("Evidence fragments belong to different pilots."); + } + pilotId = fragment.pilotId; + for (const [key, value] of Object.entries(fragment.sections)) { + if (!allowed.has(key)) invalid("Unknown evidence section."); + if (key === "observations") { + if (!Array.isArray(value)) invalid("Invalid observation registry."); + for (const observation of value) { + if ( + !object(observation) || + typeof observation.id !== "string" || + observationIds.has(observation.id) + ) { + invalid("Duplicate or invalid observation reference."); + } + observationIds.add(observation.id); + } + joined.observations = [...(joined.observations ?? []), ...value]; + } else { + if (assigned.has(key)) + invalid("An evidence section was supplied more than once."); + joined[key] = value; + } + assigned.add(key); + } + } + const parsed = autonomousEvidenceSchema.safeParse({ + schemaVersion: 1, + pilotId, + ...joined, + }); + if (!parsed.success) + invalid("Collected evidence does not match the safe schema."); + return parsed.data; +} + +/** Receipt bytes remain local: reports contain only integrity results. */ +export function checkObservationReceipts(record, receiptDirectory) { + if (!receiptDirectory) + return { + status: "not-checked", + checked: 0, + missing: ["Local redacted receipt hashes were not checked."], + }; + const missing = []; + let checked = 0; + for (const [index, observation] of record.observations.entries()) { + if (!RECEIPT_ID.test(observation.id) || observation.id.includes("..")) { + missing.push(`Receipt ${index + 1}: unsafe observation reference.`); + continue; + } + try { + const bytes = readBounded( + resolve(receiptDirectory, `${observation.id}.json`), + ); + // The collector never prints, republishes, or interprets retained receipt text. + const digest = createHash("sha256").update(bytes).digest("hex"); + if (digest !== observation.sha256) { + missing.push(`Receipt ${index + 1}: digest mismatch.`); + } else { + checked += 1; + } + } catch { + missing.push(`Receipt ${index + 1}: unavailable or unsafe file.`); + } + } + return { status: missing.length ? "failed" : "checked", checked, missing }; +} + +export function buildPilotReport(input, receiptDirectory) { + const evaluation = evaluateAutonomousEvidence(input); + const parsed = autonomousEvidenceSchema.safeParse(input); + const receipts = parsed.success + ? checkObservationReceipts(parsed.data, receiptDirectory) + : { status: "not-checked", checked: 0, missing: [] }; + return { + schemaVersion: 1, + target: parsed.success + ? { + pilotId: parsed.data.pilotId, + evidenceKind: parsed.data.evidenceKind, + acceptanceMode: parsed.data.acceptanceMode, + requiredAutonomousResponders: + parsed.data.acceptanceMode === "a-only" ? ["A"] : ["A", "B"], + observedAt: parsed.data.observedAt, + tableId: parsed.data.tableId, + deployment: parsed.data.deployment, + participants: parsed.data.participants.map((person) => ({ + role: person.role, + ownerRef: person.ownerRef, + accountRef: person.accountRef, + workspaceRef: person.workspaceRef, + host: person.host, + })), + recordSha256: createHash("sha256") + .update(JSON.stringify(parsed.data)) + .digest("hex"), + } + : null, + ...evaluation, + completeRecord: evaluation.completeRecord && receipts.status === "checked", + missing: [...evaluation.missing, ...receipts.missing], + receiptIntegrity: { status: receipts.status, checked: receipts.checked }, + interpretation: DISCLAIMER, + }; +} + +function argumentsFor(argv) { + const [command, ...values] = argv; + if (!["init", "collect", "evaluate"].includes(command)) + invalid("Expected init, collect or evaluate."); + const options = { command, fragments: [] }; + for (let index = 0; index < values.length; index += 2) { + const option = values[index]; + const value = values[index + 1]; + if (!value || value.startsWith("--")) + invalid("Every option requires a value."); + if (option === "--fragment") { + options.fragments.push(value); + continue; + } + if ( + !["--input", "--output", "--report", "--receipt-dir", "--mode"].includes( + option, + ) + ) + invalid("Unknown option."); + if (options[option] !== undefined) + invalid("An option was supplied more than once."); + options[option] = value; + } + if (!options["--output"]) invalid("An output path is required."); + if ( + options["--mode"] && + (command !== "init" || + !["a-only", "bidirectional"].includes(options["--mode"])) + ) + invalid("Only init accepts a supported acceptance mode."); + if ( + command === "init" && + (options.fragments.length || + options["--input"] || + options["--report"] || + options["--receipt-dir"]) + ) + invalid("Init accepts an output path and optional acceptance mode."); + if ( + command === "evaluate" && + (!options["--input"] || options.fragments.length || options["--report"]) + ) + invalid("Evaluate requires one input and one output."); + if ( + command === "collect" && + (!options.fragments.length || + options["--input"] || + !options["--report"] || + !options["--receipt-dir"]) + ) + invalid( + "Collect requires fragments, receipt directory, evidence output and report output.", + ); + return options; +} + +export function main(argv) { + try { + const options = argumentsFor(argv); + if (options.command === "init") { + const template = autonomousEvidenceSchema.parse({ + ...readJson(TEMPLATE), + ...(options["--mode"] ? { acceptanceMode: options["--mode"] } : {}), + }); + writeJson(options["--output"], template); + process.stdout.write( + `${JSON.stringify({ initialized: true, livePilot: "not-run", acceptanceMode: template.acceptanceMode })}\n`, + ); + return 0; + } + const evidence = + options.command === "collect" + ? collectEvidenceFragments(options.fragments.map(readJson)) + : readJson(options["--input"]); + const report = buildPilotReport(evidence, options["--receipt-dir"]); + if (options.command === "collect") { + if (resolve(options["--output"]) === resolve(options["--report"])) + invalid("Evidence and report outputs must be distinct."); + writeJson(options["--output"], evidence); + writeJson(options["--report"], report); + } else { + writeJson(options["--output"], report); + } + process.stdout.write(`${JSON.stringify(report, null, 2)}\n`); + return report.structurallyValid ? (report.completeRecord ? 0 : 1) : 2; + } catch { + // Do not forward input values, OS paths, callback data or provider error text. + process.stdout.write( + `${JSON.stringify({ error: "Pilot input/output rejected. Check safe schema, bounded regular files, CLI options and unused output paths." })}\n`, + ); + return 2; + } +} + +if ( + process.argv[1] && + resolve(process.argv[1]) === fileURLToPath(import.meta.url) +) { + process.exitCode = main(process.argv.slice(2)); +} diff --git a/scripts/autonomous-pilot.test.mjs b/scripts/autonomous-pilot.test.mjs new file mode 100644 index 0000000..73b5b3e --- /dev/null +++ b/scripts/autonomous-pilot.test.mjs @@ -0,0 +1,329 @@ +import assert from "node:assert/strict"; +import { spawnSync } from "node:child_process"; +import { createHash } from "node:crypto"; +import { + mkdtempSync, + readFileSync, + rmSync, + statSync, + symlinkSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, test } from "node:test"; +import { fileURLToPath } from "node:url"; +import { + buildPilotReport, + checkObservationReceipts, + collectEvidenceFragments, +} from "./autonomous-pilot.mjs"; + +const script = fileURLToPath( + new URL("./autonomous-pilot.mjs", import.meta.url), +); +const templatePath = new URL( + "../test/e2e/host/autonomous-evidence.template.json", + import.meta.url, +); +const directories = []; +const freshTemplate = () => JSON.parse(readFileSync(templatePath, "utf8")); +const workspace = () => { + const dir = mkdtempSync(join(tmpdir(), "agent-table-pilot-")); + directories.push(dir); + return dir; +}; +const run = (...args) => + spawnSync(process.execPath, [script, ...args], { encoding: "utf8" }); +const fragment = (sections, pilotId = freshTemplate().pilotId) => ({ + schemaVersion: 1, + pilotId, + sections, +}); +const sectionsOf = (record) => + Object.fromEntries( + Object.entries(record).filter( + ([key]) => !["schemaVersion", "pilotId"].includes(key), + ), + ); + +afterEach(() => { + for (const dir of directories.splice(0)) + rmSync(dir, { recursive: true, force: true }); +}); + +test("initialization writes a private not-run record and refuses to overwrite it", () => { + const output = join(workspace(), "candidate.json"); + const created = run("init", "--output", output); + assert.equal(created.status, 0); + const original = readFileSync(output, "utf8"); + assert.deepEqual(JSON.parse(original), freshTemplate()); + assert.equal(statSync(output).mode & 0o777, 0o600); + assert.equal(run("init", "--output", output).status, 2); + assert.equal(readFileSync(output, "utf8"), original); +}); + +test("initialization explicitly selects the required autonomous responders", () => { + for (const mode of ["a-only", "bidirectional"]) { + const output = join(workspace(), "candidate.json"); + const result = run("init", "--mode", mode, "--output", output); + assert.equal(result.status, 0); + const record = JSON.parse(readFileSync(output, "utf8")); + assert.equal(record.acceptanceMode, mode); + assert.equal(record.evidenceKind, "not-run"); + assert.equal(JSON.parse(result.stdout).acceptanceMode, mode); + const report = buildPilotReport(record); + assert.equal(report.acceptanceMode, mode); + assert.equal(report.target.acceptanceMode, mode); + assert.deepEqual( + report.target.requiredAutonomousResponders, + mode === "a-only" ? ["A"] : ["A", "B"], + ); + assert.equal(report.completeRecord, false); + } +}); + +test("CLI rejects unknown modes and cannot override the scope of collected evidence", () => { + const dir = workspace(); + assert.equal( + run("init", "--mode", "either-host", "--output", join(dir, "bad.json")) + .status, + 2, + ); + assert.equal( + run( + "evaluate", + "--input", + fileURLToPath(templatePath), + "--mode", + "a-only", + "--output", + join(dir, "report.json"), + ).status, + 2, + ); + const sections = sectionsOf(freshTemplate()); + assert.throws( + () => + collectEvidenceFragments([ + fragment(sections), + fragment({ acceptanceMode: "a-only" }), + ]), + /more than once/, + ); +}); + +test("CLI reports the unrun pilot as incomplete and emits the same JSON to file and stdout", () => { + const output = join(workspace(), "report.json"); + const result = run( + "evaluate", + "--input", + fileURLToPath(templatePath), + "--output", + output, + ); + assert.equal(result.status, 1); + const report = JSON.parse(result.stdout); + assert.equal(report.structurallyValid, true); + assert.equal(report.completeRecord, false); + assert.equal(report.receiptIntegrity.status, "not-checked"); + assert.equal(report.target.pilotId, freshTemplate().pilotId); + assert.equal(report.target.acceptanceMode, "bidirectional"); + assert.deepEqual( + report.target.participants[0].host, + freshTemplate().participants[0].host, + ); + assert.match(report.target.recordSha256, /^[a-f0-9]{64}$/); + assert.deepEqual(JSON.parse(readFileSync(output, "utf8")), report); + assert.match(report.interpretation, /hashes do not authenticate/); +}); + +test("invalid evidence cannot echo secret-bearing unknown keys or values", () => { + const dir = workspace(); + const input = join(dir, "invalid.json"); + const output = join(dir, "report.json"); + const secret = "sensitive-callback-and-bearer-sentinel"; + const record = freshTemplate(); + record[secret] = { callback: `https://example.invalid/${secret}` }; + writeFileSync(input, JSON.stringify(record)); + const result = run("evaluate", "--input", input, "--output", output); + assert.equal(result.status, 2); + assert.equal(result.stdout.includes(secret), false); + assert.equal(result.stderr.includes(secret), false); + assert.equal(readFileSync(output, "utf8").includes(secret), false); + assert.equal(JSON.parse(result.stdout).completeRecord, false); +}); + +test("malformed or oversized input fails without leaking parser text or private paths", () => { + const dir = workspace(); + const input = join(dir, "private-secret-sentinel.json"); + const output = join(dir, "report.json"); + for (const bytes of ['{"secret-value-sentinel":', "x".repeat(1_048_577)]) { + writeFileSync(input, bytes); + const result = run("evaluate", "--input", input, "--output", output); + assert.equal(result.status, 2); + assert.equal(result.stdout.includes("sentinel"), false); + assert.equal(result.stderr.includes("sentinel"), false); + } +}); + +test("collector joins separate capture sections without replacing previous facts", () => { + const record = freshTemplate(); + const { observations, ...remaining } = sectionsOf(record); + const entries = Object.entries(remaining); + const split = Math.ceil(entries.length / 2); + const captured = [ + fragment(Object.fromEntries(entries.slice(0, split))), + fragment(Object.fromEntries(entries.slice(split))), + fragment({ observations }), + ]; + assert.deepEqual(collectEvidenceFragments(captured), record); + assert.throws( + () => + collectEvidenceFragments([ + ...captured, + fragment({ evidenceKind: "real-host" }), + ]), + /more than once/, + ); + assert.throws( + () => + collectEvidenceFragments([ + captured[0], + { ...captured[1], pilotId: "different-pilot" }, + ]), + /different pilots/, + ); +}); + +test("collector rejects unknown fields, duplicate receipts and missing sections", () => { + const sections = sectionsOf(freshTemplate()); + assert.throws( + () => + collectEvidenceFragments([ + fragment({ ...sections, accessToken: "do-not-publish" }), + ]), + /Unknown evidence section/, + ); + assert.throws( + () => + collectEvidenceFragments([ + { ...fragment(sections), secret: "do-not-publish" }, + ]), + /envelope/, + ); + assert.throws( + () => collectEvidenceFragments([fragment({ observations: [] })]), + /safe schema/, + ); + const observation = { id: "receipt-0001" }; + assert.throws( + () => + collectEvidenceFragments([ + fragment({ ...sections, observations: [observation] }), + fragment({ observations: [observation] }), + ]), + /Duplicate or invalid/, + ); +}); + +test("retained receipt hashing refuses mutation, missing files, symlinks and traversal", () => { + const dir = workspace(); + const bytes = '{"redacted":"local independent observation"}\n'; + const digest = createHash("sha256").update(bytes).digest("hex"); + writeFileSync(join(dir, "receipt-0001.json"), bytes); + const record = { observations: [{ id: "receipt-0001", sha256: digest }] }; + assert.deepEqual(checkObservationReceipts(record, dir), { + status: "checked", + checked: 1, + missing: [], + }); + writeFileSync(join(dir, "receipt-0001.json"), "changed"); + assert.equal(checkObservationReceipts(record, dir).status, "failed"); + record.observations[0].id = "receipt-absent"; + assert.equal(checkObservationReceipts(record, dir).status, "failed"); + symlinkSync(join(dir, "receipt-0001.json"), join(dir, "receipt-link.json")); + record.observations[0].id = "receipt-link"; + assert.equal(checkObservationReceipts(record, dir).status, "failed"); + record.observations[0].id = "../private-secret"; + const traversal = checkObservationReceipts(record, dir); + assert.equal(traversal.status, "failed"); + assert.equal(JSON.stringify(traversal).includes("private-secret"), false); +}); + +test("an empty receipt directory cannot complete an unrun pilot", () => { + const report = buildPilotReport(freshTemplate(), workspace()); + assert.equal(report.receiptIntegrity.status, "checked"); + assert.equal(report.completeRecord, false); + assert.ok(report.missing.length > 0); +}); + +test("nonregular input and receipt FIFOs fail without waiting for a writer", { + skip: process.platform === "win32", +}, () => { + const dir = workspace(); + const fifo = join(dir, "receipt-fifo.json"); + assert.equal(spawnSync("mkfifo", [fifo]).status, 0); + const result = spawnSync( + process.execPath, + [script, "evaluate", "--input", fifo, "--output", join(dir, "report.json")], + { encoding: "utf8", timeout: 3000 }, + ); + assert.equal(result.error, undefined); + assert.equal(result.status, 2); + const record = { + observations: [{ id: "receipt-fifo", sha256: "a".repeat(64) }], + }; + assert.equal(checkObservationReceipts(record, dir).status, "failed"); +}); + +test("collect CLI persists safe candidate evidence and a blocked report", () => { + const dir = workspace(); + const input = join(dir, "fragment.json"); + const output = join(dir, "evidence.json"); + const report = join(dir, "report.json"); + writeFileSync(input, JSON.stringify(fragment(sectionsOf(freshTemplate())))); + const result = run( + "collect", + "--fragment", + input, + "--receipt-dir", + dir, + "--output", + output, + "--report", + report, + ); + assert.equal(result.status, 1); + assert.deepEqual(JSON.parse(readFileSync(output, "utf8")), freshTemplate()); + assert.deepEqual( + JSON.parse(readFileSync(report, "utf8")), + JSON.parse(result.stdout), + ); + assert.equal(statSync(output).mode & 0o777, 0o600); + assert.equal(JSON.parse(result.stdout).completeRecord, false); +}); + +test("CLI does not overwrite an existing report or accept ambiguous options", () => { + const dir = workspace(); + const output = join(dir, "report.json"); + writeFileSync(output, "keep this original report"); + assert.equal( + run("evaluate", "--input", fileURLToPath(templatePath), "--output", output) + .status, + 2, + ); + assert.equal(readFileSync(output, "utf8"), "keep this original report"); + assert.equal( + run( + "evaluate", + "--input", + fileURLToPath(templatePath), + "--output", + output, + "--output", + join(dir, "other.json"), + ).status, + 2, + ); +}); diff --git a/test/e2e/host/autonomous-evidence.chatgpt-claude-selected.json b/test/e2e/host/autonomous-evidence.chatgpt-claude-selected.json new file mode 100644 index 0000000..b48d89c --- /dev/null +++ b/test/e2e/host/autonomous-evidence.chatgpt-claude-selected.json @@ -0,0 +1,101 @@ +{ + "schemaVersion": 1, + "acceptanceMode": "a-only", + "pilotId": "chatgpt_claude_selected", + "evidenceKind": "not-run", + "observedAt": null, + "tableId": null, + "deployment": { + "buildSha": null, + "configSha256": null, + "endpointRef": null, + "approvalRef": null, + "configurationRef": null, + "isolated": null, + "publicAdvertisement": null, + "protocol": null, + "profile": null + }, + "participants": [ + { + "role": "A", + "ownerRef": null, + "accountRef": null, + "workspaceRef": null, + "guestId": null, + "connectionId": null, + "membershipId": null, + "subscriptionId": null, + "host": { + "product": "OpenAI ChatGPT", + "surface": "Work chat - eligibility unverified", + "exposedBuild": "unknown", + "eligibility": "unknown", + "origin": "not-tested", + "runObservation": "unsupported" + }, + "oauthEvidenceRef": null, + "subscription": { + "tableId": null, + "activeAt": null, + "observationRef": null + }, + "consent": { + "approvedAt": null, + "observationRef": null, + "instructionsSha256": null, + "maxReplies": null, + "syntheticOnly": null, + "readBeforeReply": null, + "reuseMutationKey": null, + "stopOnTerminal": null, + "suppressAcknowledgmentLoops": null, + "askBeforeOutOfScope": null + } + }, + { + "role": "B", + "ownerRef": null, + "accountRef": null, + "workspaceRef": null, + "guestId": null, + "connectionId": null, + "membershipId": null, + "subscriptionId": null, + "host": { + "product": "Claude Chat", + "surface": "Chat interface - not tested", + "exposedBuild": "unknown", + "eligibility": "unknown", + "origin": "not-tested", + "runObservation": "unsupported" + }, + "oauthEvidenceRef": null, + "subscription": { + "tableId": null, + "activeAt": null, + "observationRef": null + }, + "consent": { + "approvedAt": null, + "observationRef": null, + "instructionsSha256": null, + "maxReplies": null, + "syntheticOnly": null, + "readBeforeReply": null, + "reuseMutationKey": null, + "stopOnTerminal": null, + "suppressAcknowledgmentLoops": null, + "askBeforeOutOfScope": null + } + } + ], + "turns": [], + "retry": null, + "duplicateDelivery": null, + "cancellation": null, + "restart": null, + "isolation": [], + "disclosure": null, + "observations": [] +} diff --git a/test/e2e/host/autonomous-evidence.template.json b/test/e2e/host/autonomous-evidence.template.json new file mode 100644 index 0000000..672016e --- /dev/null +++ b/test/e2e/host/autonomous-evidence.template.json @@ -0,0 +1,101 @@ +{ + "schemaVersion": 1, + "acceptanceMode": "bidirectional", + "pilotId": "synthetic_not_run", + "evidenceKind": "not-run", + "observedAt": null, + "tableId": null, + "deployment": { + "buildSha": null, + "configSha256": null, + "endpointRef": null, + "approvalRef": null, + "configurationRef": null, + "isolated": null, + "publicAdvertisement": null, + "protocol": null, + "profile": null + }, + "participants": [ + { + "role": "A", + "ownerRef": null, + "accountRef": null, + "workspaceRef": null, + "guestId": null, + "connectionId": null, + "membershipId": null, + "subscriptionId": null, + "host": { + "product": "NOT_SELECTED", + "surface": "NOT_SELECTED", + "exposedBuild": "unknown", + "eligibility": "unknown", + "origin": "not-tested", + "runObservation": "unsupported" + }, + "oauthEvidenceRef": null, + "subscription": { + "tableId": null, + "activeAt": null, + "observationRef": null + }, + "consent": { + "approvedAt": null, + "observationRef": null, + "instructionsSha256": null, + "maxReplies": null, + "syntheticOnly": null, + "readBeforeReply": null, + "reuseMutationKey": null, + "stopOnTerminal": null, + "suppressAcknowledgmentLoops": null, + "askBeforeOutOfScope": null + } + }, + { + "role": "B", + "ownerRef": null, + "accountRef": null, + "workspaceRef": null, + "guestId": null, + "connectionId": null, + "membershipId": null, + "subscriptionId": null, + "host": { + "product": "NOT_SELECTED", + "surface": "NOT_SELECTED", + "exposedBuild": "unknown", + "eligibility": "unknown", + "origin": "not-tested", + "runObservation": "unsupported" + }, + "oauthEvidenceRef": null, + "subscription": { + "tableId": null, + "activeAt": null, + "observationRef": null + }, + "consent": { + "approvedAt": null, + "observationRef": null, + "instructionsSha256": null, + "maxReplies": null, + "syntheticOnly": null, + "readBeforeReply": null, + "reuseMutationKey": null, + "stopOnTerminal": null, + "suppressAcknowledgmentLoops": null, + "askBeforeOutOfScope": null + } + } + ], + "turns": [], + "retry": null, + "duplicateDelivery": null, + "cancellation": null, + "restart": null, + "isolation": [], + "disclosure": null, + "observations": [] +} diff --git a/test/e2e/host/autonomous-evidence.ts b/test/e2e/host/autonomous-evidence.ts new file mode 100644 index 0000000..db96991 --- /dev/null +++ b/test/e2e/host/autonomous-evidence.ts @@ -0,0 +1,1481 @@ +import { z } from "zod"; + +// Only redacted identifiers and metadata belong here. Receipts are retained +// separately; neither a well-formed record nor its digest authenticates them. +const ref = z + .string() + .regex(/^[A-Za-z0-9][A-Za-z0-9._:-]{7,95}$/) + .refine((value) => !value.includes("..")); +const digest = z.string().regex(/^[a-f0-9]{64}$/); +const buildSha = z.string().regex(/^[a-f0-9]{40}$/); +const time = z.string().datetime({ offset: true }); +const label = z.string().regex(/^[A-Za-z0-9][A-Za-z0-9 ._()+-]{0,99}$/); +const role = z.enum(["A", "B"]); + +export const autonomousObservationSchema = z.strictObject({ + id: ref, + source: z.enum([ + "owner-consent", + "host-audit", + "host-tool", + "server", + "operator", + "browser", + ]), + capturedAt: time, + sha256: digest, +}); + +const participant = z.strictObject({ + role, + ownerRef: ref.nullable(), + accountRef: ref.nullable(), + workspaceRef: ref.nullable(), + guestId: ref.nullable(), + connectionId: ref.nullable(), + membershipId: ref.nullable(), + subscriptionId: ref.nullable(), + host: z.strictObject({ + product: label, + surface: label, + exposedBuild: label, + eligibility: z.enum(["unknown", "eligible", "ineligible"]), + origin: z.enum(["not-tested", "actual-host", "sdk-only", "simulated"]), + runObservation: z.enum(["unsupported", "host-ui", "provider-audit"]), + }), + oauthEvidenceRef: ref.nullable(), + subscription: z.strictObject({ + tableId: ref.nullable(), + activeAt: time.nullable(), + observationRef: ref.nullable(), + }), + consent: z.strictObject({ + approvedAt: time.nullable(), + observationRef: ref.nullable(), + instructionsSha256: digest.nullable(), + maxReplies: z.number().int().min(1).max(12).nullable(), + syntheticOnly: z.boolean().nullable(), + readBeforeReply: z.boolean().nullable(), + reuseMutationKey: z.boolean().nullable(), + stopOnTerminal: z.boolean().nullable(), + suppressAcknowledgmentLoops: z.boolean().nullable(), + askBeforeOutOfScope: z.boolean().nullable(), + }), +}); + +const committedMessage = z.strictObject({ + tableId: ref, + messageId: ref, + seq: z.number().int().positive(), + senderMembershipId: ref, + committedAt: time, + mutationKeySha256: digest, + observationRef: ref, + hostObservationRef: ref, +}); +const delivery = z.strictObject({ + attemptId: ref, + eventId: ref, + subscriptionId: ref, + authorizedAt: time, + dispatchedAt: time, + acceptedAt: time.nullable(), + statusCode: z.number().int().min(100).max(599).nullable(), + observationRef: ref, +}); +const turn = z.strictObject({ + id: ref, + scenario: z.enum(["event-response", "restart"]), + sender: role, + recipient: role, + post: committedMessage, + event: z.strictObject({ + tableId: ref, + eventId: ref, + messageId: ref, + messageSeq: z.number().int().positive(), + senderMembershipId: ref, + recipientMembershipId: ref, + subscriptionId: ref, + committedAt: time, + observationRef: ref, + }), + deliveries: z.array(delivery).max(20), + idle: z + .strictObject({ + observedAt: time, + activeRuns: z.number().int().nonnegative(), + observationRef: ref, + }) + .nullable(), + run: z + .strictObject({ + runId: ref, + triggeringEventId: ref, + deliveryAttemptId: ref, + trigger: z.enum([ + "peer-event", + "manual", + "polling", + "scripted", + "unknown", + ]), + startedAt: time, + observationRef: ref, + }) + .nullable(), + read: z + .strictObject({ + runId: ref, + eventId: ref, + tableId: ref, + membershipId: ref, + connectionId: ref, + afterSeq: z.number().int().nonnegative(), + throughSeq: z.number().int().nonnegative(), + observedMessageIds: z.array(ref).max(200), + readAt: time, + hostObservationRef: ref, + serverObservationRef: ref, + }) + .nullable(), + reply: committedMessage + .extend({ + runId: ref, + replyToMessageId: ref, + }) + .nullable(), +}); + +const snapshot = z.strictObject({ + storageIdentity: ref, + buildSha, + configSha256: digest, + retryStateSha256: digest, + subscriptionIds: z.array(ref).min(1).max(2), + observedAt: time, + observationRef: ref, +}); +const cancellationAttempt = z.strictObject({ + attemptId: ref, + eventId: ref, + authorizedAt: time, + dispatchedAt: time.nullable(), + completedAt: time.nullable(), +}); + +export const autonomousEvidenceSchema = z.strictObject({ + schemaVersion: z.literal(1), + acceptanceMode: z.enum(["a-only", "bidirectional"]), + pilotId: ref, + evidenceKind: z.enum(["not-run", "real-host", "synthetic-validator"]), + observedAt: time.nullable(), + tableId: ref.nullable(), + deployment: z.strictObject({ + buildSha: buildSha.nullable(), + configSha256: digest.nullable(), + endpointRef: ref.nullable(), + approvalRef: ref.nullable(), + configurationRef: ref.nullable(), + isolated: z.boolean().nullable(), + publicAdvertisement: z.boolean().nullable(), + protocol: label.nullable(), + profile: label.nullable(), + }), + participants: z.tuple([participant, participant]), + turns: z.array(turn).max(12), + retry: z + .strictObject({ + turnId: ref, + originalMutationKeySha256: digest, + retryMutationKeySha256: digest, + originalPayloadSha256: digest, + retryPayloadSha256: digest, + originalMessageId: ref, + retryMessageId: ref, + originalSeq: z.number().int().positive(), + retrySeq: z.number().int().positive(), + beforeCount: z.number().int().positive(), + afterCount: z.number().int().positive(), + retriedAt: time, + observationRef: ref, + }) + .nullable(), + duplicateDelivery: z + .strictObject({ + turnId: ref, + eventId: ref, + acceptedAttemptIds: z.array(ref).min(1).max(20), + observedRunIds: z.array(ref).min(1).max(20), + acceptedReplyMessageIds: z.array(ref).min(1).max(20), + behavior: z.enum(["not-observed", "no-additional-run", "additional-run"]), + through: time, + hostObservationRef: ref, + serverObservationRef: ref, + }) + .nullable(), + cancellation: z + .strictObject({ + tableId: ref, + recipient: role, + recipientGuestId: ref, + action: z.enum(["close", "unsubscribe"]), + subscriptionId: ref, + pendingEventId: ref, + pendingAt: time, + decisionAt: time, + observationRef: ref, + attempts: z.array(cancellationAttempt).max(20), + coverageFrom: time, + coverageThrough: time, + coverageObservationRef: ref, + postDecisionProbeAt: time, + refusalObservationRef: ref, + }) + .nullable(), + restart: z + .strictObject({ + mode: z.enum(["retained-storage", "fresh-storage", "unknown"]), + tableId: ref, + startedAt: time, + readyAt: time, + operationRef: ref, + before: snapshot, + after: snapshot, + newTurnId: ref, + terminalTableId: ref, + terminalSubscriptionId: ref, + terminalStateBefore: z.enum([ + "closed", + "deleted", + "expired", + "unsubscribed", + ]), + terminalStateAfter: z.enum([ + "closed", + "deleted", + "expired", + "unsubscribed", + ]), + terminalCheckAt: time, + terminalDispatchCount: z.number().int().nonnegative(), + terminalObservationRef: ref, + }) + .nullable(), + isolation: z + .array( + z.strictObject({ + action: z.enum([ + "metadata", + "transcript", + "inbox", + "renew-subscription", + "unsubscribe", + ]), + outsiderGuestId: ref, + outsiderConnectionId: ref, + tableId: ref, + targetMembershipId: ref, + targetSubscriptionId: ref.nullable(), + observedAt: time, + outcome: z.enum(["denied", "allowed", "unknown"]), + privateDataReturned: z.boolean(), + subscriptionChanged: z.boolean(), + clientObservationRef: ref, + serverObservationRef: ref, + }), + ) + .max(20), + disclosure: z + .strictObject({ + syntheticPayloadsSha256: digest, + postedPayloadsSha256: digest, + scope: z.enum(["synthetic-only", "unknown", "other-content"]), + ownerReviewRefs: z.tuple([ref, ref]), + transcriptObservationRef: ref, + }) + .nullable(), + observations: z.array(autonomousObservationSchema).max(250), +}); + +export type AutonomousEvidence = z.infer; +export type AutonomousObservation = z.infer; +export type EvidenceStatus = "not-run" | "blocked" | "failed" | "passed"; +export const AUTONOMOUS_SCENARIOS = [ + "setup", + "eventResponse", + "exactRetry", + "duplicateDelivery", + "cancellation", + "restart", + "isolation", + "disclosure", +] as const; +type Scenario = (typeof AUTONOMOUS_SCENARIOS)[number]; +export interface AutonomousEvidenceResult { + acceptanceMode: AutonomousEvidence["acceptanceMode"] | null; + structurallyValid: boolean; + completeRecord: boolean; + missing: string[]; + layers: { + tools: EvidenceStatus; + notificationReceipt: EvidenceStatus; + autonomousReply: EvidenceStatus; + }; + scenarios: Record; +} + +const milliseconds = (value: string | null) => + value === null ? Number.NaN : Date.parse(value); +const isKnown = (value: string) => + !/^(unknown|unavailable|not.selected|not.available|not.exposed)$/i.test( + value, + ); +const knownBuild = (value: string) => + value === "not-exposed-in-tested-host" || + (isKnown(value) && + !/^(unknown|unavailable|not.exposed|not.available)/i.test(value)); +const unique = (values: unknown[]) => new Set(values).size === values.length; + +/** + * Check completeness and causal joins in redacted observations, not their + * authenticity. The CLI separately verifies local receipt digests. A reviewer + * must still inspect independently captured owner, host and server evidence. + */ +export function evaluateAutonomousEvidence( + input: unknown, +): AutonomousEvidenceResult { + const parsed = autonomousEvidenceSchema.safeParse(input); + const scenarios = Object.fromEntries( + AUTONOMOUS_SCENARIOS.map((name) => [name, "not-run"]), + ) as Record; + const layers: AutonomousEvidenceResult["layers"] = { + tools: "not-run", + notificationReceipt: "not-run", + autonomousReply: "not-run", + }; + if (!parsed.success) { + // Even unknown field names can contain secrets; never echo schema input. + return { + acceptanceMode: null, + structurallyValid: false, + completeRecord: false, + missing: [ + "Invalid autonomous evidence schema; inspect the input locally.", + ], + layers, + scenarios, + }; + } + + const record = parsed.data; + const missing: string[] = []; + const issues = new Map(); + const failed = new Set(); + const observations = new Map( + record.observations.map((item) => [item.id, item]), + ); + const check = ( + scenario: Scenario, + condition: unknown, + detail: string, + observedFailure = false, + ) => { + if (condition) return true; + const message = `${scenario}: ${detail}`; + missing.push(message); + issues.set(scenario, [...(issues.get(scenario) ?? []), message]); + if (observedFailure) failed.add(scenario); + return false; + }; + const receipt = ( + scenario: Scenario, + id: string | null, + source: AutonomousObservation["source"], + detail: string, + occurredAt?: string | null, + ) => { + const observation = id === null ? undefined : observations.get(id); + const valid = check(scenario, observation?.source === source, detail); + if (observation && occurredAt) + check( + scenario, + milliseconds(observation.capturedAt) >= milliseconds(occurredAt), + "receipt capture does not predate its observed fact", + true, + ); + return valid; + }; + const finish = (scenario: Scenario, ran: boolean) => { + scenarios[scenario] = !ran + ? "not-run" + : failed.has(scenario) + ? "failed" + : issues.has(scenario) + ? "blocked" + : "passed"; + }; + + check( + "setup", + unique(record.observations.map((item) => item.id)), + "unique receipt identifiers", + ); + check( + "setup", + unique(record.turns.map((item) => item.id)), + "unique turn identifiers", + ); + check("setup", record.observedAt !== null, "dated pilot observation"); + // Schema validation already restricted timestamp fields and unknown keys. + const observedTimes: number[] = []; + const collectTimes = (value: unknown, key = "") => { + if ( + typeof value === "string" && + /(?:At|From|Through|^through)$/.test(key) + ) { + observedTimes.push(milliseconds(value)); + } else if (Array.isArray(value)) { + for (const entry of value) collectTimes(entry); + } else if (value !== null && typeof value === "object") { + for (const [name, entry] of Object.entries(value)) + collectTimes(entry, name); + } + }; + collectTimes(record); + check( + "setup", + record.observedAt && + observedTimes.every((value) => value <= milliseconds(record.observedAt)), + "report date covers all fact and receipt capture timestamps", + record.observedAt !== null, + ); + check("setup", record.tableId !== null, "one primary private table"); + const deployment = record.deployment; + check( + "setup", + deployment.buildSha && deployment.configSha256 && deployment.endpointRef, + "pinned deployment build, configuration and endpoint reference", + ); + check( + "setup", + deployment.isolated === true && deployment.publicAdvertisement === false, + "approved isolated deployment without public support advertisement", + ); + check( + "setup", + deployment.protocol && + deployment.profile && + isKnown(deployment.protocol) && + isKnown(deployment.profile), + "exact tested protocol and notification profile", + ); + receipt( + "setup", + deployment.approvalRef, + "operator", + "isolated deployment approval receipt", + ); + receipt( + "setup", + deployment.configurationRef, + "operator", + "deployed build/configuration receipt", + ); + const [a, b] = record.participants; + const requiredReceivers = record.acceptanceMode === "a-only" ? [a] : [a, b]; + const expectedResponseTurns = record.acceptanceMode === "a-only" ? 1 : 2; + check( + "setup", + a.role === "A" && b.role === "B", + "ordered A and B participant records", + ); + for (const key of [ + "ownerRef", + "accountRef", + "guestId", + "connectionId", + "membershipId", + ] as const) { + check( + "setup", + a[key] && b[key] && a[key] !== b[key], + `two independent participant ${key} values`, + ); + } + check( + "setup", + a.consent.observationRef !== b.consent.observationRef, + "separate owner consent receipts", + ); + check( + "setup", + requiredReceivers.every((person) => person.subscriptionId !== null) && + unique(requiredReceivers.map((person) => person.subscriptionId)), + "distinct active subscriptions for the mode's required receivers", + ); + for (const person of record.participants) { + check( + "setup", + person.workspaceRef && + person.host.eligibility === "eligible" && + person.host.origin === "actual-host", + "eligible actual-host account/workspace for each owner", + ); + check( + "setup", + isKnown(person.host.product) && + isKnown(person.host.surface) && + knownBuild(person.host.exposedBuild), + "exact host identity or explicit not-exposed-in-tested-host build marker", + ); + if (requiredReceivers.includes(person)) + check( + "setup", + person.host.runObservation !== "unsupported", + "supported independent idle/run observation surface for each required receiver", + ); + receipt( + "setup", + person.oauthEvidenceRef, + "host-tool", + "actual-host guest OAuth/tool receipt for each participant", + ); + const subscription = person.subscription; + if (requiredReceivers.includes(person)) { + check( + "setup", + subscription.tableId === record.tableId && subscription.activeAt, + "active subscriptions on the same primary table", + ); + receipt( + "setup", + subscription.observationRef, + "server", + "active subscription authority receipt for each participant", + subscription.activeAt, + ); + } + const consent = person.consent; + check( + "setup", + consent.approvedAt && consent.instructionsSha256 && consent.maxReplies, + "separate dated bounded standing instructions", + ); + receipt( + "setup", + consent.observationRef, + "owner-consent", + "independent owner approval of standing instructions", + consent.approvedAt, + ); + check( + "setup", + consent.syntheticOnly && + consent.readBeforeReply && + consent.reuseMutationKey && + consent.stopOnTerminal && + consent.suppressAcknowledgmentLoops && + consent.askBeforeOutOfScope, + "synthetic disclosure, read/retry/stop/authority and acknowledgment bounds", + ); + if (requiredReceivers.includes(person)) + check( + "setup", + milliseconds(consent.approvedAt) <= milliseconds(subscription.activeAt), + "owner consent precedes active subscription", + ); + const participantMessages = record.turns + .flatMap((item) => [item.post, ...(item.reply ? [item.reply] : [])]) + .filter((message) => message.senderMembershipId === person.membershipId); + check( + "setup", + consent.approvedAt !== null && + participantMessages.every( + (message) => + milliseconds(consent.approvedAt) <= + milliseconds(message.committedAt), + ), + "each owner's approval precedes their recorded posts and replies", + ); + } + check( + "setup", + unique(record.turns.map((item) => item.event.eventId)), + "distinct committed event identifiers across all turns", + true, + ); + check( + "setup", + unique(record.turns.flatMap((item) => (item.run ? [item.run.runId] : []))), + "distinct new host run identifiers across all turns", + true, + ); + check( + "setup", + unique( + record.turns.flatMap((item) => + item.deliveries.map((entry) => entry.attemptId), + ), + ), + "distinct dispatch attempt identifiers across all turns", + true, + ); + const messages = new Map(); + const sequences = new Map(); + const mutationKeys = new Map(); + let immutableMessagesValid = true; + for (const item of record.turns) { + const scenario = item.scenario === "restart" ? "restart" : "eventResponse"; + for (const message of [item.post, ...(item.reply ? [item.reply] : [])]) { + const immutable = JSON.stringify([ + message.tableId, + message.seq, + message.senderMembershipId, + message.mutationKeySha256, + message.committedAt, + ]); + if ( + messages.has(message.messageId) && + messages.get(message.messageId) !== immutable + ) + immutableMessagesValid = false; + check( + scenario, + !messages.has(message.messageId) || + messages.get(message.messageId) === immutable, + "one immutable table/sequence/sender/key/time for each accepted message", + true, + ); + messages.set(message.messageId, immutable); + const seqKey = `${message.tableId}:${message.seq}`; + if (sequences.has(seqKey) && sequences.get(seqKey) !== message.messageId) + immutableMessagesValid = false; + check( + scenario, + !sequences.has(seqKey) || sequences.get(seqKey) === message.messageId, + "one accepted message identity for each table sequence", + true, + ); + sequences.set(seqKey, message.messageId); + const mutationKey = `${message.tableId}:${message.senderMembershipId}:${message.mutationKeySha256}`; + if ( + mutationKeys.has(mutationKey) && + mutationKeys.get(mutationKey) !== message.messageId + ) + immutableMessagesValid = false; + check( + scenario, + !mutationKeys.has(mutationKey) || + mutationKeys.get(mutationKey) === message.messageId, + "one accepted message per sender mutation key", + true, + ); + mutationKeys.set(mutationKey, message.messageId); + } + } + for (const person of record.participants) { + const replies = new Set( + record.turns + .filter((item) => item.recipient === person.role) + .flatMap((item) => (item.reply ? [item.reply.messageId] : [])), + ); + const duplicatedTurn = record.turns.find( + (item) => item.id === record.duplicateDelivery?.turnId, + ); + if (duplicatedTurn?.recipient === person.role) { + for (const id of record.duplicateDelivery?.acceptedReplyMessageIds ?? []) + replies.add(id); + } + check( + "setup", + person.consent.maxReplies !== null && + replies.size <= person.consent.maxReplies, + "all observed replies including duplicates stay within each owner's approved bound", + person.consent.maxReplies !== null, + ); + } + finish("setup", record.evidenceKind !== "not-run"); + + const validateTurn = ( + item: AutonomousEvidence["turns"][number], + scenario: Scenario, + ) => { + const start = missing.length; + const sender = item.sender === "A" ? a : b; + const recipient = item.recipient === "A" ? a : b; + const { post, event } = item; + check( + scenario, + requiredReceivers.includes(recipient), + "event targets a receiver required by the explicit acceptance mode", + true, + ); + check( + scenario, + item.sender !== item.recipient, + "peer membership rather than self-wakeup", + ); + check( + scenario, + post.tableId === record.tableId && event.tableId === post.tableId, + "committed post/event on the primary table", + ); + check( + scenario, + post.senderMembershipId === sender.membershipId && + event.senderMembershipId === sender.membershipId && + event.recipientMembershipId === recipient.membershipId && + event.subscriptionId === recipient.subscriptionId, + "event sender, recipient and active subscription association", + ); + check( + scenario, + event.messageId === post.messageId && + event.messageSeq === post.seq && + milliseconds(event.committedAt) >= milliseconds(post.committedAt), + "same committed event/message and ordered sequence", + ); + check( + scenario, + requiredReceivers.every( + (person) => + milliseconds(person.subscription.activeAt) <= + milliseconds(post.committedAt), + ), + "required receiver subscriptions active before peer post", + ); + receipt( + scenario, + post.observationRef, + "server", + "committed post receipt", + post.committedAt, + ); + receipt( + scenario, + post.hostObservationRef, + "host-tool", + "actual initiating agent post tool receipt with committed message result", + post.committedAt, + ); + receipt( + scenario, + event.observationRef, + "server", + "committed peer event receipt", + event.committedAt, + ); + check( + scenario, + unique(item.deliveries.map((entry) => entry.attemptId)), + "unique delivery attempt identifiers", + ); + const accepted = item.deliveries.filter( + (entry) => + entry.statusCode !== null && + entry.statusCode >= 200 && + entry.statusCode < 300 && + entry.acceptedAt !== null, + ); + check( + scenario, + accepted.length > 0, + "observed successful receiver acceptance", + ); + for (const attempt of item.deliveries) { + check( + scenario, + attempt.eventId === event.eventId && + attempt.subscriptionId === event.subscriptionId, + "delivery/event/subscription association", + ); + check( + scenario, + milliseconds(attempt.authorizedAt) >= milliseconds(event.committedAt) && + milliseconds(attempt.dispatchedAt) >= + milliseconds(attempt.authorizedAt) && + (attempt.acceptedAt === null || + milliseconds(attempt.acceptedAt) >= + milliseconds(attempt.dispatchedAt)), + "ordered delivery authorization, dispatch and response", + ); + receipt( + scenario, + attempt.observationRef, + "server", + "safe server delivery observation", + attempt.acceptedAt ?? attempt.dispatchedAt, + ); + } + const notificationValid = missing.length === start; + const { idle, run, read, reply } = item; + check( + scenario, + idle !== null && run !== null && read !== null && reply !== null, + "independent idle, new run, authenticated read and committed reply observations", + ); + if (!idle || !run || !read || !reply) + return { notificationValid, autonomousValid: false }; + const initiatingAttempt = accepted.find( + (entry) => entry.attemptId === run.deliveryAttemptId, + ); + check( + scenario, + idle.activeRuns === 0 && + milliseconds(idle.observedAt) >= + milliseconds(recipient.subscription.activeAt) && + initiatingAttempt && + milliseconds(idle.observedAt) <= milliseconds(post.committedAt), + "recipient idle before the initiating post", + true, + ); + check( + scenario, + run.trigger === "peer-event" && + run.triggeringEventId === event.eventId && + initiatingAttempt && + milliseconds(run.startedAt) >= + milliseconds(initiatingAttempt.dispatchedAt), + "new host run caused by this peer event without human or polling substitute", + true, + ); + receipt( + scenario, + idle.observationRef, + "host-audit", + "supported host idle-state receipt", + idle.observedAt, + ); + receipt( + scenario, + run.observationRef, + "host-audit", + "independent host event-to-new-run receipt", + run.startedAt, + ); + check( + scenario, + read.runId === run.runId && + read.eventId === event.eventId && + read.tableId === post.tableId && + read.membershipId === recipient.membershipId && + read.connectionId === recipient.connectionId && + read.afterSeq < post.seq && + read.throughSeq >= post.seq && + read.observedMessageIds.includes(post.messageId) && + milliseconds(read.readAt) >= milliseconds(run.startedAt), + "same-run authenticated read of the triggering committed message", + ); + receipt( + scenario, + read.hostObservationRef, + "host-tool", + "host authenticated-read tool receipt", + read.readAt, + ); + receipt( + scenario, + read.serverObservationRef, + "server", + "server authenticated-read receipt", + read.readAt, + ); + check( + scenario, + reply.runId === run.runId && + reply.tableId === post.tableId && + reply.senderMembershipId === recipient.membershipId && + reply.replyToMessageId === post.messageId && + reply.messageId !== post.messageId && + reply.seq > post.seq && + milliseconds(reply.committedAt) >= milliseconds(read.readAt), + "same-run committed peer reply after authenticated read", + ); + receipt( + scenario, + reply.observationRef, + "server", + "committed reply receipt", + reply.committedAt, + ); + receipt( + scenario, + reply.hostObservationRef, + "host-tool", + "same-run host reply tool receipt with committed message result", + reply.committedAt, + ); + return { notificationValid, autonomousValid: missing.length === start }; + }; + + const responses = record.turns.filter( + (item) => item.scenario === "event-response", + ); + check( + "eventResponse", + responses.length === expectedResponseTurns, + "exact event-response turn count for the explicit acceptance mode", + ); + const responseResults = responses.map((item) => + validateTurn(item, "eventResponse"), + ); + if (record.acceptanceMode === "bidirectional") { + const first = responses.find( + (item) => item.sender === "A" && item.recipient === "B", + ); + const back = responses.find( + (item) => item.sender === "B" && item.recipient === "A", + ); + const firstReply = first?.reply; + check( + "eventResponse", + first && + back && + firstReply && + back.post.messageId === firstReply.messageId && + back.post.seq === firstReply.seq && + back.post.senderMembershipId === firstReply.senderMembershipId && + back.post.mutationKeySha256 === firstReply.mutationKeySha256 && + back.post.committedAt === firstReply.committedAt, + "B reply is the exact committed post causing the return A turn", + ); + check( + "eventResponse", + first?.run && + back?.run && + first.run.runId !== back.run.runId && + first.event.eventId !== back.event.eventId, + "distinct event-driven host runs in both directions", + ); + } else { + check( + "eventResponse", + responses.length === 1 && + responses[0].sender === "B" && + responses[0].recipient === "A", + "A-only acceptance requires an actual B post followed by an event-driven A reply", + responses.length > 0, + ); + } + finish("eventResponse", responses.length > 0); + + const retry = record.retry; + check("exactRetry", retry !== null, "observed exact mutation-key retry"); + if (retry) { + const original = record.turns.find( + (item) => item.id === retry.turnId, + )?.post; + check( + "exactRetry", + original && + retry.originalMutationKeySha256 === original.mutationKeySha256 && + retry.retryMutationKeySha256 === retry.originalMutationKeySha256 && + retry.originalPayloadSha256 === retry.retryPayloadSha256, + "unchanged original mutation key and payload", + ); + check( + "exactRetry", + original && + retry.originalMessageId === original.messageId && + retry.retryMessageId === original.messageId && + retry.originalSeq === original.seq && + retry.retrySeq === original.seq, + "the original accepted message/sequence receipt on retry", + ); + check( + "exactRetry", + original && + retry.beforeCount >= original.seq && + retry.beforeCount === retry.afterCount && + milliseconds(retry.retriedAt) >= milliseconds(original.committedAt), + "no additional accepted transcript message on exact retry", + ); + receipt( + "exactRetry", + retry.observationRef, + "server", + "retry and unchanged transcript receipt", + retry.retriedAt, + ); + } + finish("exactRetry", retry !== null); + + const duplicates = record.duplicateDelivery; + check( + "duplicateDelivery", + duplicates !== null, + "actual duplicate-delivery behavior or explicit not-observed record", + ); + if (duplicates) { + const observedTurn = record.turns.find( + (item) => item.id === duplicates.turnId, + ); + const accepted = + observedTurn?.deliveries.filter( + (entry) => + entry.acceptedAt && + entry.statusCode !== null && + entry.statusCode >= 200 && + entry.statusCode < 300, + ) ?? []; + check( + "duplicateDelivery", + observedTurn && + duplicates.eventId === observedTurn.event.eventId && + accepted.length === duplicates.acceptedAttemptIds.length && + accepted.every((item) => + duplicates.acceptedAttemptIds.includes(item.attemptId), + ), + "duplicate observations joined to accepted attempts for the same event", + ); + check( + "duplicateDelivery", + accepted.every( + (item) => + milliseconds(item.acceptedAt) <= milliseconds(duplicates.through), + ), + "duplicate observation window covers every included receiver acceptance", + true, + ); + check( + "duplicateDelivery", + unique(duplicates.acceptedAttemptIds) && + unique(duplicates.observedRunIds) && + unique(duplicates.acceptedReplyMessageIds), + "unique observed duplicate attempt/run/reply identifiers", + ); + check( + "duplicateDelivery", + observedTurn?.run && + observedTurn.reply && + duplicates.observedRunIds.includes(observedTurn.run.runId) && + duplicates.acceptedReplyMessageIds.includes( + observedTurn.reply.messageId, + ) && + milliseconds(duplicates.through) >= + milliseconds(observedTurn.reply.committedAt), + "host run/reply observation window for the same event", + ); + check( + "duplicateDelivery", + duplicates.behavior === "not-observed" + ? accepted.length === 1 + : accepted.length > 1 && + (duplicates.behavior === "additional-run" + ? duplicates.observedRunIds.length > 1 + : duplicates.observedRunIds.length === 1), + "reported duplicate behavior matches observed attempts and runs", + ); + receipt( + "duplicateDelivery", + duplicates.hostObservationRef, + "host-audit", + "host duplicate-delivery behavior receipt", + duplicates.through, + ); + receipt( + "duplicateDelivery", + duplicates.serverObservationRef, + "server", + "server duplicate delivery and accepted reply receipt", + duplicates.through, + ); + } + finish("duplicateDelivery", duplicates !== null); + + const cancellation = record.cancellation; + check( + "cancellation", + cancellation !== null, + "pending-work closure or unsubscribe exercise", + ); + if (cancellation) { + const cancellationRecipient = cancellation.recipient === "A" ? a : b; + check( + "cancellation", + requiredReceivers.includes(cancellationRecipient) && + cancellation.recipientGuestId === cancellationRecipient.guestId, + "cancellation targets the independently authorized guest of a required receiver", + true, + ); + const decision = milliseconds(cancellation.decisionAt); + check( + "cancellation", + cancellation.tableId !== record.tableId && + milliseconds(cancellation.pendingAt) < decision, + "separate cancellation table with work pending before the decision", + ); + check( + "cancellation", + milliseconds(cancellation.coverageFrom) <= + milliseconds(cancellation.pendingAt) && + milliseconds(cancellation.coverageThrough) >= + milliseconds(cancellation.postDecisionProbeAt) && + milliseconds(cancellation.postDecisionProbeAt) > decision, + "attempt-ledger coverage spanning pending work and post-decision refusal", + ); + check( + "cancellation", + unique(cancellation.attempts.map((item) => item.attemptId)), + "unique cancellation attempt identifiers", + ); + for (const attempt of cancellation.attempts) { + check( + "cancellation", + milliseconds(attempt.authorizedAt) < decision, + "no newly authorized delivery after the terminal decision", + true, + ); + check( + "cancellation", + attempt.eventId === cancellation.pendingEventId, + "attempt ledger is scoped to the same pending event", + true, + ); + check( + "cancellation", + milliseconds(attempt.authorizedAt) >= + milliseconds(cancellation.coverageFrom) && + (attempt.dispatchedAt === null || + milliseconds(attempt.dispatchedAt) >= + milliseconds(attempt.authorizedAt)) && + (attempt.completedAt === null || + (attempt.dispatchedAt !== null && + milliseconds(attempt.completedAt) >= + milliseconds(attempt.dispatchedAt) && + milliseconds(attempt.completedAt) <= + milliseconds(cancellation.coverageThrough))), + "authorization, dispatch and completion remain distinct in observed ledger window", + ); + } + receipt( + "cancellation", + cancellation.observationRef, + "server", + "authoritative terminal decision receipt", + cancellation.decisionAt, + ); + receipt( + "cancellation", + cancellation.coverageObservationRef, + "server", + "complete scoped attempt-ledger observation", + cancellation.coverageThrough, + ); + receipt( + "cancellation", + cancellation.refusalObservationRef, + "server", + "post-decision delivery refusal receipt", + cancellation.postDecisionProbeAt, + ); + } + finish("cancellation", cancellation !== null); + + const restart = record.restart; + check( + "restart", + restart !== null, + "retained-state restart and fresh real-host progress", + ); + if (restart) { + const { before, after } = restart; + check( + "restart", + restart.mode === "retained-storage" && + restart.tableId === record.tableId && + before.storageIdentity === after.storageIdentity && + before.retryStateSha256 === after.retryStateSha256, + "same retained storage and unchanged server retry state", + true, + ); + check( + "restart", + before.buildSha === deployment.buildSha && + after.buildSha === deployment.buildSha && + before.configSha256 === deployment.configSha256 && + after.configSha256 === deployment.configSha256, + "same pinned build/configuration across service restart", + ); + const originalSubscriptions = requiredReceivers.map( + (person) => person.subscriptionId, + ); + check( + "restart", + before.subscriptionIds.length === originalSubscriptions.length && + after.subscriptionIds.length === originalSubscriptions.length && + before.subscriptionIds.every( + (id, i) => id === originalSubscriptions[i], + ) && + after.subscriptionIds.every((id, i) => id === originalSubscriptions[i]), + "original active subscriptions survive without replacement", + ); + check( + "restart", + milliseconds(before.observedAt) <= milliseconds(restart.startedAt) && + milliseconds(restart.readyAt) >= milliseconds(restart.startedAt) && + milliseconds(after.observedAt) >= milliseconds(restart.readyAt), + "ordered before/restart/ready/after observations", + ); + receipt( + "restart", + restart.operationRef, + "operator", + "storage-preserving restart operation receipt", + restart.readyAt, + ); + receipt( + "restart", + before.observationRef, + "server", + "pre-restart durable-state receipt", + before.observedAt, + ); + receipt( + "restart", + after.observationRef, + "server", + "post-restart durable-state receipt", + after.observedAt, + ); + const resumed = record.turns.find( + (item) => item.id === restart.newTurnId && item.scenario === "restart", + ); + const previousMessages = responses.flatMap((item) => [ + item.post, + ...(item.reply ? [item.reply] : []), + ]); + check( + "restart", + record.turns.filter((item) => item.scenario === "restart").length === 1 && + resumed && + previousMessages.every( + (message) => + message.messageId !== resumed.post.messageId && + message.seq < resumed.post.seq && + milliseconds(message.committedAt) <= + milliseconds(resumed.post.committedAt), + ), + "one genuinely new post and ordered sequence after the event-response phase", + resumed !== undefined, + ); + check( + "restart", + resumed && + milliseconds(resumed.post.committedAt) > milliseconds(after.observedAt), + "fresh eligible peer post after retained-state restart", + ); + if (resumed) validateTurn(resumed, "restart"); + check( + "restart", + cancellation && + restart.terminalTableId === cancellation.tableId && + restart.terminalSubscriptionId === cancellation.subscriptionId && + milliseconds(cancellation.decisionAt) < + milliseconds(restart.startedAt) && + restart.terminalStateBefore === restart.terminalStateAfter && + restart.terminalStateBefore === + (cancellation.action === "close" ? "closed" : "unsubscribed"), + "the original terminal scope remains terminal after restart", + ); + check( + "restart", + milliseconds(restart.terminalCheckAt) >= milliseconds(restart.readyAt) && + restart.terminalDispatchCount === 0, + "no terminal-scope dispatch resurrection after restart", + ); + receipt( + "restart", + restart.terminalObservationRef, + "server", + "retained terminal-state and denied-dispatch receipt", + restart.terminalCheckAt, + ); + } + finish("restart", restart !== null); + + const isolationActions = [ + "metadata", + "transcript", + "inbox", + "renew-subscription", + "unsubscribe", + ] as const; + for (const person of record.participants) { + for (const action of isolationActions) { + if ( + (action === "renew-subscription" || action === "unsubscribe") && + !requiredReceivers.includes(person) + ) + continue; + check( + "isolation", + record.isolation.some( + (item) => + item.action === action && + item.targetMembershipId === person.membershipId && + item.targetSubscriptionId === person.subscriptionId, + ), + "third-guest private-read coverage for both memberships and subscription maintenance for required receivers", + ); + } + } + const outsider = record.isolation[0]; + for (const item of record.isolation) { + check( + "isolation", + item.outsiderGuestId === outsider?.outsiderGuestId && + item.outsiderConnectionId === outsider.outsiderConnectionId && + record.participants.every( + (person) => + item.outsiderGuestId !== person.guestId && + item.outsiderConnectionId !== person.connectionId, + ), + "one independently authenticated unaffiliated third guest", + ); + check( + "isolation", + item.tableId === record.tableId && + item.outcome === "denied" && + !item.privateDataReturned && + !item.subscriptionChanged, + "denied access without private content or subscription mutation", + true, + ); + receipt( + "isolation", + item.clientObservationRef, + "host-tool", + "third-guest authenticated client refusal receipt", + item.observedAt, + ); + receipt( + "isolation", + item.serverObservationRef, + "server", + "server denial and unchanged subscription receipt", + item.observedAt, + ); + } + finish("isolation", record.isolation.length > 0); + + const disclosure = record.disclosure; + check( + "disclosure", + disclosure !== null, + "synthetic-only shared transcript review", + ); + if (disclosure) { + check( + "disclosure", + disclosure.scope === "synthetic-only" && + disclosure.syntheticPayloadsSha256 === disclosure.postedPayloadsSha256, + "only deliberately supplied synthetic payloads crossed the relay", + ); + check( + "disclosure", + disclosure.ownerReviewRefs[0] !== disclosure.ownerReviewRefs[1], + "separate owner disclosure reviews", + ); + for (const id of disclosure.ownerReviewRefs) + receipt( + "disclosure", + id, + "owner-consent", + "independent owner synthetic disclosure review", + ); + receipt( + "disclosure", + disclosure.transcriptObservationRef, + "server", + "committed synthetic transcript receipt", + ); + } + finish("disclosure", disclosure !== null); + + const actual = record.evidenceKind === "real-host"; + if (!actual) + missing.push( + "Actual independently observed real-host pilot evidence is required; fixtures are not compatibility results.", + ); + const hasReceipt = ( + id: string | null, + source: AutonomousObservation["source"], + ) => id !== null && observations.get(id)?.source === source; + // Tools and receipt compatibility do not depend on host run observability. + const toolsVerified = + actual && + unique(record.observations.map((item) => item.id)) && + record.observedAt && + observedTimes.every((value) => value <= milliseconds(record.observedAt)) && + deployment.isolated === true && + deployment.buildSha && + deployment.configSha256 && + deployment.endpointRef && + deployment.protocol && + deployment.profile && + hasReceipt(deployment.approvalRef, "operator") && + hasReceipt(deployment.configurationRef, "operator") && + a.role === "A" && + b.role === "B" && + (["ownerRef", "accountRef", "guestId", "connectionId"] as const).every( + (key) => a[key] && b[key] && a[key] !== b[key], + ) && + record.participants.every( + (person) => + person.workspaceRef && + person.host.eligibility === "eligible" && + person.host.origin === "actual-host" && + isKnown(person.host.product) && + isKnown(person.host.surface) && + knownBuild(person.host.exposedBuild) && + hasReceipt(person.oauthEvidenceRef, "host-tool"), + ); + const notificationPrerequisite = + toolsVerified && + immutableMessagesValid && + unique(record.turns.map((item) => item.event.eventId)) && + unique( + record.turns.flatMap((item) => + item.deliveries.map((entry) => entry.attemptId), + ), + ) && + record.tableId && + requiredReceivers.every( + (person) => + person.subscription.tableId === record.tableId && + person.subscription.activeAt && + hasReceipt(person.subscription.observationRef, "server"), + ); + const prerequisite = scenarios.setup === "passed" && actual; + layers.tools = toolsVerified + ? "passed" + : record.evidenceKind === "not-run" + ? "not-run" + : "blocked"; + layers.notificationReceipt = + notificationPrerequisite && + responseResults.length === expectedResponseTurns && + responseResults.every((item) => item.notificationValid) + ? "passed" + : responses.length + ? "blocked" + : "not-run"; + layers.autonomousReply = + prerequisite && scenarios.eventResponse === "passed" + ? "passed" + : scenarios.eventResponse === "failed" + ? "failed" + : responses.length + ? "blocked" + : "not-run"; + return { + acceptanceMode: record.acceptanceMode, + structurallyValid: true, + completeRecord: actual && missing.length === 0, + missing: [...new Set(missing)], + layers, + scenarios, + }; +} diff --git a/test/unit/autonomous-evidence.test.ts b/test/unit/autonomous-evidence.test.ts new file mode 100644 index 0000000..2b7027f --- /dev/null +++ b/test/unit/autonomous-evidence.test.ts @@ -0,0 +1,1186 @@ +import { describe, expect, it } from "vitest"; +import { + AUTONOMOUS_SCENARIOS, + type AutonomousEvidence, + type AutonomousObservation, + autonomousEvidenceSchema, + evaluateAutonomousEvidence, +} from "../e2e/host/autonomous-evidence"; +import selectedChatPair from "../e2e/host/autonomous-evidence.chatgpt-claude-selected.json"; +import template from "../e2e/host/autonomous-evidence.template.json"; + +// This fixture is invented validator input, never a real compatibility record. +// Changing its kind in a test exercises only the metadata gate's positive branch. +function syntheticFixture( + acceptanceMode: AutonomousEvidence["acceptanceMode"] = "bidirectional", +): AutonomousEvidence { + const record = autonomousEvidenceSchema.parse(template); + record.acceptanceMode = acceptanceMode; + const aOnly = acceptanceMode === "a-only"; + record.evidenceKind = "synthetic-validator"; + const at = (seconds: number) => + new Date(Date.parse("2026-11-12T00:00:00Z") + seconds * 1000).toISOString(); + let nextId = 0; + const receipt = (source: AutonomousObservation["source"]) => { + const n = ++nextId; + const id = `fixture-observation-${n}`; + record.observations.push({ + id, + source, + capturedAt: at(250), + sha256: n.toString(16).padStart(64, "0"), + }); + return id; + }; + record.pilotId = "synthetic-validator-only"; + record.observedAt = at(300); + record.tableId = "primary-table-0001"; + record.deployment = { + buildSha: "a".repeat(40), + configSha256: "b".repeat(64), + endpointRef: "isolated-endpoint-0001", + approvalRef: receipt("operator"), + configurationRef: receipt("operator"), + isolated: true, + publicAdvertisement: false, + protocol: "mcp-2025-11-25", + profile: "synthetic-events-profile", + }; + record.participants = record.participants.map((person) => ({ + ...person, + ownerRef: `fixture-owner-${person.role}`, + accountRef: `fixture-account-${person.role}`, + workspaceRef: `fixture-workspace-${person.role}`, + guestId: `fixture-guest-${person.role}`, + connectionId: `fixture-connection-${person.role}`, + membershipId: `fixture-membership-${person.role}`, + subscriptionId: `fixture-subscription-${person.role}`, + host: { + product: `Synthetic Host ${person.role}`, + surface: "Synthetic private app", + exposedBuild: "not-exposed-in-tested-host", + eligibility: "eligible", + origin: "actual-host", + runObservation: "provider-audit", + }, + oauthEvidenceRef: receipt("host-tool"), + subscription: { + tableId: record.tableId, + activeAt: at(1), + observationRef: receipt("server"), + }, + consent: { + approvedAt: at(0), + observationRef: receipt("owner-consent"), + instructionsSha256: "c".repeat(64), + maxReplies: 3, + syntheticOnly: true, + readBeforeReply: true, + reuseMutationKey: true, + stopOnTerminal: true, + suppressAcknowledgmentLoops: true, + askBeforeOutOfScope: true, + }, + })) as AutonomousEvidence["participants"]; + const [a, b] = record.participants; + if (aOnly) { + b.subscriptionId = null; + b.subscription = { tableId: null, activeAt: null, observationRef: null }; + b.host.runObservation = "unsupported"; + } + const makeTurn = ( + id: string, + sender: "A" | "B", + start: number, + seq: number, + scenario: "event-response" | "restart", + ): AutonomousEvidence["turns"][number] => { + const from = sender === "A" ? a : b; + const to = sender === "A" ? b : a; + const eventId = `${id}-event`; + const attemptId = `${id}-attempt`; + const runId = `${id}-run`; + const post = { + tableId: record.tableId as string, + messageId: `${id}-post`, + seq, + senderMembershipId: from.membershipId as string, + committedAt: at(start), + mutationKeySha256: seq.toString(16).repeat(64), + observationRef: receipt("server"), + hostObservationRef: receipt("host-tool"), + }; + return { + id, + scenario, + sender, + recipient: sender === "A" ? "B" : "A", + post, + event: { + tableId: post.tableId, + eventId, + messageId: post.messageId, + messageSeq: seq, + senderMembershipId: post.senderMembershipId, + recipientMembershipId: to.membershipId as string, + subscriptionId: to.subscriptionId as string, + committedAt: at(start), + observationRef: receipt("server"), + }, + deliveries: [ + { + attemptId, + eventId, + subscriptionId: to.subscriptionId as string, + authorizedAt: at(start + 1), + dispatchedAt: at(start + 2), + acceptedAt: at(start + 3), + statusCode: 204, + observationRef: receipt("server"), + }, + ], + idle: { + observedAt: at(start - 1), + activeRuns: 0, + observationRef: receipt("host-audit"), + }, + run: { + runId, + triggeringEventId: eventId, + deliveryAttemptId: attemptId, + trigger: "peer-event", + startedAt: at(start + 4), + observationRef: receipt("host-audit"), + }, + read: { + runId, + eventId, + tableId: post.tableId, + membershipId: to.membershipId as string, + connectionId: to.connectionId as string, + afterSeq: seq - 1, + throughSeq: seq, + observedMessageIds: [post.messageId], + readAt: at(start + 5), + hostObservationRef: receipt("host-tool"), + serverObservationRef: receipt("server"), + }, + reply: { + tableId: post.tableId, + runId, + messageId: `${id}-reply`, + seq: seq + 1, + senderMembershipId: to.membershipId as string, + replyToMessageId: post.messageId, + committedAt: at(start + 6), + mutationKeySha256: (seq + 1).toString(16).repeat(64), + observationRef: receipt("server"), + hostObservationRef: receipt("host-tool"), + }, + }; + }; + const first = makeTurn( + "fixture-first", + aOnly ? "B" : "A", + 10, + 1, + "event-response", + ); + const firstReply = first.reply as NonNullable; + record.turns = [first]; + if (!aOnly) { + const back = makeTurn("fixture-return", "B", 16, 2, "event-response"); + back.post = { + tableId: firstReply.tableId, + messageId: firstReply.messageId, + seq: firstReply.seq, + senderMembershipId: firstReply.senderMembershipId, + committedAt: firstReply.committedAt, + mutationKeySha256: firstReply.mutationKeySha256, + observationRef: firstReply.observationRef, + hostObservationRef: firstReply.hostObservationRef, + }; + back.event.messageId = back.post.messageId; + if (back.read) back.read.observedMessageIds = [back.post.messageId]; + (back.reply as NonNullable).replyToMessageId = + back.post.messageId; + record.turns.push(back); + } + const resumed = makeTurn( + "fixture-resumed", + aOnly ? "B" : "A", + 110, + aOnly ? 3 : 4, + "restart", + ); + record.turns.push(resumed); + record.retry = { + turnId: first.id, + originalMutationKeySha256: first.post.mutationKeySha256, + retryMutationKeySha256: first.post.mutationKeySha256, + originalPayloadSha256: "1".repeat(64), + retryPayloadSha256: "1".repeat(64), + originalMessageId: first.post.messageId, + retryMessageId: first.post.messageId, + originalSeq: 1, + retrySeq: 1, + beforeCount: aOnly ? 2 : 3, + afterCount: aOnly ? 2 : 3, + retriedAt: at(30), + observationRef: receipt("server"), + }; + record.duplicateDelivery = { + turnId: first.id, + eventId: first.event.eventId, + acceptedAttemptIds: [first.deliveries[0].attemptId], + observedRunIds: [first.run?.runId as string], + acceptedReplyMessageIds: [firstReply.messageId], + behavior: "not-observed", + through: at(60), + hostObservationRef: receipt("host-audit"), + serverObservationRef: receipt("server"), + }; + record.cancellation = { + tableId: "cancellation-table-0001", + recipient: "A", + recipientGuestId: a.guestId as string, + action: "close", + subscriptionId: "cancellation-subscription-0001", + pendingEventId: "cancellation-pending-0001", + pendingAt: at(40), + decisionAt: at(50), + observationRef: receipt("server"), + attempts: [ + { + attemptId: "cancellation-inflight-0001", + eventId: "cancellation-pending-0001", + authorizedAt: at(41), + dispatchedAt: at(42), + completedAt: at(55), + }, + ], + coverageFrom: at(35), + coverageThrough: at(70), + coverageObservationRef: receipt("server"), + postDecisionProbeAt: at(65), + refusalObservationRef: receipt("server"), + }; + const before = { + storageIdentity: "retained-storage-0001", + buildSha: record.deployment.buildSha as string, + configSha256: record.deployment.configSha256 as string, + retryStateSha256: "2".repeat(64), + subscriptionIds: aOnly + ? [a.subscriptionId as string] + : [a.subscriptionId as string, b.subscriptionId as string], + observedAt: at(100), + observationRef: receipt("server"), + }; + record.restart = { + mode: "retained-storage", + tableId: record.tableId, + startedAt: at(101), + readyAt: at(102), + operationRef: receipt("operator"), + before, + after: { + ...before, + observedAt: at(103), + observationRef: receipt("server"), + }, + newTurnId: resumed.id, + terminalTableId: record.cancellation.tableId, + terminalSubscriptionId: record.cancellation.subscriptionId, + terminalStateBefore: "closed", + terminalStateAfter: "closed", + terminalCheckAt: at(120), + terminalDispatchCount: 0, + terminalObservationRef: receipt("server"), + }; + for (const person of record.participants) { + for (const action of [ + "metadata", + "transcript", + "inbox", + "renew-subscription", + "unsubscribe", + ] as const) { + if ( + aOnly && + person.role === "B" && + (action === "renew-subscription" || action === "unsubscribe") + ) + continue; + record.isolation.push({ + action, + outsiderGuestId: "unaffiliated-guest-0001", + outsiderConnectionId: "unaffiliated-connection-0001", + tableId: record.tableId, + targetMembershipId: person.membershipId as string, + targetSubscriptionId: person.subscriptionId, + observedAt: at(80), + outcome: "denied", + privateDataReturned: false, + subscriptionChanged: false, + clientObservationRef: receipt("host-tool"), + serverObservationRef: receipt("server"), + }); + } + } + record.disclosure = { + syntheticPayloadsSha256: "3".repeat(64), + postedPayloadsSha256: "3".repeat(64), + scope: "synthetic-only", + ownerReviewRefs: [receipt("owner-consent"), receipt("owner-consent")], + transcriptObservationRef: receipt("server"), + }; + return record; +} + +function realHostShapedFixture( + acceptanceMode: AutonomousEvidence["acceptanceMode"] = "bidirectional", +) { + const record = syntheticFixture(acceptanceMode); + record.evidenceKind = "real-host"; + return record; +} + +describe("autonomous pilot metadata gate", () => { + it.each([ + ["generic template", template], + ["owner-selected ChatGPT and Claude Chat", selectedChatPair], + ])("keeps the unrun %s explicit and incomplete", (_name, record) => { + const result = evaluateAutonomousEvidence(record); + expect(result.structurallyValid).toBe(true); + expect(result.completeRecord).toBe(false); + expect(Object.values(result.layers)).toEqual([ + "not-run", + "not-run", + "not-run", + ]); + expect(result.missing.length).toBeGreaterThan(0); + }); + + it("never turns a complete synthetic fixture into host compatibility evidence", () => { + const result = evaluateAutonomousEvidence(syntheticFixture()); + expect(result.structurallyValid).toBe(true); + for (const scenario of AUTONOMOUS_SCENARIOS) + expect(result.scenarios[scenario]).toBe("passed"); + expect(result.completeRecord).toBe(false); + expect(result.layers.autonomousReply).toBe("blocked"); + }); + + it("accepts a complete real-host-shaped metadata join without authenticating its receipts", () => { + // Positive validator branch only: this invented record is not saved as evidence. + const result = evaluateAutonomousEvidence(realHostShapedFixture()); + expect(result.missing).toEqual([]); + expect(result.completeRecord).toBe(true); + expect(result.layers).toEqual({ + tools: "passed", + notificationReceipt: "passed", + autonomousReply: "passed", + }); + }); + + it("reports tools and receipt separately when receiver 2xx has no observed host run", () => { + const record = realHostShapedFixture(); + for (const item of record.turns) { + item.idle = null; + item.run = null; + item.read = null; + item.reply = null; + } + const result = evaluateAutonomousEvidence(record); + expect(result.layers.tools).toBe("passed"); + expect(result.layers.notificationReceipt).toBe("passed"); + expect(result.layers.autonomousReply).toBe("blocked"); + expect(result.completeRecord).toBe(false); + }); + + it("does not certify unsupported run observation from previously verified tools", () => { + const record = realHostShapedFixture(); + record.participants[1].host.runObservation = "unsupported"; + const result = evaluateAutonomousEvidence(record); + expect(result.layers.tools).toBe("passed"); + expect(result.layers.notificationReceipt).toBe("passed"); + expect(result.layers.autonomousReply).toBe("blocked"); + expect(result.completeRecord).toBe(false); + }); + + const mutations: [string, (record: AutonomousEvidence) => void][] = [ + [ + "one owner operating both accounts", + (r) => { + r.participants[1].ownerRef = r.participants[0].ownerRef; + }, + ], + [ + "one grant relabeled as a second participant", + (r) => { + r.participants[1].connectionId = r.participants[0].connectionId; + }, + ], + [ + "missing independent owner consent", + (r) => { + r.participants[1].consent.observationRef = null; + }, + ], + [ + "unapproved synthetic disclosure scope", + (r) => { + r.participants[1].consent.syntheticOnly = false; + }, + ], + [ + "SDK-only replacement", + (r) => { + r.participants[1].host.origin = "sdk-only"; + }, + ], + [ + "unisolated deployment", + (r) => { + r.deployment.isolated = false; + }, + ], + [ + "public support advertisement during private verification", + (r) => { + r.deployment.publicAdvertisement = true; + }, + ], + [ + "missing deployed build configuration", + (r) => { + r.deployment.configSha256 = null; + }, + ], + [ + "another subscription", + (r) => { + r.turns[0].event.subscriptionId = "unrelated-subscription"; + }, + ], + [ + "another event", + (r) => { + r.turns[0].deliveries[0].eventId = "unrelated-event-0001"; + }, + ], + [ + "another committed message", + (r) => { + r.turns[0].event.messageId = "unrelated-message-0001"; + }, + ], + [ + "manual continuation", + (r) => { + if (r.turns[0].run) r.turns[0].run.trigger = "manual"; + }, + ], + [ + "scheduled polling", + (r) => { + if (r.turns[0].run) r.turns[0].run.trigger = "polling"; + }, + ], + [ + "scripted host replacement", + (r) => { + if (r.turns[0].run) r.turns[0].run.trigger = "scripted"; + }, + ], + [ + "model claim replacing host audit", + (r) => { + const observation = r.observations.find( + (item) => item.id === r.turns[0].run?.observationRef, + ); + if (observation) observation.source = "host-tool"; + }, + ], + [ + "already active target", + (r) => { + if (r.turns[0].idle) r.turns[0].idle.activeRuns = 1; + }, + ], + [ + "run before event dispatch", + (r) => { + if (r.turns[0].run) r.turns[0].run.startedAt = "2026-11-12T00:00:00Z"; + }, + ], + [ + "read in unrelated run", + (r) => { + if (r.turns[0].read) r.turns[0].read.runId = "unrelated-run-0001"; + }, + ], + [ + "read using another connection", + (r) => { + if (r.turns[0].read) + r.turns[0].read.connectionId = r.participants[0] + .connectionId as string; + }, + ], + [ + "read before triggering message", + (r) => { + if (r.turns[0].read) r.turns[0].read.throughSeq = 0; + }, + ], + [ + "empty read at a snapshot upper bound containing the trigger", + (r) => { + if (r.turns[0].read) r.turns[0].read.observedMessageIds = []; + }, + ], + [ + "read returning only unrelated messages", + (r) => { + if (r.turns[0].read) + r.turns[0].read.observedMessageIds = ["unrelated-message-0001"]; + }, + ], + [ + "read cursor skipping the triggering message", + (r) => { + if (r.turns[0].read) r.turns[0].read.afterSeq = r.turns[0].post.seq; + }, + ], + [ + "reply to unrelated message", + (r) => { + if (r.turns[0].reply) + r.turns[0].reply.replyToMessageId = "unrelated-message-0001"; + }, + ], + [ + "reply using sender membership", + (r) => { + if (r.turns[0].reply) + r.turns[0].reply.senderMembershipId = r.participants[0] + .membershipId as string; + }, + ], + [ + "disconnected return turn", + (r) => { + r.turns[1].post.messageId = "disconnected-message-0001"; + r.turns[1].event.messageId = r.turns[1].post.messageId; + if (r.turns[1].reply) + r.turns[1].reply.replyToMessageId = r.turns[1].post.messageId; + }, + ], + [ + "new mutation key on retry", + (r) => { + if (r.retry) r.retry.retryMutationKeySha256 = "9".repeat(64); + }, + ], + [ + "changed payload on retry", + (r) => { + if (r.retry) r.retry.retryPayloadSha256 = "9".repeat(64); + }, + ], + [ + "extra accepted retry message", + (r) => { + if (r.retry) r.retry.afterCount += 1; + }, + ], + [ + "new retry receipt identity", + (r) => { + if (r.retry) r.retry.retryMessageId = "unrelated-message-0001"; + }, + ], + [ + "unreported duplicate acceptance", + (r) => { + if (r.duplicateDelivery) + r.duplicateDelivery.acceptedAttemptIds.push( + "unobserved-attempt-0001", + ); + }, + ], + [ + "invented exactly-once duplicate behavior", + (r) => { + if (r.duplicateDelivery) + r.duplicateDelivery.behavior = "no-additional-run"; + }, + ], + [ + "post-terminal delivery authorization", + (r) => { + if (r.cancellation) + r.cancellation.attempts[0].authorizedAt = "2026-11-12T00:00:51Z"; + }, + ], + [ + "cancellation without pending work", + (r) => { + if (r.cancellation) r.cancellation.pendingAt = "2026-11-12T00:01:00Z"; + }, + ], + [ + "truncated post-terminal coverage", + (r) => { + if (r.cancellation) + r.cancellation.coverageThrough = "2026-11-12T00:00:49Z"; + }, + ], + [ + "fresh-storage replacement", + (r) => { + if (r.restart) r.restart.mode = "fresh-storage"; + }, + ], + [ + "retry budget reset", + (r) => { + if (r.restart) r.restart.after.retryStateSha256 = "9".repeat(64); + }, + ], + [ + "replacement active subscription", + (r) => { + if (r.restart) + r.restart.after.subscriptionIds[0] = "replacement-subscription"; + }, + ], + [ + "replacement table after restart", + (r) => { + if (r.restart) r.restart.tableId = "replacement-table-0001"; + }, + ], + [ + "terminal dispatch resurrection", + (r) => { + if (r.restart) r.restart.terminalDispatchCount = 1; + }, + ], + [ + "missing fresh real-host resumed turn", + (r) => { + r.turns = r.turns.filter((item) => item.scenario !== "restart"); + }, + ], + [ + "third guest transcript leak", + (r) => { + r.isolation[1].privateDataReturned = true; + }, + ], + [ + "third guest modifying subscription", + (r) => { + r.isolation[3].subscriptionChanged = true; + }, + ], + [ + "missing other owner's unsubscribe denial", + (r) => { + r.isolation.pop(); + }, + ], + [ + "member reused as unaffiliated guest", + (r) => { + r.isolation[0].outsiderGuestId = r.participants[0].guestId as string; + }, + ], + [ + "unapproved content crossing relay", + (r) => { + if (r.disclosure) r.disclosure.postedPayloadsSha256 = "9".repeat(64); + }, + ], + [ + "missing named external receipt", + (r) => { + r.observations.pop(); + }, + ], + [ + "missing same-run host reply tool receipt", + (r) => { + r.observations = r.observations.filter( + (item) => item.id !== r.turns[0].reply?.hostObservationRef, + ); + }, + ], + [ + "missing actual initiating agent post tool receipt", + (r) => { + r.observations = r.observations.filter( + (item) => item.id !== r.turns[0].post.hostObservationRef, + ); + }, + ], + [ + "server receipt substituted for host reply tool trace", + (r) => { + const observation = r.observations.find( + (item) => item.id === r.turns[0].reply?.hostObservationRef, + ); + if (observation) observation.source = "server"; + }, + ], + [ + "host reply tool trace predating committed reply", + (r) => { + const observation = r.observations.find( + (item) => item.id === r.turns[0].reply?.hostObservationRef, + ); + if (observation) observation.capturedAt = "2026-11-12T00:00:15Z"; + }, + ], + [ + "duplicate receipt identifier", + (r) => { + r.observations.push({ ...r.observations[0] }); + }, + ], + [ + "ambiguous undisclosed build", + (r) => { + r.participants[0].host.exposedBuild = "not-exposed-unspecified"; + }, + ], + [ + "idle observed only after the triggering post", + (r) => { + if (r.turns[0].idle) + r.turns[0].idle.observedAt = "2026-11-12T00:00:11Z"; + }, + ], + [ + "report predating its evidence", + (r) => { + r.observedAt = "2026-11-12T00:00:01Z"; + }, + ], + [ + "host run receipt captured before the run", + (r) => { + const observation = r.observations.find( + (item) => item.id === r.turns[0].run?.observationRef, + ); + if (observation) observation.capturedAt = "2026-11-12T00:00:01Z"; + }, + ], + [ + "unrelated cancellation ledger event", + (r) => { + if (r.cancellation) + r.cancellation.attempts[0].eventId = "unrelated-event-0001"; + }, + ], + [ + "exceeded independently approved reply bound", + (r) => { + r.participants[1].consent.maxReplies = 1; + }, + ], + [ + "reused event across resumed host turns", + (r) => { + r.turns[2].event.eventId = r.turns[0].event.eventId; + r.turns[2].deliveries[0].eventId = r.turns[0].event.eventId; + if (r.turns[2].run) + r.turns[2].run.triggeringEventId = r.turns[0].event.eventId; + if (r.turns[2].read) r.turns[2].read.eventId = r.turns[0].event.eventId; + }, + ], + [ + "reused host run after restart", + (r) => { + if ( + r.turns[2].run && + r.turns[2].read && + r.turns[2].reply && + r.turns[0].run + ) { + r.turns[2].run.runId = r.turns[0].run.runId; + r.turns[2].read.runId = r.turns[0].run.runId; + r.turns[2].reply.runId = r.turns[0].run.runId; + } + }, + ], + [ + "reused delivery attempt after restart", + (r) => { + r.turns[2].deliveries[0].attemptId = r.turns[0].deliveries[0].attemptId; + if (r.turns[2].run) + r.turns[2].run.deliveryAttemptId = r.turns[0].deliveries[0].attemptId; + }, + ], + [ + "same accepted message relabeled as fresh after restart", + (r) => { + r.turns[2].post.messageId = r.turns[0].post.messageId; + r.turns[2].event.messageId = r.turns[0].post.messageId; + if (r.turns[2].reply) + r.turns[2].reply.replyToMessageId = r.turns[0].post.messageId; + }, + ], + [ + "prior sequence committed only after the restarted peer post", + (r) => { + if (r.turns[1].reply) + r.turns[1].reply.committedAt = "2026-11-12T00:03:20Z"; + if (r.retry) { + r.retry.beforeCount = 2; + r.retry.afterCount = 2; + } + }, + ], + [ + "same sender mutation key reused for new post", + (r) => { + r.turns[2].post.mutationKeySha256 = r.turns[0].post.mutationKeySha256; + }, + ], + [ + "second unvalidated restart turn", + (r) => { + r.turns.push({ ...r.turns[2], id: "extra-restart-turn-0001" }); + }, + ], + ]; + it.each(mutations)("rejects %s", (_name, mutate) => { + const record = realHostShapedFixture(); + mutate(record); + const result = evaluateAutonomousEvidence(record); + expect(result.completeRecord).toBe(false); + expect(result.missing.length).toBeGreaterThan(0); + }); + + it("separates pre-decision dispatched traffic from forbidden new authorization", () => { + const record = realHostShapedFixture(); + expect(record.cancellation?.attempts[0].completedAt).toBe( + "2026-11-12T00:00:55.000Z", + ); + expect(evaluateAutonomousEvidence(record).scenarios.cancellation).toBe( + "passed", + ); + }); + + it("distinguishes failed observed behavior from missing blocked observations", () => { + const manual = realHostShapedFixture(); + if (manual.turns[0].run) manual.turns[0].run.trigger = "manual"; + expect(evaluateAutonomousEvidence(manual).scenarios.eventResponse).toBe( + "failed", + ); + const missing = realHostShapedFixture(); + missing.turns[0].run = null; + expect(evaluateAutonomousEvidence(missing).scenarios.eventResponse).toBe( + "blocked", + ); + const leaked = realHostShapedFixture(); + leaked.isolation[0].outcome = "allowed"; + expect(evaluateAutonomousEvidence(leaked).scenarios.isolation).toBe( + "failed", + ); + }); + + it("records duplicated delivery with additional observed host runs honestly", () => { + const record = realHostShapedFixture(); + const first = record.turns[0]; + first.deliveries.push({ + ...first.deliveries[0], + attemptId: "duplicate-attempt-0002", + }); + const duplicates = record.duplicateDelivery as NonNullable< + typeof record.duplicateDelivery + >; + duplicates.acceptedAttemptIds.push("duplicate-attempt-0002"); + duplicates.observedRunIds.push("duplicate-host-run-0002"); + duplicates.acceptedReplyMessageIds.push("duplicate-reply-0002"); + duplicates.behavior = "additional-run"; + expect(evaluateAutonomousEvidence(record).scenarios.duplicateDelivery).toBe( + "passed", + ); + }); + + it("rejects duplicate behavior asserted before the duplicate was accepted", () => { + const record = realHostShapedFixture(); + const first = record.turns[0]; + first.deliveries.push({ + ...first.deliveries[0], + attemptId: "late-duplicate-attempt", + authorizedAt: "2026-11-12T00:03:18Z", + dispatchedAt: "2026-11-12T00:03:19Z", + acceptedAt: "2026-11-12T00:03:20Z", + }); + const duplicates = record.duplicateDelivery as NonNullable< + typeof record.duplicateDelivery + >; + duplicates.acceptedAttemptIds.push("late-duplicate-attempt"); + duplicates.behavior = "no-additional-run"; + const result = evaluateAutonomousEvidence(record); + expect(result.completeRecord).toBe(false); + expect(result.scenarios.duplicateDelivery).toBe("failed"); + expect(result.missing).toContain( + "duplicateDelivery: duplicate observation window covers every included receiver acceptance", + ); + }); + + it("rejects secret-bearing unknown keys without echoing keys, values or URLs", () => { + const sentinel = "SECRET_DO_NOT_ECHO_123456789"; + const input = { + ...syntheticFixture(), + [sentinel]: `https://private.example/${sentinel}`, + }; + const result = evaluateAutonomousEvidence(input); + expect(result.structurallyValid).toBe(false); + expect(JSON.stringify(result)).not.toContain(sentinel); + const nested = syntheticFixture(); + Object.assign(nested.turns[0].deliveries[0], { + callbackUrl: `https://private.example/${sentinel}`, + }); + expect(evaluateAutonomousEvidence(nested).structurallyValid).toBe(false); + }); +}); + +describe("explicit A-only autonomous acceptance", () => { + it("selects modes explicitly in both unrun templates", () => { + expect(template.acceptanceMode).toBe("bidirectional"); + expect(selectedChatPair.acceptanceMode).toBe("a-only"); + expect(selectedChatPair.participants[0].host.product).toBe( + "OpenAI ChatGPT", + ); + expect(selectedChatPair.participants[1].host.product).toBe("Claude Chat"); + expect(evaluateAutonomousEvidence(selectedChatPair).acceptanceMode).toBe( + "a-only", + ); + expect(evaluateAutonomousEvidence(selectedChatPair).completeRecord).toBe( + false, + ); + }); + + it("accepts A event responses while B has tools-only manual source capability", () => { + // Invented metadata exercises the positive branch, not actual host evidence. + const record = realHostShapedFixture("a-only"); + expect(record.participants[1].host.runObservation).toBe("unsupported"); + expect(record.participants[1].subscriptionId).toBeNull(); + expect(record.participants[1].subscription.activeAt).toBeNull(); + expect(record.turns).toHaveLength(2); + for (const turn of record.turns) { + expect(turn.sender).toBe("B"); + expect(turn.recipient).toBe("A"); + expect(turn.run?.trigger).toBe("peer-event"); + } + const result = evaluateAutonomousEvidence(record); + expect(result.missing).toEqual([]); + expect(result.completeRecord).toBe(true); + expect(result.acceptanceMode).toBe("a-only"); + expect(result.scenarios.eventResponse).toBe("passed"); + expect(result.scenarios).not.toHaveProperty("roundTrip"); + expect(result.layers).toEqual({ + tools: "passed", + notificationReceipt: "passed", + autonomousReply: "passed", + }); + }); + + it("never presents an A-only synthetic fixture as actual compatibility", () => { + const result = evaluateAutonomousEvidence(syntheticFixture("a-only")); + expect(result.acceptanceMode).toBe("a-only"); + expect(result.completeRecord).toBe(false); + expect(result.layers.autonomousReply).toBe("blocked"); + }); + + it("rejects a missing mode rather than silently falling back", () => { + const record = realHostShapedFixture("a-only"); + Reflect.deleteProperty(record, "acceptanceMode"); + const result = evaluateAutonomousEvidence(record); + expect(result.structurallyValid).toBe(false); + expect(result.acceptanceMode).toBeNull(); + expect(result.completeRecord).toBe(false); + }); + + it("cannot promote A-only evidence to bidirectional acceptance", () => { + const record = realHostShapedFixture("a-only"); + record.acceptanceMode = "bidirectional"; + const result = evaluateAutonomousEvidence(record); + expect(result.acceptanceMode).toBe("bidirectional"); + expect(result.completeRecord).toBe(false); + expect(result.scenarios.eventResponse).not.toBe("passed"); + expect(result.scenarios.restart).not.toBe("passed"); + }); + + const mutations: [string, (record: AutonomousEvidence) => void][] = [ + [ + "A manually woken for the primary response", + (r) => { + if (r.turns[0].run) r.turns[0].run.trigger = "manual"; + }, + ], + [ + "A manually woken after restart", + (r) => { + if (r.turns[1].run) r.turns[1].run.trigger = "manual"; + }, + ], + [ + "primary response in the wrong direction", + (r) => { + r.turns[0].sender = "A"; + r.turns[0].recipient = "B"; + }, + ], + [ + "restart response in the wrong direction", + (r) => { + r.turns[1].sender = "A"; + r.turns[1].recipient = "B"; + }, + ], + [ + "a second primary response disguised as A-only", + (r) => { + r.turns.push({ ...r.turns[0], id: "extra-response-0001" }); + }, + ], + [ + "missing A active subscription", + (r) => { + r.participants[0].subscriptionId = null; + r.participants[0].subscription = { + tableId: null, + activeAt: null, + observationRef: null, + }; + }, + ], + [ + "unsupported A idle/run observation", + (r) => { + r.participants[0].host.runObservation = "unsupported"; + }, + ], + [ + "missing retained-state restart", + (r) => { + r.restart = null; + }, + ], + [ + "missing post-restart A response", + (r) => { + r.turns = r.turns.filter((item) => item.scenario !== "restart"); + }, + ], + [ + "another receiver subscription in restart snapshot", + (r) => { + if (r.restart) + r.restart.after.subscriptionIds.push("unexpected-subscription"); + }, + ], + [ + "replacement A subscription after restart", + (r) => { + if (r.restart) + r.restart.after.subscriptionIds[0] = "replacement-subscription"; + }, + ], + [ + "cancellation targeting B", + (r) => { + if (r.cancellation) { + r.cancellation.recipient = "B"; + r.cancellation.recipientGuestId = r.participants[1].guestId as string; + } + }, + ], + [ + "A cancellation labeled with B guest identity", + (r) => { + if (r.cancellation) + r.cancellation.recipientGuestId = r.participants[1].guestId as string; + }, + ], + [ + "unverified B source host account", + (r) => { + r.participants[1].host.eligibility = "unknown"; + }, + ], + [ + "SDK replacement for the B source host", + (r) => { + r.participants[1].host.origin = "sdk-only"; + }, + ], + [ + "missing B OAuth/tool receipt", + (r) => { + r.participants[1].oauthEvidenceRef = null; + }, + ], + [ + "missing independent B bounded consent", + (r) => { + r.participants[1].consent.observationRef = null; + }, + ], + [ + "B consent only after its first post", + (r) => { + r.participants[1].consent.approvedAt = "2026-11-12T00:00:11Z"; + }, + ], + [ + "missing actual B source-post tool receipt", + (r) => { + r.observations = r.observations.filter( + (item) => item.id !== r.turns[0].post.hostObservationRef, + ); + }, + ], + [ + "missing private transcript denial for B membership", + (r) => { + r.isolation = r.isolation.filter( + (item) => + !( + item.action === "transcript" && + item.targetMembershipId === r.participants[1].membershipId + ), + ); + }, + ], + [ + "missing A subscription-maintenance denial", + (r) => { + r.isolation = r.isolation.filter( + (item) => item.action !== "unsubscribe", + ); + }, + ], + [ + "A exceeding its independently approved reply bound", + (r) => { + r.participants[0].consent.maxReplies = 1; + }, + ], + ]; + it.each(mutations)("rejects %s", (_name, mutate) => { + const record = realHostShapedFixture("a-only"); + mutate(record); + const result = evaluateAutonomousEvidence(record); + expect(result.completeRecord).toBe(false); + expect(result.acceptanceMode).toBe("a-only"); + expect(result.missing.length).toBeGreaterThan(0); + }); +});