Skip to content

[Conformance] ACS-Core over the wire: the reference Guardian accepts unsigned, replayed and stale envelopes and publishes no chain head #184

Open

Description

@astrogilda

[Conformance] ACS-Core over the wire: the reference Guardian accepts unsigned, replayed and stale envelopes, publishes no chain head, and answers system/ping, four minimum hooks and wrapped MCP with method-not-dispatched

Filed against #92. Every row below is a single reproduction against the reference Guardian, and every request and response pair is held as a capture file.

What you ran

what value
repository GenAI-Security-Project/agent-control-standard at bfdb898be4a9bcaedd90529b480ae5b62e94f29a, main, the merge of #105 on 2026-09-10
tree under test reference-implementations/agt/
worktree clean throughout, with no uncommitted change on any run
host Linux 7.0.0-31-generic x86_64
Bun 1.3.13, where CI pins 1.3.11. The shim behaviour #111 reports under 1.4.2 did not occur here
other runtimes Node v24.19.0, Python 3.14.3 for the adapter, jq 1.8, trash-cli 7.2.0 installed the way the workflow does
AGT SDK agent-control-specification@0.3.1-beta.0 from the lockfile; OPA from the bundled linux-x64 binary; no OPA path override needed
Guardian ACS_GUARDIAN_PORT=18787 bun run guardian, posture proceed, the default, with policy/manifest.yaml, mapping.yaml and policy/lib unmodified
port 8787 was already held by an unrelated local HTTP service on this workstation, so the Guardian moved to 18787. OBS-1 records what the shim did before the port was moved
full pin list the capture 00-environment.txt

Two harnesses ran, the control first and the adapter second.

  1. The tree's own, as a positive control. bun test over the whole suite, then the README's "Drive one hook by hand" payloads, rm -rf / and ls -la, through the shipped hook at hosts/claude-code/acs-hook.ts.

  2. An independent host adapter, acs_vector_adapter.py, which imports nothing from the tree and runs on the standard library alone. It builds envelopes from the published request-envelope and tool-call-request schemas. It signs them per Specification section 10: JCS of the envelope with the signature removed, HMAC-SHA256 under an HKDF session key, with the IKM recorded in the results file. It performs the handshake once per session, posts to the Guardian, and records every request and response pair.

the adapter drove detail
the ACS-Core negative-conformance corpus 28 members at probityai/agent-evidence-vectors/vectors-acs-core, corpus digest 7eeedf34, read at ACS 9d4a9da. 16 members carry a wire request; 12 are framework-, approver- or deployment-role and carry none
fourteen named probes one per ACS-Core obligation the corpus cannot express as a single request
run 1, vectors as written kept because its uniform deny evaluation_failed is itself a finding, D10
run 2, tools mapped the corpus's abstract records.* tools rewritten to Bash and ls so policy content cannot mask a wire-level verdict. This is the run scored below

What you expected

The conformance page, under "ACS-Core (mandatory baseline)", says a v0.1.0-conformant deployment MUST implement the handshake, the envelope, the six-hook minimum taxonomy, all five dispositions, SessionContext with the Guardian publishing the chain head on responses, replay protection, the baseline HMAC-SHA256 signature, decision honoring, system/ping, and wrapped MCP. The tree's README claims exactly one profile, acs-core, calls the claim "qualified", and carries a status table that already concedes several rows. This report measures each row on the wire. It also adds what the table does not list.

Per row, the sentence that set the expectation:

Section 10: "A signature over the section 10 canonical input is REQUIRED in ACS-Core ... A verifier MUST recompute this canonical form and MUST reject a signature that does not cover it (SIGNATURE_INVALID, 17.1)."
Section 10.3: "Guardians MUST reject requests whose timestamp is more than the negotiated skew window ... returning TIMESTAMP_OUT_OF_WINDOW (-32006), MUST reject duplicate request_id values within the session with REPLAY_DETECTED (-32005), and SHOULD reject duplicate nonce values."
Section 8.6: "For every step where the Guardian writes a ContextEntry ... the Guardian MUST include the resulting chain_hash in its response, and that chain_hash MUST be covered by the response signature."
Section 13: "Guardians MUST always return decision: "allow" for system/ping regardless of policy ... system/ping MUST NOT return an ACS-specific error."
Section 4: "Version mismatch terminates with UNSUPPORTED_VERSION (-32001)." conformance.md: "ClientHello includes profiles_supported; ServerHello includes profiles_accepted." Section 10.3: "skew_window_ms in ServerHello."
conformance.md hook taxonomy: "At minimum: sessionStart, userMessage or agentTrigger, toolCallRequest, toolCallResult, agentResponse, sessionEnd."
conformance.md: "Wrapped MCP: protocols/MCP/*."
README "Verify": "bun test: the whole suite; exactly one test skips."

What happened

Positive control

step result capture
bun install 82 packages, exit 0 00-bun-install.log
bun test (fresh clone, no trash) 1084 pass, 1 skip, 26 fail across 1111 tests; every failure is bun: command not found: trash from trashDir() in packages/conformance/test/*.test.ts 01-bun-test.log
bun add -g trash-cli@7.2.0, then bun test packages/conformance 125 pass, 7 fail, all "a beforeEach/afterEach hook timed out" (5000 ms default); trash takes ~1.6 s per directory on this host and the after-hooks trash several 02b-bun-test-conformance-with-trash.log
same, --timeout 30000 132 pass, 0 fail 02c-bun-test-conformance-timeout30s.log
Guardian banner the README's four lines exactly, on 18787 03e-guardian-stdout-port18787.log
shim, rm -rf / {"hookSpecificOutput":{...,"permissionDecision":"deny","permissionDecisionReason":"This command was blocked because it matches a destructive-shell-command pattern. Policy: destructive_shell_command_blocked, from AGT's stock bundle (agt_stock). Matched at offset 0."}}, exit 0, 4 envelopes, 1 chain entry 03f-positive-control-hook-deny.log, 03h-positive-control-envelopes.jsonl
shim, ls -la {"hookSpecificOutput":{"hookEventName":"PreToolUse","permissionDecision":"allow"}}, exit 0 03g-positive-control-hook-allow.log
adapter P0a/P0b (same two commands through the independent envelope builder) deny with reason_codes: ["destructive_shell_command_blocked"]; allow 05-adapter-run-toolmap-Bash/wire/*-P0a-deny.json, *-P0b-allow.json

So the tree does what its README says it does, byte for byte, and the adapter reproduces it without the tree's code. Everything below is measured against that baseline. Across the 350 envelopes the Guardian logged for the whole session, its stderr carried zero schema-failure lines: every response it sent was schema-valid.

Probes (Guardian obligations, one reproduction each)

Wire files are captured one per probe under the run directory; each holds the exact request body and the exact response.

# probe spec expected observed verdict
P1 request with no signature 10, 17.1 -32004 SIGNATURE_INVALID decision: allow FAIL
P2 HMAC with every byte inverted 10 -32004 decision: allow FAIL
P3 same request_id twice in one session 10.3 -32005 REPLAY_DETECTED on the second decision: allow, twice FAIL
P4 timestamp 3600 s in the past 10.3 -32006 TIMESTAMP_OUT_OF_WINDOW decision: allow FAIL
P5 timestamp 3600 s in the future 10.3 -32006 decision: allow FAIL
P6 chain_hash and signature on a content-bearing response 8.6 both present neither present (the chain entry is written to .acs/session-context.jsonl) FAIL
P7 client metadata.session_state.chain_hash = f x 64 8.6 (MAY) chain_mismatch deny or -32007 decision: allow, no reason code observation, MAY
P8 system/ping 13 decision: allow -32011 method not dispatched by this Guardian: system/ping FAIL
P9 ClientHello with acs_version / supported_versions = 9.9.9 4 -32001 UNSUPPORTED_VERSION ServerHello negotiated_version: 0.1.0 FAIL
P10 ServerHello fields conformance.md, 10.3 profiles_accepted, skew_window_ms keys: methods_evaluated, negotiated_version, on_decision_failure, selected_transport, timeout_config FAIL
P11 steps/sessionStart conformance.md taxonomy a decision -32011 FAIL
P12 protocols/MCP/tools/call conformance.md wrapped MCP a decision -32011 FAIL
P13 same nonce twice 10.3 (SHOULD) -32005 allow observation, SHOULD
P14 body > 1 MiB tree's own cap -32010 -32010 PASS (project choice, not spec)

Corpus, vectors-acs-core with tools mapped to Bash

status n members
PASS 5 v07180a3 R-007 accept, v3e946bc R-002 accept, va6d6e95 R-003 accept (-60 s, inside the window), vf3679e6 R-013 accept, vff6b4d4 R-011 accept
FAIL 9 vb87b86b and vd67cd20 R-001 (invalid / subset-coverage signature, allowed), vef655ce R-002 (replay, allowed), vb10df32 R-003 (-3600 s, allowed), v25f01d8 R-006 (no chain_hash, no signature on the response), v100f749 R-011, v661266c and vfcab242 R-012/R-013, vc333473 R-007 (intent, approval and approver identity ride beside the request in the corpus and have no wire field on steps/toolCallRequest; the Guardian never receives them and allows)
NOT_DRIVEN 10 framework/agent/deployment-role members with no wire request or whose property is the Observed Agent's behaviour (R-004, R-005 x2, R-008, R-009, R-010, R-013 x2 budget, R-014, R-015); each was still sent where it had a request and the Guardian's answer is on file
NOT_APPLICABLE 2 R-016 x2: acs-audit is not claimed
UNMEASURABLE 2 v82b6d11 (revocation, unmeasurable by the corpus's own declaration); v7b0b32f (signature-covers-chain_hash cannot be tested when neither field is on the wire)

Run as written, with the corpus tools records.lookup and records.read and the argument id, every one of the 16 wire members came back deny with the reason code evaluation_failed and a message saying that mapping.yaml reads that tool's policy target from an argument the call did not send. See D10.

Defect table

Severity is against the ACS-Core MUST list. "One-line fix" names the function and the change; each is small because the surrounding machinery already exists (the chain is computed, the session store is keyed by session, validateEnvelope is total, errorResponse takes a code).

id what where reproduction one-line fix sev
D1 No signature verification. An unsigned envelope and one whose HMAC is byte-inverted both receive a policy decision. The schema keeps params.signature optional and nothing in the Guardian reads it. packages/guardian/src/validate-envelope.ts (signature not required), server.ts dispatch (no verify step); handshake.ts negotiates no key P1 wire/*-P1-unsigned.json; P2 wire/*-P2-badsig.json; corpus vb87b86b, vd67cd20 In dispatch, after validateEnvelope and before the method switch: verifySignature(envelope, sessionKey(session_id)) over jcs(envelopeWithoutSignature), else errorResponse(rpcId, -32004, "SIGNATURE_INVALID"); key from HKDF(deployment IKM env var, session_id). MUST
D2 No replay protection. The same request_id is accepted twice in one session; the same nonce too. server.ts evaluateStep -> appendContextEntry; session-context-store.ts already holds every request_id per session P3 wire/*-P3-replay.json; P13; corpus vef655ce In appendContextEntry (or just before it), if (session.entries.some(e => e.request_id === id)) throw new ReplayError() mapped to -32005. MUST
D3 No timestamp window. +/-3600 s is accepted; the ServerHello never declares skew_window_ms. server.ts dispatch; handshake.ts buildServerHello P4, P5; corpus vb10df32 if (Math.abs(Date.now() - Date.parse(params.timestamp)) > SKEW_MS) return errorResponse(rpcId, -32006, ..., {skew_window_ms: SKEW_MS}); add skew_window_ms: 300000 to the ServerHello. MUST
D4 Chain head not published. The chain entry is computed and written to disk, then dropped; the response carries no chain_hash and no signature, so 8.6 and the response half of 10 are both unmet. server.ts evaluateStep (ignores appendContextEntry's return), acs-result.ts finalResult P6 wire/*-P6-chain.json; the Guardian's own session-context.jsonl has the hash the response omits const entry = appendContextEntry(...) and finalResult(envelope.params, decision, entry.hash) setting chain_hash; then sign the result envelope with the same session key as D1. MUST
D5 system/ping is refused with an ACS-specific error, the one thing section 13 says it MUST NOT do. #129 carries this defect on its own. server.ts dispatch, falls through to METHOD_NOT_DISPATCHED_CODE P8 wire/*-P8-ping.json Before the isToolCallRequest branch: if (envelope.method === "system/ping") return successResponse(envelope.id, {type:"final", acs_version, request_id, decision:"allow", payload:{status:"ok", echo: payload.echo, server_timestamp: new Date().toISOString()}}), and skip the chain entry. MUST
D6 Handshake is not negotiated. A ClientHello declaring only 9.9.9 gets negotiated_version: 0.1.0; profiles_supported is never read and profiles_accepted never sent. The README states the ClientHello is not read; the consequence is that the version and profile rules in section 4 and conformance.md are unenforced. handshake.ts buildServerHello (takes no ClientHello) P9 wire/*-P9-version.json; P10 buildServerHello(clientHello, env): intersect supported_versions with ["0.1.0"], else -32001 with data.supported_versions; echo profiles_accepted: clientHello.profiles_supported intersected with ["acs-core"]. MUST
D7 Four of the six minimum hooks are not dispatched. steps/sessionStart (and by the same code path userMessage, agentResponse, sessionEnd) return -32011. mapping.yaml already maps them to AGT points. server.ts dispatch (two predicates, then -32011) P11 wire/*-P11-sessionStart.json Add a third branch for the four mapped-not-evaluated methods that appends the chain entry and returns allow (the ServerHello already tells the client they are allow-by-default; answering them keeps the chain complete). MUST
D8 Wrapped MCP not dispatched. same P12 Route protocols/MCP/tools/call through the toolCallRequest assembler with tool.name from the MCP payload. MUST
D9 Test suite needs two things the README "Verify" step does not say. (a) Without trash on PATH, 26 tests fail; README "Install" lists trash only for scripts/. (b) With trash-cli 7.2.0 installed, 7 tests in packages/conformance/test/{upstream-watch,surfaces}.test.ts still fail because their afterEach trashes several directories at ~1.6 s each against Bun's 5000 ms hook default; bun test --timeout 30000 is green. The workflow comment says the undeclared dependency "is tracked as an issue". README.md Install/Verify; packages/conformance/test/*.test.ts trashDir; package.json scripts.test 01-bun-test.log, 02b-...log, 02c-...log README: bun add -g trash-cli@7.2.0 under Install and bun test --timeout 30000 under Verify; or "test": "bun test --timeout 30000" in package.json. (#110 reports the Windows half of the same class.) docs
D10 An unregistered tool is denied by a mapping message, not a policy. Any steps/toolCallRequest whose tool is not in policy_target_argument.by_tool falls to default: command; a call with no command argument is denied evaluation_failed with a message about mapping.yaml. Fail-closed is the right posture, but the reason code says the Guardian broke, when what happened is that the deployment governs no such tool. Every corpus member as written hit this. packages/guardian/src/map-verdict.ts resolvePolicyTargetArgument; server.ts evaluateStep catch wire/*-v3e946bcfde26bbe2-step1.json, run as written In the catch, or before assembly: if the tool is neither in by_tool nor in the manifest's tools:, deny with reason_codes: ["tool_unregistered"] and a reason naming the tool, leaving evaluation_failed for real throws. low

Observations that are not defects against the text:

OBS-1, a foreign endpoint at the Guardian URL. The README default port 8787 was held by an unrelated local HTTP service, so the Guardian exited EADDRINUSE. That is correct. The shim, pointed at 8787, returned permissionDecision allow and exit 0. Its audit entry classified the failure kind as transport, in the capture 03d-audit-foreign-endpoint-on-8787.jsonl.

Section 6.4 is satisfied, since the proceed is audited. One line is worth adding to the host README: a parsed-but-non-envelope body is classified as transport, so the audit log cannot separate a Guardian that is down from something else answering on that port. 8787 is a common default for local proxies.

OBS-2. P7 and P13 are MAY and SHOULD and are listed for completeness only.

OBS-3, the corpus's own gap. Nine corpus members carry Intent, an approval, or an approver response beside the request. steps/toolCallRequest has no field for any of them, so a Guardian that receives only the wire cannot be measured on R-007/R-011/R-012/R-013 by a host adapter; those rows in the FAIL bucket are a statement about the wire format as much as about this Guardian. That is recorded here so the count is not read as nine independent Guardian defects.

Where behaviour and the tree's own table agree, and where they do not

The README's ACS-Core status table already concedes replay, signature, ping, wrapped MCP, the four hooks and the chain head. This run confirms each concession on the wire and adds three the table does not carry: the handshake accepts any version (D6), system/ping is answered with an ACS-specific error where the specification says it MUST NOT be (D5), and the test suite's own verify step is not reproducible from the README (D9). D1 to D8 are one tree; D2, D3, D5 and D7 are each an afternoon because the state they need already exists.

Current Priority Scope

Feeds the runnable Guardian reference implementation (D1 to D8, D10) and feeds conformance evidence (the adapter, the probe set and the corpus run are reusable against any Guardian). D9 is documentation.

Activity

  1. added
    scope:in-focusFeeds the ninety-day committed outcome. Maintainers only
    status:acceptedIn the backlog. Maintainers only
    and removed
    status:needs-triageNot yet triaged. Applied by the issue forms
    on Oct 5, 2026
  2. jrcrittenden commented on Oct 6, 2026

    @jrcrittenden

    A partial implementation is now available in #229.

    It addresses the concrete D1-D10 wire probes in this report, including authenticated request and response envelopes, stale/replay rejection, published chain heads, system/ping, handshake negotiation fields, the six-hook minimum, wrapped MCP tools/call, and explicit tool_unregistered decisions. The PR also adds adversarial and integration coverage and updates the adapter documentation.

    This is not a claim that the full issue is resolved. Known follow-up work includes persisting replay state across Guardian restarts, enforcing negotiated session capabilities on subsequent calls, and preserving chain continuity when the bounded session-context store evicts a session.

    The change has passed local repository guards, the strict documentation build, AGT typechecking, and the focused changed-area tests. It has not yet been externally reviewed by a maintainer or independent reviewer, so the implementation and its conformance claims should be treated as proposed pending review. This issue should remain open while #229 is reviewed and the remaining gaps are assessed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    scope:in-focusFeeds the ninety-day committed outcome. Maintainers onlystatus:acceptedIn the backlog. Maintainers onlytype:conformanceConformance or dogfooding report

    Type

    No type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions