diff --git a/docs/identity/overview.md b/docs/identity/overview.md index a95ba772..38f6e864 100644 --- a/docs/identity/overview.md +++ b/docs/identity/overview.md @@ -121,7 +121,7 @@ A concrete example that illustrates several of these challenges is Palo Alto Net | 1 | **Chain Integrity**
*Can the full delegation chain be named and verified end-to-end?* | A single agent task can chain through five to ten identities: user → orchestrator → sub-agent → tool → downstream API. Two failures compound: the chain is not *named* (audit collapses into "the agent did it"), and the chain is not *verifiable* (downstream resources cannot confirm the chain without trusting every intermediary). In practice the full chain is often not reconstructable end-to-end, and there is a known class of attacks where the chain itself can be spliced. | **[OAuth 2.0](https://www.rfc-editor.org/rfc/rfc6749) access tokens** identify the bearer, not the chain.
**[OIDC Core 1.0](https://openid.net/specs/openid-connect-core-1_0.html) ID tokens** name the human at the start but lose granularity at each hop.
**[SPIFFE SVIDs](https://spiffe.io/docs/latest/spiffe-about/spiffe-concepts/)** identify the workload but not which call within it.
**[OAuth Token Exchange (RFC 8693)](https://www.rfc-editor.org/rfc/rfc8693)** defines nested `act` claims that can express a chain, but per [§4.1](https://www.rfc-editor.org/rfc/rfc8693#section-4.1) *"prior actors identified by any nested `act` claims are informational only and are not to be considered in access control decisions."* The chain is recorded, not normatively enforceable.
**[OAuth Identity Chaining draft](https://datatracker.ietf.org/doc/draft-ietf-oauth-identity-chaining/)** and **[Transaction Tokens draft](https://datatracker.ietf.org/doc/draft-ietf-oauth-transaction-tokens/)** address parts but specify neither chain verification rules nor revocation propagation as required runtime behavior.
**[W3C Trace Context](https://www.w3.org/TR/trace-context/)** correlates but [does not authenticate](https://www.w3.org/TR/trace-context/#privacy-of-traceparent-field). | Every action must produce, at the moment of execution, a signed record that names the specific link in the chain and lets any participant (including the downstream resource) verify the entire chain end-to-end without trusting any intermediary. Cryptographic chain structure, mandatory scope attenuation at every hop, revocation propagation. | Pending | | 2 | **Over-Privilege**
*Was this action within the agent's purpose?* | Agents inherit broad standing privileges. Service accounts. Long-lived API keys. OAuth scopes designed for human applications. One compromised agent has the union of every permission anyone has ever granted to it. The fix is not bigger scopes; the fix is no standing privilege. | **[OAuth scopes (RFC 6749 §3.3)](https://www.rfc-editor.org/rfc/rfc6749#section-3.3)** are coarse, deployment-defined strings (e.g. `read`, `write`, `repo`). Per **[RFC 9396 §1](https://www.rfc-editor.org/rfc/rfc9396#section-1)**, scope is "sufficient... for coarse-grained authorization requests" but "not sufficient to specify fine-grained authorization requirements" — it does not express per-action, per-resource, per-data-cell intent.
**RBAC and ABAC** scope by role or attribute, granted once, evaluated coarsely.
**AIMS** does not require **[RAR (RFC 9396)](https://www.rfc-editor.org/rfc/rfc9396)** and treats scope semantics as deployment-specific. | Credentials minted just before the call, scoped to one downstream callee, structured (per-action, per-resource), expired in minutes. Authorization evaluated per-call against the immutable intent for the run, not per-session. | Pending | | 3 | **Token Theft Resistance**
*If a token leaks from agent memory, is it useless to the attacker?* | The agent threat model breaks OAuth's bearer-token assumption entirely. Prompt injection can exfiltrate tokens from memory. Tool outputs can leak tokens. Sub-agents can replay tokens. Logs and crash dumps capture tokens. Long-lived refresh tokens in agent memory are the worst case: a durable credential held in storage that any indirect prompt injection can read. | **[OAuth 2.0 Bearer Tokens (RFC 6750 §1.2)](https://www.rfc-editor.org/rfc/rfc6750#section-1.2)** explicitly treat possession as sufficient: *"Using a bearer token does not require a bearer to prove possession of cryptographic key material."*
**API keys** are bearer secrets by definition.
**[OAuth refresh tokens (RFC 6749 §1.5)](https://www.rfc-editor.org/rfc/rfc6749#section-1.5)** were designed for the same user returning later to resume a session, which does not describe how agents work.
**[DPoP (RFC 9449)](https://www.rfc-editor.org/rfc/rfc9449)** and **[OAuth 2.0 Mutual-TLS Certificate-Bound Access Tokens (RFC 8705)](https://www.rfc-editor.org/rfc/rfc8705)** solve sender-constraint but are optional extensions that most deployments do not enforce.
None of these are required by the OAuth core. | All non-trivial tokens sender-constrained (DPoP or mTLS) so a leaked token is useless without the corresponding private key. Aggressive lifetime ceilings measured in minutes or seconds, not hours. No long-lived secrets in model-visible context. Credentials fetched at task start, used within a tight window, discarded at task end. | Pending | -| 4 | **Last-Mile Enforcement**
*Did the resource actually enforce the decision, on every call, inside the runtime?* | The moment the action reaches a production system. All upstream identity work either holds here or fails here. Most systems today cannot distinguish an authorized action by an agent acting in good faith from the same call by an agent hijacked by indirect prompt injection. Gateway-only enforcement misses tool calls and sub-agent invocations that never cross a gateway. Cached decisions miss revocation, scope change, and freshly-discovered taint. | **API gateways and service meshes** enforce at the edge; agent tool calls never reach them.
**[OAuth 2.0 resource servers (RFC 6749 §1.1)](https://www.rfc-editor.org/rfc/rfc6749#section-1.1)** validate the token but not the action's relationship to intent.
**[mTLS (RFC 8705)](https://www.rfc-editor.org/rfc/rfc8705)** and **[SPIFFE](https://spiffe.io/docs/latest/spiffe-about/spiffe-concepts/)** establish workload identity but do not enforce per-action policy.
None fire inside the agent runtime where the decision is made. | Enforcement fires at every tool call, inside the agent runtime, before the action executes. The runtime verifies identity, scope, taint, freshness, and intent on every call. The LLM call cost dwarfs the re-authorization cost, so caching decisions is not required for performance. Default-deny verified by a negative test in CI. | **Partially specified.** The ACS Instrument spec already defines the honoring-decisions contract in [§6.4 (Honoring Decisions, Normative)](../spec/instrument/specification.md#64-honoring-decisions-normative): the Observed Agent MUST wait for the Guardian's decision up to the negotiated timeout, MUST apply it, and MUST record every fail-open proceed as an audit event. Note the spec's current default on timeout is **fail-open with audit**, which is in tension with this row's default-deny / no-cache goal; whether identity-tier actions should tighten that to fail-closed is an open work item for this workstream. | +| 4 | **Last-Mile Enforcement**
*Did the resource actually enforce the decision, on every call, inside the runtime?* | The moment the action reaches a production system. All upstream identity work either holds here or fails here. Most systems today cannot distinguish an authorized action by an agent acting in good faith from the same call by an agent hijacked by indirect prompt injection. Gateway-only enforcement misses tool calls and sub-agent invocations that never cross a gateway. Cached decisions miss revocation, scope change, and freshly-discovered taint. | **API gateways and service meshes** enforce at the edge; agent tool calls never reach them.
**[OAuth 2.0 resource servers (RFC 6749 §1.1)](https://www.rfc-editor.org/rfc/rfc6749#section-1.1)** validate the token but not the action's relationship to intent.
**[mTLS (RFC 8705)](https://www.rfc-editor.org/rfc/rfc8705)** and **[SPIFFE](https://spiffe.io/docs/latest/spiffe-about/spiffe-concepts/)** establish workload identity but do not enforce per-action policy.
None fire inside the agent runtime where the decision is made. | Enforcement fires at every tool call, inside the agent runtime, before the action executes. The runtime verifies identity, scope, taint, freshness, and intent on every call. The LLM call cost dwarfs the re-authorization cost, so caching decisions is not required for performance. Default-deny verified by a negative test in CI. | **Partially specified.** The ACS Instrument spec already defines the honoring-decisions contract in [§6.4 (Honoring Decisions, Normative)](../spec/instrument/specification.md#64-honoring-decisions-normative): the Observed Agent MUST wait for the Guardian's decision up to the negotiated timeout, MUST apply it, and MUST record every decision failure as an audit event. Note the spec's current default on timeout is **fail-open with audit**, which is in tension with this row's default-deny / no-cache goal; whether identity-tier actions should tighten that to fail-closed is an open work item for this workstream. | | 5 | **Proof of Intent**
*Was the action bound to an immutable, declared purpose that survives mid-task reframing?* | OAuth scopes say what the user *can* do. They do not say what the user *is doing right now*. Without intent binding at runtime, every agent action is a guess about user authorization. The agent's reasoning can be reframed mid-task by an instruction hidden in a document or a tool output. The original intent has to survive that. This is the hardest of the five, because it is the only one OAuth cannot patch with an extension: the underlying problem is non-deterministic client behavior under adversarial inputs. | **None at the identity layer.** Intent is implicit in the OAuth consent screen and discarded after token issuance.
**[RFC 9396 RAR](https://www.rfc-editor.org/rfc/rfc9396)** can carry purpose claims as `authorization_details` data but does not bind them as runtime invariants against the LLM client.
**IBAC (Intent-Based Access Control)** is research-stage with no IETF/W3C standard track.
No production standard treats intent as a first-class, immutable, normatively-enforced runtime invariant. | Intent treated as a first-class governance concept with a normative immutability invariant. Intent must flow across all three ACS pillars: Instrument fires decisions against it, Trace records it as a span attribute, Inspect catalogs what it authorized. Intent extension permitted only through an explicit, human-approved path. | Pending | --- diff --git a/docs/spec/conformance-claim-template.md b/docs/spec/conformance-claim-template.md new file mode 100644 index 00000000..bccf9a5a --- /dev/null +++ b/docs/spec/conformance-claim-template.md @@ -0,0 +1,152 @@ +# ACS conformance claim + +One page. Copy it once per implementation, fill every cell, publish the filled copy where a +reader can fetch it unchanged. + +## 1. Claim header + +| Field | Value | +|---|---| +| Implementation under test | `` | +| Specification version claimed against | `` at `` | +| Date of claim | `` | +| Claimant | `` | +| Published at | `` | +| SHA-256 of this filled copy, computed with the digest replaced by the literal `` | `` and `` | + +Pin the specification by tag and full commit SHA. A tag can move, and `version.txt` has +disagreed with the latest tag before (`Issue #143`). A full SHA is computed from the bytes and +cannot name anything else. + +## 2. Trust topology + +| Field | Value | +|---|---| +| Who operates the Observed Agent | `` | +| Who operates the Guardian | `` | +| Same party operates both | `` | +| If no, which profile carries integrity beyond the shared secret | `` | + +This block is not optional and it is not a formality. ACS-Core's baseline integrity is +HMAC-SHA256 with an HKDF-derived per-session key from deployment-provided key material. It is +symmetric, so it authenticates possession of that key but cannot give the non-repudiation and +external verifiability that ACS-Crypto adds. Both ends hold the same key, so a signed decision +is not attributable to the Guardian against the Observed Agent (§8.6). + +Two deployments can fill the rest of this page identically and truthfully while one operates +both ends and the other governs a customer's agents behind an independently operated Guardian. +The second is making a much weaker claim, and without this block the page does not show it. + +## 3. Profile matrix + +One row per profile, one column per role. The unit is the profile times the role, not the +deployment: `profiles_supported` is declared by the Observed Agent in ClientHello and +`profiles_accepted` by the Guardian in ServerHello, so a profile active at one end and not +claimed at the other is a normal outcome of per-session negotiation, not an inconsistency. + +Every cell carries a state token and, where the state is not `EXERCISED`, a reason code from +section 4. **A blank cell is not a claim of any kind and voids the row.** + +| Profile | Observed Agent (`profiles_supported`) | Guardian (`profiles_accepted`) | +|---|---|---| +| `acs-core` | | | +| `acs-trace` | | | +| `acs-inspect` | | | +| `acs-inspect-dynamic` | | | +| `acs-provenance` | | | +| `acs-crypto` | | | +| `acs-audit` | | | + +**States.** + +`EXERCISED` the profile is declared and something in the deployment demonstrates it. +`NOT CLAIMED` deliberately not declared. Carries a reason code. +`NOT ESTABLISHED` the claimant cannot say either way from what they have. Carries a reason code. + +The third state is the one that disappears when a form offers a checkbox, because an unticked +box reads as the second. A reader who cannot separate a deliberate non-claim from an unmeasured +one sees a clean sheet in both cases. + +## 4. Reason codes + +Closed vocabulary. One code per non-exercised cell, no free text in the matrix. Free text turns +a one-pager into prose; no reason at all turns it back into a checkbox. A closed list is also +what lets two claims be compared later, which is the property a registry would need. + +For `NOT CLAIMED`: + +| Code | Meaning | +|---|---| +| `not-implemented` | The methods are not implemented. | +| `not-offered` | The product does not offer this profile. | +| `governed-elsewhere` | The control is provided outside this deployment. | +| `out-of-scope` | Deliberately outside the scope of this claim. | + +For `NOT ESTABLISHED`: + +| Code | Meaning | +|---|---| +| `no-test` | No test in this run covers it. | +| `no-sink` | The required sink or recorder is not installed. | +| `not-negotiated` | The peer never negotiated the profile in any observed session. | +| `not-observed` | The path exists but no traffic was seen traversing it. | + +Anything that does not fit a code above is `NOT ESTABLISHED` plus a note in section 6, not a new +code invented on the page. + +## 5. ACS-Core decision posture + +Required whenever `acs-core` is `EXERCISED`. ACS-Core mandates decision honoring, and its +defaults are permissive, so a claim that omits this reports the default posture as evidence of +enforcement. + +| Field | Value | +|---|---| +| `on_decision_failure` posture as configured (§6.4) | `` | +| Startup posture on handshake failure (§4.1) | `` | +| Sessions observed in this run | `` | +| Decision failures recorded, by failure class (`sessionEnd` `summary.decision_failures`) | `` | +| Sessions started unguarded after handshake failure | `` | +| Where the decision-failure audit events were recorded | `` | + +An action permitted by policy and an action permitted because no decision happened are +indistinguishable on the wire and both look like conformance. Section 6.4 requires every +decision failure to be recorded as an audit event, fail-open or fail-closed, and `sessionEnd` +reports the count for each failure class in `summary.decision_failures`. A session started +unguarded has no Guardian to receive that event, so it belongs in the deployment's own log. + +## 6. Limits of this claim + +State what the claim does not cover, in the claimant's own words. At minimum, any cell marked +`NOT ESTABLISHED`, whether the environment in which the run happened still exists, and who +authored the policy and answered the ASKs. + +| Field | Value | +|---|---| +| Policy authorship | `` | +| ASK responses during this run | `` | + +ACS-Core checks that the channel is authenticated and the Observed Agent honors decisions. It +does not check who wrote the policy or who answered an ASK. A deployment whose policy was +generated and whose ASKs were all answered by a service is fully conformant, and a claim that +stays silent on it lets a reader supply oversight that ACS never checked. + +`` + +## 7. What this page is + +Quoted from `docs/spec/conformance.md` so that it travels with the claim: + +> What "ACS-Core conformant" guarantees: the channel is authenticated and the Observed Agent +> honors the Guardian's decisions. It does NOT assert that a deployment's policies are strict, +> nor that the audit chain is tamper-evident against a compromised Guardian (that is the +> ACS-Crypto and ACS-Audit profiles, since the HMAC baseline is symmetric). A permissive Guardian +> is a conformant but permissive deployment, not a violation. + +> Who verifies a conformance claim: nobody, in v0.1.0. `profiles_supported` and +> `profiles_accepted` are self-declaration on the wire. This release ships no conformance test +> suite, no registry of conformant implementations, and no steward body to arbitrate a disputed +> claim. + +This page records what an implementer asserts about itself. A claim form that does not repeat +that limit inside itself will be quoted without it. diff --git a/docs/spec/conformance.md b/docs/spec/conformance.md index dd937e95..08a9f7a1 100644 --- a/docs/spec/conformance.md +++ b/docs/spec/conformance.md @@ -34,7 +34,7 @@ A v0.1.0-conformant deployment MUST implement ACS-Core. ACS-Core comprises: - **SessionContext and Intent**: `session_id`, `chain_hash` (rolling SHA-256), append-only ContextEntry chain, with the Guardian publishing the chain head (`chain_hash`) on responses for content-bearing steps ([§8](./instrument/specification.md#8-sessioncontext-and-intent)). Intent is optional but normative when IBAC is the enforcement paradigm. - **Replay protection** — `request_id` (UUID) and `timestamp` on every request; Guardians MUST reject replays per [§10.3](./instrument/specification.md#103-replay-protection). - **Baseline integrity**: every request and response carries a signature over the canonical envelope ([§10](./instrument/specification.md#10-cryptographic-signatures)). `HMAC-SHA256` with an HKDF-derived per-session key from deployment-provided key material is the baseline; asymmetric and post-quantum algorithms are the ACS-Crypto profile. -- **Decision honoring**: the Observed Agent MUST wait for the Guardian's decision up to the negotiated timeout and apply it; on a decision failure (timeout, transport failure, or an error without a decision) it applies the `on_decision_failure` posture (default `proceed`, fail-open) and records every fail-open proceed as an audit event ([§6.4](./instrument/specification.md#64-honoring-decisions-normative)). Handshake failure follows the deployment's startup posture ([§4.1](./instrument/specification.md#41-handshake-failure-normative)). +- **Decision honoring**: the Observed Agent MUST wait for the Guardian's decision up to the negotiated timeout and apply it; on a decision failure (timeout, transport failure, or an error without a decision) it applies the `on_decision_failure` posture (default `proceed`, fail-open) and records every decision failure as an audit event, fail-open or fail-closed ([§6.4](./instrument/specification.md#64-honoring-decisions-normative)). Handshake failure follows the deployment's startup posture ([§4.1](./instrument/specification.md#41-handshake-failure-normative)). - **Liveness** — `system/ping` SHOULD be implemented: the Observed Agent declares it in `methods_implemented` and sends it, and the Guardian answers it per [§13](./instrument/specification.md#13-liveness-system-methods). A deployment MAY omit it only if the Observed Agent relies on an alternative liveness mechanism that the deployment's configuration names (for example: TCP keepalive or HTTP/2 PING on HTTP transports; process supervision or an out-of-band heartbeat on stdio; or continuous observed hook traffic where the session's own cadence acts as the probe). ACS does not put that declaration on the wire in v0.1, as [§9.2](./instrument/specification.md#92-approver-incapable-clients-normative) does not for approver capability. Omitting `system/ping` without a declared alternative is non-conformant: the chain of defaults (`startup_posture: proceed`, `on_decision_failure: proceed`, no liveness) produces silent fail-open on Guardian outage, which the Core baseline cannot allow. - **Wrapped MCP** — `protocols/MCP/*` ([Hooks](./instrument/hooks.md#protocolsmcp)) MUST be implemented by deployments whose sessions involve MCP at any point (tools, resources, prompts, notifications), so that `protocols/MCP/resources/read` — the canonical prompt-injection ingress for attacker-controlled external content — is not bypassed. MCP `tools/call` MAY instead flow through the generic `steps/toolCallRequest` / `steps/toolCallResult` hooks when tool-level policy suffices; the requirement exists for the surfaces the generic hooks cannot see (capability negotiation, prompt fetches, resource reads, notifications). Support is declared in the handshake via `wrapped_protocols`. Deployments whose sessions never involve MCP MAY omit the namespace. **Known v0.1 limitation:** `wrapped_protocols` is declared once, at handshake, and v0.1 defines no renegotiation. A session that declared no MCP and then gains an MCP server mid-session (a mutation `agbom/changed` reports, and the `acs-inspect-dynamic` profile exists to surface) cannot begin wrapping mid-session: the generic tool hooks still govern that server's tool calls, the addition is visible through `agbom/changed`, and wrapped coverage begins at the next session's handshake. Renegotiation is v0.2 scope. @@ -96,7 +96,7 @@ Profiles compose. A deployment that wants full observability and supply-chain in | Profile | What it adds | When to claim | |---|---|---| | `acs-core` | Handshake, envelope, minimum hook set (incl. `subagentStart` for subagent-capable clients), four MUST-support dispositions (ALLOW/DENY/ASK/DEFER) with MODIFY SHOULD-support (§6.5), SessionContext + published chain head, replay protection, baseline signature (HMAC-SHA256), decision honoring, declared liveness mechanism, Wrapped MCP when sessions involve MCP | Always (mandatory) | -| `acs-trace` | OTel + OCSF event emission per step | Cross-vendor observability or SIEM integration | +| `acs-trace` | OTel or OCSF Trace events per supported step, decisions and provenance included | Cross-vendor observability or SIEM integration | | `acs-inspect` | `agbom/snapshot` + canonical AgBOM serialization | Policy depends on component inventory | | `acs-inspect-dynamic` | `agbom/changed` on mutation | Agent hot-swaps components mid-session | | `acs-provenance` | Field-level Provenance objects | Enforcing FIDES, CaMeL, or AARM-style information-flow paradigms | diff --git a/docs/spec/instrument/hooks.md b/docs/spec/instrument/hooks.md index 9187081f..c3ee53dc 100644 --- a/docs/spec/instrument/hooks.md +++ b/docs/spec/instrument/hooks.md @@ -201,7 +201,9 @@ Schema: [`hooks/tool-call-result.json`](https://genai-security-project.github.io Fires after tool execution, before the result is ingested into the agent's reasoning context. Provenance: `origin: tool_output`, with `derived_from` set to the originating `toolCallRequest`'s provenance ids when the tool's output is data-derived. -**Payload:** `execution_id` (correlated to the request), outputs, exit status, duration. +**Payload:** `request_id_ref` (the originating request), outputs, exit status, duration. + +`request_id_ref`, when present, is the `request_id` of the originating `toolCallRequest`. A Guardian SHOULD resolve it against the `toolCallRequest` records it holds for the session, and MUST record the reference in its audit record when it cannot resolve it. Whether an unresolvable reference is grounds to deny is the deployment's policy. The record is required because an unresolvable reference is evidence, so it has to survive even when the result is allowed. **Decision:** ALLOW / DENY / MODIFY. @@ -315,7 +317,7 @@ Schema: [`hooks/session-end.json`](https://genai-security-project.github.io/agen Session termination, audit finalization. The Guardian seals the chain at this point. -**Payload:** `session_reason` (`completed`, `cancelled`, `error`, `timeout`), final aggregates. +**Payload:** `reason` (`completed`, `cancelled`, `error`, `timeout`, `abandoned`), and an optional `summary` with the outcome, step count, errors, and `decision_failures`, the number of decision failures in each failure class (Specification §6.4). **Decision:** Audit only. diff --git a/docs/spec/instrument/specification.md b/docs/spec/instrument/specification.md index 7292a076..12e98bc8 100644 --- a/docs/spec/instrument/specification.md +++ b/docs/spec/instrument/specification.md @@ -62,7 +62,7 @@ A worked example appears in [ACS in Action](../../topics/ACS_in_action_example.m | Element | Choice | |---|---| -| Envelope | JSON-RPC 2.0, with top-level `acs_version` | +| Envelope | JSON-RPC 2.0, with `acs_version` in `params` (requests) and `result` (responses) | | Transports | HTTP(S) and stdio (Content-Length framing, UTF-8) | | Method namespaces | `steps/*`, `protocols/A2A/*`, `protocols/MCP/*`, `agbom/*`, `system/*`, `handshake/*`, `trace/*` (reserved) | | Wrapped methods | `protocols/MCP/*` is the canonical v0.1 namespace for MCP wrapping (e.g. `protocols/MCP/tools/call`). The `wrapped:` prefix is an alternative explicit-version form for deployments that pin a specific protocol version on the wire (e.g. `wrapped:mcp-2025-06-18/tools/call`). Both are valid; `protocols/*` is preferred. A2A wrapping deferred to v0.2. | @@ -99,7 +99,7 @@ sequenceDiagram `on_decision_failure` is negotiated in the handshake, so it cannot govern a handshake that never completes. When `handshake/hello` fails — the Guardian is unreachable, silent past the deployment's connection timeout, or errors without producing a ServerHello — the Observed Agent applies its **startup posture**, configured out of band: `proceed` starts the session unguarded, `refuse` does not start it. The default is `proceed`, matching `on_decision_failure`, so a Guardian outage does not block new sessions. -A Guardian that answers with a refusal (`SESSION_REFUSED`, `UNSUPPORTED_VERSION`, `PROVENANCE_REQUIRED`) has decided; the startup posture applies only when no answer arrives. A session started unguarded MUST be recorded in the deployment's own audit log — there is no Guardian to receive the event — and SHOULD be surfaced on Trace events when the deployment claims ACS-Trace. The Observed Agent SHOULD retry the handshake for subsequent sessions. Whether a Guardian can attach mid-flight to a session that started unguarded is undefined in v0.1; renegotiation is deferred to v0.2 alongside `system/handshake_renegotiate` (§13). +A Guardian that answers with a refusal (`SESSION_REFUSED`, `UNSUPPORTED_VERSION`, `PROVENANCE_REQUIRED`) has decided; the startup posture applies only when no answer arrives. A session started unguarded MUST be recorded in the deployment's own audit log — there is no Guardian to receive the event — and SHOULD be surfaced on Trace as an `acs.decision.failure` event with class `handshake_failure` and posture `fail_open` on the session span when the deployment claims ACS-Trace. The Observed Agent SHOULD retry the handshake for subsequent sessions. Whether a Guardian can attach mid-flight to a session that started unguarded is undefined in v0.1; renegotiation is deferred to v0.2 alongside `system/handshake_renegotiate` (§13). ## 5. Hook Taxonomy @@ -199,7 +199,7 @@ A hook is a control point only if the Observed Agent waits for the verdict and a A step suffers a **decision failure** when no usable decision arrives within the negotiated timeout: the Guardian stays silent, the transport fails (connection refused, TLS failure, malformed response), or the Guardian returns an error instead of a decision. All three resolve the same way: the Observed Agent applies the deployment's failure posture, declared as `on_decision_failure` in the handshake (§4) and defaulting to `proceed` (fail-open) so that a slow, erroring, or unreachable Guardian does not halt production. A deployment MAY set `on_decision_failure: deny` (fail-closed). The negotiated timeout bounds every failure mode: an error from the §17.1 registry carries a recovery action the agent MAY attempt within the remaining budget, and an unambiguous failure (a refused connection) MAY resolve immediately rather than waiting out the clock. -Every step that proceeds without a decision MUST be recorded as an audit event, so the bypass is visible rather than silent. When a decision does arrive within the timeout, the agent MUST honor it regardless of the posture. Fail-open trades enforcement for availability under disruption: an adversary who can disrupt the channel converts control into audit. Deployments for which that trade is unacceptable set `on_decision_failure: deny`. +Every decision failure MUST be recorded as an audit event, under either posture. Under `proceed` the record makes the bypass visible rather than silent, and under `deny` it shows why the step was blocked. The Observed Agent records it in the deployment's own audit log, the rule §4.1 sets for a session started unguarded, and SHOULD also emit it as an `acs.decision.failure` event when the deployment claims ACS-Trace ([Trace Events](../trace/events.md)). At `sessionEnd`, the Observed Agent SHOULD report how many failures it recorded in each failure class, in `summary.decision_failures`. A reader treats a session that ends without that count as unmeasurable. When a decision does arrive within the timeout, the agent MUST honor it regardless of the posture. Fail-open trades enforcement for availability under disruption: an adversary who can disrupt the channel converts control into audit. Deployments for which that trade is unacceptable set `on_decision_failure: deny`. ### 6.5 MODIFY-incapable clients (normative) @@ -310,7 +310,7 @@ For every step where the Guardian writes a ContextEntry (the content-bearing ste A `subagentStop` payload MAY omit `final_chain_hash`: a framework that maintains no session-chain omits the field rather than fabricate a value, since fabrication corrupts the artifact the field exists to produce. A Guardian MUST treat the omission as "chain not maintained by this framework," not as an integrity failure; integrity claims about that subagent's chain are simply unavailable. -Tamper-evidence here is bounded by the signature in use. Under the HMAC baseline the published, signed head gives integrity against a network tamperer and lets a key-holder verify the chain, but it does not bind the Guardian itself: the Guardian holds the symmetric key and can re-sign a rewritten head. Non-repudiation, proving to a third party that a specific Guardian issued a specific head, requires the asymmetric ACS-Crypto profile. The baseline detects accidental divergence and cross-Guardian disagreement; defeating a determined, compromised Guardian is a profile-level guarantee, not a Core one. +Tamper-evidence here is bounded by the signature in use. Under the HMAC baseline the published, signed head gives integrity against a network tamperer and lets a key-holder verify the chain, but it binds neither party. The Guardian and the Observed Agent derive the same per-session key (§10), so the Guardian can re-sign a rewritten head, and the Observed Agent can produce a validly signed response the Guardian never sent, an `allow` or a `chain_hash` included. A valid HMAC shows that a holder of the session key produced the message, not which holder, so under HMAC a Guardian decision is not attributable to the Guardian against the Observed Agent. Non-repudiation, proving to a third party that a specific Guardian issued a specific head or decision, requires the asymmetric ACS-Crypto profile. The baseline detects accidental divergence and cross-Guardian disagreement; defeating a determined, compromised Guardian is a profile-level guarantee, not a Core one. ## 9. Approver Model @@ -349,6 +349,10 @@ This rule preserves the security guarantee (actions that would have been `ASK`'d A signature over the §10 canonical input is REQUIRED in ACS-Core. The signature envelope is `{ algorithm, value, key_id }`, the algorithm drawn from the registry below and the per-direction supported set declared in the handshake. `HMAC-SHA256` is the baseline that satisfies the requirement; ACS-Core does not require asymmetric or post-quantum signatures, which are the ACS-Crypto profile (§10.1). The per-session HMAC key is `HKDF`-derived from deployment-provided input keying material (a pre-shared secret, or a transport channel binding such as a TLS exporter) together with the `session_id`. ACS v0.1 defines no in-band key-exchange: a deployment with neither a shared secret nor a secure transport cannot use the HMAC baseline and satisfies the requirement with an ACS-Crypto algorithm whose key distribution it manages out of band. +**HMAC key derivation.** For `HMAC-SHA256`, the per-session key is HKDF ([RFC 5869](https://www.rfc-editor.org/rfc/rfc5869)) with SHA-256, no salt (RFC 5869's default, a string of 32 zero bytes, which gives the same key as an empty salt), `info` set to the UTF-8 bytes of `metadata.session_id`, and a 32-byte output, applied to the secret that `key_id` names. Both parties MUST derive the key with exactly these parameters, so independent implementations holding the same secret derive the same key. `info` is the `session_id` string exactly as the request carries it, not parsed or normalized first: an uppercase or `urn:uuid:` spelling of the same UUID derives a different key. A response carries no `session_id`, so it is signed and verified with the key of the request it answers. The secret MUST be at least 32 bytes from a cryptographically secure random source. HKDF does no key stretching, so a passphrase used as the secret can be recovered offline from one captured signature. This derivation is defined for a pre-shared secret. v0.1 defines no exporter label or context for a channel-binding input, so that input is not yet interoperable between implementations. For example, the 32-byte secret `000102…1e1f` (the bytes 0 through 31) and the `session_id` `6b1f2c9e-4d7a-4e3b-8a51-9c0d2e7f1a36` give the key `80f0dc8c611279bfc2af2d5f374c5cb685888192ce1af5c3e0673d8fa748e8a4`. + +**`key_id` for HMAC.** For `HMAC-SHA256`, `key_id` is an opaque label, chosen by the deployment, that names one configured secret. This specification reserves no values, and an implementation takes the label from its configuration. It does not hardcode one. A verifier MUST reject an unknown `key_id` (`SIGNATURE_INVALID`, §17.1) and MUST NOT fall back to another secret. `key_id` MUST NOT be derived from the secret. A Guardian that signs its response with `HMAC-SHA256` uses the secret, and the `key_id`, that the request named. The `SIGNATURE_INVALID` error for an unknown `key_id` is unsigned, because no configured secret matches it. A deployment rotating its secret configures the new secret under a new label, such as `"2026-10"`, so a verifier can hold both secrets while the rotation is in progress. That is what the §17.1 recovery for `SIGNATURE_INVALID`, re-resolving `key_id`, relies on. For the asymmetric algorithms of ACS-Crypto, `key_id` identifies the public key or hybrid key descriptor (§10.2). + **Signed input.** The signed input for every algorithm is the RFC 8785 (JCS) canonicalization of the request or response envelope with the `signature` field removed, encoded as UTF-8. This binds the signature to the whole envelope, including `method`, `metadata.session_id`, `request_id`, and `timestamp`, so a captured signature cannot be lifted into a different envelope (for example, re-wrapping a signed `toolCallResult` under a different `session_id`). A verifier MUST recompute this canonical form and MUST reject a signature that does not cover it (`SIGNATURE_INVALID`, §17.1). The input is fixed by this rule, not declared per message: a discoverable `signed_fields` would let a producer advertise partial coverage that omits `session_id`, reopening the gap this closes. Alternative canonicalization is not permitted in v0.1, matching the chain-hash rule in §8. ### 10.1 Algorithm registry @@ -357,7 +361,7 @@ The registry is crypto-agile: the `{ algorithm, value, key_id }` envelope suppor | Algorithm | Class | v0.1.0 status | |---|---|---| -| `HMAC-SHA256` | Symmetric MAC | RECOMMENDED. Shared-secret integrity; simplest deployment path. Sufficient for same-host and trusted-network topologies where the threat is accidental tampering or replay. | +| `HMAC-SHA256` | Symmetric MAC | RECOMMENDED. Shared-secret integrity; simplest deployment path. Sufficient for same-host and trusted-network topologies where the threat is accidental tampering or replay. Both parties hold the key, so it attributes a message to neither (§8.6). | | `ECDSA-P256` | Classical asymmetric | OPTIONAL. Strongest current ecosystem support across Java, Node, .NET, HSMs, and major cloud KMS providers. | | `RSA-PSS-SHA256` | Classical asymmetric | OPTIONAL. Legacy interop; deployments with existing RSA PKI. | | `ML-DSA-65` | PQC, lattice (FIPS 204) | OPTIONAL. ~128-bit post-quantum security; ~3.3 KB signatures. Recommended for deployments shipping PQC libraries today. | diff --git a/docs/spec/trace/events.md b/docs/spec/trace/events.md index 25d271c4..c23d6440 100644 --- a/docs/spec/trace/events.md +++ b/docs/spec/trace/events.md @@ -54,7 +54,7 @@ An `acs.decision` event records the Guardian's verdict. Only the Observed Agent A decision failure gets its own event because it has no disposition and no evaluator. It goes on the step span, except `handshake_failure`, which goes on the `acs.session` span: the session started without a completed handshake, so none of its steps was governed. `signature_invalid` is a response whose signature failed verification, kept apart from `malformed_response` so a tampered response does not blend into organic failures. The Observed Agent MUST record every decision failure, fail-open or fail-closed, in the deployment's own audit log. It SHOULD emit `acs.decision.failure` when it claims ACS-Trace. A Guardian claiming ACS-Trace SHOULD emit the event for the failures it sees itself: an error it returned (`guardian_error`) and a decision it finished after the negotiated timeout (`timeout`). The mapping lists every class and posture value. -In v0.1 the Observed Agent reports this event about its own conduct. A missing event does not show that no failure happened, and a count built from these events is a lower bound. Events from both parties for one request describe one failure, so a reader counts distinct `acs.session.id` and `acs.request_id` pairs. That counts failed attempts, not steps: a step the Observed Agent recovered by retrying still shows its failed attempts. A pair with a Guardian event and no Observed Agent event still counts. A Guardian MUST NOT emit the event for a request it rejects as a replay (`REPLAY_DETECTED`), because that pair belongs to the earlier request. `acs.failure.detail` MUST be text the emitter composes itself, and MUST NOT copy bytes from the request, the response, or an error's `data`. +In v0.1 the Observed Agent reports this event about its own conduct. A missing event does not show that no failure happened, and a count built from these events is a lower bound. Events from both parties for one request describe one failure, so a reader counts distinct `acs.session.id` and `acs.request_id` pairs. That counts failed attempts, not steps: a step the Observed Agent recovered by retrying still shows its failed attempts. A pair with a Guardian event and no Observed Agent event still counts. A Guardian MUST NOT emit the event or an `acs.decision` event under the pair of a request it rejects before trusting it: an envelope that fails validation, a signature that fails verification (`SIGNATURE_INVALID`), a replay (`REPLAY_DETECTED`), or a timestamp outside the window (`TIMESTAMP_OUT_OF_WINDOW`). Such a request can carry any pair, including a genuine request's. A deployment that records rejected requests does so under attribute names other than `acs.session.id` and `acs.request_id`. `acs.failure.detail` MUST be text the emitter composes itself, and MUST NOT copy bytes from the request, the response, or an error's `data`. Some unusable responses are not decision failures, because the spec already defines their outcome. A response whose `modifications` break §6.3 gets `DENY`, whether or not schema validation also rejects it. A `MODIFY` the client cannot apply gets `DENY`, or at `postCompact` the original summary (§6.5). `malformed_response` covers a response that does not parse, or that fails schema validation for any other reason. diff --git a/docs/spec/trace/extend_opentelemetry.md b/docs/spec/trace/extend_opentelemetry.md index dfb27c04..16c460a4 100644 --- a/docs/spec/trace/extend_opentelemetry.md +++ b/docs/spec/trace/extend_opentelemetry.md @@ -24,12 +24,12 @@ The span carries the join pair (`acs.session.id` and `acs.request_id`), the tool ```json { - "traceId": "85c4878c9bda9e5b4f2fc6aae242d31f", - "spanId": "dea6530ad1681c0e", + "traceId": "13717f291e2368066d2fa3f2434ed2f6", + "spanId": "2a7efedbb7ee3881", "name": "gen_ai.tool.call", "kind": 1, - "startTimeUnixNano": "1791928279698000000", - "endTimeUnixNano": "1791928279749301083", + "startTimeUnixNano": "1792534253716796645", + "endTimeUnixNano": "1792534253780000000", "attributes": [ { "key": "acs.session.id", @@ -146,7 +146,7 @@ The span carries the join pair (`acs.session.id` and `acs.request_id`), the tool } ], "name": "acs.decision", - "timeUnixNano": "1791928279749295500", + "timeUnixNano": "1792534253780000000", "droppedAttributesCount": 0 } ], @@ -168,7 +168,7 @@ The span above is real output from the reference Guardian. This section explains ### Two streams, joined by a reader -No single process sees every outcome of a step. The Guardian sees each request it receives and the answer it gives. It exports one span per decided step, with the `acs.decision` event on it. It also reports the failures it can see itself: an error it returned instead of a decision, a decision it finished after the negotiated timeout, and a step still open when it shut down. +No single process sees every outcome of a step. The Guardian sees each request it receives and the answer it gives. It exports one span per step it trusted, with the `acs.decision` event on it, and records a request it rejected before trusting it apart, under names other than `acs.session.id` and `acs.request_id`. It also reports the failures it can see itself: an error it returned instead of a decision because it could not handle the step, a decision it finished after the negotiated timeout, and a step still open when it shut down. The host sees what the Guardian can't. An unreachable Guardian has nothing to record. A Guardian whose answer arrives after the host stopped waiting can't see what the host did in the meantime. The host adapter records both cases in its own audit log. A converter, `audit-to-otlp.ts`, turns that log into spans of the same OTLP/JSON shape, each with an `acs.decision.failure` event whose `acs.failure.reported_by` is `observed_agent`. @@ -224,7 +224,7 @@ sequenceDiagram A late decision gets both events on one span: the `acs.decision` with the verdict, then an `acs.decision.failure` with class `timeout`. When either party's stream holds a failure event for a pair, a reader doesn't count that pair's `acs.decision` as enforced. [Trace Events](./events.md) states that rule. -The Guardian never copies its error text into the trace. A failure event carries the error code, the timeout, the elapsed time and the request size when they're known, because error text can quote the request. A replay rejection gets no failure event, since its pair belongs to the earlier request. +The Guardian never copies its error text into the trace. A failure event carries the error code, the timeout, the elapsed time and the request size when they're known, because error text can quote the request. A request the Guardian rejected before trusting it, by a failed validation, signature, replay or timestamp check, gets no event under the pair it carried, since that pair can be a genuine request's. The reference Guardian records it as a separate span under its own names instead. If you only need to read a trace, you can stop here. The rest of this section is for running the pipeline or building your own. @@ -238,7 +238,7 @@ The collector path, set with `ACS_OTEL_ENDPOINT`, sends the same spans over OTLP The two Guardian outputs fail independently. The first failed write or send disables that output for the rest of the process and prints one line on stderr. The other output keeps running, and the decision goes out unchanged, as the [failure isolation](#failure-isolation) rule below requires. The [Guardian README](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/packages/guardian/README.md#troubleshooting) lists the messages you may see. -The committed [sample trace](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl) holds both streams in one file: five spans from the Guardian and one from the converter. Every value in it is synthetic. +The committed [sample trace](https://github.com/GenAI-Security-Project/agent-control-standard/blob/main/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl) holds both streams in one file: five spans from the Guardian, one of them a rejected request, and one from the converter. Every value in it is synthetic. ### What the labels don't prove diff --git a/mkdocs.yml b/mkdocs.yml index 940ba59a..e7d8d77b 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -76,6 +76,7 @@ nav: - Trust Basis: concepts/trust.md - Specification: - Conformance Profiles: spec/conformance.md + - Conformance Claim Template: spec/conformance-claim-template.md - Instrument: - Overview: spec/instrument/README.md - Supported Hooks: spec/instrument/hooks.md diff --git a/reference-implementations/README.md b/reference-implementations/README.md new file mode 100644 index 00000000..2ff701ab --- /dev/null +++ b/reference-implementations/README.md @@ -0,0 +1,93 @@ +# ACS reference implementations + +This directory holds the reference implementations of the Agent Control Standard. A reference implementation is working code that shows the ACS wire contract in use. + +## Expectations for Guardian reference implementations + +This section is a working tool for reviewing pull requests that add or change Guardian reference implementations, so that every implementation is reviewed against the same expectations. + +"Is expected to" marks a check that blocks a merge. "Should" marks a check a reviewer raises but that does not block one. + +An expectation an implementation does not meet yet is marked **Not met yet**, with the issue that tracks it. The gap is then public and tracked, so it is handled as an issue rather than as a security report ([SECURITY.md](../SECURITY.md)). A gap without an issue is not a documented gap. + +### Project-level requirements + +#### Maintained implementations + +The project maintains one Guardian reference implementation, in TypeScript, under [`agt/`](agt/). Its own [status table](agt/README.md#what-this-project-is-and-is-not) is the current record of which ACS-Core requirements it meets. Implementations in other languages join when they merge. Who keeps one working after that is decided in issue #318. + +#### Shared host integrations + +Host adapters are meant to live apart from Guardian implementations and talk to them only through the ACS wire contract, so a new Guardian language needs no host integration of its own. Today the TypeScript shims sit inside the TypeScript workspace, in [`agt/hosts/`](agt/hosts/) and [`agt/packages/host-adapter/`](agt/packages/host-adapter/), and some ports carry their own. Where shared host adapters live is decided in issue #313. + +#### Shared contract tests + +The project will maintain a language-independent contract test suite that exercises Guardians through their HTTP API, tracked in issue #311. It checks ACS protocol behavior and guarantees a host can observe. It does not require an implementation to port the TypeScript conformance toolkit. + +### Requirements for each maintained implementation + +#### ACS API + +The implementation is expected to follow the ACS wire contract, including every ACS-Core requirement in [conformance.md](../docs/spec/conformance.md#acs-core-mandatory-baseline). It is expected to declare its capabilities accurately in the handshake, and to list any ACS-Core requirement it does not meet in its README. + +**Not met yet** by the TypeScript Guardian: its ServerHello is constants, and it does not read the ClientHello (issue #165, issue #312). + +#### Policy evaluation with the AGT runtime + +Policy evaluation is expected to use AGT's policy runtime built from the AGT commit `agt_ref` in [`agt/agt.lock`](agt/agt.lock). It is published as `agent-control-specification` on npm, and other languages reach the same runtime through their own package or a native binding to its Rust core. Each implementation records the package and version it uses in its own lock file, and its CI is expected to fail when that commit differs from `agt_ref` (issue #314). + +Reimplementing the runtime's behavior, or evaluating its policy bundle directly, is not an acceptable substitute. AGT's Go module, `agent-governance-golang`, is a different policy engine and does not qualify. + +AGT's runtime is the only evaluator a maintained implementation ships, so every port runs the same policy engine and is reviewed against one rule. A test may compare AGT's decisions with another evaluator's, for example to catch drift, as long as that evaluator never answers a request. + +#### Architectural boundaries + +Transport handling, ACS processing, AGT integration, message signing, state storage, and logging should each have a clearly defined responsibility. The Guardian should expose a processing entry point that does not depend on the transport. APIs, packages, classes, and file layouts follow the language's conventions. + +**Not met yet** by the TypeScript Guardian: processing sits inside its HTTP handler (issue #312). + +#### Signing and verification + +Implementations are expected to sign and verify ACS messages as [§10 of the specification](../docs/spec/instrument/specification.md#10-cryptographic-signatures) defines, in particular its "HMAC key derivation" and "`key_id` for HMAC" paragraphs. A reviewer checks these points. + +- `key_id` comes from configuration, has no built-in default, and is neither the `session_id` nor derived from the secret. +- A verifier rejects an unknown `key_id` with `SIGNATURE_INVALID` and never falls back to another secret. +- A signed response uses the `key_id` of the request it answers. +- The secret meets §10's length and randomness rule. +- Configuration maps each `key_id` to one secret and accepts at least two at once, so a secret is replaced without an outage. The documentation describes how. + +These repository conventions sit on top of §10. + +- The secret is stored as standard base64 (RFC 4648 §4), with padding, and with surrounding whitespace ignored. The decoded bytes, not the base64 text, are the secret §10 names and the HKDF input. +- It is read from a file, an environment variable, or a secret store, and is never hard-coded. +- Neither the secret nor any key derived from it appears in ACS messages, error data, logs, or trace exports. + +**Not met yet** by the TypeScript Guardian: it does not sign or verify messages. PR #229 adds signing (issue #184, issue #312). + +#### Configurable storage + +Session and security state should be reached through a storage-provider interface that configuration selects. Storage formats are the implementation's choice, and moving a session between implementations is not required. Each storage provider's durability and multi-instance guarantees are expected to be documented. + +**Not met yet** by the TypeScript Guardian: it has one in-memory store, and only code can swap it (issue #312). + +#### Concurrency and session consistency + +The Guardian is expected to handle concurrent requests safely, whatever threading model its transport uses. Concurrent requests in one session are expected to keep §10.3's replay protection, the audit chain, and policy state consistent. No request is evaluated twice, and no state update is lost. + +**Not met yet** by the TypeScript Guardian: it rejects no replay (issue #184, PR #229). Two concurrent steps in one session can overwrite each other's information-flow labels (issue #310). + +#### Bounded processing + +Reading a request and evaluating policy are expected to be bounded by configurable limits on time, body size, and concurrent requests, with documented defaults. The default time limit is below the `timeout_config` the Guardian declares. A request that exceeds a limit gets an honoured `deny` with a reason code, not an error, so it never falls into the host's fail-open posture (§6.4). + +**Not met yet** by the TypeScript Guardian (issue #312). + +#### Bounded logging + +Every log, the audit log included, is expected to have a configurable size cap with a finite documented default, and reaching the cap never blocks a decision. A full log rotates, and the rotated file can move to an archive such as object storage or a telemetry backend. A diagnostic log may drop its oldest entries. The audit events §6.4, §6.5 and §9.2 require, and the §8 session chain, are never dropped: if the Guardian cannot rotate or archive them, it answers `deny` rather than lose an audit record. + +**Not met yet** by the TypeScript Guardian (issue #312). + +#### Verification + +Once the shared contract test suite exists (issue #311), an implementation is expected to pass the tests that apply to it. The capabilities it declares in its ServerHello decide which tests apply, and the suite checks that each declared capability behaves as declared. Until then, a port is verified the way the [root README](../README.md#port-it-python-go-rust-codex) describes: `bun run conformance` from `agt/` against the running port. diff --git a/reference-implementations/agt/README.md b/reference-implementations/agt/README.md index cdc6a862..baf7ecbe 100644 --- a/reference-implementations/agt/README.md +++ b/reference-implementations/agt/README.md @@ -312,7 +312,7 @@ Verified on macOS (arm64): natively with otelcol-contrib 0.162.0, and with the D **Run it again.** Every output here except the converter's appends, so a second run on the same `.acs` doubles every count, and the converter refuses to run at all. Stop the Guardian and the collector first, with Ctrl-C or `docker stop acs-otelcol`. A process that is still running keeps writing into the deleted file, and its spans are lost without an error. Then delete it: `rm -rf .acs && mkdir -p .acs && chmod 700 .acs`. -**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability` and `acs.request_id_ref`, because the shim doesn't send them yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). The committed sample drives the Guardian directly, so it has both. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229). +**What this is, and isn't.** The exporter covers the two hook methods this Guardian evaluates, `steps/toolCallRequest` and `steps/toolCallResult`. It isn't an ACS-Trace conformance claim. Traces driven through the shim also lack `acs.capability`, because the shim doesn't send it yet ([issue #274](https://github.com/GenAI-Security-Project/agent-control-standard/issues/274)). They carry `acs.request_id_ref` on a result when the host supplied its call id. The committed sample drives the Guardian directly, so it carries `acs.capability`, and `acs.request_id_ref` on its results. Spans also lack `acs.evaluator` until the Guardian sends `metadata.evaluator` (PR #229). The trace files, the audit log and the collector's port can be written by any process running as you, the observed agent included, and in a container by other containers on the same Docker network, so the output isn't tamper-evident. `service.name` (`acs-reference-guardian`, `acs-reference-audit-converter`) is a label each writer sets for itself, not authentication, and loopback isn't authentication either. These labels belong to the reference implementation, not to the ACS specification. @@ -420,7 +420,7 @@ The `.acs/` directory holds four kinds of local file. None is ever committed. | File | Written by | Holds | |---|---|---| -| `envelopes.jsonl` | Guardian | Every envelope, both directions, before validation | +| `envelopes.jsonl` | Guardian | Every envelope, both directions, before validation, plus a `finding` line for a fact the spec requires the Guardian to record, such as a `request_id_ref` it cannot resolve | | `session-context.jsonl` | Guardian | One hash-chain entry per governed step | | `audit.jsonl` | Host shim | Every fail-open proceed and every posture-driven block | | `sessions/.json` | Host shim, Claude Code only | The negotiated ServerHello, so the next hook subprocess finds it | @@ -492,9 +492,10 @@ The suite has 68 test files. Most of them drive a real Guardian, the real host s These are known limits of the tree as it stands. None is fixed. -- **Memory and disk grow without bound.** The Guardian's session store never evicts a session. The envelope log, the audit log and the session-context log have no rotation and no size cap. A long-lived Guardian grows until something else stops it. +- **Memory and disk grow without bound.** The Guardian's session store keeps at most 1024 sessions and evicts the least recently written one, but it bounds sessions, not the steps within one. The envelope log, the audit log and the session-context log have no rotation and no size cap. A long-lived Guardian grows until something else stops it. - **Log writes sit on the decision path.** The Guardian writes two synchronous file appends per request in a single-threaded server. A slow disk blocks every in-flight decision. - **The wire is unauthenticated.** See [What this project is, and is not](#what-this-project-is-and-is-not). Loopback by default is the only protection. +- **A well-formed forgery is trusted.** The Guardian checks a request against the schema and nothing else, so a forged or replayed request that passes the schema is decided, recorded on the trace under the ids it carries, and appended to that session's hash chain. Signature, replay and timestamp checks arrive with PR #229. Until then, a trace's request pairs, and the chain, are only as trustworthy as access to the Guardian's port. - **The egress gate has three measured soft spots.** A shell command whose destination the extractor cannot parse falls through to `allow`, not `deny`. With the shipped allowlist, any command whose text contains an off-list URL is denied, whether or not it reaches that URL. A `WebFetch` URL reaches AGT's gate with no parsing, so four userinfo-style URL shapes pass on the fetch route and fail on the shell route. Closing the last one means editing a `.rego` file, which this project does not do. - **An `ask` never carries `ask_details`.** The Guardian's decision type has no such field. Every `ask` fails the response schema. A `defer` would fail the same way if any AGT verdict mapped to it. - **The handshake is not negotiated.** The Guardian returns constants and never reads the ClientHello. Neither side declares ACS profiles. @@ -522,6 +523,11 @@ These findings are open on this tree. They are recorded here so a reader does no - A `steps/skillLoad` hook that drives AGT's `content_hash` gate is planned, not built. No code for it exists in the tree, and the ServerHello does not name it. - Two AGT intervention points, `pre_model_call` and `post_model_call`, have no ACS v0.1.0 method. Whether this project proposes `steps/modelCall` for ACS v0.2 is open. - Whether to report a fail-open to AGT upstream is open: a `./`-prefixed `bundle:` path silently disables all policy and returns `allow`. This tree throws on that path as a backstop. +- A finding is recorded on the step's own entry in the hash chain (`session-context.jsonl`), as `findings` plus the unresolved `request_id_ref`, and the entry's hash covers both. Entries with no findings hash exactly as before. A readable copy goes to the envelope log, or, when there is no envelope log or it has disabled itself, to stderr as `acs-finding ` followed by one JSON object with the same fields plus `recorded_at`, `method` and `rpc_id`. Findings don't reach the OTel export. The chain is tamper-evident only as far as §8.6 allows, since this Guardian publishes no chain head. +- A reference fails to resolve for three ordinary reasons as well as a suspicious one: the request was recorded before a Guardian restart, its session was evicted, or the request never reached this Guardian and the host proceeded under its posture. The finding doesn't say which. Run one Guardian per session, or route by session, or every result that lands on a different Guardian is a finding. +- A reference to a request the Guardian denied counts as resolved, so a result for a denied tool call gets no finding. +- Specification §10.3 says a Guardian MUST reject a duplicate `request_id` in a session. This Guardian records a `request_id_duplicate` finding instead, for a `steps/toolCallRequest` or `steps/toolCallResult` that reuses an id any earlier step in the session carried, and still decides the step. It sees only the two methods it dispatches. +- The shim derives a request's `request_id` from the host's call id, so asking twice about one call repeats the id. If a Guardian answers that with `REPLAY_DETECTED`, the shim retries that request once with a random id. The result hook runs on its own and still names the derived id, so the result resolves against the first request the host sent for that call. The JSON-RPC `id` stays random either way, so the envelope log never pairs two requests under one id, and the host audit log records the `request_id` separately as `request_id`. - Six required Trace-pillar attributes exist on the wire but are optional there. Requiring them is an ask for v0.2. The Guardian also leaves `AcsResult.metadata` empty, which it could fill today. ## Verify diff --git a/reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml b/reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml index 0f227b7c..e77ea83c 100644 --- a/reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml +++ b/reference-implementations/agt/hosts/claude-code/claude-code.hookmap.yaml @@ -19,6 +19,9 @@ hooks: PreToolUse: acs_method: steps/toolCallRequest tool_name: $.tool_name + # Claude Code's id for this one tool call, the same on PreToolUse and + # PostToolUse, so the result can name the request it answers. + call_id: $.tool_use_id arguments: $.tool_input # ACS's own raw_command: the command line this step IS, verbatim, which is # a different thing from the argument bag above and is why it gets its own @@ -128,6 +131,9 @@ hooks: PostToolUse: acs_method: steps/toolCallResult tool_name: $.tool_name + # Claude Code's id for this one tool call, the same on PreToolUse and + # PostToolUse, so the result can name the request it answers. + call_id: $.tool_use_id # THIS GATE'S MATCHER STAYS SINGLE-TOOL WHILE `outputs.from` IS A SINGLE # PATH, and that is the fact to read before widening `^Bash$` in # settings.json to match the request gate's `^(Bash|WebFetch)$`. JSON diff --git a/reference-implementations/agt/hosts/claude-code/test/pairing.test.ts b/reference-implementations/agt/hosts/claude-code/test/pairing.test.ts new file mode 100644 index 00000000..50ed04df --- /dev/null +++ b/reference-implementations/agt/hosts/claude-code/test/pairing.test.ts @@ -0,0 +1,72 @@ +import { describe, expect, it } from "bun:test"; +import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +// Test-only import, the same precedent as hook.test.ts: a real Guardian, so +// this proves the pairing on the wire rather than against a copied shape. +import { startGuardian } from "guardian"; +import { + createGuardianClient, + createMemorySessionConfigStore, + DEFAULT_TIMEOUT_MS, + governStep, + loadHookmap, + NULL_AUDIT_SINK, + resolveSessionConfig, + toSessionUuid, +} from "host-adapter"; + +const HOOKMAP_PATH = fileURLToPath(new URL("../claude-code.hookmap.yaml", import.meta.url)); + +// The shipped hookmap names `tool_use_id` as the call id. Claude Code's hooks +// reference shows the field on both PreToolUse and PostToolUse, and a +// transcript check found the same value on both hooks of one call. +function findingLines(path: string): unknown[] { + if (!existsSync(path)) { + return []; + } + return readFileSync(path, "utf8") + .split("\n") + .filter((line) => line.trim() !== "") + .map((line) => JSON.parse(line) as { direction: string; envelope: unknown }) + .filter((entry) => entry.direction === "finding") + .map((entry) => entry.envelope); +} + +async function runCall(sessionId: string, beforeId: string, afterId: string): Promise { + const dir = mkdtempSync(join(tmpdir(), "acs-pairing-")); + const logPath = join(dir, "envelopes.jsonl"); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath: logPath }); + try { + const hookmap = loadHookmap(HOOKMAP_PATH); + const client = createGuardianClient(guardian.url); + const session = await resolveSessionConfig( + { guardian: client, agentId: hookmap.host, sessionId: toSessionUuid(sessionId), timeoutMs: DEFAULT_TIMEOUT_MS }, + createMemorySessionConfigStore(), + ); + const common = { session_id: sessionId, tool_name: "Bash", tool_input: { command: "ls" } }; + for (const [hookEventName, payload] of [ + ["PreToolUse", { ...common, hook_event_name: "PreToolUse", tool_use_id: beforeId }], + ["PostToolUse", { ...common, hook_event_name: "PostToolUse", tool_use_id: afterId, tool_response: { stdout: "a.txt" } }], + ] as const) { + await governStep({ hookEventName, payload, hookmap, guardian: client, session, sessionId, audit: NULL_AUDIT_SINK }); + } + return findingLines(logPath); + } finally { + await guardian.close(); + rmSync(dir, { recursive: true, force: true }); + } +} + +describe("Claude Code: a result names the request it answers", () => { + it("records no finding when PostToolUse carries the same tool_use_id as PreToolUse", async () => { + expect(await runCall("sess-raw-not-a-uuid", "toolu_01", "toolu_01")).toHaveLength(0); + }); + + it("records an unresolved reference when the result's call never reached the Guardian", async () => { + const findings = await runCall("sess-raw-not-a-uuid", "toolu_01", "toolu_02"); + expect(findings).toHaveLength(1); + expect(findings[0]).toMatchObject({ finding: "request_id_ref_unresolved" }); + }); +}); diff --git a/reference-implementations/agt/hosts/opencode/opencode.hookmap.yaml b/reference-implementations/agt/hosts/opencode/opencode.hookmap.yaml index aeba2105..8d556d1e 100644 --- a/reference-implementations/agt/hosts/opencode/opencode.hookmap.yaml +++ b/reference-implementations/agt/hosts/opencode/opencode.hookmap.yaml @@ -129,6 +129,9 @@ hooks: tool.execute.before: acs_method: steps/toolCallRequest tool_name: $.tool + # OpenCode's id for this one tool call, the same before and after it, + # so the result can name the request it answers. + call_id: $.callID arguments: $.args # The same declaration Claude Code's hookmap makes, at this host's own # field: OpenCode hands the plugin a mutable `{args}`, so the shell tool's @@ -209,6 +212,9 @@ hooks: tool.execute.after: acs_method: steps/toolCallResult tool_name: $.tool + # OpenCode's id for this one tool call, the same before and after it, + # so the result can name the request it answers. + call_id: $.callID # SCOPED TO bash ONLY, and this is data because OpenCode has no matcher to # get it from. OpenCode fires this hook for EVERY tool, and `metadata` is # PER-TOOL -- measured on 1.18.15 across four tools: `bash`'s metadata is diff --git a/reference-implementations/agt/hosts/opencode/test/hookmap.test.ts b/reference-implementations/agt/hosts/opencode/test/hookmap.test.ts index 26ad1873..2ecdf356 100644 --- a/reference-implementations/agt/hosts/opencode/test/hookmap.test.ts +++ b/reference-implementations/agt/hosts/opencode/test/hookmap.test.ts @@ -11,7 +11,7 @@ */ import { describe, expect, it } from "bun:test"; import { fileURLToPath } from "node:url"; -import { buildEnvelope, loadHookmap } from "host-adapter"; +import { buildEnvelope, loadHookmap, toInvocationRequestId } from "host-adapter"; import { validateEnvelope } from "guardian"; const HOOKMAP = fileURLToPath(new URL("../opencode.hookmap.yaml", import.meta.url)); @@ -138,7 +138,7 @@ describe("opencode.hookmap.yaml", () => { expect(hooks["tool.execute.before"]?.tools).toEqual(["bash", "webfetch"]); }); - it("builds a wire payload of exactly {tool, exit_status, outputs:[{value}]}, with no mirrors anywhere in it", () => { + it("builds a wire payload of exactly {tool, request_id_ref, exit_status, outputs:[{value}]}, with no mirrors anywhere in it", () => { // This is the first hookmap that actually declares `outputs.mirrors`, so // this is the first place a mirror leaking onto the wire could be caught. // `buildPayload` never reads `mirrors` -- this asserts the envelope it @@ -167,6 +167,8 @@ describe("opencode.hookmap.yaml", () => { expect(() => validateEnvelope(envelope)).not.toThrow(); expect(envelope.params.payload).toEqual({ tool: { name: "bash" }, + // From the hookmap's call_id: the request this result answers. + request_id_ref: toInvocationRequestId("6c616a11-495a-4f05-878a-c1bfaa29f0e5", "call-1"), exit_status: "success", outputs: [{ value: "TOKEN=ghp_ABCDEF123456" }], }); diff --git a/reference-implementations/agt/packages/guardian/README.md b/reference-implementations/agt/packages/guardian/README.md index 4c604f7f..ec7c864d 100644 --- a/reference-implementations/agt/packages/guardian/README.md +++ b/reference-implementations/agt/packages/guardian/README.md @@ -38,11 +38,13 @@ To see a decision without an agent client, pipe a hook payload into the Claude C 1. The raw envelope is appended to the envelope log, before anything else. 2. The envelope is validated against `request-envelope.json`. A `steps/toolCallRequest` or `steps/toolCallResult` is also validated against its own payload schema. An invalid envelope on a `steps/*` method is answered with an honoured `deny`, not a bare error, so the host has a decision to act on. -3. The AGT intervention point for the method is resolved from `mapping.yaml`, and so is the argument the policy target is read from for the tool the payload names. -4. The AGT snapshot is assembled, one shape per gate, and evaluated through the bridge. -5. The verdict is mapped to an ACS decision with the tables in `mapping.yaml`. A `transform` becomes a `modify` whose modification differs by gate: a parameter override at the request gate, a redaction on the result payload at the result gate. -6. One entry is appended to the session's hash chain. Information-flow labels AGT emitted are stored, so the next step of the session sends them back as input. -7. The outgoing response is checked against `response-envelope.json`. The result is logged on stderr and never thrown. The response is then recorded and sent. +3. A `steps/toolCallResult` whose `request_id_ref` names no `toolCallRequest` in the session gets the finding `request_id_ref_unresolved` on its own hash-chain entry (step 8), because `hooks.md` requires the Guardian to record a reference it cannot resolve. A readable copy goes to the envelope log as a `finding` line, or to stderr as an `acs-finding` line when there is no envelope log or it has disabled itself. The decision is left to policy. Both shipped hookmaps name the host's per-call id as `call_id` (`tool_use_id` for Claude Code, `callID` for OpenCode). The shim derives the request's `request_id` from it and sends the same value as the result's `request_id_ref`. A host that leaves the id out gets a random `request_id` and no reference. Results are governed only for the tools the hookmaps name, and Claude Code reports a failed call through `PostToolUseFailure`, which isn't mapped, so a failed call sends no result. +4. A step whose `request_id` an earlier step in the session already carried gets a `request_id_duplicate` finding the same way, and is still decided. +5. The AGT intervention point for the method is resolved from `mapping.yaml`, and so is the argument the policy target is read from for the tool the payload names. +6. The AGT snapshot is assembled, one shape per gate, and evaluated through the bridge. +7. The verdict is mapped to an ACS decision with the tables in `mapping.yaml`. A `transform` becomes a `modify` whose modification differs by gate: a parameter override at the request gate, a redaction on the result payload at the result gate. +8. One entry is appended to the session's hash chain. Information-flow labels AGT emitted are stored, so the next step of the session sends them back as input. +9. The outgoing response is checked against `response-envelope.json`. The result is logged on stderr and never thrown. The response is then recorded and sent. `handshake/hello` is answered with a constant ServerHello that names exactly the methods this Guardian dispatches: `steps/toolCallRequest` and `steps/toolCallResult`. The Guardian does not read the ClientHello. Any other method is answered with a JSON-RPC error carrying `METHOD_NOT_DISPATCHED_CODE`, which the conformance harness reads back to confirm the ServerHello tells the truth. @@ -56,7 +58,7 @@ Set `ACS_OTEL_FILE`, `ACS_OTEL_ENDPOINT`, or both. With neither set, no exporter ACS_OTEL_FILE=.acs/otel-traces.jsonl bun run guardian ``` -Each decided step becomes one span, named by [`trace/otel-mapping.json`](../../../../specification/v0.1.0/trace/otel-mapping.json): `gen_ai.tool.call` for `steps/toolCallRequest` and `gen_ai.tool.result` for `steps/toolCallResult`. The span opens when the request is recorded and closes when the response is. It carries each attribute the mapping lists for it when the envelope supplies a value. An attribute the envelope omits, such as `acs.capability`, is absent from the span. +Each step the Guardian trusted becomes one span, named by [`trace/otel-mapping.json`](../../../../specification/v0.1.0/trace/otel-mapping.json): `gen_ai.tool.call` for `steps/toolCallRequest` and `gen_ai.tool.result` for `steps/toolCallResult`. The span opens when the request is recorded and closes when the response is. It carries each attribute the mapping lists for it when the envelope supplies a value. An attribute the envelope omits, such as `acs.capability`, is absent from the span. Every span also carries `acs.session.id`, and `acs.request_id` whenever the envelope has one, so a session can be rebuilt from the trace alone. When an argument or output carries a Provenance object, the span carries `acs.provenance.origin` as an array, one element per Provenance object in wire order. `acs.provenance.source_id` is the matching array, with `""` where an object has no `source_id`, and it is left out when no object has one. Both stop at 64 objects. `acs.provenance.lineage_depth` is not emitted: `provenance.json` defines no such field. @@ -64,9 +66,13 @@ Every string attribute is cut at 4096 characters, a long `acs.reasoning` include The decision rides on the span as an `acs.decision` event. The event always carries `acs.decision`, carries `acs.reasoning` when the response has one, and carries `acs.reason_codes`, `acs.policy.ids`, `acs.policy.versions`, `acs.policy.rule_ids`, `acs.cited_provenance_ids` and `acs.chain_hash` only when the response carries them. `acs.evaluator` appears when the Guardian's response carries `metadata.evaluator`. This Guardian sends it once PR #229 is merged. -This Guardian does not send `chain_hash`, `cited_provenance_ids`, or `policy_version` on its responses yet. `acs.chain_hash` and `acs.cited_provenance_ids` are absent from its spans, and `acs.policy.versions` carries `""` for each policy reference. The three policy arrays always have the same length as `policy_references`, with `""` where a reference has no value. `handshake/hello`, methods this Guardian does not dispatch, and unparseable bodies produce no span. +This Guardian does not send `chain_hash`, `cited_provenance_ids`, or `policy_version` on its responses yet. `acs.chain_hash` and `acs.cited_provenance_ids` are absent from its spans, and `acs.policy.versions` carries `""` for each policy reference. The three policy arrays always have the same length as `policy_references`, with `""` where a reference has no value. A valid `handshake/hello`, a valid request for a method this Guardian does not dispatch, and an unparseable body produce no span. -A step that got no decision is recorded as an `acs.decision.failure` event on the same span, with `acs.failure.class`, `acs.failure.posture` and `acs.failure.reported_by`, plus the error code, the timeout, the elapsed time and the request size when known. The Guardian emits it for an error it answered instead of a decision, for a decision it finished after the negotiated timeout, and for a step still open when it shut down. It carries the error code but never the error text, because that text can quote the request. A replay rejection (`-32005`) gets no event, because its pair belongs to the earlier request. The converter below produces the same event, reported by the observed agent, from the host adapter's audit log. +A step that got no decision is recorded as an `acs.decision.failure` event on the same span, with `acs.failure.class`, `acs.failure.posture` and `acs.failure.reported_by`, plus the error code, the timeout, the elapsed time and the request size when known. The Guardian emits it for an error it answered instead of a decision, for a decision it finished after the negotiated timeout, and for a step still open when it shut down. It carries the error code but never the error text, because that text can quote the request. The converter below produces the same event, reported by the observed agent, from the host adapter's audit log. + +A request the Guardian cannot vouch for is recorded apart from the steps it governed, as a `guardian.rejected_request` span. Such a request can carry any session and request id, a genuine request's included. So the span carries no `acs.session.id`, `acs.request_id`, step attribute or `acs.decision` event. It records the ids the request claimed as `guardian.rejected.claimed_session_id` and `guardian.rejected.claimed_request_id`, with `guardian.rejected.reason` and, for an error answer, `guardian.rejected.error_code`. These names belong to this Guardian, not to the ACS mapping. + +The reasons are `envelope_invalid`, for an envelope that fails validation, and `signature_invalid`, `replay_detected` or `timestamp_out_of_window`, for a step answered with that error by whichever check produced it. The last is `guardian_fault`, for a request the Guardian could not check at all, a missing schema directory for example. The host got an error for that one, so its span also carries an `acs.decision.failure` with class `guardian_error`. This Guardian verifies no signature, replay or timestamp yet (PR #229), so a forged or replayed request that passes the schema is still recorded as a step. The file holds one OTLP `ExportTraceServiceRequest` per line, the shape the Collector's file exporter writes, with `service.name` set to `acs-reference-guardian` (the converter below uses `acs-reference-audit-converter`, and both are labels, not attribution). Spans are batched and flushed when the Guardian closes on SIGINT or SIGTERM. A SIGKILL loses the last batch, and so does an output that has not finished three seconds into shutdown: the Guardian reports it on stderr and exits instead of waiting. [`otel-collector.yaml`](../../otel-collector.yaml) at the root of this tree receives the endpoint output and writes the same file shape. @@ -116,7 +122,7 @@ It prints how many spans reached the file and exits non-zero when that is fewer ### Sample trace -[`samples/trace/acs-trace-sample.jsonl`](../../samples/trace/acs-trace-sample.jsonl) is what this export looks like, so you can read it without running the Guardian. It holds six steps: an allowed tool call and its result, linked by `acs.request_id_ref`, a denied call carrying reasoning, reason codes, policy references and provenance, a call the Guardian answered with an error, a decision that finished after the timeout, and a step the host could not deliver, converted from its audit log. The decisions come from the stock policy, and the one late decision is delayed on purpose. Every value in it is synthetic. +[`samples/trace/acs-trace-sample.jsonl`](../../samples/trace/acs-trace-sample.jsonl) is what this export looks like, so you can read it without running the Guardian. It holds five steps and one rejected request: an allowed tool call and its result, linked by `acs.request_id_ref`, a denied call carrying reasoning, reason codes, policy references and provenance, a forged request that reuses the allowed call's ids and fails the schema, recorded as a `guardian.rejected_request` span so the allowed call keeps the only span under its pair, a decision that finished after the timeout, and a step the host could not deliver, converted from its audit log. The decisions come from the stock policy, and the one late decision is delayed on purpose. Every value in it is synthetic. ```bash bun run sample:trace diff --git a/reference-implementations/agt/packages/guardian/src/audit-to-otlp.ts b/reference-implementations/agt/packages/guardian/src/audit-to-otlp.ts index 7dd92b10..9cf327bb 100644 --- a/reference-implementations/agt/packages/guardian/src/audit-to-otlp.ts +++ b/reference-implementations/agt/packages/guardian/src/audit-to-otlp.ts @@ -290,10 +290,11 @@ export async function convertAuditLog(auditPath: string, otelPath: string, mappi if (sessionId !== undefined) { attributes["acs.session.id"] = sessionUuid(sessionId); } - // The host builds every envelope with its JSON-RPC id equal to - // params.request_id (build-envelope.ts), so the entry's rpc_id is the - // request id the Guardian's span for the same request carries. - const requestId = readString(entry.rpc_id); + // The entry's request_id is the id the Guardian's span for the same + // request carries. Entries written before that field existed have only + // rpc_id, which equalled params.request_id whenever the host did not + // derive the request id from its own call id. + const requestId = readString(entry.request_id) ?? readString(entry.rpc_id); if (requestId !== undefined) { attributes["acs.request_id"] = requestId; } diff --git a/reference-implementations/agt/packages/guardian/src/envelope-log-sink.ts b/reference-implementations/agt/packages/guardian/src/envelope-log-sink.ts index e83d80f2..ed217338 100644 --- a/reference-implementations/agt/packages/guardian/src/envelope-log-sink.ts +++ b/reference-implementations/agt/packages/guardian/src/envelope-log-sink.ts @@ -28,8 +28,13 @@ import { appendFileSync, mkdirSync } from "node:fs"; import { dirname } from "node:path"; -/** Which side of the exchange one envelope-log line recorded. */ -export type EnvelopeLogDirection = "request" | "response"; +/** + * Which side of the exchange one envelope-log line recorded, or "finding" for + * a fact the Guardian established about a request that the spec requires it + * to record. A finding's `envelope` is the Guardian's own record, not a + * JSON-RPC object, and its `rpc_id` names the request it concerns. + */ +export type EnvelopeLogDirection = "request" | "response" | "finding"; /** * One line of the envelope log, named for what was recorded rather than for @@ -49,9 +54,14 @@ export type EnvelopeLogEntry = { envelope: unknown; }; -/** Where envelope-log lines go: the Guardian-side writer role. */ +/** + * Where envelope-log lines go: the Guardian-side writer role. `write` says + * whether the line landed. Most callers ignore that, because the log is + * observability. A caller recording a fact the spec requires it to keep + * reads it and puts the fact somewhere else when the log did not take it. + */ export type EnvelopeLogSink = { - write(direction: EnvelopeLogDirection, envelope: unknown, method: string | null): void; + write(direction: EnvelopeLogDirection, envelope: unknown, method: string | null, rpcId?: string | number | null): boolean; readonly path: string | null; }; @@ -68,7 +78,9 @@ export type CreateEnvelopeLogSinkOptions = { * `EnvelopeLogSink` type, not a special case. */ export const NULL_ENVELOPE_LOG_SINK: EnvelopeLogSink = { path: null, - write(): void {}, + write(): boolean { + return false; + }, }; /** The JSON-RPC id, when it is a scalar. Both request and response envelopes @@ -114,9 +126,9 @@ export function createEnvelopeLogSink({ return { path, - write(direction, envelope, method): void { + write(direction, envelope, method, rpcId): boolean { if (disabled) { - return; + return false; } try { const entry: EnvelopeLogEntry = { @@ -124,14 +136,18 @@ export function createEnvelopeLogSink({ recorded_at: now().toISOString(), direction, method, - rpc_id: extractRpcId(envelope), + // A finding carries no JSON-RPC id of its own, so the caller names + // the request it concerns. + rpc_id: rpcId === undefined ? extractRpcId(envelope) : rpcId, envelope, }; const line = `${JSON.stringify(entry)}\n`; appendFileSync(path, line); seq += 1; + return true; } catch (error) { fail(error); + return false; } }, }; diff --git a/reference-implementations/agt/packages/guardian/src/otel-mapping.ts b/reference-implementations/agt/packages/guardian/src/otel-mapping.ts index 05d9443b..0d9b900b 100644 --- a/reference-implementations/agt/packages/guardian/src/otel-mapping.ts +++ b/reference-implementations/agt/packages/guardian/src/otel-mapping.ts @@ -26,6 +26,11 @@ const PROVENANCE_COLUMN_LIMIT = 64; * earlier request, so #228 forbids a failure event for it: one would mark * that earlier, legitimate decision as not enforced. */ export const REPLAY_DETECTED_CODE = -32005; +/** JSON-RPC SIGNATURE_INVALID and TIMESTAMP_OUT_OF_WINDOW (Specification + * §17.1). Defined here beside REPLAY_DETECTED_CODE so the Guardian's checks + * and the trace's rejection mapping read one number for each. */ +export const SIGNATURE_INVALID_CODE = -32004; +export const TIMESTAMP_OUT_OF_WINDOW_CODE = -32006; // The repository's own mapping, five directories up from this file, the // same relationship SCHEMA_ROOT in validate-envelope.ts relies on. @@ -48,6 +53,46 @@ export const DECISION_FAILURE_EVENT = "acs.decision.failure"; * so a query on status text finds both sides of the same failure. */ export const GUARDIAN_ERROR_STATUS = "The Guardian returned an error instead of a decision."; +/** + * A request this Guardian cannot vouch for, recorded apart from the steps it + * governed. Trace Events forbids an acs.decision or acs.decision.failure + * under the pair of a request rejected before it was trusted, because such a + * request can carry any pair, a genuine request's included. Issue #283 ruled + * that it is still recorded, under names outside `step_to_span`, so a forged + * or replayed request stays visible without being attributed to the request + * whose ids it carries. These names belong to this reference Guardian, not + * to the ACS mapping. + */ +export const REJECTED_SPAN_NAME = "guardian.rejected_request"; +export const REJECTED_CLAIMED_SESSION_ID = "guardian.rejected.claimed_session_id"; +export const REJECTED_CLAIMED_REQUEST_ID = "guardian.rejected.claimed_request_id"; +export const REJECTED_REASON = "guardian.rejected.reason"; +export const REJECTED_ERROR_CODE = "guardian.rejected.error_code"; +export const REJECTED_STATUS = "The Guardian could not vouch for this request, so it carries no ACS ids or events."; + +/** Why a request was rejected before it was trusted. `guardian_fault` is the + * Guardian failing to check the request at all (a missing schema directory, + * say): its ids are just as unverified, so it is recorded the same way. */ +export type RejectionReason = + | "envelope_invalid" + | "guardian_fault" + | "signature_invalid" + | "replay_detected" + | "timestamp_out_of_window"; + +/** + * The §17.1 codes that mean the Guardian rejected a request before trusting + * it. dispatch reports its own rejections to the exporter directly. This is + * the backstop: a step answered with one of these codes is recorded as a + * rejected request even if the check that produced it forgot to report, so + * a rejected request's ids never land on a step span. + */ +export const PRE_TRUST_REJECTION_CODES: ReadonlyMap = new Map([ + [SIGNATURE_INVALID_CODE, "signature_invalid"], + [REPLAY_DETECTED_CODE, "replay_detected"], + [TIMESTAMP_OUT_OF_WINDOW_CODE, "timestamp_out_of_window"], +]); + /** The event's attribute keys: the three required ones first, then the * ones carried when known. */ export const FAILURE_ATTRIBUTES = { diff --git a/reference-implementations/agt/packages/guardian/src/otel-trace-exporter.ts b/reference-implementations/agt/packages/guardian/src/otel-trace-exporter.ts index b28bb85b..be3b0265 100644 --- a/reference-implementations/agt/packages/guardian/src/otel-trace-exporter.ts +++ b/reference-implementations/agt/packages/guardian/src/otel-trace-exporter.ts @@ -66,13 +66,20 @@ import { PROVENANCE_ATTRIBUTE_SOURCES, provenanceColumn, provenanceObjects, - REPLAY_DETECTED_CODE, + PRE_TRUST_REJECTION_CODES, + REJECTED_CLAIMED_REQUEST_ID, + REJECTED_CLAIMED_SESSION_ID, + REJECTED_ERROR_CODE, + REJECTED_REASON, + REJECTED_SPAN_NAME, + REJECTED_STATUS, requestParts, resultParts, STEP_ATTRIBUTE_SOURCES, type DecisionFailure, type FailureClass, type OtelMapping, + type RejectionReason, } from "./otel-mapping.ts"; import { createFileSpanExporter } from "./otlp-file-exporter.ts"; @@ -86,8 +93,10 @@ export { GUARDIAN_ERROR_STATUS, loadOtelMapping, METHODS_WITHOUT_SPAN, + REJECTED_SPAN_NAME, STEP_ATTRIBUTE_SOURCES, type DecisionFailure, + type RejectionReason, type FailureClass, type FailurePosture, type FailureReporter, @@ -149,16 +158,51 @@ export type StepSpan = { readonly method: string; readonly startedAt: number; readonly payloadBytes?: number; + /** The session and request id the request claimed. They go onto the span + * as acs.session.id and acs.request_id only when it ends as a trusted step + * (see `finish`), and under the rejected names otherwise. */ + readonly claimed: Attributes; + /** Every other step attribute, read from the request and held back the + * same way, so a span that ends as a rejected request carries none of + * them (`acs.request_id_ref` on a forged result names a genuine call). */ + readonly held: Attributes; + /** Set when the request was rejected before its span opened. */ + readonly rejected?: RejectionReason; }; +/** + * What `start` is told about the request besides its envelope. `trusted` is + * whether the Guardian accepted the envelope before acting on it. A request + * it did not accept is recorded as a rejected request, apart from the steps + * it governed, with `reason` saying why (see REJECTED_SPAN_NAME). `trusted` + * is required, not defaulted, so a caller that forgets it fails to compile + * rather than exporting an untrusted request's ids as a step. + * + * Trusted here means accepted by every check this Guardian runs. This + * reference Guardian checks the envelope against the schema and nothing + * else: it verifies no signature, replay or timestamp yet (PR #229), so a + * forgery or a replay that passes the schema is still recorded as a step. + * + * `receivedAt` is `performance.now()` when the request arrived, so the time + * the Guardian spent checking it counts toward a late decision. + */ +export type StepStart = { payloadBytes?: number; receivedAt?: number } & ( + | { trusted: true } + | { trusted: false; reason: RejectionReason } +); + /** The Guardian-side writer role, called at the envelope log's own two * points: `start` where the request is recorded, `end` where the response * is, with the handle `start` returned. */ export type OtelTraceExporter = { /** Null when this request produces no span, and `end` then does nothing. - * `payloadBytes` is the request body's size, for the failure event. */ - start(envelope: unknown, method: string | null, payloadBytes?: number): StepSpan | null; - end(stepSpan: StepSpan | null, response: unknown): void; + * `payloadBytes` is the request body's size, for the failure event. An + * untrusted request produces a rejected-request span (see StepStart). */ + start(envelope: unknown, method: string | null, step: StepStart): StepSpan | null; + /** `rejection` is set when the Guardian rejected the request before + * trusting it after the span opened (a signature, replay or timestamp + * check inside dispatch), whatever shape its answer took. */ + end(stepSpan: StepSpan | null, response: unknown, rejection?: RejectionReason): void; /** Pushes every ended span through to the exporters now rather than on * the batch timer. Tests call it; shutdown does too. */ flush(): Promise; @@ -279,6 +323,22 @@ function isolated({ label, exporter }: Output, report: (label: string, error: un }; } +/** A rejected request's attributes: the ids it claimed, under names a reader + * cannot mistake for the ACS pair, and why it was rejected. */ +function rejectedAttributes(claimed: Attributes, reason: RejectionReason, errorCode?: number): Attributes { + const attributes: Attributes = { [REJECTED_REASON]: reason }; + if (claimed["acs.session.id"] !== undefined) { + attributes[REJECTED_CLAIMED_SESSION_ID] = claimed["acs.session.id"]; + } + if (claimed["acs.request_id"] !== undefined) { + attributes[REJECTED_CLAIMED_REQUEST_ID] = claimed["acs.request_id"]; + } + if (errorCode !== undefined) { + attributes[REJECTED_ERROR_CODE] = errorCode; + } + return attributes; +} + export function createOtelTraceExporter({ filePath, endpoint, @@ -407,7 +467,32 @@ export function createOtelTraceExporter({ }); } - function begin(envelope: unknown, method: string | null, payloadBytes: number | undefined, tracer: Tracer, mapping: OtelMapping): StepSpan | null { + function begin(envelope: unknown, method: string | null, step: StepStart, tracer: Tracer, mapping: OtelMapping): StepSpan | null { + const parts = requestParts(envelope); + // Held back, not set: the pair, and every other step attribute, goes on + // only when the span ends as a trusted step, because a check that runs + // after this point can still reject the request. + const claimed: Attributes = {}; + collect(claimed, Object.keys(CORRELATION_ATTRIBUTE_SOURCES), CORRELATION_ATTRIBUTE_SOURCES, parts); + const startedAt = step.receivedAt ?? performance.now(); + // The span's own start is the arrival too, so its duration agrees with + // the elapsed time the failure event reports. + const startTime = step.receivedAt; + // A rejected request is recorded whatever its method, handled or not: + // its span needs no step_to_span entry. + if (!step.trusted) { + const rejected: StepSpan = { + span: tracer.startSpan(REJECTED_SPAN_NAME, { attributes: rejectedAttributes(claimed, step.reason), root: true, startTime }), + method: method ?? "", + startedAt, + payloadBytes: step.payloadBytes, + claimed, + held: {}, + rejected: step.reason, + }; + openSpans.add(rejected); + return rejected; + } if (method === null || !handled.has(method)) { return null; } @@ -415,9 +500,7 @@ export function createOtelTraceExporter({ if (entry === undefined) { return null; } - const parts = requestParts(envelope); const attributes: Attributes = {}; - collect(attributes, Object.keys(CORRELATION_ATTRIBUTE_SOURCES), CORRELATION_ATTRIBUTE_SOURCES, parts); collect( attributes, [...entry.required_attributes, ...(entry.conditional_attributes ?? []), ...(entry.optional_attributes ?? [])], @@ -437,17 +520,46 @@ export function createOtelTraceExporter({ } } const stepSpan: StepSpan = { - span: tracer.startSpan(entry.span_name, { attributes, root: true }), + span: tracer.startSpan(entry.span_name, { root: true, startTime }), method, - startedAt: performance.now(), - payloadBytes, + startedAt, + payloadBytes: step.payloadBytes, + claimed, + held: attributes, }; openSpans.add(stepSpan); return stepSpan; } - function finish(stepSpan: StepSpan, envelope: unknown, mapping: OtelMapping): void { + /** Ends a span as a rejected request: renamed if it opened as a step, the + * claimed ids under the rejected names, and no ACS pair, step attribute or + * decision event. A Guardian fault is still a decision failure for the + * host, which got an error, so it keeps that event, on a span with no pair. */ + function finishRejected(stepSpan: StepSpan, reason: RejectionReason, errorCode?: number): void { + const { span } = stepSpan; + span.updateName(REJECTED_SPAN_NAME); + span.setAttributes(rejectedAttributes(stepSpan.claimed, reason, errorCode)); + span.setStatus({ code: SpanStatusCode.ERROR, message: REJECTED_STATUS }); + if (reason === "guardian_fault") { + span.addEvent(DECISION_FAILURE_EVENT, failureAttributes(stepSpan, "guardian_error", { errorCode })); + } + span.end(); + } + + function finish(stepSpan: StepSpan, envelope: unknown, mapping: OtelMapping, rejection: RejectionReason | undefined): void { const { span } = stepSpan; + const error = asObject(asObject(envelope)?.error); + const errorCode = asNumber(error?.code); + // Rejected before the span opened, rejected by a check after it opened, + // or answered with a code that only a rejection carries: each ends as a + // rejected request, and no held attribute reaches it. + const reason = + stepSpan.rejected ?? rejection ?? (errorCode === undefined ? undefined : PRE_TRUST_REJECTION_CODES.get(errorCode)); + if (reason !== undefined) { + finishRejected(stepSpan, reason, errorCode); + return; + } + span.setAttributes({ ...stepSpan.claimed, ...stepSpan.held }); const decision = resultParts(envelope); if (decision === undefined) { // A JSON-RPC error: the step was answered but not decided. The first @@ -455,12 +567,8 @@ export function createOtelTraceExporter({ // carried: the message can quote the request (a schema error names its // argument keys), and #228 forbids copying response or error bytes into // the trace, so the status message is a fixed sentence too. - const error = asObject(asObject(envelope)?.error); - const errorCode = asNumber(error?.code); span.setStatus({ code: SpanStatusCode.ERROR, message: GUARDIAN_ERROR_STATUS }); - if (errorCode !== REPLAY_DETECTED_CODE) { - span.addEvent(DECISION_FAILURE_EVENT, failureAttributes(stepSpan, "guardian_error", { errorCode })); - } + span.addEvent(DECISION_FAILURE_EVENT, failureAttributes(stepSpan, "guardian_error", { errorCode })); } else { const attributes: Attributes = {}; collect(attributes, new Set([...mapping.decisionAttributes, ...DECISION_DETAIL_NAMES]), DECISION_ATTRIBUTE_SOURCES, decision); @@ -484,18 +592,18 @@ export function createOtelTraceExporter({ let shutdownPromise: Promise | undefined; return { - start(envelope, method, payloadBytes): StepSpan | null { + start(envelope, method, step): StepSpan | null { if (disabled || stopped || tracer === undefined || mapping === undefined) { return null; } try { - return begin(envelope, method, payloadBytes, tracer, mapping); + return begin(envelope, method, step, tracer, mapping); } catch (error) { fail(error); return null; } }, - end(stepSpan, response): void { + end(stepSpan, response, rejection): void { if (stepSpan === null) { return; } @@ -506,7 +614,7 @@ export function createOtelTraceExporter({ return; } try { - finish(stepSpan, response, mapping); + finish(stepSpan, response, mapping, rejection); } catch (error) { fail(error); } @@ -532,6 +640,11 @@ export function createOtelTraceExporter({ // so the step is exported at all. for (const stepSpan of openSpans) { try { + if (stepSpan.rejected !== undefined) { + finishRejected(stepSpan, stepSpan.rejected); + continue; + } + stepSpan.span.setAttributes({ ...stepSpan.claimed, ...stepSpan.held }); stepSpan.span.addEvent( DECISION_FAILURE_EVENT, failureAttributes(stepSpan, "other", { detail: "the Guardian shut down before it answered this step" }), diff --git a/reference-implementations/agt/packages/guardian/src/server.ts b/reference-implementations/agt/packages/guardian/src/server.ts index 922c11bd..8a38eed4 100644 --- a/reference-implementations/agt/packages/guardian/src/server.ts +++ b/reference-implementations/agt/packages/guardian/src/server.ts @@ -89,17 +89,21 @@ import { isToolCallResult, validateEnvelope, type AcsRequestEnvelope, + type ToolCallRequestEnvelope, + type ToolCallResultEnvelope, } from "./validate-envelope.ts"; import { checkResponse } from "./check-response.ts"; import { buildServerHello, type ServerHello } from "./handshake.ts"; import { createEnvelopeLogSink, NULL_ENVELOPE_LOG_SINK, type EnvelopeLogSink } from "./envelope-log-sink.ts"; import { createOtelTraceExporter, NULL_OTEL_TRACE_EXPORTER, type OtelTraceExporter } from "./otel-trace-exporter.ts"; +import type { RejectionReason } from "./otel-mapping.ts"; import { appendContextEntry, createMemorySessionContextStore, type IfcLabels, type SessionContextStore, } from "./session-context-store.ts"; +import type { StepFindings } from "./session-context.ts"; import { persistIfcLabels, supplySourceLabels } from "./ifc-labels.ts"; /** @@ -568,13 +572,30 @@ async function handleAcsRequest( // Recorded before validation, so an envelope that fails the schema is // visible to the Inspector rather than invisible. + const receivedAt = performance.now(); const method = extractMethod(raw); envelopeLog.write("request", raw, method); - const stepSpan = traceExporter.start(raw, method, Buffer.byteLength(body.text)); + // Validated once, here, before the span opens, because the span has to + // know whether the Guardian trusts this request: Trace Events forbids an + // acs.decision under the ids of a request rejected before it was trusted. + // A rejected request is recorded apart, as a rejected-request span (issue + // #283). dispatch acts on the same result rather than validating again. A + // signature, replay or timestamp rejection answered from inside dispatch + // is caught by its error code when the span ends (PRE_TRUST_REJECTION_CODES). + const validation = validateForDispatch(raw); + const payloadBytes = Buffer.byteLength(body.text); + const stepSpan = traceExporter.start( + raw, + method, + validation.kind === "valid" + ? { trusted: true, payloadBytes, receivedAt } + : { trusted: false, reason: validation.kind === "invalid" ? "envelope_invalid" : "guardian_fault", payloadBytes, receivedAt }, + ); let response: JsonRpcSuccess | JsonRpcFailure; + let rejection: RejectionReason | undefined; try { - response = await dispatch(raw, bridge, mapping, onDecisionFailure, sessionContextStore); + ({ response, rejection } = await dispatch(raw, validation, bridge, mapping, onDecisionFailure, sessionContextStore, envelopeLog)); } catch (error) { // The outer net. Deliberately a bare JSON-RPC error, not an ACS `deny`: // this route is `dispatch` rethrowing entirely past denyOnInvalidEnvelope @@ -616,7 +637,7 @@ async function handleAcsRequest( } envelopeLog.write("response", response, method); - traceExporter.end(stepSpan, response); + traceExporter.end(stepSpan, response, rejection); return response; } @@ -712,44 +733,175 @@ async function readCappedBody(req: Request): Promise { : { withinLimit: true, text: text + decoder.decode() }; } +/** + * What this Guardian establishes about a step as it arrives, before the step + * is appended to the chain. + * + * `request_id_ref_unresolved`: hooks.md, toolCallResult. A Guardian SHOULD + * resolve `request_id_ref` against the session's `toolCallRequest` records and + * MUST record one it cannot resolve. The decision is left to policy, so this + * only records. Resolution reads the session's own requests, because + * `request_id` is unique only within a session. A restart or an eviction + * makes an older reference unresolvable, and that is recorded too. + * + * `request_id_duplicate`: Specification §10.3 says a Guardian MUST reject a + * `request_id` the session has already carried, on any method. This Guardian + * does not enforce that yet, so it records the repeat instead. + * + * The findings go onto the step's own chain entry, which is this Guardian's + * audit record, and a copy goes to the envelope log (or stderr) for reading. + */ +function examineStep( + envelope: ToolCallRequestEnvelope | ToolCallResultEnvelope, + sessionContextStore: SessionContextStore, + envelopeLog: EnvelopeLogSink, +): StepFindings | undefined { + const sessionId = envelope.params.metadata.session_id; + const requestId = envelope.params.request_id; + const findings: StepFindings["findings"] = []; + let unresolvedRef: string | undefined; + + if (sessionContextStore.hasRequestId(sessionId, requestId)) { + findings.push("request_id_duplicate"); + recordFinding(envelopeLog, { finding: "request_id_duplicate", session_id: sessionId, request_id: requestId }, envelope.method, envelope.id); + } + + const ref = isToolCallResult(envelope) ? envelope.params.payload.request_id_ref : undefined; + if (ref !== undefined && !sessionContextStore.hasToolCallRequest(sessionId, ref)) { + findings.push("request_id_ref_unresolved"); + unresolvedRef = ref; + recordFinding( + envelopeLog, + { finding: "request_id_ref_unresolved", session_id: sessionId, request_id: requestId, request_id_ref: ref }, + envelope.method, + envelope.id, + ); + } + + if (findings.length === 0) { + return undefined; + } + return unresolvedRef === undefined ? { findings } : { findings, request_id_ref: unresolvedRef }; +} + +/** + * examineStep, unable to throw. A caller-supplied store that throws on the + * lookup costs the findings, never the decision. + */ +function examineStepSafely( + envelope: ToolCallRequestEnvelope | ToolCallResultEnvelope, + sessionContextStore: SessionContextStore, + envelopeLog: EnvelopeLogSink, +): StepFindings | undefined { + try { + return examineStep(envelope, sessionContextStore, envelopeLog); + } catch (error) { + reportCheckFailure("step findings", error); + return undefined; + } +} + +/** + * validateEnvelope's outcome, held rather than thrown so handleAcsRequest can + * read it before the trace span opens and dispatch can act on it after. + * `failed` is anything validateEnvelope threw that is not a validation + * verdict (a missing schema directory, say). dispatch rethrows it, so it + * still reaches the outer net exactly as it did when dispatch validated. + */ +type Validation = + | { kind: "valid"; envelope: AcsRequestEnvelope } + | { kind: "invalid"; error: EnvelopeValidationError } + | { kind: "failed"; error: unknown }; + +function validateForDispatch(raw: unknown): Validation { + try { + return { kind: "valid", envelope: validateEnvelope(raw) }; + } catch (error) { + return error instanceof EnvelopeValidationError ? { kind: "invalid", error } : { kind: "failed", error }; + } +} + +/** + * Writes a finding to the envelope log, or, when no envelope log was + * configured or it disabled itself after a write error, to stderr as + * `acs-finding ` followed by one JSON object carrying the same fields the log + * line would. Never throws: a finding is a record, and losing the record must + * never cost the decision it sits beside. + */ +function recordFinding( + envelopeLog: EnvelopeLogSink, + finding: Record, + method: string, + rpcId: string | number | null, +): void { + if (envelopeLog.write("finding", finding, method, rpcId)) { + return; + } + try { + console.error( + `acs-finding ${JSON.stringify({ recorded_at: new Date().toISOString(), method, rpc_id: rpcId, ...finding })}`, + ); + } catch { + // stderr itself failed (an EPIPE, say). Nothing is left to report to. + } +} + +/** One stderr line for a check that threw, itself unable to throw. */ +function reportCheckFailure(check: string, error: unknown): void { + try { + console.error(`${check} check failed: ${error instanceof Error ? error.message : String(error)}`); + } catch { + // stderr itself failed. The decision still goes ahead. + } +} + +/** + * What dispatch answered, and whether it rejected the request before trusting + * it. `rejection` is how a check inside dispatch tells the trace exporter so, + * whatever the response looks like: a signature, replay or timestamp check + * (PR #229) answering with an error, or with an honoured deny the way an + * invalid envelope is answered. The exporter then records the request apart + * from the steps instead of under the ids it claimed. + */ +type Dispatched = { response: JsonRpcSuccess | JsonRpcFailure; rejection?: RejectionReason }; + async function dispatch( raw: unknown, + validation: Validation, bridge: PolicyBridge, mapping: Mapping, onDecisionFailure: "proceed" | "deny" | undefined, sessionContextStore: SessionContextStore, -): Promise { + envelopeLog: EnvelopeLogSink, +): Promise { const rpcId = extractId(raw); - let envelope: AcsRequestEnvelope; - try { - // validateEnvelope checks the general request-envelope.json shape for - // every method, plus -- only for steps/toolCallRequest -- the - // hook-specific payload schema. Failure is a THROWN typed error, never - // a decision itself -- validateEnvelope stays total to its own contract - // (see its doc comment) -- but the catch below (denyOnInvalidEnvelope) - // turns a steps/* failure into an honoured ACS `deny` decision rather - // than a bare JSON-RPC error, since there is an identifiable step to - // answer for. A handshake failure and an undispatched method are not - // steps/*, so they always fall through to the JSON-RPC error unchanged. - envelope = validateEnvelope(raw); - } catch (error) { - if (error instanceof EnvelopeValidationError) { - if (isStepMethod(raw)) { - const denial = denyOnInvalidEnvelope(raw, { reasonCode: "envelope_invalid", message: error.message }); - const decisionResponse = asDecisionResponse(rpcId, denial); - if (decisionResponse) { - return decisionResponse; - } + // validateEnvelope checked the general request-envelope.json shape for + // every method, plus -- only for steps/toolCallRequest -- the hook-specific + // payload schema (validateForDispatch, above). An invalid steps/* envelope + // becomes an honoured ACS `deny` decision rather than a bare JSON-RPC error, + // since there is an identifiable step to answer for. A handshake failure + // and an undispatched method are not steps/*, so they always fall through + // to the JSON-RPC error unchanged. + if (validation.kind === "failed") { + throw validation.error; + } + if (validation.kind === "invalid") { + const { error } = validation; + if (isStepMethod(raw)) { + const denial = denyOnInvalidEnvelope(raw, { reasonCode: "envelope_invalid", message: error.message }); + const decisionResponse = asDecisionResponse(rpcId, denial); + if (decisionResponse) { + return { response: decisionResponse, rejection: "envelope_invalid" }; } - return errorResponse(rpcId, ENVELOPE_INVALID_CODE, error.message, { pointer: error.pointer }); } - throw error; + return { response: errorResponse(rpcId, ENVELOPE_INVALID_CODE, error.message, { pointer: error.pointer }), rejection: "envelope_invalid" }; } + const envelope = validation.envelope; if (envelope.method === HANDSHAKE_METHOD) { const env = onDecisionFailure === undefined ? undefined : { ACS_ON_DECISION_FAILURE: onDecisionFailure }; - return successResponse(envelope.id, buildServerHello(env)); + return { response: successResponse(envelope.id, buildServerHello(env)) }; } // Two gated branches, one per ACS method this Guardian assembles a snapshot @@ -773,21 +925,25 @@ async function dispatch( // shape. Anything neither predicate answers falls past both, to // METHOD_NOT_DISPATCHED_CODE below. if (isToolCallRequest(envelope)) { - return await evaluateStep(raw, envelope, assemblePreToolCallSnapshot, bridge, mapping, sessionContextStore); + // Examined before evaluateStep appends this step, so a request id is in + // the session already only if an earlier step carried it. + const findings = examineStepSafely(envelope, sessionContextStore, envelopeLog); + return { response: await evaluateStep(raw, envelope, assemblePreToolCallSnapshot, bridge, mapping, sessionContextStore, findings) }; } // The result gate, beside the request gate rather than merged into it. if (isToolCallResult(envelope)) { - return await evaluateStep(raw, envelope, assemblePostToolCallSnapshot, bridge, mapping, sessionContextStore); + const findings = examineStepSafely(envelope, sessionContextStore, envelopeLog); + return { response: await evaluateStep(raw, envelope, assemblePostToolCallSnapshot, bridge, mapping, sessionContextStore, findings) }; } // A well-formed envelope (it passed validateEnvelope: the method matched // the required prefix pattern) naming a method this Guardian has no // handler for -- distinct from a malformed envelope, which never reaches // here because validateEnvelope already threw above. - return errorResponse(rpcId, METHOD_NOT_DISPATCHED_CODE, `method not dispatched by this Guardian: ${envelope.method}`, { + return { response: errorResponse(rpcId, METHOD_NOT_DISPATCHED_CODE, `method not dispatched by this Guardian: ${envelope.method}`, { method: envelope.method, - }); + }) }; } /** @@ -857,6 +1013,7 @@ async function evaluateStep( bridge: PolicyBridge, mapping: Mapping, sessionContextStore: SessionContextStore, + findings?: StepFindings, ): Promise { try { // The chain entry is appended on arrival, before any verdict exists: a @@ -867,6 +1024,7 @@ async function evaluateStep( method: envelope.method, request_id: envelope.params.request_id, tool_name: envelope.params.payload.tool.name, + ...findings, }); // Resolved BEFORE the snapshot is assembled, where it used to be resolved diff --git a/reference-implementations/agt/packages/guardian/src/session-context-store.ts b/reference-implementations/agt/packages/guardian/src/session-context-store.ts index 11e0ca9a..e4bb2937 100644 --- a/reference-implementations/agt/packages/guardian/src/session-context-store.ts +++ b/reference-implementations/agt/packages/guardian/src/session-context-store.ts @@ -99,6 +99,8 @@ export { /** What the Guardian's request path is told and asks on every step. */ export interface SessionContextStore { context(sessionId: string): SessionContext; + hasToolCallRequest(sessionId: string, requestId: string): boolean; + hasRequestId(sessionId: string, requestId: string): boolean; append(sessionId: string, step: SessionStep): SessionContextEntry; replaceIfcLabels(sessionId: string, labels: IfcLabels): void; sourceLabels(sessionId: string): IfcLabels; @@ -226,6 +228,16 @@ export function createMemorySessionContextStore( return { session_id: sessionId, entries: [...read(sessionId).entries] }; }, + // Straight off the map rather than through `read`, which builds an empty + // state for a session it does not hold. Neither question needs one. + hasToolCallRequest(sessionId, requestId) { + return sessions.get(sessionId)?.toolCallRequestIds.has(requestId) ?? false; + }, + + hasRequestId(sessionId, requestId) { + return sessions.get(sessionId)?.requestIds.has(requestId) ?? false; + }, + append(sessionId, step) { const state = ensure(sessionId); const previous = state.entries.at(-1); @@ -237,6 +249,9 @@ export function createMemorySessionContextStore( method: step.method, request_id: step.request_id, tool_name: step.tool_name, + ...(step.findings === undefined + ? {} + : { findings: step.findings, ...(step.request_id_ref === undefined ? {} : { request_id_ref: step.request_id_ref }) }), }; const entry: SessionContextEntry = { ...withoutHash, hash: hashEntry(withoutHash) }; // Pushed onto the array this session already owns, rather than rebuilt @@ -244,6 +259,10 @@ export function createMemorySessionContextStore( // N per step, N^2 per session -- and the module doc above says what a // Guardian that slow does to the decision that never arrives. state.entries.push(entry); + state.requestIds.add(step.request_id); + if (step.method === "steps/toolCallRequest") { + state.toolCallRequestIds.add(step.request_id); + } appendLine?.(JSON.stringify(entry)); return entry; }, diff --git a/reference-implementations/agt/packages/guardian/src/session-context.ts b/reference-implementations/agt/packages/guardian/src/session-context.ts index fe114b59..d3177858 100644 --- a/reference-implementations/agt/packages/guardian/src/session-context.ts +++ b/reference-implementations/agt/packages/guardian/src/session-context.ts @@ -89,7 +89,19 @@ export type SessionProvenance = { export type Intent = { readonly text: string; readonly recorded_at: string }; /** What a step contributes to the chain. Deliberately not the whole envelope. */ -export type SessionStep = { method: string; request_id: string; tool_name: string }; +/** + * What the Guardian established about a step when it arrived, carried on the + * step's own chain entry so the published chain commits to it. `hooks.md` + * requires a Guardian to record a `request_id_ref` it cannot resolve, and the + * chain is this Guardian's audit record. `request_id_ref` is the unresolved + * reference, present only with `request_id_ref_unresolved`. + */ +export type StepFindings = { + findings: Array<"request_id_ref_unresolved" | "request_id_duplicate">; + request_id_ref?: string; +}; + +export type SessionStep = { method: string; request_id: string; tool_name: string } & Partial; export type SessionContextEntry = { session_id: string; @@ -100,7 +112,7 @@ export type SessionContextEntry = { method: string; request_id: string; tool_name: string; -}; +} & Partial; /** One session's hash chain, and nothing else the session happens to own. */ export type SessionContext = { @@ -122,6 +134,15 @@ export type SessionContext = { */ export type SessionState = { entries: SessionContextEntry[]; + /** + * The `request_id` of every `toolCallRequest` in `entries`, kept beside the + * chain so resolving a result's `request_id_ref` is a lookup, not a scan of + * the whole history on every result. + */ + toolCallRequestIds: Set; + /** Every `request_id` the session has carried, on any method, for the + * Specification §10.3 duplicate check. */ + requestIds: Set; intent: Intent | undefined; provenance: SessionProvenance; }; @@ -136,6 +157,10 @@ export const GENESIS_HASH = "0".repeat(64); * a field was declared somewhere else. */ export function hashEntry(input: Omit): string { + // Findings are appended only when present, so an entry without any hashes + // exactly as it did before findings existed and every older chain still + // verifies. An entry with findings hashes a longer array, which no entry + // without them can produce. const canonical = JSON.stringify([ input.prev_hash, input.session_id, @@ -144,6 +169,7 @@ export function hashEntry(input: Omit): string { input.method, input.request_id, input.tool_name, + ...(input.findings === undefined ? [] : [input.findings, input.request_id_ref ?? null]), ]); return createHash("sha256").update(canonical).digest("hex"); } @@ -152,6 +178,8 @@ export function hashEntry(input: Omit): string { export function emptySessionState(sessionId: string): SessionState { return { entries: [], + toolCallRequestIds: new Set(), + requestIds: new Set(), intent: undefined, // `origin` names where data entered the system, and this record is one the // Guardian synthesizes to hold a session's labels -- so "system", with the diff --git a/reference-implementations/agt/packages/guardian/test/envelope-log-sink.test.ts b/reference-implementations/agt/packages/guardian/test/envelope-log-sink.test.ts index ba642a00..1ab47860 100644 --- a/reference-implementations/agt/packages/guardian/test/envelope-log-sink.test.ts +++ b/reference-implementations/agt/packages/guardian/test/envelope-log-sink.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "bun:test"; -import { mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { mkdirSync, mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { @@ -184,3 +184,33 @@ describe("extractRpcId", () => { expect(extractRpcId("not an object")).toBeNull(); }); }); + +describe("createEnvelopeLogSink -- write says whether the line landed", () => { + it("answers true for a written line, and false once a write fails mid-run", () => { + const dir = mkdtempSync(join(tmpdir(), "acs-envelope-log-")); + const path = join(dir, "envelopes.jsonl"); + const sink = createEnvelopeLogSink({ path, onError: () => {} }); + try { + expect(sink.write("request", { id: 1 }, "steps/toolCallRequest")).toBe(true); + // A directory where the log file was: the next append throws EISDIR, + // which is a failure after earlier writes succeeded. + unlinkSync(path); + mkdirSync(path); + expect(sink.write("request", { id: 2 }, "steps/toolCallRequest")).toBe(false); + expect(sink.write("request", { id: 3 }, "steps/toolCallRequest")).toBe(false); + } finally { + // Non-recursive, like the rest of this file: the log path is a + // directory if the test got that far and a file if it did not. + try { + rmdirSync(path); + } catch { + unlinkSync(path); + } + rmdirSync(dir); + } + }); + + it("answers false from the null sink, which writes nowhere", () => { + expect(NULL_ENVELOPE_LOG_SINK.write("finding", {}, "steps/toolCallResult")).toBe(false); + }); +}); diff --git a/reference-implementations/agt/packages/guardian/test/otel-trace-exporter.test.ts b/reference-implementations/agt/packages/guardian/test/otel-trace-exporter.test.ts index 0d69ca9d..4908cdc8 100644 --- a/reference-implementations/agt/packages/guardian/test/otel-trace-exporter.test.ts +++ b/reference-implementations/agt/packages/guardian/test/otel-trace-exporter.test.ts @@ -20,11 +20,15 @@ import { METHODS_WITHOUT_SPAN, NULL_OTEL_TRACE_EXPORTER, OTEL_SERVICE_NAME, + REJECTED_SPAN_NAME, STEP_ATTRIBUTE_SOURCES, type CreateOtelTraceExporterOptions, type OtelTraceExporter, } from "../src/otel-trace-exporter.ts"; +/** What the Guardian tells `start` about a request it accepted. */ +const TRUSTED = { trusted: true } as const; + /** The live mapping, read the same way the exporter reads it, so every * expected name below comes from the file and not from a literal that could * drift from it. */ @@ -114,7 +118,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { const requestId = crypto.randomUUID(); const request = toolCallEnvelope("rm -rf /", { id: 7, requestId, sessionId }); - const step = exporter.start(request, "steps/toolCallRequest"); + const step = exporter.start(request, "steps/toolCallRequest", TRUSTED); exporter.end(step, decisionResponse(7, DENY)); await exporter.flush(); @@ -132,7 +136,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { it("records the decision as the mapping's event on that span, with acs.decision and no evaluator the response did not send", async () => { const { exporter, spans } = memoryExporter(); - exporter.end(exporter.start(toolCallEnvelope("rm -rf /", { id: 7 }), "steps/toolCallRequest"), decisionResponse(7, DENY)); + exporter.end(exporter.start(toolCallEnvelope("rm -rf /", { id: 7 }), "steps/toolCallRequest", TRUSTED), decisionResponse(7, DENY)); await exporter.flush(); const [event] = spans()[0]?.events ?? []; @@ -153,7 +157,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { it("takes acs.evaluator from the response when it carries one", async () => { const { exporter, spans } = memoryExporter(); exporter.end( - exporter.start(toolCallEnvelope("ls", { id: 7 }), "steps/toolCallRequest"), + exporter.start(toolCallEnvelope("ls", { id: 7 }), "steps/toolCallRequest", TRUSTED), decisionResponse(7, { decision: "allow", metadata: { evaluator: "agent", model_id: "m-1" } }), ); await exporter.flush(); @@ -166,7 +170,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { it("lines up policy columns in wire order, with \"\" where a reference lacks the field", async () => { const { exporter, spans } = memoryExporter(); exporter.end( - exporter.start(toolCallEnvelope("ls", { id: 7 }), "steps/toolCallRequest"), + exporter.start(toolCallEnvelope("ls", { id: 7 }), "steps/toolCallRequest", TRUSTED), decisionResponse(7, { decision: "deny", reasoning: "x", @@ -187,7 +191,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { it("pads a missing rule_id with \"\" in its own position", async () => { const { exporter, spans } = memoryExporter(); exporter.end( - exporter.start(toolCallEnvelope("ls", { id: 7 }), "steps/toolCallRequest"), + exporter.start(toolCallEnvelope("ls", { id: 7 }), "steps/toolCallRequest", TRUSTED), decisionResponse(7, { decision: "deny", reasoning: "x", @@ -210,7 +214,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { it("emits all three policy columns at the length of policy_references", async () => { const { exporter, spans } = memoryExporter(); exporter.end( - exporter.start(toolCallEnvelope("ls", { id: 7 }), "steps/toolCallRequest"), + exporter.start(toolCallEnvelope("ls", { id: 7 }), "steps/toolCallRequest", TRUSTED), decisionResponse(7, { decision: "deny", reasoning: "x", @@ -227,8 +231,8 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { it("carries acs.chain_hash only when the response does", async () => { const { exporter, spans } = memoryExporter(); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest"), decisionResponse(1, { decision: "allow", chain_hash: "a".repeat(64) })); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 2 }), "steps/toolCallRequest"), decisionResponse(2, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow", chain_hash: "a".repeat(64) })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 2 }), "steps/toolCallRequest", TRUSTED), decisionResponse(2, { decision: "allow" })); await exporter.flush(); const [withHash, withoutHash] = spans(); @@ -240,8 +244,8 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { const { exporter, spans } = memoryExporter(); // handshake/hello has no step_to_span entry. steps/sessionStart has one, // and is not in the ServerHello's methods_evaluated. - const hello = exporter.start(makeEnvelope("handshake/hello", {}, { id: 1 }), "handshake/hello"); - const sessionStart = exporter.start(makeEnvelope("steps/sessionStart", {}, { id: 2 }), "steps/sessionStart"); + const hello = exporter.start(makeEnvelope("handshake/hello", {}, { id: 1 }), "handshake/hello", TRUSTED); + const sessionStart = exporter.start(makeEnvelope("steps/sessionStart", {}, { id: 2 }), "steps/sessionStart", TRUSTED); expect(hello).toBeNull(); expect(sessionStart).toBeNull(); exporter.end(hello, { jsonrpc: "2.0", id: 1, result: { negotiated_version: "0.1.0" } }); @@ -257,7 +261,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { it("ends a span answered with a JSON-RPC error in error status, with a guardian_error failure event and no decision event", async () => { const { exporter, spans } = memoryExporter(); exporter.end( - exporter.start(toolCallEnvelope("ls", { id: 9 }), "steps/toolCallRequest"), + exporter.start(toolCallEnvelope("ls", { id: 9 }), "steps/toolCallRequest", TRUSTED), { jsonrpc: "2.0", id: 9, error: { code: -32020, message: "evaluation failed" } }, ); await exporter.flush(); @@ -276,15 +280,20 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }); // The pairing is the handle, not the id: an envelope with no JSON-RPC id - // at all still gets its span, closed by its own response. + // at all still gets its span, closed by its own response. The schema + // requires an id, so such an envelope is always one the Guardian rejected. it("pairs request and response by handle, so a request with no JSON-RPC id still gets its span", async () => { const { exporter, spans } = memoryExporter(); - const step = exporter.start(toolCallEnvelope("ls", { id: null }), "steps/toolCallRequest"); + const step = exporter.start(toolCallEnvelope("ls", { id: null }), "steps/toolCallRequest", { + trusted: false, + reason: "envelope_invalid", + }); expect(step).not.toBeNull(); exporter.end(step, { jsonrpc: "2.0", id: null, error: { code: -32010, message: "invalid" } }); await exporter.flush(); expect(spans()).toHaveLength(1); + expect(spans()[0]?.name).toBe(REJECTED_SPAN_NAME); expect(spans()[0]?.status.code).toBe(SpanStatusCode.ERROR); }); @@ -298,10 +307,10 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { onError: (error) => errors.push(error), }); - const step = exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest"); + const step = exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest", TRUSTED); expect(() => exporter.end(step, decisionResponse(1, DENY))).not.toThrow(); await exporter.flush(); - expect(() => exporter.start(toolCallEnvelope("ls", { id: 2 }), "steps/toolCallRequest")).not.toThrow(); + expect(() => exporter.start(toolCallEnvelope("ls", { id: 2 }), "steps/toolCallRequest", TRUSTED)).not.toThrow(); await exporter.flush(); await exporter.shutdown(); @@ -321,7 +330,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { methodsEvaluated: METHODS_EVALUATED, onError: (error) => errors.push(error), }); - expect(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest")).toBeNull(); + expect(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest", TRUSTED)).toBeNull(); await exporter.shutdown(); expect(errors).toHaveLength(1); } finally { @@ -346,7 +355,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }); try { for (const id of [1, 2, 3]) { - exporter.end(exporter.start(toolCallEnvelope("ls", { id }), "steps/toolCallRequest"), decisionResponse(id, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id }), "steps/toolCallRequest", TRUSTED), decisionResponse(id, { decision: "allow" })); await exporter.flush(); } await exporter.shutdown(); @@ -376,7 +385,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }); const exporter = createOtelTraceExporter({ endpoint: `http://127.0.0.1:${collector.port}/v1/traces`, methodsEvaluated: METHODS_EVALUATED }); try { - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest"), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow" })); await exporter.flush(); await exporter.shutdown(); @@ -403,7 +412,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }, { id: 1 }, ); - exporter.end(exporter.start(request, "steps/toolCallRequest"), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(request, "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow" })); await exporter.flush(); const attributes = spans()[0]?.attributes; @@ -424,7 +433,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }, { id: 1 }, ); - exporter.end(exporter.start(request, "steps/toolCallRequest"), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(request, "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow" })); await exporter.flush(); const attributes = spans()[0]?.attributes; @@ -434,7 +443,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { it("carries no provenance attribute when nothing in the payload has one", async () => { const { exporter, spans } = memoryExporter(); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest"), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow" })); await exporter.flush(); const attributes = spans()[0]?.attributes ?? {}; @@ -452,7 +461,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }, { id: 1 }, ); - exporter.end(exporter.start(request, "steps/toolCallResult"), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(request, "steps/toolCallResult", TRUSTED), decisionResponse(1, { decision: "allow" })); await exporter.flush(); const [span] = spans(); @@ -470,8 +479,8 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { const { exporter, spans } = memoryExporter(); const first = crypto.randomUUID(); const second = crypto.randomUUID(); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1, requestId: first }), "steps/toolCallRequest"), decisionResponse(1, { decision: "allow" })); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 2, requestId: second }), "steps/toolCallRequest"), decisionResponse(2, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1, requestId: first }), "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 2, requestId: second }), "steps/toolCallRequest", TRUSTED), decisionResponse(2, { decision: "allow" })); await exporter.flush(); const [spanA, spanB] = spans(); try { @@ -501,7 +510,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { const path = join(nested, "traces.jsonl"); const exporter = createOtelTraceExporter({ filePath: path, methodsEvaluated: METHODS_EVALUATED }); try { - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest"), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow" })); await exporter.shutdown(); expect((statSync(nested).mode & 0o777).toString(8)).toBe("700"); @@ -528,7 +537,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }, }; const exporter = createOtelTraceExporter({ spanExporter: slow, methodsEvaluated: METHODS_EVALUATED }); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest"), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow" })); const first = exporter.shutdown(); const second = exporter.shutdown(); @@ -571,7 +580,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { ); try { const { exporter, spans } = memoryExporter({ mappingPath }); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest"), decisionResponse(1, { decision: "allow" })); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 1 }), "steps/toolCallRequest", TRUSTED), decisionResponse(1, { decision: "allow" })); await exporter.flush(); const [span] = spans(); @@ -613,7 +622,7 @@ describe("createOtelTraceExporter -- what one exchange becomes", () => { }); it("NULL_OTEL_TRACE_EXPORTER writes nothing and never throws", async () => { - expect(NULL_OTEL_TRACE_EXPORTER.start(toolCallEnvelope("ls"), "steps/toolCallRequest")).toBeNull(); + expect(NULL_OTEL_TRACE_EXPORTER.start(toolCallEnvelope("ls"), "steps/toolCallRequest", TRUSTED)).toBeNull(); expect(() => NULL_OTEL_TRACE_EXPORTER.end(null, decisionResponse(1, DENY))).not.toThrow(); await NULL_OTEL_TRACE_EXPORTER.flush(); await NULL_OTEL_TRACE_EXPORTER.shutdown(); @@ -742,24 +751,95 @@ describe("Guardian trace export wiring", () => { } }); - // The span opens before validation, so the envelope that fails the schema - // is visible in the trace too -- as a deny, and with no chain hash, because - // the step never reached the chain. - it("gives a schema-invalid steps/* envelope a span with a deny event and no acs.chain_hash", async () => { + // Trace Events (PR #273): a Guardian MUST NOT emit acs.decision, or + // acs.decision.failure, under the pair of a request it rejects before + // trusting it, because such a request can carry any pair, a genuine + // request's included. Issue #283 ruled that it is still recorded, apart, + // under names outside step_to_span, so the attack stays visible. + it("records a schema-invalid steps/* envelope as a rejected request, and still answers it with a deny", async () => { const { exporter, spans } = memoryExporter(); const guardian = await startGuardian({ ...baseOptions, traceExporter: exporter }); try { const bad = toolCallEnvelope("rm -rf /", { id: 12 }); delete (bad.params as Record).acs_version; - await postRaw(guardian.url, bad); + const answer = await postRaw(guardian.url, bad); await exporter.flush(); + // The host still gets its honoured deny. Only the trace changes. + expect((answer.result as { decision?: string } | undefined)?.decision).toBe("deny"); const [span] = spans(); expect(spans()).toHaveLength(1); - const attributes = span?.events[0]?.attributes; - expect(attributes?.["acs.decision"]).toBe("deny"); - expect(attributes?.["acs.reason_codes"]).toEqual(["envelope_invalid"]); - expect(attributes?.["acs.chain_hash"]).toBeUndefined(); + expect(span?.name).toBe(REJECTED_SPAN_NAME); + expect(span?.attributes["acs.session.id"]).toBeUndefined(); + expect(span?.attributes["acs.request_id"]).toBeUndefined(); + expect(span?.attributes["guardian.rejected.reason"]).toBe("envelope_invalid"); + expect(span?.events).toEqual([]); + } finally { + await guardian.close(); + } + }); + + // The forgery here fails the schema. One that passes it is still recorded + // as a step: this Guardian verifies no signature, replay or timestamp yet. + for (const order of ["genuine first", "forgery first"] as const) { + it(`keeps a genuine request the only span under its pair when a schema-invalid request forges it (${order})`, async () => { + const { exporter, spans } = memoryExporter(); + const guardian = await startGuardian({ ...baseOptions, traceExporter: exporter }); + try { + const requestId = crypto.randomUUID(); + const sessionId = crypto.randomUUID(); + const genuine = toolCallEnvelope("ls -la", { id: 1, requestId, sessionId }); + const forged = toolCallEnvelope("rm -rf /", { id: 2, requestId, sessionId }); + delete (forged.params as Record).acs_version; + for (const envelope of order === "genuine first" ? [genuine, forged] : [forged, genuine]) { + await postRaw(guardian.url, envelope); + } + await exporter.flush(); + + const underPair = spans().filter( + (span) => span.attributes["acs.request_id"] === requestId && span.attributes["acs.session.id"] === sessionId, + ); + expect(underPair).toHaveLength(1); + expect(underPair[0]?.events.map((event) => event.attributes?.["acs.decision"])).toEqual(["allow"]); + const rejected = spans().filter((span) => span.name === REJECTED_SPAN_NAME); + expect(rejected).toHaveLength(1); + expect(rejected[0]?.attributes["guardian.rejected.claimed_request_id"]).toBe(requestId); + } finally { + await guardian.close(); + } + }); + } + + it("records a schema-invalid envelope for a method it does not dispatch as a rejected request too", async () => { + const { exporter, spans } = memoryExporter(); + const guardian = await startGuardian({ ...baseOptions, traceExporter: exporter }); + try { + const bad = makeEnvelope("steps/sessionStart", {}, { id: 44 }); + delete (bad.params as Record).acs_version; + await postRaw(guardian.url, bad); + await exporter.flush(); + + expect(spans().map((span) => span.name)).toEqual([REJECTED_SPAN_NAME]); + } finally { + await guardian.close(); + } + }); + + it("records an invalid envelope answered with a bare error as a rejected request, with no decision failure", async () => { + const { exporter, spans } = memoryExporter(); + const guardian = await startGuardian({ ...baseOptions, traceExporter: exporter }); + try { + // No request_id and no JSON-RPC id: denyOnInvalidEnvelope cannot address + // a decision, so the Guardian answers with a JSON-RPC error instead. + const bad = toolCallEnvelope("ls", { id: null }); + delete (bad.params as Record).request_id; + await postRaw(guardian.url, bad); + await exporter.flush(); + + const [span] = spans(); + expect(span?.name).toBe(REJECTED_SPAN_NAME); + expect(span?.events).toEqual([]); + expect(span?.attributes["guardian.rejected.error_code"]).toBe(-32010); } finally { await guardian.close(); } @@ -954,31 +1034,38 @@ describe("Guardian trace export wiring", () => { }, 15_000); // server.ts builds the exporter from the ServerHello, so the posture on a - // failure event is the one this Guardian declared to the host. An envelope - // with neither an id nor a request_id earns a deny that cannot be - // addressed to it, so the Guardian answers a bare JSON-RPC error: a - // guardian_error on the span, through the file path that production uses. + // failure event is the one this Guardian declared to the host. The failure + // here is a decision that finished after the ServerHello's own timeout, on + // a request the Guardian trusted, through the file path production uses. + // An invalid envelope would not do: a request rejected before it is + // trusted gets no failure event at all (see the schema-invalid case above). it("reports the ServerHello's posture on a failure event: a Guardian declaring deny says fail_closed", async () => { const dir = mkdtempSync(join(tmpdir(), "acs-otel-posture-")); const path = join(dir, "traces.jsonl"); - const guardian = await startGuardian({ ...baseOptions, onDecisionFailure: "deny", otelTraceFile: path }); + const late: PolicyBridge = { + async evaluate() { + await Bun.sleep(buildServerHello().timeout_config.default_ms + 200); + return { decision: "allow" }; + }, + }; + const guardian = await startGuardian({ ...baseOptions, bridge: late, onDecisionFailure: "deny", otelTraceFile: path }); try { - const response = await postRaw(guardian.url, { jsonrpc: "2.0", method: "steps/toolCallRequest", params: {} }); - expect(response.error).toBeDefined(); + const response = await postRaw(guardian.url, toolCallEnvelope("ls -la", { id: 1 })); + expect(response.result).toBeDefined(); } finally { await guardian.close(); } try { const [span] = otlpSpans(otlpFile(path)); const failure = span?.events?.find((event) => event.name === "acs.decision.failure"); - expect(otlpAttribute(failure, "acs.failure.class")).toBe("guardian_error"); + expect(otlpAttribute(failure, "acs.failure.class")).toBe("timeout"); expect(otlpAttribute(failure, "acs.failure.posture")).toBe("fail_closed"); expect(otlpAttribute(failure, "acs.failure.reported_by")).toBe("guardian"); } finally { unlinkSync(path); rmdirSync(dir); } - }); + }, 15_000); // A collector that is down must cost the endpoint and nothing else. The // real OTLPTraceExporter against a closed port, not the throwing stub: a @@ -1094,7 +1181,7 @@ describe("createOtelTraceExporter -- review fixes", () => { // to the same rule, since it lands in the same backend. it("copies no JSON-RPC error text into the failure event or the span status, and keeps the code", async () => { const { exporter, spans } = memoryExporter(); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 3 }), "steps/toolCallRequest"), { + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 3 }), "steps/toolCallRequest", TRUSTED), { jsonrpc: "2.0", id: 3, error: { code: -32020, message: `evaluation failed: ${SENTINEL}` }, @@ -1108,18 +1195,110 @@ describe("createOtelTraceExporter -- review fixes", () => { expect(span?.events[0]?.attributes?.["acs.failure.detail"]).toBeUndefined(); }); - // #228: a Guardian MUST NOT emit the failure event for a request it rejects - // as a replay, because that pair belongs to the earlier request. - it("emits no failure event for REPLAY_DETECTED (-32005)", async () => { + // #228 and PR #273: a request rejected before it was trusted gets no + // acs.decision.failure under its pair, because the pair may be a genuine + // request's. That holds even when the check ran after the span opened, as + // a signature, replay or timestamp check inside dispatch would: the span + // ends as a rejected request, renamed and without the pair. + for (const [code, reason] of [ + [-32004, "signature_invalid"], + [-32005, "replay_detected"], + [-32006, "timestamp_out_of_window"], + ] as const) { + it(`records a step answered with ${code} as a rejected request, without its pair`, async () => { + const { exporter, spans } = memoryExporter(); + const requestId = crypto.randomUUID(); + exporter.end(exporter.start(toolCallEnvelope("ls", { id: 4, requestId }), "steps/toolCallRequest", TRUSTED), { + jsonrpc: "2.0", + id: 4, + error: { code, message: "rejected" }, + }); + await exporter.flush(); + + const [span] = spans(); + expect(span?.name).toBe(REJECTED_SPAN_NAME); + expect(span?.events).toEqual([]); + expect(span?.attributes["acs.request_id"]).toBeUndefined(); + expect(span?.attributes["acs.session.id"]).toBeUndefined(); + expect(span?.attributes["guardian.rejected.claimed_request_id"]).toBe(requestId); + expect(span?.attributes["guardian.rejected.reason"]).toBe(reason); + expect(span?.attributes["guardian.rejected.error_code"]).toBe(code); + }); + } + + it("records a request the Guardian did not trust as a rejected request, with the reason it was given", async () => { const { exporter, spans } = memoryExporter(); - exporter.end(exporter.start(toolCallEnvelope("ls", { id: 4 }), "steps/toolCallRequest"), { + const requestId = crypto.randomUUID(); + const handle = exporter.start(toolCallEnvelope("ls", { id: 5, requestId }), "steps/toolCallRequest", { + trusted: false, + reason: "guardian_fault", + }); + expect(handle).not.toBeNull(); + exporter.end(handle, { jsonrpc: "2.0", id: 5, error: { code: -32020, message: "schemas missing" } }); + await exporter.flush(); + + const [span] = spans(); + expect(span?.name).toBe(REJECTED_SPAN_NAME); + expect(span?.attributes["acs.request_id"]).toBeUndefined(); + expect(span?.attributes["guardian.rejected.claimed_request_id"]).toBe(requestId); + expect(span?.attributes["guardian.rejected.reason"]).toBe("guardian_fault"); + expect(span?.status.code).toBe(SpanStatusCode.ERROR); + // The host got an error, so the step went ungoverned: still a decision + // failure, recorded on a span and event that carry no pair. + expect(span?.events.map((event) => event.attributes?.["acs.failure.class"])).toEqual(["guardian_error"]); + expect(span?.events[0]?.attributes?.["acs.request_id"]).toBeUndefined(); + }); + + it("carries none of a rejected result's step attributes, so acs.request_id_ref cannot name the genuine call", async () => { + const { exporter, spans } = memoryExporter(); + const genuineCall = crypto.randomUUID(); + const forgedResult = makeEnvelope( + "steps/toolCallResult", + { tool: { name: "run_shell" }, exit_status: "success", outputs: [{ value: "ok" }], request_id_ref: genuineCall }, + { id: 8 }, + ); + exporter.end(exporter.start(forgedResult, "steps/toolCallResult", TRUSTED), { jsonrpc: "2.0", - id: 4, + id: 8, error: { code: -32005, message: "replay" }, }); await exporter.flush(); - expect(spans()[0]?.events.map((event) => event.name)).toEqual([]); + const [span] = spans(); + expect(span?.name).toBe(REJECTED_SPAN_NAME); + expect(span?.attributes["acs.request_id_ref"]).toBeUndefined(); + expect(span?.attributes["gen_ai.tool.name"]).toBeUndefined(); + expect(Object.keys(span?.attributes ?? {}).every((key) => key.startsWith("guardian.rejected."))).toBe(true); + }); + + it("records a step dispatch rejected as a rejected request even when it answered with a deny", async () => { + const { exporter, spans } = memoryExporter(); + const handle = exporter.start(toolCallEnvelope("ls", { id: 9 }), "steps/toolCallRequest", TRUSTED); + exporter.end(handle, decisionResponse(9, { decision: "deny", reason_codes: ["signature_invalid"] }), "signature_invalid"); + await exporter.flush(); + + const [span] = spans(); + expect(span?.name).toBe(REJECTED_SPAN_NAME); + expect(span?.events).toEqual([]); + expect(span?.attributes["acs.request_id"]).toBeUndefined(); + expect(span?.attributes["guardian.rejected.reason"]).toBe("signature_invalid"); + }); + + it("counts the time before the span opened toward a late decision", async () => { + const { exporter, spans } = memoryExporter({ timeoutConfig: { default_ms: 50 } }); + const handle = exporter.start(toolCallEnvelope("ls", { id: 6 }), "steps/toolCallRequest", { + trusted: true, + receivedAt: performance.now() - 100, + }); + exporter.end(handle, decisionResponse(6, { decision: "allow" })); + await exporter.flush(); + + const [span] = spans(); + const failure = span?.events.find((event) => event.name === "acs.decision.failure"); + expect(failure?.attributes?.["acs.failure.class"]).toBe("timeout"); + // The span starts at arrival too, so its own duration covers the 100 ms. + const durationMs = (span?.duration[0] ?? 0) * 1000 + (span?.duration[1] ?? 0) / 1e6; + expect(durationMs).toBeGreaterThanOrEqual(90); }); // #228: acs.provenance.source_id is emitted only when at least one attached @@ -1130,7 +1309,7 @@ describe("createOtelTraceExporter -- review fixes", () => { tool: { name: "run_shell" }, arguments: { a: { value: "x", provenance: provenance("tool_output") }, b: { value: "y", provenance: provenance("user_input") } }, }); - exporter.end(exporter.start(envelope, "steps/toolCallRequest"), decisionResponse(1, DENY)); + exporter.end(exporter.start(envelope, "steps/toolCallRequest", TRUSTED), decisionResponse(1, DENY)); await exporter.flush(); const attributes = spans()[0]?.attributes; @@ -1153,7 +1332,7 @@ describe("createOtelTraceExporter -- review fixes", () => { { tool: { name: "run_shell" }, arguments: args }, { sessionId: "x".repeat(10_000) }, ); - exporter.end(exporter.start(envelope, "steps/toolCallRequest"), decisionResponse(1, DENY)); + exporter.end(exporter.start(envelope, "steps/toolCallRequest", TRUSTED), decisionResponse(1, DENY)); await exporter.flush(); expect(spans()).toHaveLength(1); @@ -1171,7 +1350,7 @@ describe("createOtelTraceExporter -- review fixes", () => { process.env.OTEL_SPAN_EVENT_COUNT_LIMIT = "0"; try { const { exporter, spans } = memoryExporter(); - exporter.end(exporter.start(toolCallEnvelope("ls"), "steps/toolCallRequest"), decisionResponse(1, DENY)); + exporter.end(exporter.start(toolCallEnvelope("ls"), "steps/toolCallRequest", TRUSTED), decisionResponse(1, DENY)); await exporter.flush(); expect(spans()).toHaveLength(1); @@ -1199,7 +1378,7 @@ describe("createOtelTraceExporter -- review fixes", () => { exportTimeoutMs: 100, onError: (error) => errors.push(error), }); - exporter.end(exporter.start(toolCallEnvelope("ls"), "steps/toolCallRequest"), decisionResponse(1, DENY)); + exporter.end(exporter.start(toolCallEnvelope("ls"), "steps/toolCallRequest", TRUSTED), decisionResponse(1, DENY)); const started = performance.now(); await exporter.shutdown(); @@ -1228,7 +1407,7 @@ describe("createOtelTraceExporter -- review fixes", () => { for (const mappingPath of [undefined, path]) { const { exporter, spans } = memoryExporter({ mappingPath }); exporter.end( - exporter.start(toolCallEnvelope("ls"), "steps/toolCallRequest"), + exporter.start(toolCallEnvelope("ls"), "steps/toolCallRequest", TRUSTED), decisionResponse(1, { ...DENY, reason_codes: ["blocked"], chain_hash: "a".repeat(64) }), ); await exporter.flush(); @@ -1314,7 +1493,7 @@ describe("the Guardian's exporter does not depend on its environment", () => { // Three spans ended in one synchronous turn: with a queue of one, the // SDK's default-from-environment processor drops the ones that arrive // while the first export is in flight. - for (const id of [1, 2, 3]) exporter.end(exporter.start(envelope(id), "steps/toolCallRequest"), allow(id)); + for (const id of [1, 2, 3]) exporter.end(exporter.start(envelope(id), "steps/toolCallRequest", TRUSTED), allow(id)); await exporter.shutdown(); const spans = readFileSync(path, "utf8") .split("\n") @@ -1340,7 +1519,7 @@ describe("the Guardian's exporter does not depend on its environment", () => { const path = join(dir, "trace.jsonl"); const exporter = createOtelTraceExporter({ filePath: path, methodsEvaluated: METHODS_EVALUATED }); const request = envelope(1); - exporter.end(exporter.start(request, "steps/toolCallRequest"), allow(1)); + exporter.end(exporter.start(request, "steps/toolCallRequest", TRUSTED), allow(1)); await exporter.shutdown(); const resourceSpans = JSON.parse(readFileSync(path, "utf8").split("\n")[0]!).resourceSpans[0]; expect(resourceSpans.resource.attributes).toEqual([{ key: "service.name", value: { stringValue: OTEL_SERVICE_NAME } }]); @@ -1357,7 +1536,7 @@ describe("the Guardian's exporter does not depend on its environment", () => { const collector = startCollector(); try { const exporter = createOtelTraceExporter({ endpoint: collector.url, methodsEvaluated: METHODS_EVALUATED }); - exporter.end(exporter.start(envelope(1), "steps/toolCallRequest"), allow(1)); + exporter.end(exporter.start(envelope(1), "steps/toolCallRequest", TRUSTED), allow(1)); await exporter.shutdown(); expect(collector.received.map((r) => r.headers.get("content-encoding"))).toEqual([null]); } finally { @@ -1373,7 +1552,7 @@ describe("the Guardian's exporter does not depend on its environment", () => { const collector = startCollector(); try { const exporter = createOtelTraceExporter({ endpoint: collector.url, methodsEvaluated: METHODS_EVALUATED }); - exporter.end(exporter.start(envelope(1), "steps/toolCallRequest"), allow(1)); + exporter.end(exporter.start(envelope(1), "steps/toolCallRequest", TRUSTED), allow(1)); await exporter.shutdown(); expect(collector.received.map((r) => r.headers.get("x-acs-probe"))).toEqual(["1"]); } finally { @@ -1415,7 +1594,7 @@ describe("the Guardian's exporter does not depend on its environment", () => { const endpoint = "http://127.0.0.1:" + collector.port + "/v1/traces"; keepLoopbackOffProxy(endpoint, process.env); const exporter = createOtelTraceExporter({ endpoint, methodsEvaluated: ["steps/toolCallRequest"], onError: () => {} }); - exporter.end(exporter.start(${JSON.stringify(envelope(1))}, "steps/toolCallRequest"), ${JSON.stringify(allow(1))}); + exporter.end(exporter.start(${JSON.stringify(envelope(1))}, "steps/toolCallRequest", { trusted: true }), ${JSON.stringify(allow(1))}); await exporter.shutdown(); console.log(JSON.stringify({ collected })); collector.stop(true); diff --git a/reference-implementations/agt/packages/guardian/test/request-id-ref.test.ts b/reference-implementations/agt/packages/guardian/test/request-id-ref.test.ts new file mode 100644 index 00000000..cd1698ee --- /dev/null +++ b/reference-implementations/agt/packages/guardian/test/request-id-ref.test.ts @@ -0,0 +1,233 @@ +import { describe, expect, it, spyOn } from "bun:test"; +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { startGuardian } from "../src/index.ts"; +import type { EnvelopeLogEntry } from "../src/envelope-log-sink.ts"; + +// hooks.md, toolCallResult: a Guardian SHOULD resolve request_id_ref against +// the session's toolCallRequest records and MUST record a reference it cannot +// resolve. These tests drive a running Guardian and read its envelope log, +// which is the Guardian's audit record. + +let rpcId = 0; + +function envelope(method: string, sessionId: string, payload: Record, requestId: string = crypto.randomUUID()) { + rpcId += 1; + return { + jsonrpc: "2.0", + method, + id: rpcId, + params: { + acs_version: "0.1.0", + request_id: requestId, + timestamp: new Date().toISOString(), + metadata: { agent_id: "agent-1", session_id: sessionId }, + payload, + }, + }; +} + +function toolRequest(sessionId: string, requestId: string) { + return envelope( + "steps/toolCallRequest", + sessionId, + { tool: { name: "Bash" }, arguments: { command: { value: "ls" } } }, + requestId, + ); +} + +function toolResult(sessionId: string, requestIdRef?: string) { + const payload: Record = { tool: { name: "Bash" }, exit_status: "success", outputs: [{ value: "a.txt" }] }; + if (requestIdRef !== undefined) { + payload.request_id_ref = requestIdRef; + } + return envelope("steps/toolCallResult", sessionId, payload); +} + +async function post(url: string, body: unknown): Promise> { + const res = await fetch(url, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) }); + return (await res.json()) as Record; +} + +function findings(path: string): EnvelopeLogEntry[] { + if (!existsSync(path)) { + return []; + } + return readFileSync(path, "utf8") + .split("\n") + .filter((line) => line.trim() !== "") + .map((line) => JSON.parse(line) as EnvelopeLogEntry) + .filter((entry) => entry.direction === "finding"); +} + +async function withGuardian(run: (url: string, logPath: string) => Promise): Promise { + const dir = mkdtempSync(join(tmpdir(), "acs-request-id-ref-")); + const logPath = join(dir, "envelopes.jsonl"); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath: logPath }); + try { + await run(guardian.url, logPath); + } finally { + await guardian.close(); + rmSync(dir, { recursive: true, force: true }); + } +} + +describe("request_id_ref resolution", () => { + it("records a reference the session never saw, and leaves the decision to policy", async () => { + await withGuardian(async (url, logPath) => { + const sessionId = crypto.randomUUID(); + const unknownRef = crypto.randomUUID(); + const result = toolResult(sessionId, unknownRef); + + const response = await post(url, result); + + expect(response.result).toBeDefined(); + const recorded = findings(logPath); + expect(recorded).toHaveLength(1); + const [finding] = recorded; + expect(finding?.method).toBe("steps/toolCallResult"); + expect(finding?.rpc_id).toBe(result.id); + expect(finding?.envelope).toEqual({ + finding: "request_id_ref_unresolved", + session_id: sessionId, + request_id: result.params.request_id, + request_id_ref: unknownRef, + }); + }); + }); + + it("records nothing when the reference names this session's toolCallRequest", async () => { + await withGuardian(async (url, logPath) => { + const sessionId = crypto.randomUUID(); + const requestId = crypto.randomUUID(); + await post(url, toolRequest(sessionId, requestId)); + + await post(url, toolResult(sessionId, requestId)); + + expect(findings(logPath)).toHaveLength(0); + }); + }); + + it("records nothing when the result carries no reference", async () => { + await withGuardian(async (url, logPath) => { + await post(url, toolResult(crypto.randomUUID())); + + expect(findings(logPath)).toHaveLength(0); + }); + }); + + it("treats a request from another session as unresolved, because request_id is unique only within a session", async () => { + await withGuardian(async (url, logPath) => { + const requestId = crypto.randomUUID(); + await post(url, toolRequest(crypto.randomUUID(), requestId)); + + await post(url, toolResult(crypto.randomUUID(), requestId)); + + expect(findings(logPath)).toHaveLength(1); + }); + }); + + // The record is a MUST, and the envelope log is allowed to be absent or to + // disable itself, so a finding the log did not take goes to stderr. + async function stderrFindings(envelopeLogPath: string | undefined): Promise { + const lines: string[] = []; + const spy = spyOn(console, "error").mockImplementation((...args: unknown[]) => { + lines.push(args.map(String).join(" ")); + }); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", envelopeLogPath }); + try { + await post(guardian.url, toolResult(crypto.randomUUID(), crypto.randomUUID())); + } finally { + await guardian.close(); + spy.mockRestore(); + } + return lines.filter((line) => line.startsWith("acs-finding ")).map((line) => JSON.parse(line.slice("acs-finding ".length))); + } + + it("writes the finding to stderr when no envelope log is configured", async () => { + const recorded = await stderrFindings(undefined); + expect(recorded).toHaveLength(1); + expect(recorded[0]).toMatchObject({ finding: "request_id_ref_unresolved" }); + }); + + it("writes the finding to stderr when the envelope log has disabled itself", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-request-id-ref-")); + // A file where the log's parent directory should be, so the sink cannot + // create the directory and disables itself before the first write. + const blocker = join(dir, "not-a-dir"); + writeFileSync(blocker, ""); + try { + const recorded = await stderrFindings(join(blocker, "envelopes.jsonl")); + expect(recorded).toHaveLength(1); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("does not also write to stderr when the envelope log took the finding", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-request-id-ref-")); + try { + expect(await stderrFindings(join(dir, "envelopes.jsonl"))).toHaveLength(0); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("records a toolCallRequest whose request_id the session already holds, and still decides it", async () => { + await withGuardian(async (url, logPath) => { + const sessionId = crypto.randomUUID(); + const requestId = crypto.randomUUID(); + await post(url, toolRequest(sessionId, requestId)); + + const repeat = await post(url, toolRequest(sessionId, requestId)); + + expect(repeat.result).toBeDefined(); + const recorded = findings(logPath); + expect(recorded).toHaveLength(1); + expect(recorded[0]?.envelope).toEqual({ finding: "request_id_duplicate", session_id: sessionId, request_id: requestId }); + }); + }); + + it("puts the log line's fields on a stderr finding too", async () => { + const [recorded] = await stderrFindings(undefined); + expect(recorded).toMatchObject({ method: "steps/toolCallResult", finding: "request_id_ref_unresolved" }); + expect(recorded).toHaveProperty("recorded_at"); + expect(recorded).toHaveProperty("rpc_id"); + }); + + it("records a toolCallResult that reuses an earlier request's request_id, since §10.3 covers every method", async () => { + await withGuardian(async (url, logPath) => { + const sessionId = crypto.randomUUID(); + const requestId = crypto.randomUUID(); + await post(url, toolRequest(sessionId, requestId)); + + const reused = toolResult(sessionId, requestId); + reused.params.request_id = requestId; + await post(url, reused); + + const codes = findings(logPath).map((entry) => (entry.envelope as { finding: string }).finding); + expect(codes).toEqual(["request_id_duplicate"]); + }); + }); + + it("puts the finding on the result's own chain entry, which is the Guardian's audit record", async () => { + const dir = mkdtempSync(join(tmpdir(), "acs-request-id-ref-")); + const chainPath = join(dir, "session-context.jsonl"); + const guardian = await startGuardian({ port: 0, manifestPath: "policy/manifest.yaml", sessionContextLog: chainPath }); + try { + const unknownRef = crypto.randomUUID(); + await post(guardian.url, toolResult(crypto.randomUUID(), unknownRef)); + } finally { + await guardian.close(); + } + try { + const [entry] = readFileSync(chainPath, "utf8").trim().split("\n").map((line) => JSON.parse(line)); + expect(entry.findings).toEqual(["request_id_ref_unresolved"]); + expect(entry.request_id_ref).toMatch(/^[0-9a-f-]{36}$/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + diff --git a/reference-implementations/agt/packages/guardian/test/session-context.test.ts b/reference-implementations/agt/packages/guardian/test/session-context.test.ts index 71ecb2fb..ac421a95 100644 --- a/reference-implementations/agt/packages/guardian/test/session-context.test.ts +++ b/reference-implementations/agt/packages/guardian/test/session-context.test.ts @@ -5,6 +5,7 @@ import { createMemorySessionContextStore, loadSessionContext, } from "../src/session-context-store.ts"; +import { hashEntry } from "../src/session-context.ts"; const at = (iso: string) => () => new Date(iso); const step = (n: number) => ({ method: "steps/toolCallRequest", request_id: `req-${n}`, tool_name: "Bash" }); @@ -285,3 +286,70 @@ describe("append pushes onto the chain rather than rebuilding it", () => { expect(large / small).toBeLessThan(3); }); }); + +describe("SessionContextStore.hasToolCallRequest", () => { + it("knows a session's toolCallRequest ids and no other step's", () => { + const store = createMemorySessionContextStore(); + appendContextEntry(store, "sess-a", step(1)); + appendContextEntry(store, "sess-a", { method: "steps/toolCallResult", request_id: "res-1", tool_name: "Bash" }); + expect(store.hasToolCallRequest("sess-a", "req-1")).toBe(true); + expect(store.hasToolCallRequest("sess-a", "res-1")).toBe(false); + }); + + it("answers per session, because request_id is unique only within one", () => { + const store = createMemorySessionContextStore(); + appendContextEntry(store, "sess-a", step(1)); + expect(store.hasToolCallRequest("sess-b", "req-1")).toBe(false); + }); + + it("forgets an evicted session's requests along with its chain", () => { + const store = createMemorySessionContextStore({ maxSessions: 1, onEvict: () => {} }); + appendContextEntry(store, "sess-a", step(1)); + appendContextEntry(store, "sess-b", step(2)); + expect(store.hasToolCallRequest("sess-a", "req-1")).toBe(false); + }); + + it("keeps the ids across the store's other writes to the session", () => { + const store = createMemorySessionContextStore(); + appendContextEntry(store, "sess-a", step(1)); + store.replaceIfcLabels("sess-a", ["confidential"]); + store.setIntent("sess-a", "list files"); + expect(store.hasToolCallRequest("sess-a", "req-1")).toBe(true); + }); +}); + +describe("SessionContext: findings on a chain entry", () => { + it("hashes an entry without findings exactly as before, so older chains still verify", () => { + const store = createMemorySessionContextStore({ now: at("2026-08-14T00:00:00.000Z") }); + const plain = appendContextEntry(store, "sess-a", step(1)); + const { hash, ...withoutHash } = plain; + expect(hashEntry(withoutHash)).toBe(hash); + expect("findings" in plain).toBe(false); + }); + + it("carries findings and the unresolved reference, and the hash covers both", () => { + const store = createMemorySessionContextStore({ now: at("2026-08-14T00:00:00.000Z") }); + const entry = appendContextEntry(store, "sess-a", { + method: "steps/toolCallResult", + request_id: "res-1", + tool_name: "Bash", + findings: ["request_id_ref_unresolved"], + request_id_ref: "req-9", + }); + expect(entry.findings).toEqual(["request_id_ref_unresolved"]); + expect(entry.request_id_ref).toBe("req-9"); + const { hash, ...withoutHash } = entry; + expect(hashEntry(withoutHash)).toBe(hash); + // Stripping the finding changes the digest, so a rewritten entry shows. + const { findings: _f, request_id_ref: _r, ...stripped } = withoutHash; + expect(hashEntry(stripped)).not.toBe(hash); + }); + + it("remembers every request id, on any method, for the duplicate check", () => { + const store = createMemorySessionContextStore(); + appendContextEntry(store, "sess-a", { method: "steps/toolCallResult", request_id: "res-1", tool_name: "Bash" }); + expect(store.hasRequestId("sess-a", "res-1")).toBe(true); + expect(store.hasToolCallRequest("sess-a", "res-1")).toBe(false); + }); +}); + diff --git a/reference-implementations/agt/packages/host-adapter/src/audit-sink.ts b/reference-implementations/agt/packages/host-adapter/src/audit-sink.ts index f12cf67d..4522f8f8 100644 --- a/reference-implementations/agt/packages/host-adapter/src/audit-sink.ts +++ b/reference-implementations/agt/packages/host-adapter/src/audit-sink.ts @@ -68,6 +68,13 @@ type AuditEntryIdentity = { */ method: string | null; rpc_id: string | number | null; + /** + * The envelope's `params.request_id`, or null when no request was built. + * It differs from `rpc_id` when the shim derived the request id from the + * host's call id, and it is what the Guardian's own span for the request + * carries. Optional so entries written before it existed still read. + */ + request_id?: string | null; }; /** diff --git a/reference-implementations/agt/packages/host-adapter/src/build-envelope.ts b/reference-implementations/agt/packages/host-adapter/src/build-envelope.ts index bb680234..faa49e00 100644 --- a/reference-implementations/agt/packages/host-adapter/src/build-envelope.ts +++ b/reference-implementations/agt/packages/host-adapter/src/build-envelope.ts @@ -11,7 +11,7 @@ */ import { createHash, randomUUID } from "node:crypto"; import { readFileSync } from "node:fs"; -import { resolvePath } from "./hookmap-path.ts"; +import { pathSegments, resolvePath } from "./hookmap-path.ts"; /** * A hookmap value the host sent no field for: the constant this hook always @@ -69,6 +69,17 @@ type HookmapHookEntryCommon = { acs_method: string; /** JSONPath-lite (`$.foo.bar`) into the raw hook payload for the tool/step name. */ tool_name: string; + /** + * JSONPath-lite into the raw hook payload for the host's own id of this one + * tool call, the same on the hook before the call and the hook after it. + * Each host's hookmap names its own field. Optional. When it + * resolves, the request's `request_id` is derived from it and the result + * carries the same value as `request_id_ref`, so the Guardian can pair the + * two without the shim keeping any state. When it does not resolve, the + * request gets a random `request_id` and the result no `request_id_ref`, + * which is what every hookmap without this key gets. + */ + call_id?: string; /** * This hook's own decision -> host-output mapping, consumed by * `renderDecision` and never by this module. @@ -204,6 +215,8 @@ export type AcsToolCallRequestPayload = { /** ACS's optional `raw_command`. Present only when the hookmap declared a * path for it AND that path resolved to a string. */ raw_command?: string; + /** Only a result names the request it answers. */ + request_id_ref?: never; exit_status?: never; outputs?: never; }; @@ -221,6 +234,9 @@ export type AcsToolCallResultPayload = { * exclusive union stays exclusive and `raw_command` reads as `string | * undefined` on either payload shape without a cast. */ raw_command?: never; + /** The originating request's `request_id`, present when the hookmap's + * `call_id` resolved (see HookmapHookEntryCommon). */ + request_id_ref?: string; exit_status: string; outputs: AcsOutput[]; }; @@ -433,6 +449,80 @@ function assertToolsWellFormed(hookmap: Hookmap, path: string): void { } } +/** + * Rejects a malformed `call_id` at load time, for the reason the checks + * around it give: a throw reached only from `buildEnvelope` is answered with + * the delivery posture, so under `proceed` a typo here would run every + * governed step ungoverned. `call_id` is also refused on any hook that is not + * one of the two tool gates, because the request id it derives is shared by + * exactly one request and the result that answers it. + */ +function assertPairingPathWellFormed(hookmap: Hookmap, path: string): void { + // Per tool gate: which hooks declare `call_id` and which do not. + const declared = { "steps/toolCallRequest": [] as string[], "steps/toolCallResult": [] as string[] }; + const undeclared = { "steps/toolCallRequest": [] as string[], "steps/toolCallResult": [] as string[] }; + for (const [hookEventName, entry] of Object.entries(hookmap.hooks ?? {})) { + if (!isPlainObject(entry)) { + continue; + } + const method = (entry as { acs_method?: unknown }).acs_method; + const pairingPath = (entry as { call_id?: unknown }).call_id ?? undefined; + if (pairingPath === undefined) { + if (method === "steps/toolCallRequest" || method === "steps/toolCallResult") { + undeclared[method].push(hookEventName); + } + continue; + } + if (typeof pairingPath !== "string" || !pairingPath.startsWith("$.") || pairingPath.length <= 2) { + throw new Error( + `loadHookmap: ${path}'s "hooks.${hookEventName}.call_id" is ${JSON.stringify(pairingPath)} -- when present, ` + + `"call_id" must be one JSONPath-lite string such as "$.tool_use_id"`, + ); + } + // The same segment rules resolvePath applies on every call, applied here + // once, so a reserved segment such as `$.constructor` fails at startup and + // not on every governed step. + try { + pathSegments(pairingPath); + } catch (failure) { + throw new Error( + `loadHookmap: ${path}'s "hooks.${hookEventName}.call_id" is not a usable path: ` + + `${failure instanceof Error ? failure.message : String(failure)}`, + ); + } + if (method !== "steps/toolCallRequest" && method !== "steps/toolCallResult") { + throw new Error( + `loadHookmap: ${path}'s "hooks.${hookEventName}" declares "call_id" on ${JSON.stringify(method)} -- ` + + `"call_id" pairs a tool call's request with its result, so only those two gates may declare it`, + ); + } + declared[method].push(hookEventName); + } + // Declared on one gate and not the other is a pairing that cannot work. A + // result-side gap is the dangerous half: results would stop carrying the + // reference and the Guardian would see nothing to check, with no signal. + const requestSide = declared["steps/toolCallRequest"].length > 0; + const resultSide = declared["steps/toolCallResult"].length > 0; + const missing = requestSide || resultSide + ? [...undeclared["steps/toolCallRequest"], ...undeclared["steps/toolCallResult"]] + : []; + if (missing.length > 0) { + throw new Error( + `loadHookmap: ${path} declares "call_id" on some tool gates but not on ${JSON.stringify(missing)} -- ` + + `a request and its result are paired only when both gates name the host's call id`, + ); + } + // One side declaring it and the other side absent entirely: results would + // name requests that were never sent, or requests would be paired with + // results that never come, and every result would be a finding. + if (requestSide !== resultSide) { + throw new Error( + `loadHookmap: ${path} declares "call_id" only on ${requestSide ? "request" : "result"} gates -- ` + + `a request and its result are paired only when both a request gate and a result gate name the host's call id`, + ); + } +} + /** * Rejects a hookmap entry whose `exit_status` declares both `literal` and * `from`, at load time -- the same seam `assertMirrorsWellFormed` and @@ -577,6 +667,7 @@ export function loadHookmap(path: string): Hookmap { assertToolsWellFormed(hookmap, path); assertExitStatusNotBothForms(hookmap, path); assertRequestGateDeclaresNoOutputs(hookmap, path); + assertPairingPathWellFormed(hookmap, path); return normalizeTools(hookmap); } @@ -911,13 +1002,72 @@ export function toSessionUuid(rawSessionId: string): string { .update(Buffer.concat([namespaceBytes, Buffer.from(rawSessionId, "utf8")])) .digest(); - hash[6] = (hash[6]! & 0x0f) | 0x50; // version 5 - hash[8] = (hash[8]! & 0x3f) | 0x80; // RFC 4122 variant + return formatNameBasedUuid(hash, 5); +} - const hex = hash.subarray(0, 16).toString("hex"); +/** + * The first 16 bytes of a name-based hash as a UUID string, with the version + * nibble and the RFC 9562 variant bits set. Shared by both derivations so the + * bit layout lives in one place. The digest is the caller's: SHA-1 for the + * session id's UUIDv5, as RFC 9562 defines v5, and SHA-256 for the request + * id's UUIDv8. + */ +function formatNameBasedUuid(hash: Buffer, version: 5 | 8): string { + const bytes = Buffer.from(hash.subarray(0, 16)); + bytes[6] = (bytes[6]! & 0x0f) | (version << 4); + bytes[8] = (bytes[8]! & 0x3f) | 0x80; + const hex = bytes.toString("hex"); return `${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20, 32)}`; } +// A second fixed namespace, separate from SESSION_ID_NAMESPACE, so the two +// derivations draw from unrelated spaces. Never regenerate it: doing +// so changes every derived request_id, and a result built by a newer shim +// would no longer name the request an older shim sent. +const INVOCATION_ID_NAMESPACE = "5d0f6a3e-2b71-4c8e-9a54-0e1d7c3b9f62"; + +/** + * The `request_id` for one tool call, derived from the host's call id so the + * hook before the call and the hook after it compute the same value. The + * session is part of the name because `request_id` is unique only within a + * session (Specification §10.3) and a host's call ids need not be unique + * across sessions. Not a secret: anyone who knows both ids can compute it, + * which is fine for an identifier the envelope carries in the clear. + * + * A name-based UUIDv8 over SHA-256 (RFC 9562 §5.8 and Appendix B.2) rather + * than a v5: the same construction with a current hash, so no SHA-1 enters + * new code. + */ +export function toInvocationRequestId(sessionUuid: string, invocationId: string): string { + const namespaceBytes = Buffer.from(INVOCATION_ID_NAMESPACE.replace(/-/g, ""), "hex"); + const hash = createHash("sha256") + .update(Buffer.concat([namespaceBytes, Buffer.from(`${sessionUuid}\n${invocationId}`, "utf8")])) + .digest(); + + return formatNameBasedUuid(hash, 8); +} + +/** + * The host's call id for this invocation, or undefined. Never a throw: a call + * id is optional, and a step whose host did not send one is still fully + * governable, the same reasoning `raw_command` follows in buildPayload. A + * declared path that is not a string is a hookmap fault and does throw. + */ +function invocationIdOf(event: string, entry: HookmapHookEntry, payload: Record): string | undefined { + const path = entry.call_id ?? undefined; + if (path === undefined) { + return undefined; + } + if (typeof path !== "string") { + throw new Error( + `buildEnvelope: hookmap entry for hook "${event}" declares "call_id" as ${JSON.stringify(path)} -- ` + + `"call_id" names the host's per-call id with a single JSONPath-lite string`, + ); + } + const invocationId = resolvePath(payload, path); + return typeof invocationId === "string" && invocationId.length > 0 ? invocationId : undefined; +} + /** * Builds an ACS v0.1.0 request envelope from a raw hook invocation. * `hookmap` drives every host-shape decision: the ACS method, which of the two @@ -926,7 +1076,12 @@ export function toSessionUuid(rawSessionId: string): string { * `hookmap.hooks` throws, and so does an entry naming no single payload shape -- * there is no default method, no default shape, and no partial envelope. */ -export function buildEnvelope(event: string, payload: Record, hookmap: Hookmap): AcsRequestEnvelope { +export function buildEnvelope( + event: string, + payload: Record, + hookmap: Hookmap, + options: { pairWithHostCall?: boolean } = {}, +): AcsRequestEnvelope { const entry = hookmap.hooks[event]; if (!entry) { throw new Error(`buildEnvelope: hookmap has no entry for hook "${event}"`); @@ -946,19 +1101,40 @@ export function buildEnvelope(event: string, payload: Record, h throw new Error(`buildEnvelope: hook payload for "${event}" is missing a "session_id" string field`); } - const requestId = randomUUID(); + const sessionUuid = toSessionUuid(rawSessionId); + // `pairWithHostCall: false` is governStep's retry after a REPLAY_DETECTED + // answer. Only this request gets a fresh random id. The host's result hook + // runs on its own and still names the derived id, so the result resolves + // against the earlier request that carried it: the first time the host + // asked about this call. + const invocationId = options.pairWithHostCall === false ? undefined : invocationIdOf(event, entry, payload); + const invocationRequestId = invocationId === undefined ? undefined : toInvocationRequestId(sessionUuid, invocationId); + // The request gate takes the derived id as its request_id. The result gate + // names it as the request it answers and keeps a fresh id of its own. + let requestId: string = randomUUID(); + if (invocationRequestId !== undefined) { + if (acsPayload.outputs === undefined) { + requestId = invocationRequestId; + } else { + acsPayload.request_id_ref = invocationRequestId; + } + } + // The JSON-RPC id stays random even when request_id is derived. The + // envelope log and the Inspector pair a request with its response by this + // id, and a host asking twice about one call would otherwise repeat it. + const rpcId = requestId === invocationRequestId ? randomUUID() : requestId; return { jsonrpc: "2.0", method: entry.acs_method, - id: requestId, + id: rpcId, params: { acs_version: ACS_VERSION, request_id: requestId, timestamp: new Date().toISOString(), metadata: { agent_id: hookmap.host, - session_id: toSessionUuid(rawSessionId), + session_id: sessionUuid, }, payload: acsPayload, }, diff --git a/reference-implementations/agt/packages/host-adapter/src/failure-posture.ts b/reference-implementations/agt/packages/host-adapter/src/failure-posture.ts index 9de89c63..5ef2389c 100644 --- a/reference-implementations/agt/packages/host-adapter/src/failure-posture.ts +++ b/reference-implementations/agt/packages/host-adapter/src/failure-posture.ts @@ -264,6 +264,8 @@ export type ApplyFailurePostureInput = { * knowable. Never a host's own event name -- see AuditEntry.method. */ method: string | null; rpcId: string | number | null; + /** The envelope's `params.request_id`, or null when no request was built. */ + requestId?: string | null; /** Required, not optional: §6.4 makes auditing non-skippable. */ audit: AuditSink; /** @@ -305,6 +307,7 @@ export function applyFailurePosture({ sessionId, method, rpcId, + requestId = null, audit, stage = "delivery", }: ApplyFailurePostureInput): FailureResolvedAcsDecision { @@ -338,6 +341,7 @@ export function applyFailurePosture({ session_id: sessionId, method, rpc_id: rpcId, + request_id: requestId, posture, posture_source: postureSource, outcome, diff --git a/reference-implementations/agt/packages/host-adapter/src/govern-step.ts b/reference-implementations/agt/packages/host-adapter/src/govern-step.ts index 49862c86..09477730 100644 --- a/reference-implementations/agt/packages/host-adapter/src/govern-step.ts +++ b/reference-implementations/agt/packages/host-adapter/src/govern-step.ts @@ -359,6 +359,12 @@ export function governsTool(hookmap: Hookmap, hookEventName: string, tool: strin return tools === undefined || tools.includes(tool); } +/** JSON-RPC REPLAY_DETECTED (Specification §17.1), as the raw error object + * GuardianClient hands back when no decision arrived. */ +function isReplayDetected(failure: unknown): boolean { + return typeof failure === "object" && failure !== null && (failure as { code?: unknown }).code === -32005; +} + /** * Records a step this gate declined to govern -- the `tools` skip, in the * one log that outlives the process. @@ -637,6 +643,7 @@ export async function governStep({ sessionId, method: envelope?.method ?? null, rpcId: envelope?.id ?? null, + requestId: envelope?.params.request_id ?? null, audit, stage, }); @@ -724,7 +731,27 @@ export async function governStep({ // from the envelope it just built. const modificationDocument = modificationDocumentOf(envelope); - const answer = await guardian.requestDecision(envelope, { timeoutMs }); + let answer = await guardian.requestDecision(envelope, { timeoutMs }); + // A request id derived from the host's call id (see buildEnvelope's + // `call_id`) repeats if the host ever asks twice about one call, and a + // Guardian enforcing Specification §10.3 answers the repeat with + // REPLAY_DETECTED. That code carries no decision, so under `proceed` it + // would run the step ungoverned. §17.1's recovery is to regenerate the id + // and retry, so this does that once, with a random id, inside whatever is + // left of the timeout. The result hook still names the derived id, which + // resolves against the first request the host sent for this call. Only + // the request gate derives its own id, so only it can hit this. + if ( + answer.decisionArrived === false && + isReplayDetected(answer.failure) && + envelope.params.payload.outputs === undefined + ) { + const remainingMs = timeoutMs - (performance.now() - startedAt); + if (remainingMs > 0) { + envelope = buildEnvelope(hookEventName, payload, hookmap, { pairWithHostCall: false }); + answer = await guardian.requestDecision(envelope, { timeoutMs: remainingMs }); + } + } const elapsedMs = performance.now() - startedAt; if (answer.decisionArrived === false) { diff --git a/reference-implementations/agt/packages/host-adapter/src/index.ts b/reference-implementations/agt/packages/host-adapter/src/index.ts index 43c2ce40..b223ba58 100644 --- a/reference-implementations/agt/packages/host-adapter/src/index.ts +++ b/reference-implementations/agt/packages/host-adapter/src/index.ts @@ -11,6 +11,7 @@ export { loadHookmap, modificationDocumentOf, toSessionUuid, + toInvocationRequestId, type Hookmap, type HookmapHookEntry, type HookmapOutputs, diff --git a/reference-implementations/agt/packages/host-adapter/test/build-envelope.test.ts b/reference-implementations/agt/packages/host-adapter/test/build-envelope.test.ts index de58ac5c..998753f4 100644 --- a/reference-implementations/agt/packages/host-adapter/test/build-envelope.test.ts +++ b/reference-implementations/agt/packages/host-adapter/test/build-envelope.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from "bun:test"; -import { mkdtempSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; +import { mkdtempSync, readFileSync, rmdirSync, unlinkSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { validateEnvelope } from "guardian"; @@ -7,6 +7,7 @@ import { buildEnvelope, loadHookmap, toSessionUuid, + toInvocationRequestId, modificationDocumentOf, type Hookmap, type HookmapHookEntry, @@ -1157,3 +1158,128 @@ describe("buildEnvelope", () => { }); }); }); + +describe("call_id: pairing a result with its request", () => { + const withCallId: Hookmap = { + ...hookmap, + hooks: { + PreToolUse: { ...hookmap.hooks.PreToolUse!, call_id: "$.tool_use_id" } as HookmapHookEntry, + PostToolUse: { ...hookmap.hooks.PostToolUse!, call_id: "$.tool_use_id" } as HookmapHookEntry, + }, + }; + const before = { ...preToolUsePayload, tool_use_id: "toolu_01" }; + const after = { ...before, hook_event_name: "PostToolUse", tool_response: { stdout: "done" } }; + + it("gives the result the request's request_id as request_id_ref, and both validate", () => { + const request = buildEnvelope("PreToolUse", before, withCallId); + const result = buildEnvelope("PostToolUse", after, withCallId); + + expect(() => validateEnvelope(request)).not.toThrow(); + expect(() => validateEnvelope(result)).not.toThrow(); + expect(result.params.payload.request_id_ref).toBe(request.params.request_id); + // The JSON-RPC id stays random, so two asks about one call never share it. + expect(request.id).not.toBe(request.params.request_id); + expect(buildEnvelope("PreToolUse", before, withCallId).id).not.toBe(request.id); + expect(result.params.request_id).not.toBe(request.params.request_id); + }); + + it("derives the id from the session and the call id, so another session's call does not collide", () => { + const here = buildEnvelope("PreToolUse", before, withCallId); + const elsewhere = buildEnvelope("PreToolUse", { ...before, session_id: "another-session" }, withCallId); + + expect(here.params.request_id).toBe(toInvocationRequestId(toSessionUuid("abc123"), "toolu_01")); + expect(here.params.request_id).toMatch(UUID_RE); + expect(elsewhere.params.request_id).not.toBe(here.params.request_id); + }); + + it("falls back to a random request_id and no reference when the host sends no call id", () => { + const request = buildEnvelope("PreToolUse", preToolUsePayload, withCallId); + const result = buildEnvelope("PostToolUse", { ...preToolUsePayload, tool_response: { stdout: "done" } }, withCallId); + + expect(request.params.request_id).toMatch(UUID_RE); + expect(result.params.payload.request_id_ref).toBeUndefined(); + expect(() => validateEnvelope(result)).not.toThrow(); + }); + + it("sends no reference when the hookmap declares no call_id", () => { + const result = buildEnvelope("PostToolUse", after, hookmap); + + expect(result.params.payload.request_id_ref).toBeUndefined(); + }); + + it("refuses a call_id that is not a path string, as a hookmap fault", () => { + const broken: Hookmap = { + ...hookmap, + hooks: { ...hookmap.hooks, PreToolUse: { ...hookmap.hooks.PreToolUse!, call_id: 7 } as unknown as HookmapHookEntry }, + }; + + expect(() => buildEnvelope("PreToolUse", before, broken)).toThrow(/"call_id"/); + }); +}); + + +describe("call_id: the real hookmaps, and the derivation pinned", () => { + const CLAUDE_CODE = new URL("../../../hosts/claude-code/claude-code.hookmap.yaml", import.meta.url); + const LINE = " call_id: $.tool_use_id\n"; + + function loadEdited(edit: (yaml: string) => string): Hookmap { + const dir = mkdtempSync(join(tmpdir(), "acs-hookmap-")); + const path = join(dir, "hookmap.yaml"); + writeFileSync(path, edit(readFileSync(CLAUDE_CODE, "utf8"))); + try { + return loadHookmap(path); + } finally { + unlinkSync(path); + rmdirSync(dir); + } + } + + it("pairs a Claude Code request and result through the shipped hookmap, for a session id that is not a uuid", () => { + const shipped = loadHookmap(CLAUDE_CODE.pathname); + const before = { ...preToolUsePayload, session_id: "sess-raw-not-a-uuid", tool_use_id: "toolu_01" }; + const after = { ...before, hook_event_name: "PostToolUse", tool_response: { stdout: "done" } }; + + const request = buildEnvelope("PreToolUse", before, shipped); + const result = buildEnvelope("PostToolUse", after, shipped); + + expect(result.params.payload.request_id_ref).toBe(request.params.request_id); + }); + + it("keeps the derivation stable, because a changed one unpairs every call that straddles an upgrade", () => { + expect(toSessionUuid("abc123")).toBe("4c0c070f-241c-57eb-b406-ca28fe9f1229"); + expect(toInvocationRequestId("4c0c070f-241c-57eb-b406-ca28fe9f1229", "toolu_01")).toBe( + "044e7d73-4d3f-8dc7-abdc-6200ff5a7734", + ); + }); + + it("refuses at load a call_id that is not a path string", () => { + expect(() => loadEdited((yaml) => yaml.replace(LINE, " call_id: 7\n"))).toThrow(/"call_id" must be one JSONPath-lite string/); + }); + + it("refuses at load a call_id declared on the request gate but not the result gate", () => { + const lastAt = (yaml: string) => yaml.lastIndexOf(LINE); + expect(() => + loadEdited((yaml) => yaml.slice(0, lastAt(yaml)) + yaml.slice(lastAt(yaml) + LINE.length)), + ).toThrow(/declares "call_id" on some tool gates but not on \["PostToolUse"\]/); + }); + + it("refuses at load a call_id naming a reserved segment, which resolvePath would refuse on every step", () => { + expect(() => loadEdited((yaml) => yaml.split(LINE).join(" call_id: $.constructor\n"))).toThrow( + /call_id" is not a usable path: .*reserved segment "constructor"/, + ); + }); + + it("refuses at load a call_id on result gates when the hookmap maps no request gate", () => { + // Drop the PreToolUse entry whole: everything from its key to the next hook. + const withoutRequestGate = (yaml: string) => { + const start = yaml.indexOf(" PreToolUse:\n"); + const end = yaml.indexOf(" PostToolUse:\n"); + return yaml.slice(0, start) + yaml.slice(end); + }; + expect(() => loadEdited(withoutRequestGate)).toThrow(/declares "call_id" only on result gates/); + }); + + it("loads a hookmap that declares call_id on neither gate", () => { + expect(() => loadEdited((yaml) => yaml.split(LINE).join(""))).not.toThrow(); + }); +}); diff --git a/reference-implementations/agt/packages/host-adapter/test/govern-step.test.ts b/reference-implementations/agt/packages/host-adapter/test/govern-step.test.ts index 5134115a..563c9cf2 100644 --- a/reference-implementations/agt/packages/host-adapter/test/govern-step.test.ts +++ b/reference-implementations/agt/packages/host-adapter/test/govern-step.test.ts @@ -1188,3 +1188,79 @@ describe("governsTool — the rule both hosts and governStep share", () => { expect(governsTool(scopedHookmap, "toString", "bash")).toBe(true); }); }); + +describe("governStep: a request id derived from the host's call id", () => { + // buildEnvelope derives a request's id from the host's call id, so a host + // that asks twice about one call sends the same id twice. A Guardian + // enforcing Specification §10.3 answers the repeat with REPLAY_DETECTED, + // which carries no decision and would otherwise go to the posture. + const paired: Hookmap = { host: "test-host", hooks: { OnStep: { ...ON_STEP, call_id: "$.call" } } }; + const pairedPayload = { ...payload, call: "call-1" }; + const REPLAY = { decisionArrived: false, failure: { code: -32005, message: "duplicate request_id" } } as const; + const ALLOW = { decisionArrived: true, decision: { type: "decision", decision: "allow" } } as unknown as DecisionOrFailure; + + function scripted(answers: DecisionOrFailure[]): { guardian: GuardianClient; sent: string[] } { + const sent: string[] = []; + return { + sent, + guardian: { + requestDecision: (envelope) => { + sent.push(envelope.params.request_id); + return Promise.resolve(answers.shift() ?? REPLAY); + }, + post: () => Promise.reject(new Error("governStep must not use the wire primitive")), + }, + }; + } + + it("retries once with a fresh random id, and honours the decision that arrives", async () => { + const { sink, events } = recordingSink(); + const { guardian, sent } = scripted([REPLAY, ALLOW]); + + const governed = await govern(guardian, sink, undefined, { hookmap: paired, payload: pairedPayload }); + + expect(governed.stage).toBe("honoured"); + expect(governed.decision?.decision).toBe("allow"); + expect(sent).toHaveLength(2); + expect(sent[1]).not.toBe(sent[0]); + expect(events).toHaveLength(0); + }); + + it("retries only once, and leaves a second REPLAY_DETECTED to the delivery posture (issue #32)", async () => { + const { sink, events } = recordingSink(); + const { guardian, sent } = scripted([REPLAY, REPLAY]); + + const governed = await govern(guardian, sink, undefined, { hookmap: paired, payload: pairedPayload }); + + expect(sent).toHaveLength(2); + expect(governed.stage).toBe("delivery"); + // The audit entry names the request id the retry actually sent. + expect(events).toHaveLength(1); + expect(events[0]?.request_id).toBe(sent[1]); + }); + + it("audits the derived request_id apart from the random JSON-RPC id when no decision arrives", async () => { + const { sink, events } = recordingSink(); + const { guardian, sent } = scripted([{ decisionArrived: false, failure: new GuardianTimeoutError(120) }]); + + await govern(guardian, sink, undefined, { hookmap: paired, payload: pairedPayload }); + + expect(events).toHaveLength(1); + expect(events[0]?.request_id).toBe(sent[0]); + expect(events[0]?.rpc_id).not.toBe(sent[0]); + }); + + it("does not retry at the result gate, whose own id is always random", async () => { + const { sink } = recordingSink(); + const { guardian, sent } = scripted([REPLAY, ALLOW]); + const resultHookmap: Hookmap = { host: "test-host", hooks: { OnResult: { ...ON_RESULT, call_id: "$.call" } } }; + + await governRaw(guardian, sink, undefined, { + hookmap: resultHookmap, + hookEventName: "OnResult", + payload: { ...resultPayload, call: "call-1" }, + }); + + expect(sent).toHaveLength(1); + }); +}); diff --git a/reference-implementations/agt/packages/inspector/README.md b/reference-implementations/agt/packages/inspector/README.md index b9035961..8769628b 100644 --- a/reference-implementations/agt/packages/inspector/README.md +++ b/reference-implementations/agt/packages/inspector/README.md @@ -35,7 +35,7 @@ A flag wins over its variable. The Inspector honours `NO_COLOR` and prints no co ## What each line means -An envelope prints as a header line and the envelope body. The header carries the sequence number from the log, the time it was recorded, the direction, the method and the JSON-RPC id. A response carries a decision badge above the body, with the reason codes and policy references on the same line: +An envelope prints as a header line and the envelope body. The header carries the sequence number from the log, the time it was recorded, the direction, the method and the JSON-RPC id. A `finding` line, which the Guardian writes for a `request_id_ref` it cannot resolve, prints with `! FINDING` in place of the direction. A response carries a decision badge above the body, with the reason codes and policy references on the same line: ``` ● DENY reason_codes=[destructive_shell_command_blocked] policy_references=[agt_stock#destructive_shell_command_blocked] diff --git a/reference-implementations/agt/packages/inspector/src/render.ts b/reference-implementations/agt/packages/inspector/src/render.ts index 1d01ab2b..fd8a71dc 100644 --- a/reference-implementations/agt/packages/inspector/src/render.ts +++ b/reference-implementations/agt/packages/inspector/src/render.ts @@ -233,7 +233,7 @@ export function renderOutcome(message: OutcomeMessage, options: RenderOptions = */ export function renderEnvelopeLogEntry(entry: EnvelopeLogEntry, options: RenderOptions = {}): string { const color = options.color ?? false; - const arrow = entry.direction === "request" ? "→ REQUEST " : "← RESPONSE"; + const arrow = { request: "→ REQUEST ", response: "← RESPONSE", finding: "! FINDING " }[entry.direction]; const method = entry.method ?? "(no method)"; const id = entry.rpc_id === null ? "(unpaired)" : `id=${entry.rpc_id}`; @@ -467,9 +467,13 @@ export function renderSessionChainRow( options: RenderOptions = {}, ): string { const color = options.color ?? false; + const findings = + entry.findings === undefined || entry.findings.length === 0 + ? "" + : ` ! ${entry.findings.join(", ")}${entry.request_id_ref === undefined ? "" : ` ref=${entry.request_id_ref}`}`; const row = `#${entry.seq} ${entry.tool_name} hash=${entry.hash.slice(0, SHORT_HASH_LENGTH)} ` + - `session_id=${entry.session_id}`; + `session_id=${entry.session_id}${findings}`; return link.broken ? paint(`✖ CHAIN BREAK ${row}`, RED, color) : row; } diff --git a/reference-implementations/agt/packages/inspector/src/tail-audit-log.ts b/reference-implementations/agt/packages/inspector/src/tail-audit-log.ts index ffd6a31a..2bc13868 100644 --- a/reference-implementations/agt/packages/inspector/src/tail-audit-log.ts +++ b/reference-implementations/agt/packages/inspector/src/tail-audit-log.ts @@ -70,6 +70,8 @@ type AuditEntryIdentity = { * host's own event name: this reader knows ACS and nothing else. */ method: string | null; rpc_id: string | number | null; + /** The envelope's request_id. Absent on entries written before it existed. */ + request_id?: string | null; }; /** A step the writer's negotiated posture answered, because no decision @@ -136,7 +138,8 @@ function isAuditEntryShape(value: unknown): value is AuditEntry { typeof candidate.recorded_at === "string" && typeof candidate.session_id === "string" && (typeof candidate.method === "string" || candidate.method === null) && - (typeof candidate.rpc_id === "string" || typeof candidate.rpc_id === "number" || candidate.rpc_id === null) + (typeof candidate.rpc_id === "string" || typeof candidate.rpc_id === "number" || candidate.rpc_id === null) && + (candidate.request_id === undefined || typeof candidate.request_id === "string" || candidate.request_id === null) ) ) { return false; diff --git a/reference-implementations/agt/packages/inspector/src/tail-envelope-log.ts b/reference-implementations/agt/packages/inspector/src/tail-envelope-log.ts index ca1db33f..648cbe65 100644 --- a/reference-implementations/agt/packages/inspector/src/tail-envelope-log.ts +++ b/reference-implementations/agt/packages/inspector/src/tail-envelope-log.ts @@ -59,7 +59,10 @@ */ import { closeSync, existsSync, openSync, readSync, statSync } from "node:fs"; -export type EnvelopeLogDirection = "request" | "response"; +// "finding" is a fact the Guardian established about a request that the spec +// requires it to record, such as an unresolved request_id_ref. Its envelope is +// the Guardian's own record, not a JSON-RPC object. +export type EnvelopeLogDirection = "request" | "response" | "finding"; /** One line of the envelope log the Guardian writes. */ export type EnvelopeLogEntry = { @@ -106,7 +109,7 @@ function isEnvelopeLogEntryShape(value: unknown): value is EnvelopeLogEntry { return ( typeof candidate.seq === "number" && typeof candidate.recorded_at === "string" && - (candidate.direction === "request" || candidate.direction === "response") && + (candidate.direction === "request" || candidate.direction === "response" || candidate.direction === "finding") && (typeof candidate.method === "string" || candidate.method === null) && (typeof candidate.rpc_id === "string" || typeof candidate.rpc_id === "number" || candidate.rpc_id === null) ); diff --git a/reference-implementations/agt/packages/inspector/src/tail-session-context.ts b/reference-implementations/agt/packages/inspector/src/tail-session-context.ts index 00cc6066..11360601 100644 --- a/reference-implementations/agt/packages/inspector/src/tail-session-context.ts +++ b/reference-implementations/agt/packages/inspector/src/tail-session-context.ts @@ -55,6 +55,11 @@ export type SessionContextLogEntry = { method: string; request_id: string; tool_name: string; + /** What the Guardian established about the step as it arrived, present + * only when it established something. */ + findings?: string[]; + /** The unresolved reference, present only with `request_id_ref_unresolved`. */ + request_id_ref?: string; }; export type TailSessionContextLogOptions = { @@ -97,7 +102,10 @@ function isSessionContextLogEntryShape(value: unknown): value is SessionContextL typeof candidate.recorded_at === "string" && typeof candidate.method === "string" && typeof candidate.request_id === "string" && - typeof candidate.tool_name === "string" + typeof candidate.tool_name === "string" && + (candidate.findings === undefined || + (Array.isArray(candidate.findings) && candidate.findings.every((finding) => typeof finding === "string"))) && + (candidate.request_id_ref === undefined || typeof candidate.request_id_ref === "string") ); } diff --git a/reference-implementations/agt/packages/inspector/test/render.test.ts b/reference-implementations/agt/packages/inspector/test/render.test.ts index 758fc95d..936ada14 100644 --- a/reference-implementations/agt/packages/inspector/test/render.test.ts +++ b/reference-implementations/agt/packages/inspector/test/render.test.ts @@ -202,6 +202,21 @@ describe("renderDecisionBadge", () => { }); describe("renderEnvelopeLogEntry", () => { + it("labels a finding so it reads apart from the request and response it concerns", () => { + const rendered = renderEnvelopeLogEntry( + entry({ + direction: "finding", + seq: 2, + method: "steps/toolCallResult", + envelope: { finding: "request_id_ref_unresolved", request_id_ref: "6b1f2c9e-4d7a-4e3b-8a51-9c0d2e7f1a36" }, + }), + ); + + expect(rendered.split("\n")[0]).toBe("── #2 12:04:31.221 ! FINDING steps/toolCallResult id=1"); + expect(rendered).toContain('"finding": "request_id_ref_unresolved"'); + expect(rendered).not.toContain("●"); + }); + it("renders a request as a header line plus pretty JSON, with no badge", () => { const rendered = renderEnvelopeLogEntry( entry({ diff --git a/reference-implementations/agt/packages/inspector/test/tail-envelope-log.test.ts b/reference-implementations/agt/packages/inspector/test/tail-envelope-log.test.ts index c1d6eb72..0564e1ba 100644 --- a/reference-implementations/agt/packages/inspector/test/tail-envelope-log.test.ts +++ b/reference-implementations/agt/packages/inspector/test/tail-envelope-log.test.ts @@ -6,7 +6,7 @@ import { tailEnvelopeLog, type EnvelopeLogEntry } from "../src/tail-envelope-log const POLL_MS = 10; -function entryLine(seq: number, direction: "request" | "response"): string { +function entryLine(seq: number, direction: "request" | "response" | "finding"): string { const entry: EnvelopeLogEntry = { seq, recorded_at: "2026-08-09T12:04:31.221Z", @@ -71,6 +71,25 @@ describe("tailEnvelopeLog", () => { }); }); + it("yields a finding line, which the Guardian writes for an unresolved request_id_ref", async () => { + await withTempDir(async (_dir, path) => { + writeFileSync(path, entryLine(1, "request") + entryLine(2, "finding")); + const controller = new AbortController(); + const malformed: string[] = []; + const tail = tailEnvelopeLog({ + path, + fromStart: true, + pollMs: POLL_MS, + signal: controller.signal, + onMalformedLine: (line) => malformed.push(line), + }); + + const entries = await collect(tail, 2, controller); + expect(entries.map((e) => e.direction)).toEqual(["request", "finding"]); + expect(malformed).toEqual([]); + }); + }); + it("yields pre-existing entries when fromStart is set", async () => { await withTempDir(async (_dir, path) => { writeFileSync(path, entryLine(1, "request") + entryLine(2, "response")); diff --git a/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl b/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl index 437a8f22..bb226510 100644 --- a/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl +++ b/reference-implementations/agt/samples/trace/acs-trace-sample.jsonl @@ -1,2 +1,2 @@ -{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"acs-reference-guardian"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"acs-reference-guardian"},"spans":[{"traceId":"9785c34288d27dbd92fcf12e53ac3315","spanId":"31258fb4b343b292","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792065946937000000","endTimeUnixNano":"1792065947036723875","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.capability","value":{"stringValue":"process.exec"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792065947036676541","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"112037a108c3893225309fa193536b77","spanId":"46883c95863bb8fb","name":"gen_ai.tool.result","kind":1,"startTimeUnixNano":"1792065947038000000","endTimeUnixNano":"1792065947088187667","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b02"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.exit_status","value":{"stringValue":"success"}},{"key":"acs.request_id_ref","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792065947088182292","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"981a929f7cea4c999642a38dd3d822b1","spanId":"3b199edb6364b94d","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792065947089000000","endTimeUnixNano":"1792065947141424750","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b03"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.provenance.origin","value":{"arrayValue":{"values":[{"stringValue":"user_input"},{"stringValue":"agent_generated"}]}}},{"key":"acs.provenance.source_id","value":{"arrayValue":{"values":[{"stringValue":"sample-user-1"},{"stringValue":""}]}}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"deny"}},{"key":"acs.reasoning","value":{"stringValue":"This command was blocked because it matches a destructive-shell-command pattern. Policy: destructive_shell_command_blocked, from AGT's stock bundle (agt_stock). Matched at offset 0."}},{"key":"acs.reason_codes","value":{"arrayValue":{"values":[{"stringValue":"destructive_shell_command_blocked"}]}}},{"key":"acs.policy.ids","value":{"arrayValue":{"values":[{"stringValue":"agt_stock"}]}}},{"key":"acs.policy.versions","value":{"arrayValue":{"values":[{"stringValue":""}]}}},{"key":"acs.policy.rule_ids","value":{"arrayValue":{"values":[{"stringValue":"destructive_shell_command_blocked"}]}}}],"name":"acs.decision","timeUnixNano":"1792065947141419958","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"95c92494c004b23662900ae8c17ea04c","spanId":"02d46f81d8206a0f","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792065947141000000","endTimeUnixNano":"1792065947141270000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b04"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"guardian_error"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.error_code","value":{"intValue":-32010}},{"key":"acs.failure.elapsed_ms","value":{"intValue":0}},{"key":"acs.failure.payload_bytes","value":{"intValue":340}}],"name":"acs.decision.failure","timeUnixNano":"1792065947141268125","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The Guardian returned an error instead of a decision."},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"e5703a6e87d75324de28fd7cc555724b","spanId":"e813670f52f6a811","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792065947142000000","endTimeUnixNano":"1792065949203024291","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b05"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792065949202988875","droppedAttributesCount":0},{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"timeout"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.timeout_ms","value":{"intValue":1500}},{"key":"acs.failure.elapsed_ms","value":{"intValue":2061}},{"key":"acs.failure.payload_bytes","value":{"intValue":367}}],"name":"acs.decision.failure","timeUnixNano":"1792065949203022666","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257}]}]}]} -{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"acs-reference-audit-converter"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"acs-reference-audit-converter"},"spans":[{"traceId":"9b082079ac3695104b2da1df27340e16","spanId":"99e8d283763d16d3","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792065949205000000","endTimeUnixNano":"1792065949205000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b06"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"unreachable"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"observed_agent"}},{"key":"acs.failure.detail","value":{"stringValue":"The host could not reach the Guardian."}}],"name":"acs.decision.failure","timeUnixNano":"1792065949205000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The host could not reach the Guardian."},"links":[],"droppedLinksCount":0,"flags":257}]}]}]} +{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"acs-reference-guardian"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"acs-reference-guardian"},"spans":[{"traceId":"7752aef7c63150fcc88f856974d22f5b","spanId":"7bd24ca6a5dd31f0","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792534253539454676","endTimeUnixNano":"1792534253649000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.capability","value":{"stringValue":"process.exec"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792534253649000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"180a78a6f4b17aa91f6a952195321255","spanId":"003c1193df7f552a","name":"gen_ai.tool.result","kind":1,"startTimeUnixNano":"1792534253650630391","endTimeUnixNano":"1792534253716000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b02"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.exit_status","value":{"stringValue":"success"}},{"key":"acs.request_id_ref","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792534253716000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"13717f291e2368066d2fa3f2434ed2f6","spanId":"2a7efedbb7ee3881","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792534253716796645","endTimeUnixNano":"1792534253780000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b03"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}},{"key":"acs.provenance.origin","value":{"arrayValue":{"values":[{"stringValue":"user_input"},{"stringValue":"agent_generated"}]}}},{"key":"acs.provenance.source_id","value":{"arrayValue":{"values":[{"stringValue":"sample-user-1"},{"stringValue":""}]}}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"deny"}},{"key":"acs.reasoning","value":{"stringValue":"This command was blocked because it matches a destructive-shell-command pattern. Policy: destructive_shell_command_blocked, from AGT's stock bundle (agt_stock). Matched at offset 0."}},{"key":"acs.reason_codes","value":{"arrayValue":{"values":[{"stringValue":"destructive_shell_command_blocked"}]}}},{"key":"acs.policy.ids","value":{"arrayValue":{"values":[{"stringValue":"agt_stock"}]}}},{"key":"acs.policy.versions","value":{"arrayValue":{"values":[{"stringValue":""}]}}},{"key":"acs.policy.rule_ids","value":{"arrayValue":{"values":[{"stringValue":"destructive_shell_command_blocked"}]}}}],"name":"acs.decision","timeUnixNano":"1792534253780000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"2cadee9c2b0236f4cf64ecf54933d071","spanId":"d2666e080ce88014","name":"guardian.rejected_request","kind":1,"startTimeUnixNano":"1792534253779859236","endTimeUnixNano":"1792534253781000000","attributes":[{"key":"guardian.rejected.reason","value":{"stringValue":"envelope_invalid"}},{"key":"guardian.rejected.claimed_session_id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"guardian.rejected.claimed_request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01"}},{"key":"guardian.rejected.error_code","value":{"intValue":-32010}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":2,"message":"The Guardian could not vouch for this request, so it carries no ACS ids or events."},"links":[],"droppedLinksCount":0,"flags":257},{"traceId":"f92f9aa4700acfd946b59a2fe19e489e","spanId":"2e4aa2bc8b522e06","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792534253780950247","endTimeUnixNano":"1792534255851000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b05"}},{"key":"gen_ai.tool.name","value":{"stringValue":"Bash"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.decision","value":{"stringValue":"allow"}}],"name":"acs.decision","timeUnixNano":"1792534255851000000","droppedAttributesCount":0},{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"timeout"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"guardian"}},{"key":"acs.failure.timeout_ms","value":{"intValue":1500}},{"key":"acs.failure.elapsed_ms","value":{"intValue":2070}},{"key":"acs.failure.payload_bytes","value":{"intValue":367}}],"name":"acs.decision.failure","timeUnixNano":"1792534255851000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0,"flags":257}]}]}]} +{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"acs-reference-audit-converter"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"acs-reference-audit-converter"},"spans":[{"traceId":"8ad7d2634fb9d194b1414a0033d90467","spanId":"8572a4b873e3e58e","name":"gen_ai.tool.call","kind":1,"startTimeUnixNano":"1792534255853000000","endTimeUnixNano":"1792534255853000000","attributes":[{"key":"acs.session.id","value":{"stringValue":"5a3c1e7e-0b1d-4c6a-9e2f-7d4b8c0a1f01"}},{"key":"acs.request_id","value":{"stringValue":"1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b06"}}],"droppedAttributesCount":0,"events":[{"attributes":[{"key":"acs.failure.class","value":{"stringValue":"unreachable"}},{"key":"acs.failure.posture","value":{"stringValue":"fail_open"}},{"key":"acs.failure.reported_by","value":{"stringValue":"observed_agent"}},{"key":"acs.failure.detail","value":{"stringValue":"The host could not reach the Guardian."}}],"name":"acs.decision.failure","timeUnixNano":"1792534255853000000","droppedAttributesCount":0}],"droppedEventsCount":0,"status":{"code":2,"message":"The host could not reach the Guardian."},"links":[],"droppedLinksCount":0,"flags":257}]}]}]} diff --git a/reference-implementations/agt/scripts/make-sample-trace.ts b/reference-implementations/agt/scripts/make-sample-trace.ts index de4d5e4a..daf17591 100644 --- a/reference-implementations/agt/scripts/make-sample-trace.ts +++ b/reference-implementations/agt/scripts/make-sample-trace.ts @@ -42,17 +42,26 @@ export type Scenario = { capability?: string; /** The `request_id_ref` a result carried, if any. */ requestIdRef?: string; - /** The decision the Guardian answered with, or none when it answered an - * error or was never reached. */ + /** The decision the Guardian answered with, or none when it was never + * reached. */ decision?: "allow" | "deny"; }; +/** + * A forged request that reuses the allowed call's session and request id and + * fails the schema (it carries no JSON-RPC id). The Guardian rejects it + * before trusting it, so it is recorded as a `guardian.rejected_request` + * span with the ids it claimed under rejected names, and the allowed call + * keeps the only span under its pair (Trace Events, issue #283). It is not a + * step, so it is not in SCENARIOS, which are keyed by the pair on a span. + */ +export const REJECTED = { name: "forged", claimedRequestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01", reason: "envelope_invalid" } as const; + /** Every step the sample holds, keyed by the request id its span carries. */ export const SCENARIOS: readonly Scenario[] = [ { name: "allowed", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01", capability: "process.exec", decision: "allow" }, { name: "result", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b02", requestIdRef: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b01", decision: "allow" }, { name: "denied", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b03", decision: "deny" }, - { name: "error", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b04" }, { name: "late", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b05", decision: "allow" }, { name: "unreachable", requestId: "1b6f0c2a-3d4e-4f5a-8b6c-7d8e9f0a1b06" }, ]; @@ -114,8 +123,8 @@ function requests(): Record[] { }, 3, ), - // No JSON-RPC id: the Guardian answers with an error, not a decision. - envelope("steps/toolCallRequest", requestIdOf("error"), bash("pwd"), null), + // No JSON-RPC id, so it fails the schema, and the allowed call's pair. + envelope("steps/toolCallRequest", REJECTED.claimedRequestId, bash("pwd"), null), envelope("steps/toolCallRequest", requestIdOf("late"), bash(`echo ${SLOW_MARKER}`), 5), ]; } diff --git a/reference-implementations/agt/test/audit-to-otlp.test.ts b/reference-implementations/agt/test/audit-to-otlp.test.ts index 5d8896db..0868e4d8 100644 --- a/reference-implementations/agt/test/audit-to-otlp.test.ts +++ b/reference-implementations/agt/test/audit-to-otlp.test.ts @@ -272,6 +272,15 @@ describe("audit-to-otlp: review fixes", () => { }); }); + it("prefers the entry's request_id over rpc_id, because a derived request id differs from the JSON-RPC id", async () => { + await withScratch(async (auditPath, otelPath) => { + const requestId = crypto.randomUUID(); + appendFileSync(auditPath, entryLine({ rpc_id: crypto.randomUUID(), request_id: requestId })); + await convertAuditLog(auditPath, otelPath); + expect(attribute(spansIn(otelPath)[0], "acs.request_id")).toBe(requestId); + }); + }); + it("derives the session UUID exactly as the host adapter does", () => { for (const raw of ["sess-1", "ses_opencode_abc", "6B1F2C9E-4D7A-4E3B-8A51-9C0D2E7F1A36", "セッション-é", ""]) { expect({ raw, uuid: converterSessionUuid(raw) }).toEqual({ raw, uuid: toSessionUuid(raw) }); diff --git a/reference-implementations/agt/test/sample-trace.test.ts b/reference-implementations/agt/test/sample-trace.test.ts index dbc86ab5..8ca2fda0 100644 --- a/reference-implementations/agt/test/sample-trace.test.ts +++ b/reference-implementations/agt/test/sample-trace.test.ts @@ -4,7 +4,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { CONVERTER_SERVICE_NAME } from "../packages/guardian/src/audit-to-otlp.ts"; import { OTEL_SERVICE_NAME } from "../packages/guardian/src/otel-trace-exporter.ts"; -import { generateSampleTrace, SAMPLE_PATH, SCENARIOS } from "../scripts/make-sample-trace.ts"; +import { generateSampleTrace, REJECTED, SAMPLE_PATH, SCENARIOS } from "../scripts/make-sample-trace.ts"; /** * The committed sample trace (issue #266), held to the mapping it claims to @@ -100,6 +100,10 @@ const CONVERTER_EXEMPT: Readonly> = { }; const SPANS = spansIn(SAMPLE_PATH); +/** The spans for steps the Guardian governed. A rejected request is recorded + * under a name outside step_to_span (issue #283), so the mapping's step + * rules do not apply to it; its own test is below. */ +const STEP_SPANS = SPANS.filter((span) => span.name !== "guardian.rejected_request"); function spanFor(requestId: string): OtlpSpan | undefined { return SPANS.find((span) => attributes(span)["acs.request_id"] === requestId); @@ -111,9 +115,9 @@ function failureOf(span: OtlpSpan | undefined): Record | undefi } describe("the committed sample trace matches the mapping", () => { - it("names every span with a step_to_span value", () => { - expect(SPANS.length).toBeGreaterThan(0); - for (const span of SPANS) { + it("names every step span with a step_to_span value", () => { + expect(STEP_SPANS.length).toBeGreaterThan(0); + for (const span of STEP_SPANS) { expect({ name: span.name, mapped: span.name in STEP_BY_SPAN_NAME }).toEqual({ name: span.name, mapped: true }); } }); @@ -128,7 +132,7 @@ describe("the committed sample trace matches the mapping", () => { it("carries every required attribute, apart from the named converter exemptions", () => { const always = MAPPING.properties.step_span_attributes.properties.required_attributes.default; - for (const span of SPANS) { + for (const span of STEP_SPANS) { const present = attributes(span); const fromConverter = failureOf(span)?.["acs.failure.reported_by"] === "observed_agent"; for (const attribute of [...always, ...STEP_BY_SPAN_NAME[span.name]!.required_attributes]) { @@ -182,7 +186,7 @@ describe("the committed sample trace matches the mapping", () => { expect(denied["acs.provenance.source_id"]).toContain(""); }); - it("uses only failure classes, postures and reporters the mapping lists, and covers the four failures", () => { + it("uses only failure classes, postures and reporters the mapping lists, and covers the failures it holds", () => { const allowed = MAPPING.properties.decision_failure_event.properties.attribute_values.default; const seen = new Set(); for (const span of SPANS) { @@ -197,7 +201,22 @@ describe("the committed sample trace matches the mapping", () => { } seen.add(`${failure["acs.failure.class"]}/${failure["acs.failure.reported_by"]}`); } - expect([...seen].sort()).toEqual(["guardian_error/guardian", "timeout/guardian", "unreachable/observed_agent"]); + // The sample holds no step the Guardian answered with an error. + expect([...seen].sort()).toEqual(["timeout/guardian", "unreachable/observed_agent"]); + }); + + it("holds one span per step, and keeps the forged request apart from the call whose ids it claimed", () => { + for (const scenario of SCENARIOS) { + const underPair = SPANS.filter((span) => attributes(span)["acs.request_id"] === scenario.requestId); + expect({ scenario: scenario.name, spans: underPair.length }).toEqual({ scenario: scenario.name, spans: 1 }); + } + const rejected = SPANS.filter((span) => span.name === "guardian.rejected_request"); + expect(rejected).toHaveLength(1); + const claimed = attributes(rejected[0]); + expect(claimed["guardian.rejected.claimed_request_id"]).toBe(REJECTED.claimedRequestId); + expect(claimed["guardian.rejected.reason"]).toBe(REJECTED.reason); + expect(claimed["acs.request_id"]).toBeUndefined(); + expect(rejected[0]?.events ?? []).toEqual([]); }); }); diff --git a/specification/v0.1.0/handshake.json b/specification/v0.1.0/handshake.json index 819a1e53..c901f94a 100644 --- a/specification/v0.1.0/handshake.json +++ b/specification/v0.1.0/handshake.json @@ -128,7 +128,7 @@ "type": "string", "enum": ["proceed", "deny"], "default": "proceed", - "description": "Posture the Observed Agent applies on a decision failure: no usable decision within the negotiated timeout, whether the Guardian is silent, the transport fails, or an error response carries no decision (Specification 6.4). 'proceed' (default) is fail-open, so a slow, erroring, or unreachable Guardian does not halt production; 'deny' is fail-closed. Every fail-open proceed MUST be recorded as an audit event (Specification 6.4). A decision that arrives within the timeout MUST be honored regardless of this posture." + "description": "Posture the Observed Agent applies on a decision failure: no usable decision within the negotiated timeout, whether the Guardian is silent, the transport fails, or an error response carries no decision (Specification 6.4). 'proceed' (default) is fail-open, so a slow, erroring, or unreachable Guardian does not halt production; 'deny' is fail-closed. Every decision failure MUST be recorded as an audit event, fail-open or fail-closed (Specification 6.4). A decision that arrives within the timeout MUST be honored regardless of this posture." }, "approver_types_supported": { "type": "array", diff --git a/specification/v0.1.0/hooks/session-end.json b/specification/v0.1.0/hooks/session-end.json index c1b1a873..5cf532c7 100644 --- a/specification/v0.1.0/hooks/session-end.json +++ b/specification/v0.1.0/hooks/session-end.json @@ -19,6 +19,14 @@ "errors": { "type": "array", "items": { "type": "string" } + }, + "decision_failures": { + "type": "object", + "description": "Decision failures the Observed Agent recorded in this session, counted per failure class (Specification 6.4). The classes are the acs.failure.class values in trace/otel-mapping.json. A class with no failures MAY be omitted or set to 0. A reader treats a session that ends without this object as unmeasurable.", + "propertyNames": { + "enum": ["timeout", "unreachable", "malformed_response", "guardian_error", "observed_agent_error", "signature_invalid", "handshake_failure", "other"] + }, + "additionalProperties": { "type": "integer", "minimum": 0 } } } }, diff --git a/specification/v0.1.0/hooks/tool-call-result.json b/specification/v0.1.0/hooks/tool-call-result.json index 6f4bfcd3..9de8af22 100644 --- a/specification/v0.1.0/hooks/tool-call-result.json +++ b/specification/v0.1.0/hooks/tool-call-result.json @@ -19,7 +19,7 @@ "request_id_ref": { "type": "string", "format": "uuid", - "description": "Links to the originating toolCallRequest" + "description": "The request_id of the originating toolCallRequest. How a Guardian resolves it, and what it records when it cannot, is in hooks.md, toolCallResult." }, "exit_status": { "type": "string", diff --git a/specification/v0.1.0/request-envelope.json b/specification/v0.1.0/request-envelope.json index 61218a26..2b851a81 100644 --- a/specification/v0.1.0/request-envelope.json +++ b/specification/v0.1.0/request-envelope.json @@ -122,7 +122,7 @@ }, "key_id": { "type": "string", - "description": "Identifier resolving to the public key (or the hybrid key descriptor that pins both component public keys for hybrid algorithms)." + "description": "For HMAC-SHA256, an opaque label chosen by the deployment that names one configured secret (Specification 10, key_id for HMAC). For asymmetric algorithms, the identifier resolving to the public key (or the hybrid key descriptor that pins both component public keys for hybrid algorithms)." } } } diff --git a/specification/v0.1.0/response-envelope.json b/specification/v0.1.0/response-envelope.json index 56a16a22..87f645fc 100644 --- a/specification/v0.1.0/response-envelope.json +++ b/specification/v0.1.0/response-envelope.json @@ -89,7 +89,7 @@ "chain_hash": { "type": "string", "pattern": "^[0-9a-f]{64}$", - "description": "Rolling SHA-256 audit-chain head (lowercase hex) after this step's ContextEntry was appended. The Guardian MUST include it on every response for a step where it wrote a ContextEntry, the content-bearing steps (Specification 8). Publishing the head on the wire lets an observer that records traffic detect a chain rewritten after the fact. It is covered by the response signature. Under the HMAC baseline this gives integrity and tamper-evidence to a key-holder and supports cross-Guardian comparison; non-repudiation against a compromised Guardian requires the asymmetric ACS-Crypto profile." + "description": "Rolling SHA-256 audit-chain head (lowercase hex) after this step's ContextEntry was appended. The Guardian MUST include it on every response for a step where it wrote a ContextEntry, the content-bearing steps (Specification 8). Publishing the head on the wire lets an observer that records traffic detect a chain rewritten after the fact. It is covered by the response signature. Under the HMAC baseline this gives integrity and tamper-evidence to a key-holder and supports cross-Guardian comparison. Both parties hold the key, so a signed head is attributable to neither, and non-repudiation requires the asymmetric ACS-Crypto profile (Specification 8.6)." }, "signature": { "$ref": "request-envelope.json#/$defs/Signature", diff --git a/specification/v0.1.0/trace/otel-mapping.json b/specification/v0.1.0/trace/otel-mapping.json index aa76ebed..f513fe6a 100644 --- a/specification/v0.1.0/trace/otel-mapping.json +++ b/specification/v0.1.0/trace/otel-mapping.json @@ -174,7 +174,7 @@ }, "decision_failure_event": { "type": "object", - "description": "A decision failure is recorded as a span event named 'acs.decision.failure' on the step span, separate from acs.decision, or on the acs.session span for handshake_failure. It records a step whose decision did not arrive or was not usable, and the deployment's failure posture. The Observed Agent MUST record every decision failure, fail-open or fail-closed, in the deployment's own audit log, and SHOULD emit this event when it claims ACS-Trace. A Guardian claiming ACS-Trace SHOULD emit it for the failures it sees itself: an error it returned (guardian_error) and a decision it finished after the negotiated timeout (timeout). In v0.1 the Observed Agent reports this event about its own conduct, so a missing event does not show that no failure happened, and a count built from these events is a lower bound. acs.failure.reported_by, acs.failure.posture and acs.failure.payload_bytes are the emitter's own claims. Events from both parties for one request describe one failure, so a reader counts distinct acs.session.id and acs.request_id pairs. That counts failed attempts, not steps: a step the Observed Agent recovered by retrying still shows its failed attempts. A pair with a Guardian event and no Observed Agent event is still counted. A Guardian MUST NOT emit this event for a request it rejects as a replay (REPLAY_DETECTED), because that pair belongs to the earlier request. A reader cross-checking the two sides takes them from separately produced streams. acs.failure.detail is required when acs.failure.class is other. It MUST be text the emitter composes itself, and MUST NOT copy bytes from the request, the response, or an error's data, including parser messages that quote their input. acs.failure.payload_bytes SHOULD be emitted, because when reported honestly it helps separate an induced failure from an organic one.", + "description": "A decision failure is recorded as a span event named 'acs.decision.failure' on the step span, separate from acs.decision, or on the acs.session span for handshake_failure. It records a step whose decision did not arrive or was not usable, and the deployment's failure posture. The Observed Agent MUST record every decision failure, fail-open or fail-closed, in the deployment's own audit log, and SHOULD emit this event when it claims ACS-Trace. A Guardian claiming ACS-Trace SHOULD emit it for the failures it sees itself: an error it returned (guardian_error) and a decision it finished after the negotiated timeout (timeout). In v0.1 the Observed Agent reports this event about its own conduct, so a missing event does not show that no failure happened, and a count built from these events is a lower bound. acs.failure.reported_by, acs.failure.posture and acs.failure.payload_bytes are the emitter's own claims. Events from both parties for one request describe one failure, so a reader counts distinct acs.session.id and acs.request_id pairs. That counts failed attempts, not steps: a step the Observed Agent recovered by retrying still shows its failed attempts. A pair with a Guardian event and no Observed Agent event is still counted. A Guardian MUST NOT emit this event or an acs.decision event under the pair of a request it rejects before trusting it: an envelope that fails validation, a signature that fails verification (SIGNATURE_INVALID), a replay (REPLAY_DETECTED), or a timestamp outside the window (TIMESTAMP_OUT_OF_WINDOW). Such a request can carry any pair, including a genuine request's. A deployment that records rejected requests does so under attribute names other than acs.session.id and acs.request_id. A reader cross-checking the two sides takes them from separately produced streams. acs.failure.detail is required when acs.failure.class is other. It MUST be text the emitter composes itself, and MUST NOT copy bytes from the request, the response, or an error's data, including parser messages that quote their input. acs.failure.payload_bytes SHOULD be emitted, because when reported honestly it helps separate an induced failure from an organic one.", "properties": { "event_name": { "type": "string", "const": "acs.decision.failure" }, "required_attributes": { @@ -189,7 +189,7 @@ }, "attribute_values": { "type": "object", - "description": "The values each enumerated attribute takes. timeout: the Guardian was reached and no decision arrived in time. unreachable: the request never reached the Guardian. malformed_response: an answer arrived and was not usable, for any reason other than its signature. signature_invalid: an answer arrived whose signature failed verification (Specification §10), kept apart so a tampered response does not blend into organic failures. handshake_failure: the session started without a completed handshake (Specification §4.1), so no step in it was governed; it is recorded on the acs.session span and counted under the handshake's own request_id and session_id. guardian_error: an error came back instead of a decision. observed_agent_error: the Observed Agent's own side failed before a decision could arrive. other: a failure that fits none of these. acs.failure.posture is fail_open for on_decision_failure proceed and fail_closed for deny. A response whose modifications break Specification §6.3, whether or not schema validation also rejects it, is not a decision failure, because §6.3 requires the Observed Agent to apply DENY. A MODIFY the client cannot apply is not a decision failure either, because §6.5 defines its outcome: DENY, or at postCompact the original summary. malformed_response covers a response that does not parse, or that fails schema validation for any other reason.", + "description": "The values each enumerated attribute takes. timeout: the Guardian was reached and no decision arrived in time. unreachable: the request never reached the Guardian. malformed_response: an answer arrived and was not usable, for any reason other than its signature. signature_invalid: an answer arrived whose signature failed verification (Specification §10), kept apart so a tampered response does not blend into organic failures. handshake_failure: the handshake did not complete (Specification §4.1). Under the startup posture proceed the session ran with no step governed, and under refuse it did not start. It is recorded on the acs.session span and counted under the handshake's own request_id and session_id. guardian_error: an error came back instead of a decision. observed_agent_error: the Observed Agent's own side failed before a decision could arrive. other: a failure that fits none of these. acs.failure.posture is fail_open for on_decision_failure proceed and fail_closed for deny. A handshake_failure comes before on_decision_failure is negotiated, so its posture reflects the startup posture: fail_open for proceed and fail_closed for refuse. A response whose modifications break Specification §6.3, whether or not schema validation also rejects it, is not a decision failure, because §6.3 requires the Observed Agent to apply DENY. A MODIFY the client cannot apply is not a decision failure either, because §6.5 defines its outcome: DENY, or at postCompact the original summary. malformed_response covers a response that does not parse, or that fails schema validation for any other reason.", "additionalProperties": { "type": "array", "items": { "type": "string" } }, "default": { "acs.failure.class": ["timeout", "unreachable", "malformed_response", "guardian_error", "observed_agent_error", "signature_invalid", "handshake_failure", "other"], diff --git a/tests/test_decision_failure_classes.py b/tests/test_decision_failure_classes.py new file mode 100644 index 00000000..5eb0bd87 --- /dev/null +++ b/tests/test_decision_failure_classes.py @@ -0,0 +1,28 @@ +"""The sessionEnd failure count and the Trace failure event name the same classes. + +`summary.decision_failures` in hooks/session-end.json counts decision failures per +class, and `acs.failure.class` on the `acs.decision.failure` event names the class of +each one. A class added to one list and not the other would leave a failure that one +side can record and the other cannot count. +""" + +import json +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +SESSION_END = ROOT / "specification" / "v0.1.0" / "hooks" / "session-end.json" +OTEL_MAPPING = ROOT / "specification" / "v0.1.0" / "trace" / "otel-mapping.json" + + +def test_session_end_counts_exactly_the_trace_failure_classes(): + session_end = json.loads(SESSION_END.read_text(encoding="utf-8")) + counted = session_end["properties"]["summary"]["properties"]["decision_failures"]["propertyNames"]["enum"] + + mapping = json.loads(OTEL_MAPPING.read_text(encoding="utf-8")) + failure_event = mapping["properties"].get("decision_failure_event") + assert failure_event is not None, "otel-mapping.json defines no decision_failure_event" + emitted = failure_event["properties"]["attribute_values"]["default"]["acs.failure.class"] + + assert sorted(counted) == sorted(emitted), ( + f"session-end.json counts {sorted(counted)}; acs.failure.class takes {sorted(emitted)}" + ) diff --git a/tests/test_hmac_key_derivation.py b/tests/test_hmac_key_derivation.py new file mode 100644 index 00000000..bb0d73e9 --- /dev/null +++ b/tests/test_hmac_key_derivation.py @@ -0,0 +1,54 @@ +"""The HMAC key-derivation example in Specification section 10 matches its parameters. + +The example is the one known-answer value a second implementer checks against, so +a typo in it, or a parameter change that forgets to update it, would send every +new implementation chasing a key no correct HKDF produces. This recomputes it from +the stated parameters with the standard library alone. +""" + +import hashlib +import hmac +import re +from pathlib import Path + +SPEC = Path(__file__).resolve().parent.parent / "docs" / "spec" / "instrument" / "specification.md" + + +def _hkdf_sha256(ikm: bytes, info: bytes, length: int) -> bytes: + # RFC 5869 with no salt: extract uses HashLen zero bytes as the HMAC key. + prk = hmac.new(b"\x00" * hashlib.sha256().digest_size, ikm, hashlib.sha256).digest() + okm, block, counter = b"", b"", 1 + while len(okm) < length: + block = hmac.new(prk, block + info + bytes([counter]), hashlib.sha256).digest() + okm += block + counter += 1 + return okm[:length] + + +def test_section_10_hkdf_example_matches_its_parameters(): + text = SPEC.read_text(encoding="utf-8") + example = re.search( + r"the `session_id` `(?P[0-9a-f-]{36})` give the key `(?P[0-9a-f]{64})`", text + ) + assert example, "Specification section 10 no longer carries the HKDF example in the expected form" + derived = _hkdf_sha256(bytes(range(32)), example["sid"].encode("utf-8"), 32) + assert derived.hex() == example["key"] + + +def test_hkdf_helper_matches_rfc_5869_test_case_3(): + # RFC 5869 Appendix A.3, the SHA-256 case with no salt and empty info, so + # the helper above is anchored to the RFC and not only to the spec's own example. + okm = _hkdf_sha256(b"\x0b" * 22, b"", 42) + assert okm.hex() == ( + "8da4e775a563c18f715f802a063c5a31b8a11f5c5ee1879ec3454e5f3c738d2d9d201395faa4b61a96c8" + ) + + +def test_session_id_is_used_as_sent_not_normalized(): + # Section 10 says info is the session_id string exactly as the request carries + # it. The schema's uuid format accepts other spellings of the same UUID, and + # each one derives a different key, which is why normalizing breaks interop. + sid = "6b1f2c9e-4d7a-4e3b-8a51-9c0d2e7f1a36" + as_sent = _hkdf_sha256(bytes(range(32)), sid.encode("utf-8"), 32) + for spelling in (sid.upper(), f"urn:uuid:{sid}"): + assert _hkdf_sha256(bytes(range(32)), spelling.encode("utf-8"), 32) != as_sent