From c26ac610aba4c2972ac26817e56014bd0b818c9b Mon Sep 17 00:00:00 2001 From: Christopher Date: Thu, 17 Sep 2026 14:48:03 +1000 Subject: [PATCH 01/12] docs(architecture): define coding execution gateway --- ...-agent-execution-through-an-a2a-gateway.md | 290 ++++++++++++++++++ 1 file changed, 290 insertions(+) create mode 100644 docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md new file mode 100644 index 00000000..c57f5fd3 --- /dev/null +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -0,0 +1,290 @@ +# ADR 0002: Serve coding-agent execution through an A2A gateway + +- Status: Accepted; implementation pending +- Date: 2026-09-17 + +## Context + +AllAgents already owns cross-client agent configuration, workspace knowledge, +plugins, hooks, MCP configuration, and launchers for Codex and other coding +agents. External systems also need to invoke those agents without importing +AllAgents internals or coupling to an interactive CLI process. + +The first planned consumer is AI Evals. Its +[ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) +removes AI Evals-owned coding workspaces in favor of a Promptfoo provider that +needs one remote coding-agent call to return output, usage, traces, file +changes, produced artifacts, failures, cleanup outcomes, and execution +provenance. Future clients may need the same execution boundary without +Promptfoo or evaluation semantics. + +A coding-agent execution is more than a model request. It includes immutable +source selection, repository acquisition, environment setup, credentials, +permissions, agent invocation, cancellation, evidence capture, process +termination, and cleanup. Those responsibilities need one public contract while +allowing materially different execution backends. + +The contract must not turn AllAgents into an evaluation harness. Dataset +expansion, repetition, assertions, scoring, experiment scheduling, and durable +evaluation Runs remain consumer concerns. + +## Decision + +### Add a separately deployable execution gateway + +AllAgents will provide a separately testable and deployable execution-gateway +entry point. It will not be coupled to an interactive CLI command lifecycle. + +The gateway owns: + +- authentication and authorization; +- stable Task and idempotency identity; +- deadline and cancellation propagation; +- execution-profile and backend selection; +- normalization of terminal output and evidence; +- protocol-level Task status and bounded retention; and +- enforcement of the coding-execution contract across every backend. + +The gateway is not an evaluator, grader, experiment scheduler, retry authority, +or durable evaluation Run ledger. It does not own a consumer's result store. + +### Keep the gateway separate from execution backends + +The gateway dispatches to peer execution backends. The initial design supports: + +- a direct Codex backend; and +- Agent-Conductor as an alternative backend. + +Using Codex does not require an Agent-Conductor hop. Additional backends may be +added only when they satisfy the same conformance contract. + +Execution backends own repository materialization, environment setup, agent +invocation, evidence collection, process termination, and cleanup. The gateway +must not execute evaluated agents or mount their writable repositories in the +gateway process. + +When deployed on Kubernetes, the gateway runs as its own Deployment and +ClusterIP Service, separate from consumers and execution workers. A direct +backend dispatches to a worker pool, per-invocation Job, or stronger sandbox. +Agent-Conductor remains a separate service. The protocol does not require one +worker topology. + +A separate gateway Pod is a service and failure boundary, not per-invocation +security isolation. Deployments requiring hostile-code or tenant isolation +must create or select a stronger execution boundary behind the gateway. + +### Profile A2A 1.0 instead of inventing an invocation API + +The external contract profiles the Linux Foundation +[Agent2Agent protocol](https://a2a-protocol.org/latest/specification/). The +initial profile requires the A2A 1.0 HTTP+JSON binding and retains Agent Card, +Message, Part, Task, Artifact, status, streaming, cancellation, security, and +error semantics. + +The profile narrows A2A for deterministic coding execution: + +- every accepted execution request creates exactly one addressable A2A Task; + direct-Message completion is not supported; +- the gateway implements all mandatory A2A core operations, including + `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; when its Agent Card + advertises streaming, it also implements `SendStreamingMessage` and + `SubscribeToTask`; capability-gated operations retain their standard A2A + behavior instead of being replaced by bespoke `/v1/invocations`, `/v1/runs`, + or `/v1/trials` resources; +- terminal Task results use Artifacts for output and evidence rather than + relying on transient messages or stream events; and +- each versioned Agent Card advertises one mandatory AllAgents extension version + for source and runtime identity, traces, usage and cost, file changes, + produced artifacts, typed failures, cancellation and cleanup outcomes, + evidence completeness, and provenance. + +Generic A2A conformance is insufficient. The AllAgents extension and its +conformance fixtures define the coding-execution guarantees every backend must +satisfy. + +Breaking extension changes use a new extension URI and a versioned Agent Card +or service endpoint. During migration, the gateway keeps the old card, endpoint, +and required extension serviceable while consumers move to the new profile. +Each card requires exactly one extension version. Clients pin the card they +support; the gateway never silently falls back across incompatible versions. +Retiring an old profile is a separate coordinated compatibility decision, not a +lockstep deployment requirement. + +The AAIF +[agentgateway](https://github.com/agentgateway/agentgateway) project may be used +as traffic-policy infrastructure for A2A, MCP, or model calls. It is not the +AllAgents execution service or evidence schema. Documentation uses **AllAgents +execution gateway** where the distinction matters. + +### Keep adjacent protocols at their proper boundaries + +The [Agent Client Protocol](https://agentclientprotocol.com/) may be used behind +a backend adapter when a coding agent supports it. Its session, progress, tool, +permission, terminal, diff, usage, and cancellation semantics are useful +internally, but its stdio editor-to-agent protocol is not the external gateway +API. + +[Model Context Protocol](https://modelcontextprotocol.io/) remains a tool and +resource protocol inside an execution backend. It does not represent the whole +coding-agent execution. + +[Agent Format](https://agentformat.org/) may provide an optional static agent +manifest and vocabulary. It does not define the execution transport or prove +observed execution evidence. + +The archived IBM/BeeAI Agent Communication Protocol is superseded by A2A and +will not be adopted. + +### Separate trace propagation, span semantics, and durable evidence + +Gateway calls propagate +[W3C Trace Context](https://www.w3.org/TR/trace-context/) across HTTP and process +boundaries. AllAgents uses OpenTelemetry and OTLP for operational telemetry. +AllAgents-managed agent, model, and tool spans use +[OpenInference](https://arize-ai.github.io/openinference/) semantic conventions +where corresponding attributes exist; useful backend-native attributes may be +retained alongside them. Consumer-owned evaluator spans may join the propagated +trace without becoming gateway-owned. + +These standards are complementary: + +- W3C Trace Context propagates causal trace identity; +- OpenTelemetry and OTLP represent and transport live operational telemetry; +- OpenInference describes AI operations on OpenTelemetry spans; and +- the AllAgents A2A extension returns durable coding evidence and provenance. + +An external trace backend is not the sole durable result. Sampling, redaction, +transport loss, or retention policy must not erase the terminal facts needed by +a consumer. + +### Trial ATIF only as an optional trajectory Artifact + +The Harbor +[Agent Trajectory Interchange Format](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md) +may be returned as an optional, explicitly versioned A2A Artifact when a backend +can produce or truthfully normalize an ordered agent trajectory. It is not the +A2A transport, the OpenTelemetry trace, or the AllAgents evidence envelope. +Backend-native trajectories remain available when conversion would lose +information. + +An ATIF Artifact must declare its exact schema version and correlate its A2A +Task, OpenTelemetry trace, AllAgents invocation, and backend session identities +through the versioned AllAgents extension. Reasoning content is excluded by +default. Tool arguments, observations, and media follow explicit redaction, +size, and disclosure policy. Truncation or conversion loss is reported rather +than hidden. + +ATIF remains optional until its compatibility policy, specification, tooling, +and non-Harbor conformance mature enough for a required public-contract +capability. + +Harbor's task package, Job configuration, Job/Trial result models, hosted API, +artifact manifest, registry formats, and trial-directory layout will not become +the gateway contract. They remain Harbor-native formats that a future adapter +may preserve. Harbor's ASP `.asp.json` is a draft v0 sandbox proposal and is not +adopted by this decision. + +### Make execution provenance and cleanup explicit + +The gateway and selected backend are collectively responsible for: + +1. resolving and verifying immutable source identity; +2. acquiring or restoring source through the selected transport; +3. creating a clean or explicitly reusable working location; +4. running setup before the evaluated agent action; +5. applying permissions and execution isolation; +6. invoking the agent and propagating cancellation and deadlines; +7. capturing bounded output, usage, cost, file changes, checks, and artifact + references; +8. returning terminal status, evidence completeness, and provenance; and +9. terminating processes and releasing or retaining resources according to the + documented lifecycle. + +Source transport and runtime transport are independent. A backend may use one +immutable runtime image plus a separately digest-addressed source artifact; the +contract does not require source code to be baked into the runtime image. + +Credentials remain deployment policy. Requests must not embed deployment +credentials. The gateway authenticates callers, and the selected backend scopes +source and model credentials to the execution boundary without returning +secret-bearing paths or values. + +Retries must not multiply non-idempotent agent execution. Every request carries +a caller-scoped stable invocation key through the AllAgents extension. The +gateway binds the authenticated caller, invocation key, effective execution +profile, and request digest to the created Task for a documented retry-retention +window. An identical replay returns the original Task. Reusing the key with a +different request is rejected. Backend retry suppression remains an additional +safeguard; it does not replace gateway deduplication. + +### Keep evaluation commands out of scope + +This decision does not add `allagents eval`, benchmark authoring, assertions, +scoring, datasets, or experiment scheduling. A community evaluation wrapper and +an enterprise AI Evals wrapper may share this execution service in the future, +but their product and ownership model requires a separate decision. + +## Consequences + +- AllAgents becomes a service boundary in addition to a local CLI, but retains a + narrow coding-execution responsibility. +- Consumers depend on A2A 1.0 plus a versioned AllAgents extension, not + AllAgents TypeScript modules, CLI behavior, or workspace internals. +- Direct Codex and Agent-Conductor execution are interchangeable backends behind + one conformance suite. +- Gateway and execution workers scale and fail independently. +- The gateway can remain lightweight; physical isolation and resource policy + belong to the selected execution backend. +- A2A supplies discovery and lifecycle semantics. AllAgents supplies the + coding-specific evidence contract. +- W3C Trace Context, OpenTelemetry/OTLP, OpenInference, optional ATIF, and the + terminal evidence extension remain distinct layers rather than competing + universal formats. +- Implementations must preserve bounded native evidence whenever normalization + would lose information. + +## Rejected alternatives + +### Invent a bespoke invocation, run, or trial API + +Rejected because A2A already defines remote-agent discovery, Task lifecycle, +streaming, artifacts, cancellation, errors, and web security. Coding-specific +evidence belongs in a versioned A2A extension rather than a parallel transport. + +### Run agents in the gateway Pod + +Rejected because it couples control-plane availability and credentials to +mutable repository execution, prevents independent scaling, and mistakes a +service boundary for per-invocation isolation. + +### Make Agent-Conductor mandatory + +Rejected because a direct Codex adapter and Agent-Conductor are peer backends. +Mandatory indirection adds an ownership and failure boundary without improving +the public contract. + +### Use OpenInference instead of W3C Trace Context + +Rejected as a category error. W3C Trace Context propagates trace identity; +OpenInference supplies AI semantic conventions on OpenTelemetry spans. The +gateway uses both. + +### Use ATIF as the complete gateway result + +Rejected because ATIF represents an ordered agent trajectory, not remote Task +lifecycle, repository provenance, workspace changes, produced artifacts, +cleanup, authorization, or evidence completeness. + +### Adopt Harbor's Job or Trial API + +Rejected because Harbor's formats own benchmark orchestration, verification, +and persisted runner state. The AllAgents gateway executes one coding-agent +request and does not become an evaluation harness. + +## Reconsider when + +Revisit this decision if A2A standardizes the required coding-execution evidence +without an extension, if a stable cross-vendor execution protocol subsumes the +same lifecycle and provenance guarantees, or if operational evidence shows that +the gateway and backend boundary prevents required cancellation, isolation, or +result integrity. From 8bca61e134a64631c82556cdf5dcc00aa14619d2 Mon Sep 17 00:00:00 2001 From: Christopher Date: Fri, 18 Sep 2026 08:27:43 +1000 Subject: [PATCH 02/12] docs(architecture): refine gateway backend boundaries --- ...-agent-execution-through-an-a2a-gateway.md | 46 +++++--- .../agent-host-protocol-decision-inputs.md | 109 ++++++++++++++++++ 2 files changed, 138 insertions(+), 17 deletions(-) create mode 100644 docs/research/agent-host-protocol-decision-inputs.md diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index c57f5fd3..e798ef3e 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -50,13 +50,15 @@ or durable evaluation Run ledger. It does not own a consumer's result store. ### Keep the gateway separate from execution backends -The gateway dispatches to peer execution backends. The initial design supports: +The initial design supports three peer execution backends: -- a direct Codex backend; and -- Agent-Conductor as an alternative backend. +- Codex; +- OpenCode; and +- Pi. -Using Codex does not require an Agent-Conductor hop. Additional backends may be -added only when they satisfy the same conformance contract. +Each backend implements the same conformance contract. Provider-specific +process, session, permission, cancellation, and evidence behavior remains +behind its adapter. Execution backends own repository materialization, environment setup, agent invocation, evidence collection, process termination, and cleanup. The gateway @@ -64,10 +66,10 @@ must not execute evaluated agents or mount their writable repositories in the gateway process. When deployed on Kubernetes, the gateway runs as its own Deployment and -ClusterIP Service, separate from consumers and execution workers. A direct -backend dispatches to a worker pool, per-invocation Job, or stronger sandbox. -Agent-Conductor remains a separate service. The protocol does not require one -worker topology. +ClusterIP Service, separate from consumers and execution workers. A backend +dispatches to a worker pool, per-invocation Job, or stronger sandbox according +to the selected execution profile. The protocol does not require one worker +topology. A separate gateway Pod is a service and failure boundary, not per-invocation security isolation. Deployments requiring hostile-code or tenant isolation @@ -124,6 +126,15 @@ permission, terminal, diff, usage, and cancellation semantics are useful internally, but its stdio editor-to-agent protocol is not the external gateway API. +The [Agent Host Protocol](https://microsoft.github.io/agent-host-protocol/) +may be used behind a backend adapter when a host exposes it, or beside the +gateway if AllAgents later adds a collaborative multi-client session surface. +Its host-authoritative snapshots, actions, reconnection, tools, permissions, +and changesets solve live session synchronization; they do not replace A2A +Task identity, idempotency, authorization, terminal evidence, or retention. +The supporting research and implementation consequences are captured in the +[AHP decision inputs](../research/agent-host-protocol-decision-inputs.md). + [Model Context Protocol](https://modelcontextprotocol.io/) remains a tool and resource protocol inside an execution backend. It does not represent the whole coding-agent execution. @@ -230,8 +241,8 @@ but their product and ownership model requires a separate decision. narrow coding-execution responsibility. - Consumers depend on A2A 1.0 plus a versioned AllAgents extension, not AllAgents TypeScript modules, CLI behavior, or workspace internals. -- Direct Codex and Agent-Conductor execution are interchangeable backends behind - one conformance suite. +- Codex, OpenCode, and Pi are peer execution backends behind one conformance + suite. - Gateway and execution workers scale and fail independently. - The gateway can remain lightweight; physical isolation and resource policy belong to the selected execution backend. @@ -257,12 +268,6 @@ Rejected because it couples control-plane availability and credentials to mutable repository execution, prevents independent scaling, and mistakes a service boundary for per-invocation isolation. -### Make Agent-Conductor mandatory - -Rejected because a direct Codex adapter and Agent-Conductor are peer backends. -Mandatory indirection adds an ownership and failure boundary without improving -the public contract. - ### Use OpenInference instead of W3C Trace Context Rejected as a category error. W3C Trace Context propagates trace identity; @@ -281,6 +286,13 @@ Rejected because Harbor's formats own benchmark orchestration, verification, and persisted runner state. The AllAgents gateway executes one coding-agent request and does not become an evaluation harness. +### Replace A2A with the Agent Host Protocol + +Rejected because AHP explicitly targets synchronization of independent clients +around host-owned sessions, not agent-to-agent Task execution. Its reconnect +and changeset models do not supply caller-scoped idempotency, immutable source +handling, cleanup, complete terminal evidence, or bounded Task retention. + ## Reconsider when Revisit this decision if A2A standardizes the required coding-execution evidence diff --git a/docs/research/agent-host-protocol-decision-inputs.md b/docs/research/agent-host-protocol-decision-inputs.md new file mode 100644 index 00000000..a3ebebc9 --- /dev/null +++ b/docs/research/agent-host-protocol-decision-inputs.md @@ -0,0 +1,109 @@ +# Agent Host Protocol decision inputs for the execution gateway + +## Decision + +Keep A2A 1.0 plus the versioned AllAgents extension as the execution gateway's +northbound contract. Treat the Agent Host Protocol (AHP) as an optional future +protocol behind the gateway for a compatible backend or beside it for a +collaborative session client. + +AHP does not replace ADR 0002's Task identity, caller-scoped idempotency, +authorization, immutable source handling, cleanup, terminal evidence, or +bounded result retention. + +The initial backend set is Codex, OpenCode, and Pi. They are peer execution +adapters behind one conformance contract; provider-specific process, session, +permission, cancellation, and evidence behavior stays below that seam. + +This note records the AllAgents-specific consequences. The reusable research, +source inspection, and full protocol comparison live in the AI Research Wiki: + +- [Agent Host Protocol](https://github.com/tsoyang-org/ai-research-wiki/blob/main/entities/agent-host-protocol.md) +- [Agent Host Architecture](https://github.com/tsoyang-org/ai-research-wiki/blob/main/concepts/agent-host-architecture.md) +- [Agent Host Protocol vs Agent2Agent](https://github.com/tsoyang-org/ai-research-wiki/blob/main/comparisons/agent-host-protocol-vs-agent2agent.md) +- [VS Code Agent Host source note](https://github.com/tsoyang-org/ai-research-wiki/blob/main/raw/articles/vscode-agent-host-architecture.md) + +## Boundary + +| Concern | AllAgents A2A gateway | AHP host/session layer | +|---|---|---| +| Northbound consumer | AI Evals and future remote execution clients | IDE, browser, CLI, or collaborative operator client | +| Primary lifecycle | One addressable Task per accepted execution | Long-running session/chat with shared clients | +| Public identity | Agent Card, Message, Task, Artifact, invocation key | Host, client, channel, session, chat, turn, tool call | +| State | Task status, messages, artifacts, retention | Snapshots, ordered actions, reducers, reconnect | +| Authorization | Authenticate/authorize service caller | Endpoint/resource auth and tool confirmation | +| Cancellation | Cancel Task, abort backend, terminate, clean up, report terminal outcome | Cancel interactive turn and call provider-native abort | +| Evidence | Source, output, usage/cost, traces, file changes, artifacts, failures, cleanup, completeness, provenance | Live changesets and provider/session state | +| Isolation | Selected worker/backend boundary | Not supplied by the shared host process | + +The identities must be correlated rather than reused. At minimum retain the A2A +Task ID, AllAgents invocation key, backend execution/session ID, +provider-native thread/chat ID, and trace ID. + +## Adopt now + +1. Define one narrow backend adapter contract for create/invoke, progress, + permission decisions, cancellation, terminalization, evidence collection, + shutdown, native evidence passthrough, and explicit capabilities. +2. Keep gateway responsibilities separate from worker/backend responsibilities. + The gateway owns caller authorization, Task/idempotency identity, backend + selection, normalized results, cancellation propagation, and retention. + Workers own source materialization, provider processes, mutable workspaces, + evidence capture, process termination, and cleanup. +3. Propagate `CancelTask` and deadlines through the adapter to the + provider-native abort primitive, then persist terminal status and cleanup + outcome. Transport closure is not cancellation. +4. Separate caller authorization, execution permission policy, and + provider/resource credentials. +5. Combine normalized file operations with bounded provider-native + diffs/checkpoints/trajectories. Declare attribution limits and + incompleteness rather than treating the final working-tree diff as exact + agent causality. +6. Persist terminal facts independently of progress streams and telemetry. +7. Capability-gate backend behavior instead of inferring it from provider names + or software versions. +8. Keep Task lists and metadata bounded; store large logs, diffs, traces, and + produced artifacts behind references with size, redaction, and truncation + metadata. + +## Defer + +- An AHP backend adapter until a selected backend actually exposes AHP. +- An AHP server or multi-client reducer/reconciliation engine until a + collaborative session client is a product requirement. +- Client-contributed tools and customizations for unattended evaluation + profiles. +- Active-session reconnection beyond A2A Task lookup, subscription, and + terminal result retrieval. +- AHP local endpoint discovery, SSH host selection, and tunnel multiplexing. +- Generic changeset review/operation state. +- Long-lived session/chat catalogs and provider-native session adoption. + +## Reject + +- Replacing A2A with AHP for the execution gateway. +- Running evaluated agents or writable repositories in the gateway process. +- Treating AHP changesets as the complete AllAgents evidence envelope. +- Treating AHP action replay or session restoration as execution idempotency. +- Copying VS Code's local connection-token model as gateway authentication. +- Branching on provider names above the adapter boundary. +- Making a connected interactive client a hidden prerequisite for unattended + execution. + +## Evidence + +The conclusion is based on: + +- Microsoft's [Agent Host architecture article](https://code.visualstudio.com/blogs/2026/08/26/agent-host-architecture); +- the official [Agent Host Protocol documentation](https://microsoft.github.io/agent-host-protocol/); +- direct inspection of `microsoft/vscode` commit + [`046944034292b5479b4e9a50ad1a508033ffb64f`](https://github.com/microsoft/vscode/tree/046944034292b5479b4e9a50ad1a508033ffb64f), + whose generated registry identifies AHP `0.9.0`; and +- [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md). + +The inspected implementation demonstrates host-owned state/sequencing, +provider-neutral adapters for Copilot, Claude, and Codex, layered persistence, +bounded reconnect replay, provider-native cancellation, client-owned tools, +permission translation, Git checkpoint plus SDK edit evidence, and local/remote +host placement. These observations support the architecture seams above; they +do not supply the public execution guarantees retained by ADR 0002. From 9c8074d2ea9d95cabf8bb1f1205992279e088464 Mon Sep 17 00:00:00 2001 From: Christopher Date: Fri, 18 Sep 2026 08:57:24 +1000 Subject: [PATCH 03/12] docs(plan): define execution gateway implementation --- ...0837-feat-coding-execution-gateway-plan.md | 658 ++++++++++++++++++ 1 file changed, 658 insertions(+) create mode 100644 docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md new file mode 100644 index 00000000..1d65837d --- /dev/null +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -0,0 +1,658 @@ +--- +title: "Coding-Agent Execution Gateway - Plan" +date: 2026-09-18 +deepened: 2026-09-18 +type: feat +artifact_contract: ce-unified-plan/v1 +artifact_readiness: implementation-ready +product_contract_source: ce-plan-bootstrap +execution: code +--- + +# Coding-Agent Execution Gateway - Plan + +## Goal Capsule + +- **Objective:** External systems can run Codex, OpenCode, or Pi against an immutable repository revision through one authenticated, cancellable, evidence-preserving remote contract. +- **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with three provider adapters (KTD1, KTD5, KTD7). +- **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public boundary. The A2A 1.0 specification owns core wire semantics. The versioned AllAgents extension owns coding-execution semantics. +- **Execution profile:** Build contract-first, then durable gateway state, worker lifecycle, provider adapters, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. +- **Stop conditions:** Do not execute agents in the gateway process, accept mutable source identity, put deployment credentials in requests, treat streams or telemetry as terminal evidence, or add evaluation behavior. +- **Tail ownership:** The implementing workflow runs focused contract and lifecycle tests, the complete repository quality gates, isolated gateway/worker smoke tests, provider-specific credentialed smoke tests where credentials are available, and documentation validation. + +--- + +## Product Contract + +### Summary + +AllAgents gains a remote coding-execution service without becoming an evaluation framework. Callers use A2A Tasks and one required AllAgents extension. The gateway owns caller identity, idempotency, routing, status, cancellation, evidence normalization, and bounded retention. Separate workers own repository materialization, provider processes, mutable workspaces, evidence capture, termination, and cleanup. + +### Problem Frame + +AllAgents currently configures and launches coding clients but has no service boundary for external callers. AI Evals and future consumers would otherwise need to import AllAgents internals, drive interactive CLIs, or independently reimplement repository acquisition, permissions, cancellation, evidence, and cleanup. + +The three initial runtimes expose different programmatic contracts. Codex provides a TypeScript SDK over structured JSONL events, OpenCode provides a typed HTTP SDK and SSE event stream, and Pi provides a strict JSONL RPC mode. The public service must preserve one stable lifecycle without flattening provider-specific facts into false equivalence. + +### Actors + +- A1. **Gateway caller:** An authenticated service such as AI Evals that creates, observes, lists, cancels, and retrieves coding-execution Tasks. +- A2. **Execution gateway:** The A2A server that owns caller scope, Task identity, idempotency, routing, retention, and normalized results. +- A3. **Execution worker:** A separately deployed process that owns source materialization, one mutable workspace per invocation, provider execution, evidence capture, and cleanup. +- A4. **Backend adapter:** The Codex, OpenCode, or Pi integration that translates native events, cancellation, usage, failures, and evidence into the worker contract. +- A5. **Operator:** The person or deployment system that defines profiles, credentials, limits, retention, worker endpoints, and observability policy. + +### Key Decisions + +- **Profile A2A rather than creating a public invocation API.** The service keeps standard Agent Cards, Tasks, Artifacts, operations, errors, and capability negotiation. Governs R1-R4. +- **Keep execution outside the gateway process.** Mutable repositories and provider processes belong to workers. Governs R10-R16, R21-R22. +- **Keep evaluation outside AllAgents.** Dataset expansion, repetitions, assertions, scoring, retries, and durable evaluation Runs remain caller concerns. Governs R20. + +### Requirements + +**Public protocol and compatibility** + +- R1. The gateway implements A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; it implements streaming send and task subscription when the card advertises streaming. +- R2. Every valid new request returns exactly one addressable Task. Direct-Message completion and follow-up messages to an existing Task are unsupported. Non-streaming send honors A2A `returnImmediately`; streaming always emits the durable Task first. +- R3. Every request and terminal Task uses one required, versioned AllAgents coding-execution extension URI. Unsupported required extension versions fail without fallback. +- R4. Terminal output and execution evidence are retrievable as Task Artifacts for the configured retention window even when the original stream disconnects. Active subscription emits the current Task snapshot then future events without promising replay of missed progress; terminal subscription returns the standard unsupported-operation error and callers use `GetTask`. + +**Caller identity, Task identity, and retention** + +- R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. +- R6. Authentication, required-extension validation, request validation, source/profile authorization, quota admission, and deadline validation complete before Task creation. A caller-scoped invocation key, effective profile, authenticated owner, and canonical request digest then bind atomically to one Task; identical replay returns that Task and conflicting reuse is rejected without dispatch. +- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Task state and terminal Artifact metadata survive gateway restart; nonterminal Tasks that cannot be reattached settle failed once and stale worker events cannot overwrite them. +- R8. List operations implement all A2A filters, history bounds, page-size bounds, owner/query-bound cursor pagination, and descending status-update time. One immutable expiry logically hides the Task, claim, events, and artifacts before best-effort physical deletion; expired and unauthorized IDs are indistinguishable. +- R9. Small deployments work without an external database. The built-in durable store supports one gateway replica, enforces per-owner/global admission and storage quotas, and reserves capacity for cancellation and terminal settlement; multi-replica storage is outside this delivery. + +**Execution and policy** + +- R10. Codex, OpenCode, and Pi are peer backends behind one conformance contract. (session-settled: user-directed — chosen over an additional enterprise-only adapter: the open-source gateway supports the three named runtimes directly.) +- R11. A request selects a server-defined execution profile. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. +- R12. The only initial remote source form is a canonical credential-free HTTPS Git URL plus full commit object ID and optional repository-relative subdirectory. Acquisition revalidates destination policy for every connection, disables redirects and repository-controlled secondary fetch/exec features, uses hermetic Git configuration, and verifies that the fetched object is the requested commit before setup. +- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, and retained workspaces. +- R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. +- R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. +- R16. A worker creates a fresh invocation directory and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, and runs configured checks. It then proves all invocation descendants quiescent before final evidence/artifact capture and cleanup or explicit retention. + +**Evidence and observability** + +- R17. Every terminal result contains an integrity kernel: Task/source/profile/backend identities, action outcome, cancellation or failure classification, termination and cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. +- R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. +- R19. Gateway and worker spans propagate W3C Trace Context and export OpenTelemetry data. Telemetry is operational evidence, not the only durable result. + +**Ownership and safety boundary** + +- R20. The gateway executes one coding request. It does not own eval configuration, datasets, repetition, scoring, retry policy, experiment scheduling, or a durable evaluation Run ledger. +- R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. Profiles that claim hostile-source or cross-tenant isolation are rejected until a stronger per-invocation UID, mount, PID, network, and credential boundary is configured. +- R22. Gateway admission and worker execution enforce profile limits for request rate, active/retained Tasks, subscriptions, stored bytes, source transfer/expansion, files/inodes, workspace bytes, CPU, memory, PIDs, network, phase deadlines, events, logs, and artifacts. Exhaustion is scoped to one invocation or owner and leaves capacity for terminalization and cleanup. + +### Key Flows + +- F1. **Admit, create, and stream an execution** + - **Actors:** A1, A2, A3, A4. + - **Trigger:** A caller sends a text Message with the required extension, immutable source, profile, invocation key, and deadline. + - **Steps:** Authenticate; validate and authorize the complete request; reserve quota; atomically claim idempotency and create a submitted Task; dispatch a fenced worker attempt; materialize and verify source; execute the selected backend; persist progress before emission; terminalize with Artifacts after quiescence and cleanup. + - **Outcome:** `returnImmediately: true` returns the durable current Task, false/unset waits for terminal state, and streaming starts with that Task before ordered updates. + - **Covered by:** R1-R22. +- F2. **Replay or reconnect to an invocation** + - **Actors:** A1, A2. + - **Trigger:** The owner repeats an invocation key or subscribes after a stream disconnect. + - **Steps:** Recompute the canonical digest; reject a conflict; return the existing Task; for active streaming replay/subscription emit its current snapshot then future events; for a terminal Task return it through send replay or `GetTask` without dispatch. + - **Outcome:** Retries do not multiply agent work, and reconnect never promises transient event replay. + - **Covered by:** R4, R6-R8. +- F3. **Cancel or time out an execution** + - **Actors:** A1, A2, A3, A4. + - **Trigger:** The caller invokes `CancelTask`, the effective deadline expires, or gateway shutdown claims cancellation. + - **Steps:** Atomically record the first cancellation source; if dispatch never occurred, prove no workspace exists; otherwise send one fenced worker cancel, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. + - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses late provider success and fails instead of claiming canceled when termination or cleanup cannot be verified. + - **Covered by:** R7, R14, R16-R18. +- F4. **Recover from gateway or worker loss** + - **Actors:** A2, A3. + - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, or a worker crashes. + - **Steps:** Invalidate the attempt fence; settle each non-reattachable Task failed once; reject late events/results; stop renewing leases; let workers self-abort and clean. Record cleanup complete only when the worker/process boundary proves it; otherwise record unknown. + - **Outcome:** One Task has one terminal result, no ambiguous dispatch is retried automatically, and no stale worker can overwrite durable truth. + - **Covered by:** R7, R9, R14, R16-R18, R21-R22. +- F5. **Expire retained execution data** + - **Actors:** A1, A2. + - **Trigger:** The immutable Task expiry is reached. + - **Steps:** Atomically tombstone the complete ownership aggregate; stop authorizing Task and Artifact access; retry physical cleanup independently; permit the old invocation key to create a new Task only after logical expiry. + - **Outcome:** Expired, unknown, and unauthorized identifiers are indistinguishable and no Artifact outlives Task authorization. + - **Covered by:** R5-R9. + +### Acceptance Examples + +- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile and an exact Git SHA, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same output and evidence Artifacts. +- AE2. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with the same canonical request, then the gateway returns the original Task without a second worker dispatch. +- AE3. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with a different prompt, source, profile, or deadline, then the gateway rejects the request and leaves the original Task unchanged. +- AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. +- AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository, when the worker verifies source, then provider execution never starts and the Task fails with source-verification and cleanup evidence. +- AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. +- AE7. **Covers R10.** Given equivalent profiles and fixture runtime events for Codex, OpenCode, and Pi, when each completes the same repository mutation, then all three produce the same required normalized result fields while retaining distinct native evidence. +- AE8. **Covers R4, R7, R19.** Given a caller disconnects during work, when it subscribes again, then it receives the current Task and future updates without duplicate dispatch; telemetry loss does not affect later terminal lookup. +- AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. +- AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. +- AE11. **Covers R6, R22.** Given invalid input or exhausted admission quota, the gateway returns a request/resource error and creates no Task; given capacity disappears after durable acceptance, the retained Task fails at dispatch and replay returns it without retry. +- AE12. **Covers R7, R14.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only safe operator telemetry. +- AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. +- AE14. **Covers R21-R22.** Given a profile requests pooled hostile-source or cross-tenant execution, startup/admission rejects it; a reviewed single-trust-domain profile runs one bounded execution without exposing worker control credentials to the child environment. + +### Success Criteria + +- The official A2A JavaScript client can discover the card and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, replay, cancel, and expiry behavior against the built service. +- One conformance fixture passes unchanged through Codex, OpenCode, and Pi adapters. +- Admission, replay, fencing, cancellation races, restart recovery, authorization isolation, source hardening, quotas, and evidence integrity have deterministic integration coverage. +- The gateway image contains no coding-agent runtime and cannot access worker workspace roots. +- The initial worker runs one reviewed-trust-domain execution at a time and leaves no live descendant or retained workspace unless policy requests retention. + +### Scope Boundaries + +**In scope** + +- A2A 1.0 HTTP+JSON and SSE streaming. +- One versioned AllAgents coding-execution extension and one versioned private worker protocol. +- Codex, OpenCode, and Pi backends. +- Built-in bearer authentication with OIDC/JWT and static service-token modes. +- Single-replica durable file storage, authenticated Artifact retrieval, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. +- Reviewed repositories in one configured mutual-trust domain per worker deployment. + +**Deferred to follow-up work** + +- Multi-replica database-backed Task and idempotency storage. +- Kubernetes Job dispatch, queue brokers, autoscaling controllers, and stronger hostile-source or cross-tenant sandbox providers. +- Push-notification configuration, gRPC, JSON-RPC transport, and A2A extended Agent Cards. +- AHP server/client surfaces, long-lived interactive sessions, and client-contributed tools. +- Additional coding backends and provider-session restoration after gateway restart. +- Optional ATIF conversion after the format and tooling mature. + +**Outside this product's identity** + +- Evaluation authoring, datasets, assertions, grading, repetitions, experiment scheduling, and durable evaluation Runs. +- Caller-specific result projections such as Promptfoo `ProviderResponse` mapping. + +### Sources + +- [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) +- [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) +- [AI Evals ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) +- [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) +- [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) +- [Codex TypeScript SDK](https://github.com/openai/codex/tree/main/sdk/typescript) +- [OpenCode SDK and server](https://opencode.ai/docs/sdk/) +- [Pi RPC protocol](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md) + +--- + +## Planning Contract + +### Key Technical Decisions + +- KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. The decorator owns admission, canonical Task reservation, idempotent replay, stream snapshot selection, and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. +- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes both versioned contracts, generated JSON Schemas, bounds, and fixtures. The worker protocol carries attempt identity, profile digest, dispatch acceptance, monotonic event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. +- KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates the invocation claim and submitted Task together, stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. +- KTD4. **Authenticate at HTTP ingress before A2A storage or dispatch.** Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. Unauthenticated mode is allowed only on a loopback listener. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Governs R5-R6, R13. +- KTD5. **Use fenced, separately deployable gateway and worker services.** The gateway owns A2A and durable results; the worker owns workspaces and provider processes. Every dispatch has a gateway-generated attempt ID, lease ID/epoch, short-lived capability, and event sequence. Workers idempotently accept duplicate delivery of the same attempt, reject conflicting attempts, and gateways ignore stale/out-of-order events and late terminal results. Governs R7, R10-R16, R21-R22. +- KTD6. **Make worker leases the orphan-execution fail-safe, not a replay mechanism.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry aborts and cleans the worker. Gateway restart invalidates old fences and records cleanup as unknown unless a process boundary proves it. Caller stream disconnect never affects the lease. Governs R7, R14, R16-R18. +- KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | opencode | pi` registry is the only production dispatch point. Governs R10, R14-R18, R21-R22. +- KTD8. **Use each provider's supported automation surface.** Codex uses `@openai/codex-sdk` streaming with `AbortSignal`; OpenCode uses its typed SDK against a worker-owned loopback server and session abort; Pi uses `pi --mode rpc --no-session` with a strict LF-delimited JSON parser, `agent_settled`, `get_session_stats`, and RPC abort. Governs R10, R14-R18. +- KTD9. **Make profiles the policy boundary.** Requests select a profile ID but cannot override backend credentials, executable paths, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. Profile digests enter idempotency and provenance. Governs R6, R11-R16, R21-R22. +- KTD10. **Capture Git and provider evidence as separate layers after quiescence.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded layer. Neither layer is promoted as exact causality when incomplete. Governs R16-R18. +- KTD11. **Treat Codex, OpenCode, and Pi as the complete initial backend set.** (session-settled: user-directed — chosen over adding an enterprise-only adapter: only the three named open-source gateway backends belong in this plan.) Governs R10. +- KTD12. **Separate terminal integrity from optional evidence bodies.** Identity, action outcome, failure/cancellation, termination, cleanup, Artifact index, completeness, and provenance must validate before terminal publication. Predictable budget truncation/redaction of logs, diffs, native events, or produced-file bodies may preserve completion with explicit metadata; capture failure that breaks the integrity kernel fails in the evidence phase. Governs R4, R17-R18. +- KTD13. **Harden Git acquisition as a network security boundary.** Accept canonical HTTPS origins only. Use hermetic Git configuration, disable redirects, proxies, helpers, hooks, filters, LFS smudge, submodule recursion, alternates, and non-HTTPS protocols. Revalidate normalized host/address policy for every connection, never forward credentials across origins, and verify the full object ID resolves to a commit fetched from the approved remote. Governs R12-R13, R22. +- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Stronger isolation is a separate sandbox-driver capability. Governs R16, R21-R22. +- KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. + +### High-Level Technical Design + +#### Component topology + +```mermaid +flowchart TB + Caller[Authenticated A2A caller] -->|HTTP+JSON / SSE| Gateway[execution-service gateway] + Gateway --> Auth[Auth, admission, profile policy] + Gateway --> Store[Generation-based Task and Artifact store] + Gateway -->|Fenced private protocol| Worker[Single-execution worker] + Worker --> Source[Hardened Git acquisition] + Worker --> Registry[Closed backend registry] + Registry --> Codex[Codex SDK] + Registry --> OpenCode[OpenCode SDK and server] + Registry --> Pi[Pi RPC process] + Worker --> Evidence[Quiesced checks, Git and native evidence] + Evidence -->|Bounded terminal result| Gateway + Gateway --> Telemetry[OpenTelemetry exporter] + Worker --> Telemetry +``` + +#### Admission, dispatch, and settlement sequence + +```mermaid +sequenceDiagram + participant C as Caller + participant G as Gateway decorator + participant S as Durable aggregate store + participant W as Worker + participant B as Backend adapter + + C->>G: SendMessage + required extension + G->>G: Authenticate, validate, authorize, quota, deadline + G->>S: Atomic claim + submitted Task + alt identical replay + S-->>G: Existing Task and current fence + G-->>C: Existing Task; follow active future events only + else new accepted Task + S-->>G: Task + attempt/lease fence + G->>W: Dispatch(attempt, fence, profile, source, deadline) + W-->>G: Accepted(attempt, fence) + W->>W: Materialize, verify, setup, baseline + W->>B: Invoke with isolated roots and policy + B-->>W: Progress, usage, native evidence + W-->>G: Sequenced fenced progress + G->>S: Compare-and-swap Task generation + opt cancellation or deadline wins + C->>G: CancelTask + G->>S: Persist cancellation intent once + G->>W: Fenced cancel + W->>B: Native abort + end + W->>W: Stop descendants, capture evidence, cleanup + W-->>G: Fenced terminal result + G->>S: Store blobs then atomically commit terminal manifest + G-->>C: Terminal status and Artifacts + end +``` + +#### Public A2A Task state + +```mermaid +stateDiagram-v2 + [*] --> Submitted: claim and Task committed + Submitted --> Working: worker accepts current fence + Submitted --> Canceled: cancellation proves no workspace exists + Submitted --> Failed: dispatch, restart, or source failure + Submitted --> Rejected: accepted policy refusal before work + Working --> Completed: integrity kernel and cleanup validate + Working --> Failed: provider, check, evidence, cleanup, crash, or restart failure + Working --> Rejected: known profile permission denial after stop and cleanup + Working --> Canceled: cancellation wins and stop/cleanup verify + Completed --> [*] + Failed --> [*] + Rejected --> [*] + Canceled --> [*] +``` + +Terminal states are immutable. Cancellation intent, termination, evidence capture, cleanup, and retention expiry are private record phases, not A2A Task states. + +#### Private execution-record phases + +```mermaid +stateDiagram-v2 + [*] --> Admitted + Admitted --> Dispatching + Dispatching --> Running: current fence accepted + Dispatching --> Terminalizing: dispatch rejected or unknown + Running --> CancelRequested: caller, deadline, shutdown, or lease expiry + Running --> Quiescing: provider and checks finish + CancelRequested --> Quiescing + Quiescing --> CapturingEvidence: descendants verified stopped + CapturingEvidence --> Cleaning + Cleaning --> Terminalizing + Terminalizing --> Retained + Retained --> Tombstoned: expiry + Tombstoned --> [*]: physical cleanup +``` + +### Output Structure + +```text +packages/execution-service/ + package.json + tsconfig.json + src/ + execution/ + contract.ts + extension-v1.ts + worker-protocol-v1.ts + errors.ts + profiles.ts + telemetry.ts + gateway/ + index.ts + config.ts + auth.ts + agent-card.ts + request-handler.ts + executor.ts + server.ts + worker-client.ts + store/ + gateway-repository.ts + file-gateway-repository.ts + worker/ + index.ts + config.ts + server.ts + lease.ts + workspace.ts + evidence.ts + adapters/ + types.ts + registry.ts + codex.ts + opencode.ts + pi.ts + tests/ + fixtures/execution/ + unit/execution/ + unit/gateway/ + unit/worker/ + e2e/execution-gateway.test.ts +containers/ + gateway.Dockerfile + worker.Dockerfile +examples/gateway/ + gateway.yaml + worker.yaml +docs/src/content/docs/ + guides/execution-gateway.mdx + reference/execution-gateway-configuration.mdx +``` + +### Configuration Contract + +- Gateway configuration defines listener/public URL, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker endpoints, internal capability secrets, and profiles. +- Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, and evidence budgets. +- Worker configuration fixes a private listener, one-execution concurrency, workspace root, lease grace, backend runtime constraints, trust domain, resource-control capability, and request/result limits. +- Configuration contains environment-variable names but never secret values. Startup resolves the complete graph, verifies that profile claims do not exceed deployment capabilities, and becomes ready only when store, workers, runtimes, quotas, and free-space reserves pass. + +### Error and Status Mapping + +| Condition | A2A result | Required extension detail | +|---|---|---| +| Authentication, malformed/unsupported extension, invalid source/profile, unauthorized policy, expired deadline, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | +| Identical invocation replay | Existing Task | No new Task, worker attempt, or quota reservation | +| Conflicting invocation key | Operation error; no new Task | Conflict code; existing Task unchanged | +| Worker capacity loss after acceptance | `TASK_STATE_FAILED` | `dispatch/capacity_exhausted`, retriable fact, no workspace created; gateway does not retry | +| Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | +| Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | +| Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | +| Source, setup, provider, check, mandatory evidence, worker crash, or infrastructure failure | `TASK_STATE_FAILED` | Typed phase, safe message, retriable fact, termination/cleanup/completeness | +| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors, native abort, termination, cleanup | +| Cancellation loses to terminal completion | Existing terminal Task / `TaskNotCancelableError` | No state mutation or second abort | +| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Output plus complete required evidence | +| Successful action with allowed bounded optional-evidence gap | `TASK_STATE_COMPLETED` | Per-dimension incomplete flag, reason, original/captured size, digest and redaction/truncation flags | +| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid and cleanup unknown unless proven | +| Retention expiry | Not found | Aggregate logically hidden before physical deletion; Artifact URL also invalid | + +### Phased Delivery + +1. Create the private Node 22 service package and freeze the public extension, worker protocol, profiles, fixtures, and error vocabulary. +2. Build authenticated durable A2A Task handling and fenced worker dispatch against a fake worker. +3. Build the single-execution worker lifecycle and hardened source/evidence handling against a fake adapter. +4. Add Codex, OpenCode, and Pi adapters in parallel, then compose them through the closed registry. +5. Package the services and run cross-backend, security, process, and A2A conformance before enabling a consumer. + +### System-Wide Impact + +- **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. +- **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. +- **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. +- **Security:** New review-critical surfaces are auth, owner-key derivation, source SSRF, admission/resource quotas, setup/check policy, phase-scoped secrets, internal fences, Artifact capture/serving, and reviewed-source trust enforcement. +- **Operations:** Gateway and worker health, readiness, quotas, low-space state, structured logs, traces, tombstone backlog, lease expiry, stale event rejection, and graceful shutdown need independent signals. +- **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. + +### Risks and Mitigations + +- **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup and keep captured provider fixtures versioned. +- **False idempotency or stale settlement:** Claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test duplicate delivery, cancellation races, restart, and late results. +- **Task/store corruption:** Publish immutable blobs and generations before one manifest switch; tombstone before deletion; validate owner tuples/manifests at startup; garbage-collect unreachable generations; document the one-replica limit. +- **Owner collision or path injection:** Hash a bounded canonical issuer/tenant/subject tuple, store and verify the tuple inside the owner aggregate, and use only server-generated opaque IDs in paths. +- **Orphan processes:** Combine explicit cancel, native abort, process-group termination, one-execution worker/container death, lease expiry, and quiescence proof before evidence capture. +- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral, origin-bound, and absent from repository config, process arguments, retained workspaces, logs, and errors. +- **Resource exhaustion:** Reserve per-owner/global gateway quota before claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. +- **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. +- **Evidence overclaim:** Enforce KTD12's integrity kernel and per-dimension completeness. Truncation and redaction remain independent facts. +- **Permission deadlock:** Initial profiles never prompt. Known requests resolve for one isolated invocation; unknown shapes fail closed as adapter incompatibility. +- **Trust-boundary overclaim:** Reject pooled hostile-source/cross-tenant profiles and state the reviewed mutual-trust boundary in config, readiness, Agent Card metadata, and docs. +- **Cross-platform drift:** Keep gateway/store tests cross-platform. State that worker execution and hardened evidence/source controls are Linux-only. + +### Assumptions + +- The first production deployment runs one gateway replica with persistent storage. Multi-replica transactional storage is deferred. +- Git over hardened HTTPS and exact commit object ID covers the initial consumer. Other source transports require a later extension version or capability. +- Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. +- Initial repositories are reviewed inside one configured mutual-trust domain. Strong hostile-code or cross-tenant execution remains unavailable until a stronger sandbox driver exists. +- Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, OpenCode CLI 1.18.x with its compatible SDK, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. + +--- + +## Implementation Units + +### U1. Versioned public and worker contracts + +- **Goal:** Freeze the extension, profile vocabulary, private worker protocol, canonical digest input, result envelope, typed failures, and conformance fixtures before either service endpoint. +- **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. +- **Dependencies:** None. +- **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. +- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas, one public extension URI, and one private protocol version. Include attempt/fence/lease identity, monotonic event sequence, accepted dispatch, renew/cancel, bounded terminal acknowledgement, public/private state separation, and integrity-kernel rules. Canonicalize caller input plus effective profile digest for idempotency. Generate checked-in JSON Schemas and fixtures from the same source. +- **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions, unsafe paths, invalid public states, stale fences, oversized records, and conflicting canonical inputs before implementing schemas. +- **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, and `src/core/native/types.ts` for safe error/provenance normalization. +- **Test scenarios:** + - A minimal valid request with text prompt, invocation key, profile, exact commit, and deadline parses and produces a stable digest across object-key ordering. + - Changing prompt, source object ID, profile ID/digest, artifact selection, or deadline changes the digest; trace IDs and transport metadata do not. + - A source URL with credentials, a branch/tag revision, absolute subdirectory, traversal, secret value, unknown backend, or unknown extension version is rejected safely. + - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. + - Worker fixtures reject missing/mismatched attempt IDs, lease epochs, profile digests, event sequence, bounds, and terminal acknowledgements. + - Completed, failed, canceled, and rejected results validate only with the integrity kernel; optional usage/native evidence gaps require explicit completeness reasons. + - File evidence accepts create/edit/delete/rename and rejects unsafe paths, duplicate identities, oversized inline content, and inconsistent before/after forms. +- **Verification:** Generated schemas are stable, public/private fixtures round-trip, digest vectors are cross-platform deterministic, and the private client/server fixture suite agrees before gateway or worker implementation. + +### U2. Authentication and durable gateway repository + +- **Goal:** Provide caller-scoped authentication, authorization, atomic Task/idempotency aggregates, Artifact storage, quota admission, pagination, restart fencing, logical expiry, and cleanup. +- **Requirements:** R4-R9, R13-R14, R17-R18, R22; AE2-AE4, AE6, AE8, AE10-AE13; KTD1, KTD3-KTD4, KTD12. +- **Dependencies:** U1. +- **Files:** `packages/execution-service/src/gateway/config.ts`, `packages/execution-service/src/gateway/auth.ts`, `packages/execution-service/src/gateway/store/gateway-repository.ts`, `packages/execution-service/src/gateway/store/file-gateway-repository.ts`, `packages/execution-service/tests/unit/gateway/auth.test.ts`, `packages/execution-service/tests/unit/gateway/file-gateway-repository.test.ts`. +- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Commit claim plus submitted Task in one manifest generation; publish immutable Artifact blobs before terminal manifest switch; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover and garbage-collect unreachable generations on startup. Reserve owner/global quotas before claims. Verify OIDC JWTs and constant-time static tokens before all repository access. +- **Execution note:** Implement concurrent-claim, transition-race, and crash-publication tests before request handling. Inject faults between blob, generation, manifest, tombstone, and cleanup operations. +- **Patterns to follow:** `src/core/marketplace.ts` and `src/core/profile/files.ts` for atomic publication/recovery, `src/core/mcp-http-stdio-proxy.ts` for private files and loopback safety, and the official A2A `TaskStore` owner-scoping contract. +- **Test scenarios:** + - Covers AE2-AE3. Concurrent identical claims create one aggregate; a conflicting digest returns conflict without dispatch permission. + - Covers AE4. Load/list/cancel/subscribe/Artifact lookup scopes before path/database access and gives unknown, unauthorized, and expired IDs indistinguishable behavior. + - Hostile/ambiguous issuer, tenant, subject, invocation key, Task ID, Artifact name, Unicode, case, delimiter, traversal, and Windows-reserved values cannot collide or become paths. + - All standard list filters, `historyLength`, page size 1-100, omitted Artifacts, ordering, total size, and always-present next token match A2A semantics. Tokens are owner/query-bound and reject malformed, swapped, or stale filters. + - Covers AE12. Terminal compare-and-swap wins once; stale fence, duplicate, and out-of-order updates cannot mutate the Task. + - Restart fails nonterminal Tasks once, invalidates fences, preserves terminal Tasks, and records cleanup unknown unless proven. + - Covers AE13. Exact expiry tombstones the aggregate before cleanup; failed deletion never restores visibility; same-key replay before expiry returns the old Task and after expiry creates a new Task. + - A crash between every aggregate publication step leaves either the prior or next valid manifest, never claim-without-Task or Task-with-missing-Artifact state. + - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. + - Quota-boundary races admit exactly the allowed count and preserve reserved capacity for cancel/terminal writes; low-space mode stops new claims without blocking settlement. + - Unauthenticated mode starts on loopback and refuses wildcard or non-loopback listeners. +- **Verification:** A fresh process retrieves prior records, fault recovery finds one valid aggregate generation, authorization cannot reveal neighboring owners, and expiry/quota behavior remains deterministic under concurrency. + +### U3. A2A gateway server and fenced worker client + +- **Goal:** Expose the accepted A2A profile while making admission, replay, streaming, lookup, worker fencing, failure, and cancellation use one durable state machine. +- **Requirements:** R1-R9, R11, R14-R15, R17-R22; F1-F5; AE1-AE4, AE6, AE8-AE13; KTD1-KTD7, KTD9, KTD12. +- **Dependencies:** U1, U2. +- **Files:** `packages/execution-service/src/gateway/agent-card.ts`, `packages/execution-service/src/gateway/request-handler.ts`, `packages/execution-service/src/gateway/executor.ts`, `packages/execution-service/src/gateway/server.ts`, `packages/execution-service/src/gateway/worker-client.ts`, `packages/execution-service/tests/unit/gateway/agent-card.test.ts`, `packages/execution-service/tests/unit/gateway/request-handler.test.ts`, `packages/execution-service/tests/unit/gateway/executor.test.ts`, `packages/execution-service/tests/e2e/gateway-fake-worker.test.ts`. +- **Approach:** Mount the official HTTP+JSON and Agent Card handlers behind auth. Put an AllAgents `A2ARequestHandler` decorator above `DefaultRequestHandler` so admission and canonical Task reservation happen first, identical replay bypasses new SDK Task/bus allocation, and cancellation waits for worker terminal evidence. Persist each public state before emission. Dispatch one fenced attempt, validate sequence/fence on every worker event, renew its lease, and atomically publish Artifact blobs plus terminal manifest. +- **Execution note:** Begin with an in-process fake worker and official A2A client. Prove operation errors versus accepted-Task failures, replay/subscribe behavior, fencing, cancellation races, and restart before adding providers. +- **Patterns to follow:** Official A2A sample `AgentExecutor`, `A2ARequestHandler`, `DefaultRequestHandler`, Express handlers, and cancellable-agent flow; `src/core/mcp-http-stdio-proxy.ts` for HTTP shutdown and loopback tests. +- **Test scenarios:** + - Covers AE1. `returnImmediately` true returns the submitted/working Task, false/unset waits for terminal state, and streaming starts with the same durable Task before ordered updates. + - Authentication, invalid extension/source/profile, expired deadline, and pre-claim quota failure return operation errors with no Task or worker request. + - Covers AE11. Capacity loss after acceptance fails the retained Task at `dispatch/capacity_exhausted`; ambiguous dispatch fails `dispatch_unknown`; neither is retried. + - Covers AE2-AE3. Identical send/stream replay returns the existing Task and follows only future events if active; conflict returns the documented operation error. + - Covers AE8. Active subscribe emits current snapshot then future events without missed-event replay; terminal subscribe errors and `GetTask` returns terminal truth. + - Covers AE4. Get/list/subscribe/cancel/Artifact endpoints apply owner authorization consistently. + - Covers AE6. Cancel in submitted/working, cancel versus accept/completion, caller versus deadline, duplicate cancel, and terminal cancel each produce one linearized outcome and at most one worker abort. + - Covers AE12. Duplicate, out-of-order, malformed, wrong-fence, and late terminal events cannot overwrite Task state; stale facts go only to safe telemetry. + - Known policy denial rejects only after stop/cleanup; unknown permission shape fails as adapter incompatibility. + - Caller SSE disconnect and telemetry exporter failure leave execution and terminal lookup intact. + - Graceful shutdown stops admission, claims cancellation for bounded active work, persists honest terminal state, and closes listeners. +- **Verification:** The official SDK client exercises every advertised operation against the built gateway and fake worker; persisted snapshots match streams while aggregate/fence invariants remain intact under races. + +### U4. Worker protocol and safe workspace lifecycle + +- **Goal:** Implement the single-execution worker, hardened immutable Git acquisition, profile enforcement, leases, isolated backend roots, resource controls, race-resistant evidence, termination, and cleanup independent of any provider. +- **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. +- **Dependencies:** U1. +- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. +- **Approach:** Authenticate and fence the private protocol, reserve the one execution before workspace creation, validate profile/deployment capability, and emit sequenced NDJSON. Acquire source under KTD13. Create separate workspace and backend config/data roots with a scrubbed phase-specific environment. Run setup, baseline, adapter, and checks under enforced budgets. Stop and verify the process group before descriptor-based regular-file evidence staging, then clean in `finally`. Lease expiry self-cancels. +- **Execution note:** Characterize every phase with a fake adapter, malicious fixtures, and disposable Git servers before real providers. Fault-inject dispatch acknowledgement, events, leases, acquisition, processes, evidence publication, and cleanup. +- **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, and `tests/helpers/env.ts` for isolated state. +- **Test scenarios:** + - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, and fetch failure stop before adapter invocation. + - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no secondary connection or helper execution. + - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. + - Setup changes establish the baseline; setup and checks receive no provider/control secrets; every backend gets disjoint invocation config/data roots with ambient selectors removed. + - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. + - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected; worker control credentials are absent from child environment and configured filesystem roots. + - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation and preserve worker health. + - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. + - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps; capture I/O or malformed result that breaks the kernel fails the Task. + - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. + - A worker crash before/after provider spawn reports cleanup complete only when the process/container boundary proves descendant death. +- **Verification:** A built worker mutates a disposable exact-SHA repository through the fake adapter and proves fenced dispatch, source hardening, phase isolation, budgets, quiescence, evidence integrity, and cleanup from its emitted result alone. + +### U5. Codex backend adapter + +- **Goal:** Run Codex through its supported TypeScript SDK while preserving structured progress, output, usage, file-change evidence, cancellation, and runtime identity. +- **Requirements:** R10-R22; AE1, AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. +- **Dependencies:** U4. +- **Files:** `packages/execution-service/src/worker/adapters/codex.ts`, `packages/execution-service/tests/unit/worker/adapters/codex.test.ts`, `packages/execution-service/tests/fixtures/execution/codex-events.jsonl`. +- **Approach:** Construct a fresh SDK thread in the invocation workspace with an isolated `CODEX_HOME` and scrubbed environment. Apply model, sandbox, network, approval, and writable-root settings only from the profile. Consume `runStreamed()` and pass an AbortSignal. Normalize agent messages, items, usage, failures, and file-change events while preserving the bounded native stream. +- **Execution note:** Drive the SDK through its executable override with a fixture Codex process before any credentialed smoke test. +- **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, and the SDK's `runStreamed`/AbortSignal contract. +- **Test scenarios:** + - A successful stream exposes thread ID, progress, final response, token usage, native file-change items, and terminal completion. + - Empty final response, turn failure, malformed JSONL, non-zero exit, unavailable runtime, and usage omission map to typed result/completeness fields. + - Covers AE6/AE12. Cancellation aborts the SDK process once; completion after cancel or stale fence cannot alter the selected terminal outcome. + - Profile sandbox, network, model, approval, working directory, and environment settings reach the SDK; caller input cannot override them. + - Two sequential invocations have disjoint `CODEX_HOME`, thread/session state, and writable roots; ambient selectors are removed. + - Native diffs and shared Git evidence coexist without claiming identical attribution. +- **Verification:** Fixture-driven tests cover every supported event and failure shape, followed by an isolated credentialed repository smoke test when Codex credentials are available. + +### U6. OpenCode backend adapter + +- **Goal:** Run OpenCode through its typed SDK and worker-owned loopback server while preserving session progress, output, usage/cost, diffs, permissions, cancellation, and disposal. +- **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. +- **Dependencies:** U4. +- **Files:** `packages/execution-service/src/worker/adapters/opencode.ts`, `packages/execution-service/tests/unit/worker/adapters/opencode.test.ts`, `packages/execution-service/tests/fixtures/execution/opencode-events.jsonl`. +- **Approach:** Start one loopback instance per invocation with isolated `OPENCODE_CONFIG`, `OPENCODE_CONFIG_DIR`, data/cache roots, scrubbed environment, profile configuration, and AbortSignal. Subscribe before prompting, create one session, resolve permission events from policy, collect message/session events and session diff, abort on cancellation, then delete the session and close the server in `finally`. Prevent the agent subprocess from reaching the worker control listener under the declared trust topology. +- **Execution note:** Inject SDK/server factories so protocol fixtures prove ordering and teardown without downloading or authenticating a real runtime. +- **Patterns to follow:** `src/core/profile/adapters/opencode.ts` for configuration/environment isolation and OpenCode's `createOpencode`, event subscription, session prompt/diff/abort APIs. +- **Test scenarios:** + - Successful execution collects text parts, assistant tokens/cost, session ID, events, and session diff before disposal. + - Subscription starts before prompt, ignores other session IDs, and finishes only after the target session becomes idle or errors. + - Covers AE6/AE12. Cancellation calls session abort once; late idle/completion cannot overwrite cancellation; server teardown remains idempotent. + - Covers AE9. Known permission events receive invocation-scoped `once`, `always`, or `reject` according to profile; `always` does not survive disposal and unknown types fail the adapter. + - Provider auth error, API error, aborted message, server-start timeout, SSE disconnect, and malformed SDK response map to typed failures. + - Sequential invocations have disjoint config/data/session roots; caller input cannot enable sharing, alter bind, select another project, or override provider/model/tools. +- **Verification:** Fixture tests prove session scoping, permission lifetime, cancellation races, and disposal, followed by an isolated credentialed repository smoke test when OpenCode credentials are available. + +### U7. Pi backend adapter + +- **Goal:** Run Pi through strict RPC mode while preserving settled completion, output, usage/cost, tool progress, cancellation, and process cleanup. +- **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. +- **Dependencies:** U4. +- **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. +- **Approach:** Spawn a supported Pi 0.85.x runtime with `--mode rpc --no-session`, an invocation-local `PI_CODING_AGENT_DIR`, profile model/provider, and scrubbed environment. Implement an LF-only JSONL parser rather than Node `readline`. Correlate responses, wait for `agent_settled`, read messages/stats, send RPC abort, and escalate process-group termination after the grace period. +- **Execution note:** Build parser and state-machine tests from captured RPC fixtures before process integration. +- **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing/cancellation contract. +- **Test scenarios:** + - Successful prompt acceptance streams message/tool events, stops on `agent_settled`, retrieves final messages/stats, and reports session ID, usage, and cost. + - LF framing preserves `U+2028`/`U+2029` inside JSON strings, accepts CRLF by stripping trailing CR, handles partial/multiple chunks, and rejects oversized/malformed records. + - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled events cannot overwrite the terminal fence. + - Prompt rejection, agent error, aborted stop reason, retry/compaction sequence, premature exit, stderr overflow, and stats failure map truthfully. + - Sequential invocations have disjoint `PI_CODING_AGENT_DIR` and session state; caller input cannot send extension commands, steering/follow-up, arbitrary RPC commands, or override provider/model. +- **Verification:** Fixture and fake-process tests prove framing, correlation, settled completion, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when Pi credentials are available. + +### U8. Production registry, service packaging, and observability + +- **Goal:** Compose exactly three production adapters and package independently runnable gateway and worker services with safe startup, health, shutdown, tracing, and reproducible containers. +- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD11-KTD15. +- **Dependencies:** U3-U7. +- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. +- **Approach:** Register only Codex, OpenCode, and Pi through an explicit capability/availability map. Add gateway and worker entrypoints inside the private Node 22 workspace instead of the Node 18 CLI package. Validate config, store, workers, runtime pins, trust, quotas, and resource controls before readiness. Propagate `traceparent` and instrument every phase. Build a minimal gateway image with no provider runtimes and a one-execution worker image with exact runtime versions. +- **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. +- **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. +- **Test scenarios:** + - Registry exposes exactly Codex, OpenCode, and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects unknown backend IDs before workspace creation. + - Gateway and worker start from built service outputs, become ready only after dependencies pass, and stop gracefully on SIGTERM. + - Gateway readiness fails for malformed auth, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. + - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable resource enforcement, or unsupported backend runtime. + - Trace context enters through A2A, crosses the private call, and correlates result identities; exporter failure cannot change Task status. + - Gateway image contains no Codex, OpenCode, Pi, Git workspace, or provider credential material. + - Worker image pins all runtimes, confines one workspace/config root, enforces deployment limits, and completes fake-provider health smoke tests. + - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. +- **Verification:** The registry dispatches every adapter through the same worker contract; built services and images pass lifecycle/security smoke tests; CI and publication bind immutable image tags to the release commit. + +### U9. Cross-backend conformance, documentation, and release evidence + +- **Goal:** Prove the public contract and operational workflow end to end and document deployment without leaking backend details into callers. +- **Requirements:** R1-R22; F1-F5; AE1-AE14. +- **Dependencies:** U1-U8. +- **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. +- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases. Exercise gateway and worker as separate processes. Document extension/worker protocols, profiles, auth, trust boundary, storage/HA limits, source hardening, quotas, runtime requirements, cancellation races, evidence integrity, Artifact access, retention, observability, and troubleshooting. +- **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. +- **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, and Starlight guide/reference organization under `docs/src/content/docs/docs/`. +- **Test scenarios:** + - Covers AE1-AE14 through built services with a fake backend and official A2A client. + - The same mutation fixture passes through Codex, OpenCode, and Pi event fixtures and produces contract-equivalent normalized evidence. + - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; the one-execution worker serializes admitted work. + - Gateway restart, stream reconnect, lost dispatch acknowledgement, duplicate/out-of-order events, worker crash, lease expiry, cancellation race, provider failure, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome. + - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, control-endpoint probing, and secret-exfiltration fixtures are blocked within the documented reviewed-source boundary. + - Examples validate with production schemas and reference secrets only through environment variable names. + - Docs state one gateway replica, one execution per worker, reviewed mutual-trust sources, Node/runtime floors, and no hostile-code isolation claim. + - Opt-in real-provider smoke tests record backend/runtime versions and skip only when the named credential/runtime prerequisite is absent. +- **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor, the full suites and docs pass, the official A2A client exercises every advertised operation, and release evidence records each available real backend plus explicit skipped prerequisites. + +--- + +## Verification Contract + +| Gate | Applies to | Required evidence | +|---|---|---| +| Contract generation | U1 | Public extension and private worker schema generation report no drift; positive and negative fixtures pass. | +| Focused unit tests | U1-U8 | Active-unit tests pass with fault injection, state races, limits, cancellation, and cleanup. | +| Gateway/worker integration | U3-U4, U8-U9 | Built processes agree on fenced dispatch, sequencing, leases, Task persistence, Artifacts, shutdown, and cleanup. | +| Backend conformance | U5-U9 | One shared suite passes against Codex, OpenCode, and Pi adapters with fixture runtimes. | +| Credentialed provider smoke | U5-U7, U9 | Each available provider mutates a disposable exact-SHA repository; missing credentials/runtime are recorded as skipped prerequisites, never passing coverage. | +| A2A interoperability | U3, U9 | Official `@a2a-js/sdk` client passes immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, replay, cancel races, expiry, and owner isolation. | +| Security and abuse | U2-U4, U8-U9 | Malicious identity/source/artifact/resource fixtures prove auth-before-lookup, opaque owner keys, Git SSRF controls, phase-scoped secrets, quotas, quiescence, race-resistant capture, and trust-topology rejection. | +| Service packaging | U8-U9 | Root Node 18 install, private Node 22 build, gateway/worker smoke, and both container builds pass. | +| Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | +| Documentation | U9 | `bun run docs:build` passes and examples validate against current schemas. | + +The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove protocol, durable aggregation, process isolation, fencing, cancellation, or cleanup integration. + +--- + +## Definition of Done + +### Global + +- Every R1-R22 requirement is implemented or explicitly shown in a passing conformance scenario. +- Public Agent Card/extension and private worker schemas are stable, generated from one source, and consumable without importing root AllAgents CLI modules. +- Codex, OpenCode, and Pi pass the same backend conformance suite and preserve bounded native evidence through the closed registry. +- Gateway and worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. +- Authentication precedes lookup, quota precedes Task creation, aggregate commits cannot split claims/Tasks/Artifacts, and terminal fences survive races and restart. +- Cancellation/deadlines reach one native abort, process termination, quiescence, evidence, and cleanup for all three backends. +- Source hardening, phase-scoped secrets, one-execution trust policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end. +- Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. +- Public documentation states supported topology, configuration, security boundary, storage/HA limitation, runtime pins, and deferred capabilities. +- Abandoned experiments, unused adapters, compatibility shims, generated scratch files, retained test workspaces, and stale documentation are removed. + +### Per unit + +- U1: Public/worker schemas, digest vectors, state/fence rules, typed failures, and fixtures are generated and stable. +- U2: Auth, opaque owner isolation, aggregate idempotency, CAS settlement, pagination, restart, quotas, Artifact access, tombstones, and cleanup pass fault injection. +- U3: Every advertised A2A operation agrees across stream and lookup while replay, fencing, and cancellation races preserve one Task. +- U4: Worker dispatch/source/setup/action/check/quiescence/evidence/cleanup lifecycle passes malicious and faulted disposable-repository scenarios. +- U5: Codex streaming, usage, native evidence, isolated roots, cancellation, and failure mapping pass adapter and applicable smoke verification. +- U6: OpenCode session/event/diff/permission isolation, abort, and disposal pass adapter and applicable smoke verification. +- U7: Pi strict JSONL framing, settled completion, stats, isolated roots, abort, and process cleanup pass adapter and applicable smoke verification. +- U8: Closed registry, Node-version separation, readiness, tracing, graceful shutdown, containers, and release artifacts work from built outputs. +- U9: Cross-backend E2E, A2A interoperability, abuse cases, examples, operator docs, changelog, and release evidence are complete. From a00af196261953c70d7c965747427d969ae26b6d Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Fri, 18 Sep 2026 16:51:02 +1000 Subject: [PATCH 04/12] docs(architecture): refine execution gateway scope --- ...-agent-execution-through-an-a2a-gateway.md | 100 +++++- ...0837-feat-coding-execution-gateway-plan.md | 288 +++++++++--------- 2 files changed, 234 insertions(+), 154 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index e798ef3e..1ed60865 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -50,15 +50,15 @@ or durable evaluation Run ledger. It does not own a consumer's result store. ### Keep the gateway separate from execution backends -The initial design supports three peer execution backends: +The initial design supports two execution backends, delivered in this order: -- Codex; -- OpenCode; and -- Pi. +1. Codex; and +2. Pi. Each backend implements the same conformance contract. Provider-specific -process, session, permission, cancellation, and evidence behavior remains -behind its adapter. +process, session, structured-output, cancellation, and evidence behavior +remains behind its adapter. OpenCode and other coding agents remain possible +follow-up adapters rather than part of the first delivery. Execution backends own repository materialization, environment setup, agent invocation, evidence collection, process termination, and cleanup. The gateway @@ -75,6 +75,63 @@ A separate gateway Pod is a service and failure boundary, not per-invocation security isolation. Deployments requiring hostile-code or tenant isolation must create or select a stronger execution boundary behind the gateway. +### Persist Task truth, not live provider execution + +The gateway durably stores Task identity, idempotency claims, terminal status, +Artifact metadata, and retained evidence. A provider execution itself is +ephemeral. The initial service does not checkpoint, reattach, resume, or +automatically replay an interrupted provider session. + +Gateway restart invalidates the active attempt fence and settles each +nonterminal Task failed once. A live worker that loses its lease aborts the +provider and cleans its invocation. If the worker process crashes, an external +supervisor terminates the complete execution boundary and the replacement +worker reaps or quarantines orphaned invocation roots before readiness. +Termination and filesystem cleanup are recorded separately and become complete +only when the responsible boundary proves them; otherwise the terminal record +says unknown. Durable execution and provider-session restoration require a +later decision backed by public provider guarantees. + +### Integrate providers directly + +The Codex adapter depends directly on `@openai/codex-sdk`; AllAgents does not +vendor or depend on Promptfoo's provider. Promptfoo's +[Codex provider](https://github.com/promptfoo/promptfoo/blob/main/src/providers/openai/codex-sdk.ts) +and +[tests](https://github.com/promptfoo/promptfoo/blob/main/test/providers/openai-codex-sdk.test.ts) +are characterization references for strict option mapping, minimal child +environment, working-directory validation, `AbortSignal`, structured output, +event normalization, and cleanup edge cases. + +AllAgents keeps only the gateway-owned subset: one fresh provider session per +Task, server-owned profile settings, bounded native evidence, typed failures, +and worker-proven process cleanup. It does not inherit Promptfoo configuration +layering, caching, pricing, eval retries, thread pools, or `ProviderResponse`. + +The extension defines `allagents.result-schema/v1` as a closed, bounded JSON +Schema Draft 2020-12 subset shared by admission, Codex, Pi, and terminal +validation. It requires an object root, requires every object schema to set +`additionalProperties: false`, lists every declared property in `required`, and +uses `null` unions for optional values. It allows only `type`, `properties`, +`required`, `additionalProperties` with the value `false`, `items`, `enum`, +`const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`, and rejects +remote references, format-dependent validation, and unknown keywords. The +extension version fixes byte, nesting, property, and enum limits. One shared +validator checks both the schema and the returned value, and the accepted schema +digest enters idempotency and provenance. Adapters cannot widen or narrow this +contract. + +Codex receives that schema through the SDK's per-turn `outputSchema`; Pi +implements the same terminal contract with an invocation-scoped terminating +tool. A successful structured request publishes exactly one Artifact named +`allagents.structured-result` with one A2A `Part` whose `data` field contains +the validated result object and whose `mediaType` is `application/json`. +Artifact metadata contains the result-schema version and digest. The Artifact +exists only for a valid result. The integrity +kernel always records `not_requested`, `not_produced`, `valid`, or `invalid`; +an earlier source, setup, provider, cancellation, or deadline outcome remains +the primary Task classification when no result could be produced. + ### Profile A2A 1.0 instead of inventing an invocation API The external contract profiles the Linux Foundation @@ -201,7 +258,7 @@ The gateway and selected backend are collectively responsible for: 1. resolving and verifying immutable source identity; 2. acquiring or restoring source through the selected transport; -3. creating a clean or explicitly reusable working location; +3. creating a fresh working location for one execution attempt; 4. running setup before the evaluated agent action; 5. applying permissions and execution isolation; 6. invoking the agent and propagating cancellation and deadlines; @@ -218,7 +275,9 @@ contract does not require source code to be baked into the runtime image. Credentials remain deployment policy. Requests must not embed deployment credentials. The gateway authenticates callers, and the selected backend scopes source and model credentials to the execution boundary without returning -secret-bearing paths or values. +secret-bearing paths or values. Provider and worker-control credentials must +also be absent from model-initiated command environments, tool output, retained +evidence, and repository-visible configuration. Retries must not multiply non-idempotent agent execution. Every request carries a caller-scoped stable invocation key through the AllAgents extension. The @@ -241,8 +300,14 @@ but their product and ownership model requires a separate decision. narrow coding-execution responsibility. - Consumers depend on A2A 1.0 plus a versioned AllAgents extension, not AllAgents TypeScript modules, CLI behavior, or workspace internals. -- Codex, OpenCode, and Pi are peer execution backends behind one conformance - suite. +- Codex and Pi are the initial execution backends behind one conformance suite; + Codex lands first and OpenCode is deferred. +- Durable Task and evidence records do not imply durable provider execution; + interrupted attempts fail rather than resume or replay. +- The result-schema subset, structured-result Artifact, and non-success result + states are public compatibility surface rather than adapter conventions. +- Reliable worker-crash cleanup requires an external execution supervisor and a + pre-readiness orphan-root reaper in addition to leases. - Gateway and execution workers scale and fail independently. - The gateway can remain lightweight; physical isolation and resource policy belong to the selected execution backend. @@ -293,6 +358,21 @@ around host-owned sessions, not agent-to-agent Task execution. Its reconnect and changeset models do not supply caller-scoped idempotency, immutable source handling, cleanup, complete terminal evidence, or bounded Task retention. +### Vendor Promptfoo's Codex provider + +Rejected because that provider includes Promptfoo-specific configuration +layering, caching, pricing, tracing, retry metadata, thread pooling, and result +mapping. AllAgents needs a smaller worker adapter against the Codex SDK and can +reuse Promptfoo's observable behavior as characterization evidence without +copying its implementation. + +### Treat provider session persistence as durable execution + +Rejected because a resumable provider thread does not prove workspace, +process, cancellation, evidence, or cleanup continuity across gateway or worker +failure. The initial service durably records failure and cleanup truth but does +not resume interrupted work. + ## Reconsider when Revisit this decision if A2A standardizes the required coding-execution evidence diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 1d65837d..60987efd 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -13,11 +13,11 @@ execution: code ## Goal Capsule -- **Objective:** External systems can run Codex, OpenCode, or Pi against an immutable repository revision through one authenticated, cancellable, evidence-preserving remote contract. -- **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with three provider adapters (KTD1, KTD5, KTD7). +- **Objective:** External systems can run Codex or Pi against an immutable repository revision through one authenticated, cancellable, evidence-preserving remote contract. +- **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with two direct provider adapters (KTD1, KTD5, KTD7-KTD8). - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public boundary. The A2A 1.0 specification owns core wire semantics. The versioned AllAgents extension owns coding-execution semantics. -- **Execution profile:** Build contract-first, then durable gateway state, worker lifecycle, provider adapters, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. -- **Stop conditions:** Do not execute agents in the gateway process, accept mutable source identity, put deployment credentials in requests, treat streams or telemetry as terminal evidence, or add evaluation behavior. +- **Execution profile:** Build contract-first, then durable Task/evidence state, worker lifecycle, Codex, Pi, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. +- **Stop conditions:** Do not execute agents in the gateway process, accept mutable source identity, put deployment credentials in requests, treat streams or telemetry as terminal evidence, treat provider sessions as recovery checkpoints, vendor an evaluator's provider implementation, or add evaluation behavior. - **Tail ownership:** The implementing workflow runs focused contract and lifecycle tests, the complete repository quality gates, isolated gateway/worker smoke tests, provider-specific credentialed smoke tests where credentials are available, and documentation validation. --- @@ -32,20 +32,21 @@ AllAgents gains a remote coding-execution service without becoming an evaluation AllAgents currently configures and launches coding clients but has no service boundary for external callers. AI Evals and future consumers would otherwise need to import AllAgents internals, drive interactive CLIs, or independently reimplement repository acquisition, permissions, cancellation, evidence, and cleanup. -The three initial runtimes expose different programmatic contracts. Codex provides a TypeScript SDK over structured JSONL events, OpenCode provides a typed HTTP SDK and SSE event stream, and Pi provides a strict JSONL RPC mode. The public service must preserve one stable lifecycle without flattening provider-specific facts into false equivalence. +The two initial runtimes expose different programmatic contracts. Codex provides a TypeScript SDK over structured JSONL events and native per-turn `outputSchema`; Pi provides a strict JSONL RPC mode and invocation-scoped custom tools. The public service must preserve one stable lifecycle and structured-result contract without flattening provider-specific facts into false equivalence. ### Actors - A1. **Gateway caller:** An authenticated service such as AI Evals that creates, observes, lists, cancels, and retrieves coding-execution Tasks. - A2. **Execution gateway:** The A2A server that owns caller scope, Task identity, idempotency, routing, retention, and normalized results. - A3. **Execution worker:** A separately deployed process that owns source materialization, one mutable workspace per invocation, provider execution, evidence capture, and cleanup. -- A4. **Backend adapter:** The Codex, OpenCode, or Pi integration that translates native events, cancellation, usage, failures, and evidence into the worker contract. +- A4. **Backend adapter:** The Codex or Pi integration that translates native events, structured results, cancellation, usage, failures, and evidence into the worker contract. - A5. **Operator:** The person or deployment system that defines profiles, credentials, limits, retention, worker endpoints, and observability policy. ### Key Decisions - **Profile A2A rather than creating a public invocation API.** The service keeps standard Agent Cards, Tasks, Artifacts, operations, errors, and capability negotiation. Governs R1-R4. - **Keep execution outside the gateway process.** Mutable repositories and provider processes belong to workers. Governs R10-R16, R21-R22. +- **Persist Task truth, not live executions.** Accepted Task identity and terminal evidence survive restart; provider sessions do not resume or replay. Governs R7, R14, R16-R18. - **Keep evaluation outside AllAgents.** Dataset expansion, repetitions, assertions, scoring, retries, and durable evaluation Runs remain caller concerns. Governs R20. ### Requirements @@ -61,23 +62,23 @@ The three initial runtimes expose different programmatic contracts. Codex provid - R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. - R6. Authentication, required-extension validation, request validation, source/profile authorization, quota admission, and deadline validation complete before Task creation. A caller-scoped invocation key, effective profile, authenticated owner, and canonical request digest then bind atomically to one Task; identical replay returns that Task and conflicting reuse is rejected without dispatch. -- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Task state and terminal Artifact metadata survive gateway restart; nonterminal Tasks that cannot be reattached settle failed once and stale worker events cannot overwrite them. +- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Task state and terminal Artifact metadata survive gateway restart. Every nonterminal Task present at startup settles failed once, its old attempt fence is invalidated, and stale worker events cannot overwrite it; the initial service never resumes or automatically replays interrupted provider work. - R8. List operations implement all A2A filters, history bounds, page-size bounds, owner/query-bound cursor pagination, and descending status-update time. One immutable expiry logically hides the Task, claim, events, and artifacts before best-effort physical deletion; expired and unauthorized IDs are indistinguishable. - R9. Small deployments work without an external database. The built-in durable store supports one gateway replica, enforces per-owner/global admission and storage quotas, and reserves capacity for cancellation and terminal settlement; multi-replica storage is outside this delivery. **Execution and policy** -- R10. Codex, OpenCode, and Pi are peer backends behind one conformance contract. (session-settled: user-directed — chosen over an additional enterprise-only adapter: the open-source gateway supports the three named runtimes directly.) -- R11. A request selects a server-defined execution profile. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. +- R10. Codex and Pi are the complete initial backend set behind one conformance contract, delivered Codex first and Pi second. OpenCode is deferred. (session-settled: user-directed.) +- R11. A request selects a server-defined execution profile and may include one `allagents.result-schema/v1` schema for the terminal result: a bounded JSON Schema Draft 2020-12 subset with an object root, every object schema setting `additionalProperties: false`, every declared property listed in `required`, optional values represented by `null` unions, and only `type`, `properties`, `required`, `additionalProperties` with the value `false`, `items`, `enum`, `const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`. The extension version fixes byte, depth, property, and enum limits; admission rejects remote references, format-dependent validation, and unknown keywords; one shared validator governs schema admission and returned values. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. Requests cannot supply raw provider configuration. - R12. The only initial remote source form is a canonical credential-free HTTPS Git URL plus full commit object ID and optional repository-relative subdirectory. Acquisition revalidates destination policy for every connection, disables redirects and repository-controlled secondary fetch/exec features, uses hermetic Git configuration, and verifies that the fetched object is the requested commit before setup. -- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, and retained workspaces. +- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, and every model-initiated command or tool environment. - R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. - R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. -- R16. A worker creates a fresh invocation directory and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, and runs configured checks. It then proves all invocation descendants quiescent before final evidence/artifact capture and cleanup or explicit retention. +- R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves all invocation descendants quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. An external supervisor terminates the complete execution boundary when the worker process crashes, and a replacement worker reaps or quarantines orphaned roots before readiness. **Evidence and observability** -- R17. Every terminal result contains an integrity kernel: Task/source/profile/backend identities, action outcome, cancellation or failure classification, termination and cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. +- R17. Every terminal result contains an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. Pre-output source, setup, provider, cancellation, or deadline outcomes retain their primary Task classification and record `not_produced` secondarily. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. - R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. - R19. Gateway and worker spans propagate W3C Trace Context and export OpenTelemetry data. Telemetry is operational evidence, not the only durable result. @@ -91,8 +92,8 @@ The three initial runtimes expose different programmatic contracts. Codex provid - F1. **Admit, create, and stream an execution** - **Actors:** A1, A2, A3, A4. - - **Trigger:** A caller sends a text Message with the required extension, immutable source, profile, invocation key, and deadline. - - **Steps:** Authenticate; validate and authorize the complete request; reserve quota; atomically claim idempotency and create a submitted Task; dispatch a fenced worker attempt; materialize and verify source; execute the selected backend; persist progress before emission; terminalize with Artifacts after quiescence and cleanup. + - **Trigger:** A caller sends a text Message with the required extension, immutable source, profile, invocation key, deadline, and optional bounded result schema. + - **Steps:** Authenticate; validate and authorize the complete request and result-schema subset; reserve quota; atomically claim idempotency and create a submitted Task; dispatch a fenced worker attempt; materialize and verify source; execute the selected backend; validate structured output with the shared validator when requested; persist progress before emission; terminalize with the fixed-name structured-result Artifact only for a valid result and with evidence Artifacts after quiescence and cleanup. - **Outcome:** `returnImmediately: true` returns the durable current Task, false/unset waits for terminal state, and streaming starts with that Task before ordered updates. - **Covered by:** R1-R22. - F2. **Replay or reconnect to an invocation** @@ -107,11 +108,11 @@ The three initial runtimes expose different programmatic contracts. Codex provid - **Steps:** Atomically record the first cancellation source; if dispatch never occurred, prove no workspace exists; otherwise send one fenced worker cancel, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses late provider success and fails instead of claiming canceled when termination or cleanup cannot be verified. - **Covered by:** R7, R14, R16-R18. -- F4. **Recover from gateway or worker loss** +- F4. **Settle after gateway or worker loss** - **Actors:** A2, A3. - - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, or a worker crashes. - - **Steps:** Invalidate the attempt fence; settle each non-reattachable Task failed once; reject late events/results; stop renewing leases; let workers self-abort and clean. Record cleanup complete only when the worker/process boundary proves it; otherwise record unknown. - - **Outcome:** One Task has one terminal result, no ambiguous dispatch is retried automatically, and no stale worker can overwrite durable truth. + - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, a live worker loses its lease, or a worker process crashes. + - **Steps:** Invalidate the attempt fence and settle every affected Task failed once without provider-session reattachment or automatic replay. A live worker that loses its lease self-aborts and cleans. On worker-process crash, the external supervisor terminates the complete execution boundary; the replacement worker proves termination, then reaps or quarantines orphaned roots before readiness. Reject late events/results and record termination and filesystem cleanup separately as complete only when the responsible boundary proves each outcome. + - **Outcome:** One Task has one terminal result, interrupted work is never presented as resumed, no stale worker can overwrite durable truth, and a crashed worker cannot leave an unowned process or reusable workspace. - **Covered by:** R7, R9, R14, R16-R18, R21-R22. - F5. **Expire retained execution data** - **Actors:** A1, A2. @@ -122,28 +123,28 @@ The three initial runtimes expose different programmatic contracts. Codex provid ### Acceptance Examples -- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile and an exact Git SHA, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same output and evidence Artifacts. +- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, an exact Git SHA, and an optional result schema, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same validated output and evidence Artifacts. - AE2. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with the same canonical request, then the gateway returns the original Task without a second worker dispatch. - AE3. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with a different prompt, source, profile, or deadline, then the gateway rejects the request and leaves the original Task unchanged. - AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. - AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository, when the worker verifies source, then provider execution never starts and the Task fails with source-verification and cleanup evidence. - AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. -- AE7. **Covers R10.** Given equivalent profiles and fixture runtime events for Codex, OpenCode, and Pi, when each completes the same repository mutation, then all three produce the same required normalized result fields while retaining distinct native evidence. +- AE7. **Covers R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. - AE8. **Covers R4, R7, R19.** Given a caller disconnects during work, when it subscribes again, then it receives the current Task and future updates without duplicate dispatch; telemetry loss does not affect later terminal lookup. - AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. - AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. - AE11. **Covers R6, R22.** Given invalid input or exhausted admission quota, the gateway returns a request/resource error and creates no Task; given capacity disappears after durable acceptance, the retained Task fails at dispatch and replay returns it without retry. -- AE12. **Covers R7, R14.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only safe operator telemetry. +- AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only safe operator telemetry. Given the worker is killed with live descendants and an invocation root, its supervisor terminates the execution boundary and the replacement worker reaps or quarantines the root before readiness without changing the failed Task. - AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. - AE14. **Covers R21-R22.** Given a profile requests pooled hostile-source or cross-tenant execution, startup/admission rejects it; a reviewed single-trust-domain profile runs one bounded execution without exposing worker control credentials to the child environment. ### Success Criteria - The official A2A JavaScript client can discover the card and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, replay, cancel, and expiry behavior against the built service. -- One conformance fixture passes unchanged through Codex, OpenCode, and Pi adapters. -- Admission, replay, fencing, cancellation races, restart recovery, authorization isolation, source hardening, quotas, and evidence integrity have deterministic integration coverage. +- One conformance fixture passes unchanged through the Codex and Pi adapters. +- Admission, replay, fencing, cancellation races, restart terminalization without resume, supervised worker-crash cleanup, authorization isolation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. - The gateway image contains no coding-agent runtime and cannot access worker workspace roots. -- The initial worker runs one reviewed-trust-domain execution at a time and leaves no live descendant or retained workspace unless policy requests retention. +- The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools receive no provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed or crashed attempt. ### Scope Boundaries @@ -151,7 +152,7 @@ The three initial runtimes expose different programmatic contracts. Codex provid - A2A 1.0 HTTP+JSON and SSE streaming. - One versioned AllAgents coding-execution extension and one versioned private worker protocol. -- Codex, OpenCode, and Pi backends. +- Codex and Pi backends. - Built-in bearer authentication with OIDC/JWT and static service-token modes. - Single-replica durable file storage, authenticated Artifact retrieval, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. - Reviewed repositories in one configured mutual-trust domain per worker deployment. @@ -159,10 +160,11 @@ The three initial runtimes expose different programmatic contracts. Codex provid **Deferred to follow-up work** - Multi-replica database-backed Task and idempotency storage. +- Durable provider execution, checkpointing, provider-session restoration, and automatic replay after gateway or worker restart. +- OpenCode and additional coding backends. - Kubernetes Job dispatch, queue brokers, autoscaling controllers, and stronger hostile-source or cross-tenant sandbox providers. - Push-notification configuration, gRPC, JSON-RPC transport, and A2A extended Agent Cards. - AHP server/client surfaces, long-lived interactive sessions, and client-contributed tools. -- Additional coding backends and provider-session restoration after gateway restart. - Optional ATIF conversion after the format and tooling mature. **Outside this product's identity** @@ -178,8 +180,14 @@ The three initial runtimes expose different programmatic contracts. Codex provid - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) - [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) - [Codex TypeScript SDK](https://github.com/openai/codex/tree/main/sdk/typescript) -- [OpenCode SDK and server](https://opencode.ai/docs/sdk/) +- [Codex configuration reference](https://developers.openai.com/codex/config-reference) +- [Promptfoo Codex provider documentation](https://github.com/promptfoo/promptfoo/blob/main/site/docs/providers/openai-codex-sdk.md) +- [Promptfoo Codex provider implementation](https://github.com/promptfoo/promptfoo/blob/main/src/providers/openai/codex-sdk.ts) +- [Promptfoo Codex provider tests](https://github.com/promptfoo/promptfoo/blob/main/test/providers/openai-codex-sdk.test.ts) - [Pi RPC protocol](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md) +- [Pi CLI reference](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/README.md#cli-reference) +- [Pi extension API](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/extensions.md) +- [Pi provider credentials](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/providers.md) --- @@ -188,20 +196,20 @@ The three initial runtimes expose different programmatic contracts. Codex provid ### Key Technical Decisions - KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. The decorator owns admission, canonical Task reservation, idempotent replay, stream snapshot selection, and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. -- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes both versioned contracts, generated JSON Schemas, bounds, and fixtures. The worker protocol carries attempt identity, profile digest, dispatch acceptance, monotonic event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. +- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes both versioned contracts, generated JSON Schemas, bounds, and fixtures. The public contract carries the `allagents.result-schema/v1` closed subset, its canonical digest, four structured-result states, and the fixed `allagents.structured-result` Artifact containing one A2A `Part` with `data` and `mediaType: application/json`. The worker protocol carries attempt identity, profile digest, dispatch acceptance, monotonic event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. - KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates the invocation claim and submitted Task together, stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. - KTD4. **Authenticate at HTTP ingress before A2A storage or dispatch.** Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. Unauthenticated mode is allowed only on a loopback listener. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Governs R5-R6, R13. -- KTD5. **Use fenced, separately deployable gateway and worker services.** The gateway owns A2A and durable results; the worker owns workspaces and provider processes. Every dispatch has a gateway-generated attempt ID, lease ID/epoch, short-lived capability, and event sequence. Workers idempotently accept duplicate delivery of the same attempt, reject conflicting attempts, and gateways ignore stale/out-of-order events and late terminal results. Governs R7, R10-R16, R21-R22. -- KTD6. **Make worker leases the orphan-execution fail-safe, not a replay mechanism.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry aborts and cleans the worker. Gateway restart invalidates old fences and records cleanup as unknown unless a process boundary proves it. Caller stream disconnect never affects the lease. Governs R7, R14, R16-R18. -- KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | opencode | pi` registry is the only production dispatch point. Governs R10, R14-R18, R21-R22. -- KTD8. **Use each provider's supported automation surface.** Codex uses `@openai/codex-sdk` streaming with `AbortSignal`; OpenCode uses its typed SDK against a worker-owned loopback server and session abort; Pi uses `pi --mode rpc --no-session` with a strict LF-delimited JSON parser, `agent_settled`, `get_session_stats`, and RPC abort. Governs R10, R14-R18. -- KTD9. **Make profiles the policy boundary.** Requests select a profile ID but cannot override backend credentials, executable paths, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. Profile digests enter idempotency and provenance. Governs R6, R11-R16, R21-R22. +- KTD5. **Use fenced, separately deployable gateway and worker services.** The gateway owns A2A and durable Task/results truth; the worker owns ephemeral execution attempts, workspaces, and provider processes. Every dispatch has a gateway-generated attempt ID, lease ID/epoch, short-lived capability, and event sequence. Workers idempotently accept duplicate delivery of the same attempt, reject conflicting attempts, and gateways ignore stale/out-of-order events and late terminal results. Governs R7, R10-R16, R21-R22. +- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Worker-process exit makes the external supervisor terminate the complete execution boundary; before readiness the replacement worker proves termination and reaps or quarantines orphaned invocation roots. Termination and filesystem cleanup remain separate outcomes and are unknown until proved. The gateway never reattaches to or resumes a provider session. Caller stream disconnect never affects the lease. Governs R7, R14, R16-R18. +- KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, optional structured result, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, schema validation, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | pi` registry is the only production dispatch point. Governs R10-R11, R14-R18, R21-R22. +- KTD8. **Use each provider's supported automation surface directly.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, consumes streamed events, and applies a pinned shell-environment policy that excludes provider/control credentials from model-initiated commands. Pi uses `pi --mode rpc --no-session --no-extensions --no-builtin-tools` with a strict LF-delimited JSON parser, `agent_settled`, `get_session_stats`, RPC abort, an invocation-local credential store rather than credential environment variables, and one explicitly loaded worker-owned policy extension outside the repository. That extension supplies workspace-confined filesystem/command tools and the terminating result tool; no repository extension or unrestricted built-in tool loads. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. +- KTD9. **Make profiles the policy boundary.** Requests select a profile ID and may provide only an `allagents.result-schema/v1` schema. They cannot override backend credentials, executable paths, provider config, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. Profile digests and the canonical result-schema digest enter idempotency and provenance. Governs R6, R11-R16, R21-R22. - KTD10. **Capture Git and provider evidence as separate layers after quiescence.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded layer. Neither layer is promoted as exact causality when incomplete. Governs R16-R18. -- KTD11. **Treat Codex, OpenCode, and Pi as the complete initial backend set.** (session-settled: user-directed — chosen over adding an enterprise-only adapter: only the three named open-source gateway backends belong in this plan.) Governs R10. -- KTD12. **Separate terminal integrity from optional evidence bodies.** Identity, action outcome, failure/cancellation, termination, cleanup, Artifact index, completeness, and provenance must validate before terminal publication. Predictable budget truncation/redaction of logs, diffs, native events, or produced-file bodies may preserve completion with explicit metadata; capture failure that breaks the integrity kernel fails in the evidence phase. Governs R4, R17-R18. +- KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. +- KTD12. **Separate terminal integrity from optional evidence bodies.** Identity, action outcome, the four-state structured-result record and fixed Artifact rule, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance must validate before terminal publication. A pre-output failure records `not_produced` without replacing its primary phase classification. Predictable budget truncation/redaction of logs, diffs, native events, or produced-file bodies may preserve completion with explicit metadata; capture failure that breaks the integrity kernel fails in the evidence phase. Governs R4, R17-R18. - KTD13. **Harden Git acquisition as a network security boundary.** Accept canonical HTTPS origins only. Use hermetic Git configuration, disable redirects, proxies, helpers, hooks, filters, LFS smudge, submodule recursion, alternates, and non-HTTPS protocols. Revalidate normalized host/address policy for every connection, never forward credentials across origins, and verify the full object ID resolves to a commit fetched from the approved remote. Governs R12-R13, R22. -- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Stronger isolation is a separate sandbox-driver capability. Governs R16, R21-R22. -- KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. +- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Provider/source credentials are absent from setup/check phases, model-initiated commands and tools, and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load, and its replacement tools confine paths to the invocation workspace and spawn commands with the phase allowlist. Stronger isolation is a separate sandbox-driver capability. Governs R13, R16, R21-R22. +- KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. ### High-Level Technical Design @@ -216,7 +224,6 @@ flowchart TB Worker --> Source[Hardened Git acquisition] Worker --> Registry[Closed backend registry] Registry --> Codex[Codex SDK] - Registry --> OpenCode[OpenCode SDK and server] Registry --> Pi[Pi RPC process] Worker --> Evidence[Quiesced checks, Git and native evidence] Evidence -->|Bounded terminal result| Gateway @@ -312,6 +319,7 @@ packages/execution-service/ execution/ contract.ts extension-v1.ts + result-schema-v1.ts worker-protocol-v1.ts errors.ts profiles.ts @@ -332,6 +340,8 @@ packages/execution-service/ index.ts config.ts server.ts + supervisor.ts + reaper.ts lease.ts workspace.ts evidence.ts @@ -339,8 +349,8 @@ packages/execution-service/ types.ts registry.ts codex.ts - opencode.ts pi.ts + pi-policy-extension.ts tests/ fixtures/execution/ unit/execution/ @@ -362,7 +372,7 @@ docs/src/content/docs/ - Gateway configuration defines listener/public URL, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker endpoints, internal capability secrets, and profiles. - Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, and evidence budgets. -- Worker configuration fixes a private listener, one-execution concurrency, workspace root, lease grace, backend runtime constraints, trust domain, resource-control capability, and request/result limits. +- Worker configuration fixes a private listener, one-execution concurrency, workspace root, execution-supervisor mechanism, pre-readiness orphan policy, lease grace, backend runtime constraints, trust domain, resource-control capability, and request/result limits. - Configuration contains environment-variable names but never secret values. Startup resolves the complete graph, verifies that profile claims do not exceed deployment capabilities, and becomes ready only when store, workers, runtimes, quotas, and free-space reserves pass. ### Error and Status Mapping @@ -376,39 +386,43 @@ docs/src/content/docs/ | Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | | Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | | Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | -| Source, setup, provider, check, mandatory evidence, worker crash, or infrastructure failure | `TASK_STATE_FAILED` | Typed phase, safe message, retriable fact, termination/cleanup/completeness | -| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors, native abort, termination, cleanup | +| Source, setup, provider, check, mandatory evidence, worker crash, or infrastructure failure | `TASK_STATE_FAILED` | Typed primary phase, safe message, retriable fact, structured result `not_produced` when requested, separate termination/cleanup/completeness | +| Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` or `structured_result/invalid`, no structured-result Artifact | +| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors, native abort, structured result `not_produced` unless already valid, termination, cleanup | | Cancellation loses to terminal completion | Existing terminal Task / `TaskNotCancelableError` | No state mutation or second abort | -| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Output plus complete required evidence | +| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Output plus complete required evidence; a requested valid result uses the fixed-name Artifact with one A2A `Part` containing `data` and `mediaType: application/json` | | Successful action with allowed bounded optional-evidence gap | `TASK_STATE_COMPLETED` | Per-dimension incomplete flag, reason, original/captured size, digest and redaction/truncation flags | -| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid and cleanup unknown unless proven | +| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid, structured result `not_produced` unless already committed, and cleanup unknown unless proven | | Retention expiry | Not found | Aggregate logically hidden before physical deletion; Artifact URL also invalid | ### Phased Delivery -1. Create the private Node 22 service package and freeze the public extension, worker protocol, profiles, fixtures, and error vocabulary. -2. Build authenticated durable A2A Task handling and fenced worker dispatch against a fake worker. -3. Build the single-execution worker lifecycle and hardened source/evidence handling against a fake adapter. -4. Add Codex, OpenCode, and Pi adapters in parallel, then compose them through the closed registry. -5. Package the services and run cross-backend, security, process, and A2A conformance before enabling a consumer. +1. Create the private Node 22 service package and freeze the public extension, portable result-schema subset, structured-result Artifact, worker protocol, profiles, fixtures, and error vocabulary. +2. Build authenticated durable A2A Task handling and fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. +3. Build the supervised single-execution worker lifecycle, pre-readiness orphan reaper, and hardened source/evidence handling against a fake adapter. +4. Add the direct Codex SDK adapter and prove structured output, cancellation, provider-credential exclusion from model commands, environment isolation, and native evidence. +5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing confined tools plus the terminating result tool. +6. Package the services and run cross-backend, security, process, and A2A conformance before enabling a consumer. ### System-Wide Impact - **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. - **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. - **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. -- **Security:** New review-critical surfaces are auth, owner-key derivation, source SSRF, admission/resource quotas, setup/check policy, phase-scoped secrets, internal fences, Artifact capture/serving, and reviewed-source trust enforcement. -- **Operations:** Gateway and worker health, readiness, quotas, low-space state, structured logs, traces, tombstone backlog, lease expiry, stale event rejection, and graceful shutdown need independent signals. +- **Security:** New review-critical surfaces are auth, owner-key derivation, source SSRF, admission/resource quotas, setup/check policy, provider-credential exclusion from model tools, Pi extension/tool replacement, phase-scoped secrets, internal fences, Artifact capture/serving, and reviewed-source trust enforcement. +- **Operations:** Gateway and worker health, readiness, quotas, low-space state, structured logs, traces, tombstone backlog, lease expiry, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. - **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. ### Risks and Mitigations -- **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup and keep captured provider fixtures versioned. +- **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup, keep captured provider fixtures versioned, and use Promptfoo's Codex tests as characterization input rather than vendored implementation. - **False idempotency or stale settlement:** Claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test duplicate delivery, cancellation races, restart, and late results. - **Task/store corruption:** Publish immutable blobs and generations before one manifest switch; tombstone before deletion; validate owner tuples/manifests at startup; garbage-collect unreachable generations; document the one-replica limit. - **Owner collision or path injection:** Hash a bounded canonical issuer/tenant/subject tuple, store and verify the tuple inside the owner aggregate, and use only server-generated opaque IDs in paths. -- **Orphan processes:** Combine explicit cancel, native abort, process-group termination, one-execution worker/container death, lease expiry, and quiescence proof before evidence capture. -- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral, origin-bound, and absent from repository config, process arguments, retained workspaces, logs, and errors. +- **Orphan processes and roots:** Combine explicit cancel, native abort, process-group termination, one-execution supervisor/container death, lease expiry, pre-readiness orphan reaping or quarantine, and separate termination/filesystem proof before evidence or readiness. +- **False recovery claims:** Persist Task and evidence truth only. Startup fails active Tasks, invalidates fences, and relies on lease expiry or supervisor-boundary proof instead of resuming provider sessions. +- **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in idempotency/provenance, pass the exact accepted schema through each adapter's supported mechanism, validate with one shared validator, publish only the fixed Artifact shape, and fail rather than publish missing or invalid JSON. +- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral, origin-bound, and absent from repository config, process arguments, model-initiated command/tool environments, retained workspaces, logs, and errors; Pi repository extensions and unrestricted built-in tools never load, and policy tools cannot access Pi config/data roots. - **Resource exhaustion:** Reserve per-owner/global gateway quota before claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. - **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. - **Evidence overclaim:** Enforce KTD12's integrity kernel and per-dimension completeness. Truncation and redaction remain independent facts. @@ -422,7 +436,7 @@ docs/src/content/docs/ - Git over hardened HTTPS and exact commit object ID covers the initial consumer. Other source transports require a later extension version or capability. - Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. - Initial repositories are reviewed inside one configured mutual-trust domain. Strong hostile-code or cross-tenant execution remains unavailable until a stronger sandbox driver exists. -- Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, OpenCode CLI 1.18.x with its compatible SDK, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. +- Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. --- @@ -434,16 +448,16 @@ docs/src/content/docs/ - **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. - **Dependencies:** None. - **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. -- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas, one public extension URI, and one private protocol version. Include attempt/fence/lease identity, monotonic event sequence, accepted dispatch, renew/cancel, bounded terminal acknowledgement, public/private state separation, and integrity-kernel rules. Canonicalize caller input plus effective profile digest for idempotency. Generate checked-in JSON Schemas and fixtures from the same source. +- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas, one public extension URI, and one private protocol version. Define the exact `allagents.result-schema/v1` keyword allowlist and bounds, canonical schema digest, four structured-result states, fixed-name Artifact containing one A2A `Part` with `data` and `mediaType: application/json`, and shared schema/result validator. Include attempt/fence/lease identity, monotonic event sequence, accepted dispatch, renew/cancel, bounded terminal acknowledgement, public/private state separation, and integrity-kernel rules. Canonicalize caller input plus effective profile and result-schema digests for idempotency. Generate checked-in JSON Schemas and fixtures from the same source. - **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions, unsafe paths, invalid public states, stale fences, oversized records, and conflicting canonical inputs before implementing schemas. - **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, and `src/core/native/types.ts` for safe error/provenance normalization. - **Test scenarios:** - - A minimal valid request with text prompt, invocation key, profile, exact commit, and deadline parses and produces a stable digest across object-key ordering. - - Changing prompt, source object ID, profile ID/digest, artifact selection, or deadline changes the digest; trace IDs and transport metadata do not. - - A source URL with credentials, a branch/tag revision, absolute subdirectory, traversal, secret value, unknown backend, or unknown extension version is rejected safely. + - A minimal valid request with text prompt, invocation key, profile, exact commit, deadline, and optional `allagents.result-schema/v1` schema parses and produces a stable digest across object-key ordering. + - Changing prompt, source object ID, profile ID/digest, result schema, artifact selection, or deadline changes the digest; trace IDs and transport metadata do not. + - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - Worker fixtures reject missing/mismatched attempt IDs, lease epochs, profile digests, event sequence, bounds, and terminal acknowledgements. - - Completed, failed, canceled, and rejected results validate only with the integrity kernel; optional usage/native evidence gaps require explicit completeness reasons. + - `not_requested`, `not_produced`, `valid`, and `invalid` cover success, pre-output failure, cancellation, missing output, and invalid output without replacing the primary Task classification; only `valid` permits one `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data`, `mediaType: application/json`, and a matching schema digest in Artifact metadata. - File evidence accepts create/edit/delete/rename and rejects unsafe paths, duplicate identities, oversized inline content, and inconsistent before/after forms. - **Verification:** Generated schemas are stable, public/private fixtures round-trip, digest vectors are cross-platform deterministic, and the private client/server fixture suite agrees before gateway or worker implementation. @@ -453,7 +467,7 @@ docs/src/content/docs/ - **Requirements:** R4-R9, R13-R14, R17-R18, R22; AE2-AE4, AE6, AE8, AE10-AE13; KTD1, KTD3-KTD4, KTD12. - **Dependencies:** U1. - **Files:** `packages/execution-service/src/gateway/config.ts`, `packages/execution-service/src/gateway/auth.ts`, `packages/execution-service/src/gateway/store/gateway-repository.ts`, `packages/execution-service/src/gateway/store/file-gateway-repository.ts`, `packages/execution-service/tests/unit/gateway/auth.test.ts`, `packages/execution-service/tests/unit/gateway/file-gateway-repository.test.ts`. -- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Commit claim plus submitted Task in one manifest generation; publish immutable Artifact blobs before terminal manifest switch; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover and garbage-collect unreachable generations on startup. Reserve owner/global quotas before claims. Verify OIDC JWTs and constant-time static tokens before all repository access. +- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Commit claim plus submitted Task in one manifest generation; publish immutable Artifact blobs before atomically switching the manifest to a new generation; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover and garbage-collect unreachable generations on startup. Startup recovery is a readiness barrier: invalidate every old fence and terminalize every nonterminal Task before admission, subscriptions, dispatch, or lease renewal begin. Reserve owner/global quotas before claims. Verify OIDC JWTs and constant-time static tokens before all repository access. - **Execution note:** Implement concurrent-claim, transition-race, and crash-publication tests before request handling. Inject faults between blob, generation, manifest, tombstone, and cleanup operations. - **Patterns to follow:** `src/core/marketplace.ts` and `src/core/profile/files.ts` for atomic publication/recovery, `src/core/mcp-http-stdio-proxy.ts` for private files and loopback safety, and the official A2A `TaskStore` owner-scoping contract. - **Test scenarios:** @@ -462,7 +476,7 @@ docs/src/content/docs/ - Hostile/ambiguous issuer, tenant, subject, invocation key, Task ID, Artifact name, Unicode, case, delimiter, traversal, and Windows-reserved values cannot collide or become paths. - All standard list filters, `historyLength`, page size 1-100, omitted Artifacts, ordering, total size, and always-present next token match A2A semantics. Tokens are owner/query-bound and reject malformed, swapped, or stale filters. - Covers AE12. Terminal compare-and-swap wins once; stale fence, duplicate, and out-of-order updates cannot mutate the Task. - - Restart fails nonterminal Tasks once, invalidates fences, preserves terminal Tasks, and records cleanup unknown unless proven. + - Restart, including repeated failure during startup recovery, completes the recovery barrier before serving: it fails every nonterminal Task once, invalidates fences, never renews an old lease or requests provider reattachment/replay, preserves terminal Tasks, and records cleanup unknown unless proven. - Covers AE13. Exact expiry tombstones the aggregate before cleanup; failed deletion never restores visibility; same-key replay before expiry returns the old Task and after expiry creates a new Task. - A crash between every aggregate publication step leaves either the prior or next valid manifest, never claim-without-Task or Task-with-missing-Artifact state. - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. @@ -495,117 +509,104 @@ docs/src/content/docs/ ### U4. Worker protocol and safe workspace lifecycle -- **Goal:** Implement the single-execution worker, hardened immutable Git acquisition, profile enforcement, leases, isolated backend roots, resource controls, race-resistant evidence, termination, and cleanup independent of any provider. +- **Goal:** Implement the supervised single-execution worker, hardened immutable Git acquisition, profile enforcement, leases, isolated backend roots, resource controls, race-resistant evidence, termination, and cleanup independent of any provider. - **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. - **Dependencies:** U1. -- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. -- **Approach:** Authenticate and fence the private protocol, reserve the one execution before workspace creation, validate profile/deployment capability, and emit sequenced NDJSON. Acquire source under KTD13. Create separate workspace and backend config/data roots with a scrubbed phase-specific environment. Run setup, baseline, adapter, and checks under enforced budgets. Stop and verify the process group before descriptor-based regular-file evidence staging, then clean in `finally`. Lease expiry self-cancels. +- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. +- **Approach:** Authenticate and fence the private protocol, reserve the one execution before workspace creation, validate profile/deployment capability, and emit sequenced NDJSON. Run the worker server inside a deployment-approved supervisor boundary that kills all invocation descendants if the server exits. Before readiness, inspect invocation manifests, require supervisor proof that prior descendants are dead, and delete or quarantine orphaned roots; an unprovable root blocks reuse and reports degraded readiness. Acquire source under KTD13. Create separate workspace and backend config/data roots with a scrubbed phase-specific environment. Run setup, baseline, adapter, and checks under enforced budgets. Stop and verify the process group before descriptor-based regular-file evidence staging, then clean in `finally`. Lease expiry self-cancels. - **Execution note:** Characterize every phase with a fake adapter, malicious fixtures, and disposable Git servers before real providers. Fault-inject dispatch acknowledgement, events, leases, acquisition, processes, evidence publication, and cleanup. - **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, and `tests/helpers/env.ts` for isolated state. - **Test scenarios:** - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, and fetch failure stop before adapter invocation. - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no secondary connection or helper execution. - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. - - Setup changes establish the baseline; setup and checks receive no provider/control secrets; every backend gets disjoint invocation config/data roots with ambient selectors removed. + - Setup changes establish the baseline; setup, checks, and model-initiated tools receive no provider/control secrets; every backend gets disjoint invocation config/data roots with ambient selectors removed. - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected; worker control credentials are absent from child environment and configured filesystem roots. - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation and preserve worker health. - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps; capture I/O or malformed result that breaks the kernel fails the Task. - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. - - A worker crash before/after provider spawn reports cleanup complete only when the process/container boundary proves descendant death. -- **Verification:** A built worker mutates a disposable exact-SHA repository through the fake adapter and proves fenced dispatch, source hardening, phase isolation, budgets, quiescence, evidence integrity, and cleanup from its emitted result alone. + - Covers AE12. SIGKILL the worker before and after provider spawn with live descendants and a persistent invocation root; the supervisor proves descendant death, the replacement reaper deletes or quarantines the root before readiness, and the gateway retains one failed Task with separate termination and cleanup outcomes. +- **Verification:** A built supervised worker mutates a disposable exact-SHA repository through the fake adapter and proves fenced dispatch, source hardening, phase isolation, budgets, quiescence, evidence integrity, worker-crash containment, orphan-root handling, and cleanup from its emitted result plus supervisor proof. ### U5. Codex backend adapter -- **Goal:** Run Codex through its supported TypeScript SDK while preserving structured progress, output, usage, file-change evidence, cancellation, and runtime identity. +- **Goal:** Run Codex directly through its supported TypeScript SDK while preserving structured progress, validated output, usage, file-change evidence, cancellation, and runtime identity. - **Requirements:** R10-R22; AE1, AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. - **Dependencies:** U4. - **Files:** `packages/execution-service/src/worker/adapters/codex.ts`, `packages/execution-service/tests/unit/worker/adapters/codex.test.ts`, `packages/execution-service/tests/fixtures/execution/codex-events.jsonl`. -- **Approach:** Construct a fresh SDK thread in the invocation workspace with an isolated `CODEX_HOME` and scrubbed environment. Apply model, sandbox, network, approval, and writable-root settings only from the profile. Consume `runStreamed()` and pass an AbortSignal. Normalize agent messages, items, usage, failures, and file-change events while preserving the bounded native stream. -- **Execution note:** Drive the SDK through its executable override with a fixture Codex process before any credentialed smoke test. -- **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, and the SDK's `runStreamed`/AbortSignal contract. +- **Approach:** Depend directly on a pinned `@openai/codex-sdk` and fail worker readiness when it is unavailable or incompatible. Construct one fresh SDK thread in the invocation workspace with an isolated `CODEX_HOME` and a minimal allowlisted environment. Apply model, sandbox, network, approval, working-directory, writable-root, and a pinned `shell_environment_policy` only from the profile. The Codex runtime may receive its scoped provider credential, but model-initiated shell commands receive only named non-secret variables and never provider or worker-control credentials. Consume `runStreamed()`, pass the invocation `AbortSignal`, and pass the exact accepted result schema as per-turn `outputSchema`. Parse and validate the final JSON with the shared worker validator before publishing the canonical result Artifact. Normalize agent messages, items, usage, failures, and file-change events while preserving the bounded native stream. Never call `resumeThread`, pool threads, or reuse provider sessions after interruption. +- **Execution note:** Wrap the SDK behind an injectable factory and drive it through fixture events and its executable override before any credentialed smoke test. Use Promptfoo's provider and tests to enumerate observable edge cases, not as copied code or a runtime dependency. +- **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, the SDK's `startThread`/`runStreamed`/`AbortSignal`/`outputSchema` and shell-environment policy contracts, and Promptfoo's Codex provider tests for characterization of option forwarding, environment isolation, cancellation, structured output, and cleanup. - **Test scenarios:** - A successful stream exposes thread ID, progress, final response, token usage, native file-change items, and terminal completion. + - A structured request forwards the exact accepted schema to `outputSchema`; valid JSON becomes the fixed-name Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`, while missing, malformed, or schema-invalid output fails with a typed structured-result error. - Empty final response, turn failure, malformed JSONL, non-zero exit, unavailable runtime, and usage omission map to typed result/completeness fields. - - Covers AE6/AE12. Cancellation aborts the SDK process once; completion after cancel or stale fence cannot alter the selected terminal outcome. - - Profile sandbox, network, model, approval, working directory, and environment settings reach the SDK; caller input cannot override them. - - Two sequential invocations have disjoint `CODEX_HOME`, thread/session state, and writable roots; ambient selectors are removed. + - Covers AE6/AE12. A pre-aborted signal prevents start; in-flight cancellation aborts the SDK once; worker escalation proves descendant termination; completion after cancel or stale fence cannot alter the selected terminal outcome. + - Profile sandbox, network, model, approval, working directory, shell-environment policy, and environment settings reach the SDK; caller input cannot override them or supply raw Codex config. + - The Codex runtime receives only its scoped credential and minimal runtime environment. A model-initiated command that attempts to print provider/control credential names observes no values, and output, errors, and retained evidence contain none. + - Two sequential invocations create fresh threads with disjoint `CODEX_HOME`, session state, and writable roots; no resume or thread-persistence API is called. - Native diffs and shared Git evidence coexist without claiming identical attribution. -- **Verification:** Fixture-driven tests cover every supported event and failure shape, followed by an isolated credentialed repository smoke test when Codex credentials are available. +- **Verification:** Fixture-driven tests cover every supported event and failure shape, SDK option/schema/signal forwarding, environment isolation, fresh-thread behavior, and cleanup escalation, followed by an isolated credentialed repository smoke test when Codex credentials are available. -### U6. OpenCode backend adapter +### U6. Pi backend adapter -- **Goal:** Run OpenCode through its typed SDK and worker-owned loopback server while preserving session progress, output, usage/cost, diffs, permissions, cancellation, and disposal. +- **Goal:** Run Pi through strict RPC mode while preserving settled completion, schema-backed terminal output, usage/cost, tool progress, cancellation, and process cleanup. - **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. -- **Dependencies:** U4. -- **Files:** `packages/execution-service/src/worker/adapters/opencode.ts`, `packages/execution-service/tests/unit/worker/adapters/opencode.test.ts`, `packages/execution-service/tests/fixtures/execution/opencode-events.jsonl`. -- **Approach:** Start one loopback instance per invocation with isolated `OPENCODE_CONFIG`, `OPENCODE_CONFIG_DIR`, data/cache roots, scrubbed environment, profile configuration, and AbortSignal. Subscribe before prompting, create one session, resolve permission events from policy, collect message/session events and session diff, abort on cancellation, then delete the session and close the server in `finally`. Prevent the agent subprocess from reaching the worker control listener under the declared trust topology. -- **Execution note:** Inject SDK/server factories so protocol fixtures prove ordering and teardown without downloading or authenticating a real runtime. -- **Patterns to follow:** `src/core/profile/adapters/opencode.ts` for configuration/environment isolation and OpenCode's `createOpencode`, event subscription, session prompt/diff/abort APIs. -- **Test scenarios:** - - Successful execution collects text parts, assistant tokens/cost, session ID, events, and session diff before disposal. - - Subscription starts before prompt, ignores other session IDs, and finishes only after the target session becomes idle or errors. - - Covers AE6/AE12. Cancellation calls session abort once; late idle/completion cannot overwrite cancellation; server teardown remains idempotent. - - Covers AE9. Known permission events receive invocation-scoped `once`, `always`, or `reject` according to profile; `always` does not survive disposal and unknown types fail the adapter. - - Provider auth error, API error, aborted message, server-start timeout, SSE disconnect, and malformed SDK response map to typed failures. - - Sequential invocations have disjoint config/data/session roots; caller input cannot enable sharing, alter bind, select another project, or override provider/model/tools. -- **Verification:** Fixture tests prove session scoping, permission lifetime, cancellation races, and disposal, followed by an isolated credentialed repository smoke test when OpenCode credentials are available. - -### U7. Pi backend adapter - -- **Goal:** Run Pi through strict RPC mode while preserving settled completion, output, usage/cost, tool progress, cancellation, and process cleanup. -- **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. -- **Dependencies:** U4. -- **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. -- **Approach:** Spawn a supported Pi 0.85.x runtime with `--mode rpc --no-session`, an invocation-local `PI_CODING_AGENT_DIR`, profile model/provider, and scrubbed environment. Implement an LF-only JSONL parser rather than Node `readline`. Correlate responses, wait for `agent_settled`, read messages/stats, send RPC abort, and escalate process-group termination after the grace period. -- **Execution note:** Build parser and state-machine tests from captured RPC fixtures before process integration. -- **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing/cancellation contract. +- **Dependencies:** U4, U5. +- **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/src/worker/adapters/pi-policy-extension.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-policy-extension.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. +- **Approach:** Spawn a supported Pi 0.85.x runtime with `--mode rpc --no-session --no-extensions --no-builtin-tools`, an invocation-local `PI_CODING_AGENT_DIR`, profile model/provider, and scrubbed environment. Materialize the scoped provider credential only in Pi's supported invocation-local credential store with private permissions, not in the process environment. Explicitly load one worker-owned policy extension from outside the repository and verify the loaded extension/tool inventory before accepting work. The extension registers workspace-confined read/write/edit/search and sandboxed command tools plus the terminating result tool; command children receive only the phase allowlist and cannot access the Pi config/data roots. Implement an LF-only JSONL parser rather than Node `readline`. Correlate responses, wait for `agent_settled`, read messages/stats, send RPC abort, and escalate process-group termination after the grace period. For a structured request, generate the terminating tool from the exact accepted schema. The first observed tool call atomically claims the result candidate before validation: valid arguments produce the canonical Artifact; malformed or schema-invalid arguments fail the Task; later calls cannot replace the candidate. A prior cancel, deadline, or stale fence suppresses the call. Settling without a call fails as missing structured output. +- **Execution note:** Build parser, extension/tool-inventory, terminating-tool, policy-tool, and state-machine tests from captured RPC fixtures before process integration. Reuse the contract established by U5 rather than adding Pi-shaped public fields. +- **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing, `--no-extensions` plus explicit `--extension`, `--no-builtin-tools`, custom-tool, credential-store, and cancellation contracts. - **Test scenarios:** - Successful prompt acceptance streams message/tool events, stops on `agent_settled`, retrieves final messages/stats, and reports session ID, usage, and cost. + - A structured request exposes only the invocation-scoped terminating tool in addition to the policy tools. The first observed call claims the candidate; valid arguments produce the fixed-name Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; an invalid first call fails without replacement; a later duplicate cannot replace the result; a cancel/deadline/fence that wins first suppresses the call; and settled completion without a call fails as missing output. - LF framing preserves `U+2028`/`U+2029` inside JSON strings, accepts CRLF by stripping trailing CR, handles partial/multiple chunks, and rejects oversized/malformed records. - - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled events cannot overwrite the terminal fence. + - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled or terminating-tool events cannot overwrite the terminal fence. - Prompt rejection, agent error, aborted stop reason, retry/compaction sequence, premature exit, stderr overflow, and stats failure map truthfully. - - Sequential invocations have disjoint `PI_CODING_AGENT_DIR` and session state; caller input cannot send extension commands, steering/follow-up, arbitrary RPC commands, or override provider/model. -- **Verification:** Fixture and fake-process tests prove framing, correlation, settled completion, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when Pi credentials are available. + - Sequential invocations have disjoint `PI_CODING_AGENT_DIR`, tool registration, and session state; the provider credential exists only in the private invocation-local store; policy tools cannot read that root and their command children observe no provider/control credential; repository `.pi/extensions` and unrestricted built-in tools do not load; and caller input cannot send extension commands, steering/follow-up, arbitrary RPC commands, or override provider/model. +- **Verification:** Fixture and fake-process tests prove framing, correlation, deterministic terminating-tool selection, shared validation, exact extension/tool inventory, workspace confinement, command-environment and credential isolation, settled completion, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when Pi credentials are available. -### U8. Production registry, service packaging, and observability +### U7. Production registry, service packaging, and observability -- **Goal:** Compose exactly three production adapters and package independently runnable gateway and worker services with safe startup, health, shutdown, tracing, and reproducible containers. +- **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with safe startup, health, shutdown, tracing, and reproducible containers. - **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD11-KTD15. -- **Dependencies:** U3-U7. -- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. -- **Approach:** Register only Codex, OpenCode, and Pi through an explicit capability/availability map. Add gateway and worker entrypoints inside the private Node 22 workspace instead of the Node 18 CLI package. Validate config, store, workers, runtime pins, trust, quotas, and resource controls before readiness. Propagate `traceparent` and instrument every phase. Build a minimal gateway image with no provider runtimes and a one-execution worker image with exact runtime versions. +- **Dependencies:** U3-U6. +- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. +- **Approach:** Register only Codex and Pi through an explicit capability/availability map. Add gateway and supervised worker entrypoints inside the private Node 22 workspace instead of the Node 18 CLI package. Pin both runtimes as direct service dependencies, validate config, store, workers, runtime availability, supervisor boundary, orphan roots, trust, quotas, and resource controls before readiness, and never discover a missing provider only after Task acceptance. Propagate `traceparent` and instrument every phase. Build a minimal gateway image with no provider runtimes and a one-execution worker image whose init/runtime kills the complete execution boundary when the worker server exits. - **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. - **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. - **Test scenarios:** - - Registry exposes exactly Codex, OpenCode, and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects unknown backend IDs before workspace creation. - - Gateway and worker start from built service outputs, become ready only after dependencies pass, and stop gracefully on SIGTERM. + - Registry exposes exactly Codex and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects OpenCode or unknown backend IDs before workspace creation. + - Gateway and supervised worker start from built service outputs, become ready only after dependencies and orphan recovery pass, and stop gracefully on SIGTERM. - Gateway readiness fails for malformed auth, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable resource enforcement, or unsupported backend runtime. + - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. + - Killing the worker server while an adapter child and invocation root exist makes the supervisor kill the boundary; replacement readiness waits for root deletion or quarantine and never reuses it. - Trace context enters through A2A, crosses the private call, and correlates result identities; exporter failure cannot change Task status. - - Gateway image contains no Codex, OpenCode, Pi, Git workspace, or provider credential material. - - Worker image pins all runtimes, confines one workspace/config root, enforces deployment limits, and completes fake-provider health smoke tests. + - Gateway image contains no Codex, Pi, Git workspace, or provider credential material. + - Worker image pins both runtimes, confines one workspace/config root, excludes credentials from model tools, disables repository Pi extensions and unrestricted built-in tools, enforces deployment limits, and completes fake-provider health smoke tests. - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. -- **Verification:** The registry dispatches every adapter through the same worker contract; built services and images pass lifecycle/security smoke tests; CI and publication bind immutable image tags to the release commit. +- **Verification:** The registry dispatches both adapters through the same worker contract; built services and images pass lifecycle/security smoke tests; CI and publication bind immutable image tags to the release commit. -### U9. Cross-backend conformance, documentation, and release evidence +### U8. Cross-backend conformance, documentation, and release evidence - **Goal:** Prove the public contract and operational workflow end to end and document deployment without leaking backend details into callers. - **Requirements:** R1-R22; F1-F5; AE1-AE14. -- **Dependencies:** U1-U8. +- **Dependencies:** U1-U7. - **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. -- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases. Exercise gateway and worker as separate processes. Document extension/worker protocols, profiles, auth, trust boundary, storage/HA limits, source hardening, quotas, runtime requirements, cancellation races, evidence integrity, Artifact access, retention, observability, and troubleshooting. +- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases. Exercise gateway and supervised worker as separate processes. Document extension/worker protocols, the portable result-schema subset, fixed structured-result Artifact, four result states, profiles, auth, trust boundary, storage/HA limits, explicit lack of execution resume, worker-crash supervision/orphan recovery, source hardening, quotas, runtime requirements, cancellation races, evidence integrity, Artifact access, retention, observability, and troubleshooting. - **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. - **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, and Starlight guide/reference organization under `docs/src/content/docs/docs/`. - **Test scenarios:** - Covers AE1-AE14 through built services with a fake backend and official A2A client. - - The same mutation fixture passes through Codex, OpenCode, and Pi event fixtures and produces contract-equivalent normalized evidence. + - The same accepted schema, valid result, invalid result, missing result, and pre-output failure fixtures pass through Codex and Pi adapters with identical validator decisions, integrity states, and Artifact presence/shape while retaining distinct native evidence. - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; the one-execution worker serializes admitted work. - - Gateway restart, stream reconnect, lost dispatch acknowledgement, duplicate/out-of-order events, worker crash, lease expiry, cancellation race, provider failure, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome. - - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, control-endpoint probing, and secret-exfiltration fixtures are blocked within the documented reviewed-source boundary. + - Gateway restart, stream reconnect, lost dispatch acknowledgement, duplicate/out-of-order events, worker crash, lease expiry, cancellation race, provider failure, invalid structured output, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome without provider reattachment or replay. + - A worker SIGKILL with live descendants and an invocation root proves supervisor termination and pre-readiness deletion/quarantine; repeated gateway crashes cannot serve until startup terminalization completes. + - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, control-endpoint probing, provider-credential echo/read attempts, repository Pi extensions, unrestricted Pi built-in tools, and secret-exfiltration fixtures are blocked within the documented reviewed-source boundary. - Examples validate with production schemas and reference secrets only through environment variable names. - - Docs state one gateway replica, one execution per worker, reviewed mutual-trust sources, Node/runtime floors, and no hostile-code isolation claim. + - Docs state one gateway replica, one supervised execution per worker, reviewed mutual-trust sources, Node/runtime floors, ephemeral provider sessions, and no hostile-code isolation claim. - Opt-in real-provider smoke tests record backend/runtime versions and skip only when the named credential/runtime prerequisite is absent. - **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor, the full suites and docs pass, the official A2A client exercises every advertised operation, and release evidence records each available real backend plus explicit skipped prerequisites. @@ -616,15 +617,15 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| | Contract generation | U1 | Public extension and private worker schema generation report no drift; positive and negative fixtures pass. | -| Focused unit tests | U1-U8 | Active-unit tests pass with fault injection, state races, limits, cancellation, and cleanup. | -| Gateway/worker integration | U3-U4, U8-U9 | Built processes agree on fenced dispatch, sequencing, leases, Task persistence, Artifacts, shutdown, and cleanup. | -| Backend conformance | U5-U9 | One shared suite passes against Codex, OpenCode, and Pi adapters with fixture runtimes. | -| Credentialed provider smoke | U5-U7, U9 | Each available provider mutates a disposable exact-SHA repository; missing credentials/runtime are recorded as skipped prerequisites, never passing coverage. | -| A2A interoperability | U3, U9 | Official `@a2a-js/sdk` client passes immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, replay, cancel races, expiry, and owner isolation. | -| Security and abuse | U2-U4, U8-U9 | Malicious identity/source/artifact/resource fixtures prove auth-before-lookup, opaque owner keys, Git SSRF controls, phase-scoped secrets, quotas, quiescence, race-resistant capture, and trust-topology rejection. | -| Service packaging | U8-U9 | Root Node 18 install, private Node 22 build, gateway/worker smoke, and both container builds pass. | +| Focused unit tests | U1-U7 | Active-unit tests pass with fault injection, state races, limits, cancellation, and cleanup. | +| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on fenced dispatch, sequencing, leases, Task persistence, Artifacts, shutdown, and cleanup. | +| Backend conformance | U5-U8 | One shared suite passes against Codex and Pi adapters with fixture runtimes, including identical acceptance and validation of the versioned result-schema subset, four result states, and fixed Artifact shape. | +| Credentialed provider smoke | U5-U6, U8 | Each available provider mutates a disposable exact-SHA repository; missing credentials/runtime are recorded as skipped prerequisites, never passing coverage. | +| A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, replay, cancel races, expiry, and owner isolation. | +| Security and abuse | U2-U4, U7-U8 | Malicious identity/source/artifact/resource fixtures prove auth-before-lookup, opaque owner keys, Git SSRF controls, phase-scoped secrets, provider-credential exclusion from model tools, disabled repository Pi extensions/built-ins, policy-tool confinement, quotas, quiescence, race-resistant capture, and trust-topology rejection. | +| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, worker-crash containment/orphan recovery, and both container builds pass. | | Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | -| Documentation | U9 | `bun run docs:build` passes and examples validate against current schemas. | +| Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove protocol, durable aggregation, process isolation, fencing, cancellation, or cleanup integration. @@ -636,23 +637,22 @@ The authoritative behavioral proof is the built-process E2E path with the offici - Every R1-R22 requirement is implemented or explicitly shown in a passing conformance scenario. - Public Agent Card/extension and private worker schemas are stable, generated from one source, and consumable without importing root AllAgents CLI modules. -- Codex, OpenCode, and Pi pass the same backend conformance suite and preserve bounded native evidence through the closed registry. -- Gateway and worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. -- Authentication precedes lookup, quota precedes Task creation, aggregate commits cannot split claims/Tasks/Artifacts, and terminal fences survive races and restart. -- Cancellation/deadlines reach one native abort, process termination, quiescence, evidence, and cleanup for all three backends. -- Source hardening, phase-scoped secrets, one-execution trust policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end. +- Codex and Pi pass the same backend conformance suite, accept the same versioned result-schema subset, validate with the same shared validator, publish the same fixed structured-result Artifact shape and result states, and preserve bounded native evidence through the closed registry. +- Gateway and supervised worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. +- Authentication precedes lookup, quota precedes Task creation, aggregate commits cannot split claims/Tasks/Artifacts, startup recovery completes before serving, and terminal fences survive races and restart without claiming provider-session recovery. +- Cancellation/deadlines reach one native abort, process termination, quiescence, evidence, and cleanup for both backends; worker-process death triggers supervisor termination and pre-readiness orphan deletion or quarantine. +- Source hardening, phase-scoped secrets, provider-credential exclusion from model tools, disabled repository Pi extensions/unrestricted built-ins, Pi policy-tool confinement, one-execution trust policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end. - Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. -- Public documentation states supported topology, configuration, security boundary, storage/HA limitation, runtime pins, and deferred capabilities. +- Public documentation states supported topology, configuration, security boundary, storage/HA limitation, runtime pins, structured-result contract, worker-crash recovery, and deferred capabilities. - Abandoned experiments, unused adapters, compatibility shims, generated scratch files, retained test workspaces, and stale documentation are removed. ### Per unit -- U1: Public/worker schemas, digest vectors, state/fence rules, typed failures, and fixtures are generated and stable. -- U2: Auth, opaque owner isolation, aggregate idempotency, CAS settlement, pagination, restart, quotas, Artifact access, tombstones, and cleanup pass fault injection. +- U1: Public/worker schemas, result-schema subset, structured-result Artifact/states, digest vectors, fence rules, typed failures, and fixtures are generated and stable. +- U2: Auth, opaque owner isolation, aggregate idempotency, CAS settlement, pagination, startup recovery barrier, quotas, Artifact access, tombstones, and cleanup pass fault injection. - U3: Every advertised A2A operation agrees across stream and lookup while replay, fencing, and cancellation races preserve one Task. -- U4: Worker dispatch/source/setup/action/check/quiescence/evidence/cleanup lifecycle passes malicious and faulted disposable-repository scenarios. -- U5: Codex streaming, usage, native evidence, isolated roots, cancellation, and failure mapping pass adapter and applicable smoke verification. -- U6: OpenCode session/event/diff/permission isolation, abort, and disposal pass adapter and applicable smoke verification. -- U7: Pi strict JSONL framing, settled completion, stats, isolated roots, abort, and process cleanup pass adapter and applicable smoke verification. -- U8: Closed registry, Node-version separation, readiness, tracing, graceful shutdown, containers, and release artifacts work from built outputs. -- U9: Cross-backend E2E, A2A interoperability, abuse cases, examples, operator docs, changelog, and release evidence are complete. +- U4: Supervision, orphan recovery, worker dispatch/source/setup/action/check/quiescence/evidence/cleanup lifecycle pass malicious, crashed, and faulted disposable-repository scenarios. +- U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, usage, native evidence, minimal environment, model-tool credential exclusion, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. +- U6: Pi strict JSONL framing, deterministic terminating-tool output, exact policy-extension/tool inventory, disabled repository extensions and built-ins, credential-store and workspace confinement, settled completion, stats, isolated roots, abort, and process cleanup pass adapter and applicable smoke verification. +- U7: Closed registry, Node-version separation, runtime readiness, tracing, graceful shutdown, containers, and release artifacts work from built outputs. +- U8: Cross-backend E2E, A2A interoperability, abuse cases, examples, operator docs, changelog, and release evidence are complete. From d3df7ba9925c173ccdb1045cac32aa8a4f13d9ba Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Fri, 18 Sep 2026 17:57:05 +1000 Subject: [PATCH 05/12] docs(architecture): harden execution gateway contract --- ...-agent-execution-through-an-a2a-gateway.md | 19 +- ...0837-feat-coding-execution-gateway-plan.md | 384 ++++++++++-------- .../agent-host-protocol-decision-inputs.md | 6 +- 3 files changed, 227 insertions(+), 182 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 1ed60865..599d72e8 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -207,12 +207,23 @@ will not be adopted. Gateway calls propagate [W3C Trace Context](https://www.w3.org/TR/trace-context/) across HTTP and process -boundaries. AllAgents uses OpenTelemetry and OTLP for operational telemetry. +boundaries. AllAgents uses OpenTelemetry and OTLP for metadata-only operational +telemetry by default. An explicit allowlist limits structured logs and spans to +non-content operational metadata. Prompts and model outputs, tool arguments and +results, file bodies and source fragments, and secret-bearing attributes are +prohibited before export. A bounded filtering and redaction step must run before +any structured log or span processor so disallowed content cannot enter the +telemetry pipeline. + AllAgents-managed agent, model, and tool spans use [OpenInference](https://arize-ai.github.io/openinference/) semantic conventions -where corresponding attributes exist; useful backend-native attributes may be -retained alongside them. Consumer-owned evaluator spans may join the propagated -trace without becoming gateway-owned. +only for attributes that pass this allowlist. Backend-native attributes must +pass the same allowlist. Owner correlation is limited to an opaque identifier +appropriate for the telemetry operators' access; it does not expose caller +identity or grant access to a Task or Artifact. Telemetry access and retention +are governed separately from Task and Artifact access and retention. +Consumer-owned evaluator spans may join the propagated trace without becoming +gateway-owned. These standards are complementary: diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 60987efd..d373b358 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -55,14 +55,14 @@ The two initial runtimes expose different programmatic contracts. Codex provides - R1. The gateway implements A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; it implements streaming send and task subscription when the card advertises streaming. - R2. Every valid new request returns exactly one addressable Task. Direct-Message completion and follow-up messages to an existing Task are unsupported. Non-streaming send honors A2A `returnImmediately`; streaming always emits the durable Task first. -- R3. Every request and terminal Task uses one required, versioned AllAgents coding-execution extension URI. Unsupported required extension versions fail without fallback. +- R3. The public extension URI is `https://allagents.dev/a2a/extensions/coding-execution/v1`. The Agent Card advertises it as required; HTTP clients opt in with `A2A-Extensions`; each request sets `Message.extensions` to include the URI and puts the schema-defined request only at `Message.metadata[uri]`. Every terminal Task contains exactly one fixed-name `allagents.execution-integrity` Artifact whose `extensions` includes the URI and whose single `Part` contains the schema-defined integrity envelope in `data` with `mediaType: application/json`. The Task uses only the standard A2A fields and never adds `Task.extensions`. Unsupported or missing required extension versions fail without fallback. - R4. Terminal output and execution evidence are retrievable as Task Artifacts for the configured retention window even when the original stream disconnects. Active subscription emits the current Task snapshot then future events without promising replay of missed progress; terminal subscription returns the standard unsupported-operation error and callers use `GetTask`. **Caller identity, Task identity, and retention** -- R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. -- R6. Authentication, required-extension validation, request validation, source/profile authorization, quota admission, and deadline validation complete before Task creation. A caller-scoped invocation key, effective profile, authenticated owner, and canonical request digest then bind atomically to one Task; identical replay returns that Task and conflicting reuse is rejected without dispatch. -- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Task state and terminal Artifact metadata survive gateway restart. Every nonterminal Task present at startup settles failed once, its old attempt fence is invalidated, and stale worker events cannot overwrite it; the initial service never resumes or automatically replays interrupted provider work. +- R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. Production public ingress reaches the gateway through TLS terminated at the configured named trusted boundary; an unauthenticated loopback-only development listener is the sole plaintext exception. Remote gateway-worker links use mTLS or an explicitly configured equivalent authenticated encrypted overlay, while a same-host Unix socket is acceptable. The authenticated worker identity is bound to its route, capabilities, and attempt fence, and readiness fails for plaintext or identity-mismatched remote endpoints. +- R6. After authentication, required-extension checks, and bounded canonical parsing, the gateway first resolves the owner-scoped invocation claim. A retained claim compares the canonical caller request and result-schema digest against the originals and returns its existing Task only while its stored original effective-profile and schema bindings remain intact; a mismatch conflicts without dispatch. Current source/profile authorization, profile resolution/readiness, quota, and deadline checks apply only when atomically creating a new claim that binds the authenticated owner, canonical caller request digest, original effective-profile digest, original result-schema digest, and submitted Task. +- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Acceptance of the current worker fence moves a submitted Task to working before source materialization or setup, so a subsequent source/setup failure transitions from working to failed. Task state and terminal Artifact metadata survive gateway restart. Every nonterminal Task present at startup settles failed once, its old attempt fence is invalidated, and stale worker events cannot overwrite it; the initial service never resumes or automatically replays interrupted provider work. - R8. List operations implement all A2A filters, history bounds, page-size bounds, owner/query-bound cursor pagination, and descending status-update time. One immutable expiry logically hides the Task, claim, events, and artifacts before best-effort physical deletion; expired and unauthorized IDs are indistinguishable. - R9. Small deployments work without an external database. The built-in durable store supports one gateway replica, enforces per-owner/global admission and storage quotas, and reserves capacity for cancellation and terminal settlement; multi-replica storage is outside this delivery. @@ -71,48 +71,48 @@ The two initial runtimes expose different programmatic contracts. Codex provides - R10. Codex and Pi are the complete initial backend set behind one conformance contract, delivered Codex first and Pi second. OpenCode is deferred. (session-settled: user-directed.) - R11. A request selects a server-defined execution profile and may include one `allagents.result-schema/v1` schema for the terminal result: a bounded JSON Schema Draft 2020-12 subset with an object root, every object schema setting `additionalProperties: false`, every declared property listed in `required`, optional values represented by `null` unions, and only `type`, `properties`, `required`, `additionalProperties` with the value `false`, `items`, `enum`, `const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`. The extension version fixes byte, depth, property, and enum limits; admission rejects remote references, format-dependent validation, and unknown keywords; one shared validator governs schema admission and returned values. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. Requests cannot supply raw provider configuration. - R12. The only initial remote source form is a canonical credential-free HTTPS Git URL plus full commit object ID and optional repository-relative subdirectory. Acquisition revalidates destination policy for every connection, disables redirects and repository-controlled secondary fetch/exec features, uses hermetic Git configuration, and verifies that the fetched object is the requested commit before setup. -- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, and every model-initiated command or tool environment. +- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, structured logs/spans before processing or export, and every model-initiated command or tool environment. Credentialed profiles additionally require an OS-enforced provider/tool credential boundary: the credential-bearing provider runtime and model-invoked tools use distinct UID/process/mount policy that prevents tool access to provider processes, procfs entries, and backend config/data roots, or an equivalent credential broker keeps reusable credentials out of the agent runtime. Worker readiness fails when the declared boundary cannot be proved; environment filtering alone is not credential isolation. - R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. - R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. -- R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves all invocation descendants quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. An external supervisor terminates the complete execution boundary when the worker process crashes, and a replacement worker reaps or quarantines orphaned roots before readiness. +- R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves the complete invocation process set quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. If bounded termination escalation cannot prove quiescence, the worker persists termination as unknown/failed, poisons admission, and exits so the external supervisor destroys the complete process boundary; replacement readiness performs orphan recovery before accepting work. The same supervisor boundary handles a worker crash. **Evidence and observability** -- R17. Every terminal result contains an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. Pre-output source, setup, provider, cancellation, or deadline outcomes retain their primary Task classification and record `not_produced` secondarily. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. +- R17. Every terminal Task contains the required `allagents.execution-integrity` Artifact carrying an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one additional `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. `not_produced` is legal only before a result candidate is produced. Once validation selects `valid` or `invalid`, later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the fixed structured-result Artifact while the later phase remains the primary Task failure classification. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. - R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. -- R19. Gateway and worker spans propagate W3C Trace Context and export OpenTelemetry data. Telemetry is operational evidence, not the only durable result. +- R19. Gateway and worker calls propagate W3C Trace Context and export metadata-only OpenTelemetry data. One explicit pre-processor allowlist admits only bounded non-content operational metadata; OpenInference and backend-native attributes pass the same allowlist and bounded filtering/redaction before any structured log or span processor. Prompts, model outputs, tool arguments/results, file bodies, source fragments, and secret-bearing attributes are prohibited before export. Owner correlation uses only an opaque identifier appropriate to telemetry-operator access, never caller identity or Task/Artifact authorization. Telemetry access and retention are configured separately from Task and Artifact access and retention, and telemetry is neither durable result truth nor required for terminal lookup. **Ownership and safety boundary** - R20. The gateway executes one coding request. It does not own eval configuration, datasets, repetition, scoring, retry policy, experiment scheduling, or a durable evaluation Run ledger. -- R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. Profiles that claim hostile-source or cross-tenant isolation are rejected until a stronger per-invocation UID, mount, PID, network, and credential boundary is configured. +- R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. R13's narrow OS-enforced provider/tool credential boundary is required for credentialed profiles but does not claim hostile-source or cross-tenant isolation. Profiles making either stronger claim are rejected until a full per-invocation UID, mount, PID, network, and credential isolation boundary is configured. - R22. Gateway admission and worker execution enforce profile limits for request rate, active/retained Tasks, subscriptions, stored bytes, source transfer/expansion, files/inodes, workspace bytes, CPU, memory, PIDs, network, phase deadlines, events, logs, and artifacts. Exhaustion is scoped to one invocation or owner and leaves capacity for terminalization and cleanup. ### Key Flows - F1. **Admit, create, and stream an execution** - **Actors:** A1, A2, A3, A4. - - **Trigger:** A caller sends a text Message with the required extension, immutable source, profile, invocation key, deadline, and optional bounded result schema. - - **Steps:** Authenticate; validate and authorize the complete request and result-schema subset; reserve quota; atomically claim idempotency and create a submitted Task; dispatch a fenced worker attempt; materialize and verify source; execute the selected backend; validate structured output with the shared validator when requested; persist progress before emission; terminalize with the fixed-name structured-result Artifact only for a valid result and with evidence Artifacts after quiescence and cleanup. + - **Trigger:** A caller opts into `https://allagents.dev/a2a/extensions/coding-execution/v1` and sends a text Message whose `extensions` includes that URI and whose `metadata[uri]` contains the immutable source, profile, invocation key, deadline, and optional bounded result schema. + - **Steps:** Authenticate, check extension negotiation, and bounded-canonicalize the request; resolve an owner-scoped retained claim and return or conflict against its original request/profile/schema bindings before mutable admission checks. For a new claim only, validate current source/profile authorization, profile/readiness, quota, and deadline; atomically create the claim and submitted Task; dispatch a fenced worker attempt; accept the current fence and move the Task to working; materialize and verify source; execute the selected backend; validate structured output with the shared validator when requested; persist progress before emission; and terminalize with the required integrity Artifact plus the fixed-name structured-result Artifact only for a valid result after quiescence and cleanup. - **Outcome:** `returnImmediately: true` returns the durable current Task, false/unset waits for terminal state, and streaming starts with that Task before ordered updates. - **Covered by:** R1-R22. - F2. **Replay or reconnect to an invocation** - **Actors:** A1, A2. - **Trigger:** The owner repeats an invocation key or subscribes after a stream disconnect. - - **Steps:** Recompute the canonical digest; reject a conflict; return the existing Task; for active streaming replay/subscription emit its current snapshot then future events; for a terminal Task return it through send replay or `GetTask` without dispatch. + - **Steps:** After authentication and bounded canonical parsing, resolve the owner-scoped claim; compare the request and schema digest with the stored originals and verify the retained Task's original effective-profile/schema bindings without resolving the current profile. Reject a mismatch; otherwise return the existing Task before current authorization, quota, readiness, profile, or deadline checks. For active streaming replay/subscription emit its current snapshot then future events; for a terminal Task return it through send replay or `GetTask` without dispatch. - **Outcome:** Retries do not multiply agent work, and reconnect never promises transient event replay. - **Covered by:** R4, R6-R8. - F3. **Cancel or time out an execution** - **Actors:** A1, A2, A3, A4. - **Trigger:** The caller invokes `CancelTask`, the effective deadline expires, or gateway shutdown claims cancellation. - - **Steps:** Atomically record the first cancellation source; if dispatch never occurred, prove no workspace exists; otherwise send one fenced worker cancel, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. - - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses late provider success and fails instead of claiming canceled when termination or cleanup cannot be verified. + - **Steps:** Atomically record the first cancellation source; send one revisioned fenced worker cancel even when dispatch delivery is unconfirmed, so an unseen attempt is tombstoned before any delayed dispatch can create a workspace. If work exists, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. + - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses stale dispatch and late provider success. If bounded escalation cannot prove the complete invocation process set empty, the Task fails rather than claiming canceled, the worker poisons admission and exits, and its supervisor destroys the boundary. - **Covered by:** R7, R14, R16-R18. - F4. **Settle after gateway or worker loss** - **Actors:** A2, A3. - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, a live worker loses its lease, or a worker process crashes. - - **Steps:** Invalidate the attempt fence and settle every affected Task failed once without provider-session reattachment or automatic replay. A live worker that loses its lease self-aborts and cleans. On worker-process crash, the external supervisor terminates the complete execution boundary; the replacement worker proves termination, then reaps or quarantines orphaned roots before readiness. Reject late events/results and record termination and filesystem cleanup separately as complete only when the responsible boundary proves each outcome. - - **Outcome:** One Task has one terminal result, interrupted work is never presented as resumed, no stale worker can overwrite durable truth, and a crashed worker cannot leave an unowned process or reusable workspace. + - **Steps:** Invalidate the attempt fence and settle every affected Task failed once without provider-session reattachment or automatic replay. A live worker that loses its lease self-aborts and cleans. On worker-process crash or unproved quiescence after bounded escalation, poison admission and exit the worker so the external supervisor terminates the complete execution boundary; the replacement worker proves termination, then reaps or quarantines orphaned roots before readiness. Reject late events/results and record termination and filesystem cleanup separately as complete only when the responsible boundary proves each outcome. + - **Outcome:** One Task has one terminal result, interrupted work is never presented as resumed, no stale worker can overwrite durable truth, and a failed quiescence proof cannot leave the poisoned worker available for another reservation. - **Covered by:** R7, R9, R14, R16-R18, R21-R22. - F5. **Expire retained execution data** - **Actors:** A1, A2. @@ -124,27 +124,27 @@ The two initial runtimes expose different programmatic contracts. Codex provides ### Acceptance Examples - AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, an exact Git SHA, and an optional result schema, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same validated output and evidence Artifacts. -- AE2. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with the same canonical request, then the gateway returns the original Task without a second worker dispatch. -- AE3. **Covers R6.** Given an existing Task, when its owner reuses the invocation key with a different prompt, source, profile, or deadline, then the gateway rejects the request and leaves the original Task unchanged. +- AE2. **Covers R6.** Given a retained Task whose original absolute deadline has passed or whose profile is now disabled, changed, or no longer authorized for new work, when its owner reuses the invocation key with the same canonical request and result schema, then the gateway returns the original Task from its stored original bindings before mutable admission checks and makes no second worker dispatch. +- AE3. **Covers R6.** Given a retained Task, when its owner reuses the invocation key with a different prompt, source, profile ID, deadline, or result schema, or the stored original profile/schema binding is inconsistent, then the gateway rejects the request and leaves the original Task unchanged. - AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. -- AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository, when the worker verifies source, then provider execution never starts and the Task fails with source-verification and cleanup evidence. -- AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. -- AE7. **Covers R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. -- AE8. **Covers R4, R7, R19.** Given a caller disconnects during work, when it subscribes again, then it receives the current Task and future updates without duplicate dispatch; telemetry loss does not affect later terminal lookup. +- AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository or setup fails, when the worker has already accepted the current fence, then provider execution never starts, the selected public trace is `Submitted -> Working -> Failed`, and the Task retains source/setup-failure and cleanup evidence. +- AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. Given cancel reaches a worker before its delayed dispatch, the worker tombstones the unseen attempt and the stale dispatch creates no workspace or provider process. +- AE7. **Covers R3, R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both publish the required fixed-name integrity Artifact at the schema-defined extension carrier, validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. +- AE8. **Covers R4, R7, R19.** Given canary secrets and cross-owner content fragments in prompts, model output, tool arguments/results, source files, stale events, and errors, when agent, model, tool, stale-event, and error telemetry is processed, then the exporter receives only allowlisted bounded metadata plus the correct opaque owner correlation and receives none of those canaries, fragments, or raw caller identities. Given a caller or exporter disconnects during work, reconnect still returns the current Task and future updates without duplicate dispatch, and telemetry loss does not affect terminal lookup. - AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. -- AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. +- AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. Given output validation has already selected `valid` or `invalid` and a later check or mandatory-evidence phase fails, the failed Task preserves that result state and a valid result preserves its one fixed structured-result Artifact; only a failure before candidate production records `not_produced`. - AE11. **Covers R6, R22.** Given invalid input or exhausted admission quota, the gateway returns a request/resource error and creates no Task; given capacity disappears after durable acceptance, the retained Task fails at dispatch and replay returns it without retry. -- AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only safe operator telemetry. Given the worker is killed with live descendants and an invocation root, its supervisor terminates the execution boundary and the replacement worker reaps or quarantines the root before readiness without changing the failed Task. +- AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only allowlisted metadata-only operator telemetry. Given bounded escalation cannot stop a descendant that starts a new session and ignores graceful signals, the worker persists termination unknown/failed, refuses another reservation, exits, and its supervisor destroys the boundary; replacement readiness performs orphan recovery without changing the failed Task. - AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. -- AE14. **Covers R21-R22.** Given a profile requests pooled hostile-source or cross-tenant execution, startup/admission rejects it; a reviewed single-trust-domain profile runs one bounded execution without exposing worker control credentials to the child environment. +- AE14. **Covers R5, R13, R21-R22.** Given a production public listener or remote worker route lacks its configured trusted transport or authenticated peer identity, readiness fails; a same-host Unix worker socket is accepted. Given a credentialed reviewed-domain profile, model tools cannot inspect provider process environments, process listings, backend config/data roots, or exfiltrate provider/control credentials across the configured OS boundary. Hostile-source or cross-tenant claims remain rejected. ### Success Criteria -- The official A2A JavaScript client can discover the card and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, replay, cancel, and expiry behavior against the built service. +- The official A2A JavaScript client can discover the required extension, negotiate it through `A2A-Extensions`, use the standard Message and Artifact extension carriers, and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, retained replay, cancel, and expiry behavior against the built service without `Task.extensions`. - One conformance fixture passes unchanged through the Codex and Pi adapters. -- Admission, replay, fencing, cancellation races, restart terminalization without resume, supervised worker-crash cleanup, authorization isolation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. +- Admission, retained replay, monotonic worker commands, fencing, acceptance-before-materialization source/setup failure, cancellation races, trace-order/fence/multiplicity constraints, failed-quiescence recycling, restart terminalization without resume, supervised worker-crash cleanup, trusted transports, authorization isolation, metadata-only telemetry export, OS-enforced provider/tool credential separation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. - The gateway image contains no coding-agent runtime and cannot access worker workspace roots. -- The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools receive no provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed or crashed attempt. +- The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools are OS-isolated from provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed, failed-quiescence, or crashed attempt. ### Scope Boundaries @@ -153,9 +153,9 @@ The two initial runtimes expose different programmatic contracts. Codex provides - A2A 1.0 HTTP+JSON and SSE streaming. - One versioned AllAgents coding-execution extension and one versioned private worker protocol. - Codex and Pi backends. -- Built-in bearer authentication with OIDC/JWT and static service-token modes. -- Single-replica durable file storage, authenticated Artifact retrieval, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. -- Reviewed repositories in one configured mutual-trust domain per worker deployment. +- Built-in bearer authentication with OIDC/JWT and static service-token modes behind the named production TLS boundary. +- Single-replica durable file storage, authenticated Artifact retrieval, authenticated encrypted remote worker transport or same-host Unix sockets, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. +- Reviewed repositories in one configured mutual-trust domain per worker deployment, with the narrow OS-enforced provider/tool credential boundary required for credentialed profiles. **Deferred to follow-up work** @@ -188,6 +188,10 @@ The two initial runtimes expose different programmatic contracts. Codex provides - [Pi CLI reference](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/README.md#cli-reference) - [Pi extension API](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/extensions.md) - [Pi provider credentials](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/providers.md) +- [Buzz pure Kubernetes state classifier](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-backend-kubernetes/src/classify.rs) +- [Buzz non-secret intent fingerprint](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-backend-kubernetes/src/intent.rs) +- [Buzz conformance coverage checker](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-conformance/src/checker.rs) +- [Buzz bounded process-tree cancellation](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-dev-mcp/src/shell.rs) --- @@ -195,20 +199,20 @@ The two initial runtimes expose different programmatic contracts. Codex provides ### Key Technical Decisions -- KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. The decorator owns admission, canonical Task reservation, idempotent replay, stream snapshot selection, and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. -- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes both versioned contracts, generated JSON Schemas, bounds, and fixtures. The public contract carries the `allagents.result-schema/v1` closed subset, its canonical digest, four structured-result states, and the fixed `allagents.structured-result` Artifact containing one A2A `Part` with `data` and `mediaType: application/json`. The worker protocol carries attempt identity, profile digest, dispatch acceptance, monotonic event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. -- KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates the invocation claim and submitted Task together, stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. -- KTD4. **Authenticate at HTTP ingress before A2A storage or dispatch.** Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. Unauthenticated mode is allowed only on a loopback listener. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Governs R5-R6, R13. -- KTD5. **Use fenced, separately deployable gateway and worker services.** The gateway owns A2A and durable Task/results truth; the worker owns ephemeral execution attempts, workspaces, and provider processes. Every dispatch has a gateway-generated attempt ID, lease ID/epoch, short-lived capability, and event sequence. Workers idempotently accept duplicate delivery of the same attempt, reject conflicting attempts, and gateways ignore stale/out-of-order events and late terminal results. Governs R7, R10-R16, R21-R22. -- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Worker-process exit makes the external supervisor terminate the complete execution boundary; before readiness the replacement worker proves termination and reaps or quarantines orphaned invocation roots. Termination and filesystem cleanup remain separate outcomes and are unknown until proved. The gateway never reattaches to or resumes a provider session. Caller stream disconnect never affects the lease. Governs R7, R14, R16-R18. +- KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. After authentication, required-extension checks, and bounded canonical parsing, the decorator resolves an owner-scoped retained claim before mutable admission; identical replay bypasses current profile/deadline/quota/readiness checks and any new SDK Task/bus allocation. New requests then pass mutable admission and canonical Task reservation. The decorator also owns stream snapshot selection and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. +- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes `https://allagents.dev/a2a/extensions/coding-execution/v1`, its standard Agent Card/header/Message/Artifact negotiation, `Message.metadata[uri]` request location, and the single-Part `allagents.execution-integrity` Artifact data location; no schema or implementation adds `Task.extensions`. The public contract also carries the `allagents.result-schema/v1` closed subset, its canonical digest, four structured-result states, and the separate fixed `allagents.structured-result` Artifact. The worker protocol carries worker identity, attempt identity, profile digest, monotonic command revision and tombstone state, dispatch acceptance, event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. +- KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates a new invocation claim and submitted Task together after mutable admission, storing the canonical caller request and the original effective-profile and result-schema digests needed for retained replay. It stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. +- KTD4. **Authenticate at a named trusted HTTP ingress before A2A storage or dispatch.** Production traffic reaches the gateway through TLS terminated by the configured gateway or named trusted reverse-proxy boundary; plaintext is allowed only for an unauthenticated loopback development listener. Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Readiness rejects a production public URL whose trusted TLS boundary is absent or inconsistent. Governs R5-R6, R13. +- KTD5. **Use fenced, separately deployable gateway and worker services.** Remote gateway-worker routes use mTLS or an explicitly equivalent authenticated encrypted overlay; a same-host Unix socket is acceptable. The authenticated worker identity is pinned to the configured route/capability set, and every short-lived attempt capability is bound to that identity, attempt ID, lease ID/epoch, and fence. Each worker keeps one minimal durable monotonic command record scoped to its worker identity and lease: `Cancel(attempt, fence, revision)` tombstones even an unseen attempt, and `Dispatch` for a tombstoned or lower-revision attempt is rejected before workspace creation. Dispatch/cancel I/O conditionally verifies the persisted command/outbox revision immediately before any mutating or terminating effect. Duplicate delivery is idempotent; conflicting, stale, out-of-order, or identity-mismatched commands/events are rejected. Gateway and worker transition selectors remain pure and executors re-enter from persisted or freshly observed state. This record is worker-local fence state, not a new durable execution subsystem. Governs R5, R7, R10-R16, R21-R22. +- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Worker-process exit makes the external supervisor terminate the complete execution boundary. If bounded escalation cannot prove the complete invocation process set empty, the worker records termination unknown/failed, poisons admission, and exits rather than accepting another reservation; its supervisor destroys the boundary. Before readiness the replacement proves termination and reaps or quarantines orphaned invocation roots. Production readiness accepts a dedicated worker container process namespace under a minimal init/reaper as the baseline; a non-container deployment must prove an equivalent systemd/cgroup boundary. The gateway never reattaches to or resumes a provider session. Governs R7, R14, R16-R18. - KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, optional structured result, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, schema validation, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | pi` registry is the only production dispatch point. Governs R10-R11, R14-R18, R21-R22. -- KTD8. **Use each provider's supported automation surface directly.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, consumes streamed events, and applies a pinned shell-environment policy that excludes provider/control credentials from model-initiated commands. Pi uses `pi --mode rpc --no-session --no-extensions --no-builtin-tools` with a strict LF-delimited JSON parser, `agent_settled`, `get_session_stats`, RPC abort, an invocation-local credential store rather than credential environment variables, and one explicitly loaded worker-owned policy extension outside the repository. That extension supplies workspace-confined filesystem/command tools and the terminating result tool; no repository extension or unrestricted built-in tool loads. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. -- KTD9. **Make profiles the policy boundary.** Requests select a profile ID and may provide only an `allagents.result-schema/v1` schema. They cannot override backend credentials, executable paths, provider config, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. Profile digests and the canonical result-schema digest enter idempotency and provenance. Governs R6, R11-R16, R21-R22. -- KTD10. **Capture Git and provider evidence as separate layers after quiescence.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded layer. Neither layer is promoted as exact causality when incomplete. Governs R16-R18. +- KTD8. **Use each provider's supported automation surface directly behind the credential boundary.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, and consumes streamed events. Pi uses strict RPC with an invocation-local credential store and one explicitly loaded worker-owned policy extension; repository extensions and unrestricted built-ins never load. For either adapter, a credentialed provider runtime is separated from every model-invoked tool by the R13 OS-enforced UID/process/mount boundary or an equivalent credential broker; shell-environment filtering is defense in depth, not the boundary. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. +- KTD9. **Make profiles the new-admission policy boundary.** Requests select a profile ID and may provide only an `allagents.result-schema/v1` schema. They cannot override backend credentials, executable paths, provider config, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. For a new claim, resolve a versioned canonical `EffectiveProfileIntent`, compute its digest without resolved secrets or per-attempt state, and persist it with the canonical caller request and result-schema digest. Retained replay compares those stored original bindings and never substitutes or re-resolves the current profile. Governs R6, R11-R16, R21-R22. +- KTD10. **Keep durable evidence and operational telemetry as separate bounded layers.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded evidence layer; neither Git nor provider evidence is promoted as exact causality when incomplete. Telemetry is a third, non-durable metadata-only channel: one small shared pre-export sanitizer applies an explicit operational-metadata allowlist plus bounded filtering/redaction before every structured log or span processor, and only opaque owner correlation may cross the separately governed operator boundary. OpenInference and backend-native attributes receive no bypass. This is an export guard, not a telemetry framework or alternate evidence store. Governs R13, R16-R19. - KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. -- KTD12. **Separate terminal integrity from optional evidence bodies.** Identity, action outcome, the four-state structured-result record and fixed Artifact rule, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance must validate before terminal publication. A pre-output failure records `not_produced` without replacing its primary phase classification. Predictable budget truncation/redaction of logs, diffs, native events, or produced-file bodies may preserve completion with explicit metadata; capture failure that breaks the integrity kernel fails in the evidence phase. Governs R4, R17-R18. +- KTD12. **Separate terminal integrity from optional evidence bodies.** The fixed `allagents.execution-integrity` Artifact validates identity, action outcome, the four-state structured-result record, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance before terminal publication. `not_produced` applies only before result-candidate production. Once validation selects `valid` or `invalid`, a later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the separate fixed structured-result Artifact while retaining the later phase as the primary Task failure. Predictable optional-body truncation/redaction may preserve completion; failure that breaks the integrity kernel fails in the evidence phase. Governs R3-R4, R17-R18. - KTD13. **Harden Git acquisition as a network security boundary.** Accept canonical HTTPS origins only. Use hermetic Git configuration, disable redirects, proxies, helpers, hooks, filters, LFS smudge, submodule recursion, alternates, and non-HTTPS protocols. Revalidate normalized host/address policy for every connection, never forward credentials across origins, and verify the full object ID resolves to a commit fetched from the approved remote. Governs R12-R13, R22. -- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Provider/source credentials are absent from setup/check phases, model-initiated commands and tools, and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load, and its replacement tools confine paths to the invocation workspace and spawn commands with the phase allowlist. Stronger isolation is a separate sandbox-driver capability. Governs R13, R16, R21-R22. +- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Credentialed profiles still require R13's narrower OS-enforced provider/tool separation: model tools cannot inspect provider processes, procfs entries, or backend config/data roots, and readiness fails without that capability. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load. This credential boundary does not imply hostile-source or cross-tenant isolation; that stronger sandbox-driver capability remains deferred. Governs R13, R16, R21-R22. - KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. ### High-Level Technical Design @@ -217,10 +221,10 @@ The two initial runtimes expose different programmatic contracts. Codex provides ```mermaid flowchart TB - Caller[Authenticated A2A caller] -->|HTTP+JSON / SSE| Gateway[execution-service gateway] - Gateway --> Auth[Auth, admission, profile policy] + Caller[Authenticated A2A caller] -->|TLS at named trusted ingress| Gateway[execution-service gateway] + Gateway --> Auth[Auth, retained replay, new admission] Gateway --> Store[Generation-based Task and Artifact store] - Gateway -->|Fenced private protocol| Worker[Single-execution worker] + Gateway -->|mTLS/authenticated overlay or same-host Unix socket| Worker[Single-execution worker] Worker --> Source[Hardened Git acquisition] Worker --> Registry[Closed backend registry] Registry --> Codex[Codex SDK] @@ -241,31 +245,37 @@ sequenceDiagram participant W as Worker participant B as Backend adapter - C->>G: SendMessage + required extension - G->>G: Authenticate, validate, authorize, quota, deadline - G->>S: Atomic claim + submitted Task - alt identical replay - S-->>G: Existing Task and current fence - G-->>C: Existing Task; follow active future events only - else new accepted Task + C->>G: SendMessage + header/Message extension + metadata[uri] + G->>G: Authenticate, check extension, canonicalize within bounds + G->>S: Resolve owner-scoped invocation claim + alt retained identical replay + S-->>G: Existing Task + original request/profile/schema bindings + G-->>C: Existing Task before current admission checks + else conflicting retained claim + G-->>C: Conflict; existing Task unchanged + else no retained claim + G->>G: Current authorization, profile/readiness, quota, deadline + G->>S: Atomic new claim + submitted Task + original digests S-->>G: Task + attempt/lease fence - G->>W: Dispatch(attempt, fence, profile, source, deadline) + G->>W: Dispatch(attempt, fence, command revision) + W->>W: Verify command record before workspace creation W-->>G: Accepted(attempt, fence) W->>W: Materialize, verify, setup, baseline - W->>B: Invoke with isolated roots and policy + W->>B: Invoke with isolated roots and credential boundary B-->>W: Progress, usage, native evidence W-->>G: Sequenced fenced progress G->>S: Compare-and-swap Task generation opt cancellation or deadline wins C->>G: CancelTask G->>S: Persist cancellation intent once - G->>W: Fenced cancel + G->>W: Cancel(attempt, fence, newer command revision) + W->>W: Persist tombstone before effects W->>B: Native abort end W->>W: Stop descendants, capture evidence, cleanup W-->>G: Fenced terminal result G->>S: Store blobs then atomically commit terminal manifest - G-->>C: Terminal status and Artifacts + G-->>C: Terminal status and extension Artifacts end ``` @@ -276,10 +286,10 @@ stateDiagram-v2 [*] --> Submitted: claim and Task committed Submitted --> Working: worker accepts current fence Submitted --> Canceled: cancellation proves no workspace exists - Submitted --> Failed: dispatch, restart, or source failure + Submitted --> Failed: dispatch or restart failure Submitted --> Rejected: accepted policy refusal before work Working --> Completed: integrity kernel and cleanup validate - Working --> Failed: provider, check, evidence, cleanup, crash, or restart failure + Working --> Failed: source, setup, provider, check, evidence, cleanup, crash, or restart failure Working --> Rejected: known profile permission denial after stop and cleanup Working --> Canceled: cancellation wins and stop/cleanup verify Completed --> [*] @@ -296,12 +306,14 @@ Terminal states are immutable. Cancellation intent, termination, evidence captur stateDiagram-v2 [*] --> Admitted Admitted --> Dispatching - Dispatching --> Running: current fence accepted - Dispatching --> Terminalizing: dispatch rejected or unknown + Dispatching --> Running: current command revision accepted + Dispatching --> Terminalizing: dispatch rejected, tombstoned, or unknown Running --> CancelRequested: caller, deadline, shutdown, or lease expiry Running --> Quiescing: provider and checks finish CancelRequested --> Quiescing - Quiescing --> CapturingEvidence: descendants verified stopped + Quiescing --> CapturingEvidence: complete process set verified empty + Quiescing --> Poisoned: bounded escalation cannot prove empty + Poisoned --> [*]: persist unknown/failed and exit boundary CapturingEvidence --> Cleaning Cleaning --> Terminalizing Terminalizing --> Retained @@ -350,6 +362,7 @@ packages/execution-service/ registry.ts codex.ts pi.ts + pi-rpc.ts pi-policy-extension.ts tests/ fixtures/execution/ @@ -370,64 +383,69 @@ docs/src/content/docs/ ### Configuration Contract -- Gateway configuration defines listener/public URL, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker endpoints, internal capability secrets, and profiles. -- Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, and evidence budgets. -- Worker configuration fixes a private listener, one-execution concurrency, workspace root, execution-supervisor mechanism, pre-readiness orphan policy, lease grace, backend runtime constraints, trust domain, resource-control capability, and request/result limits. -- Configuration contains environment-variable names but never secret values. Startup resolves the complete graph, verifies that profile claims do not exceed deployment capabilities, and becomes ready only when store, workers, runtimes, quotas, and free-space reserves pass. +- Gateway configuration defines the listener/public URL, a named trusted TLS termination boundary for production ingress, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker routes, internal capability secrets, and profiles. Each remote worker route declares mTLS or an explicitly equivalent authenticated encrypted overlay, pinned worker identity/capabilities, and trust material; a same-host route may declare a Unix socket. Plaintext remote URLs are invalid. +- Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, evidence budgets, and the required provider/tool credential-boundary capability for credentialed execution. +- Worker configuration fixes a private listener, worker identity, one-execution concurrency, workspace root, minimal worker-local command-record location, execution-supervisor mechanism, pre-readiness orphan policy, lease grace, backend runtime constraints, trust domain, resource-control and provider/tool credential-boundary capabilities, and request/result limits. +- Production worker readiness requires authenticated route identity, protected remote transport or a same-host Unix socket, an enforceable credential boundary for every credentialed profile, and a supervisor that proves complete descendant termination and root ownership. The supported supervisor baseline is a dedicated worker container process namespace under a minimal init/reaper; bare-host deployment requires an equivalent systemd/cgroup mechanism. +- Telemetry configuration defines the OTLP destination, filtering/redaction bounds, opaque owner-correlation derivation, and telemetry-specific operator access and retention. The service version fixes the metadata allowlist; configuration cannot extend it to prompt/output/tool/source/file-body attributes, secret-bearing fields, raw caller identity, or unfiltered backend-native/OpenInference attribute passthrough. +- Configuration contains environment-variable names but never secret values. Startup resolves the complete graph and becomes ready only when trusted ingress, worker transports/identities, store, runtimes, quotas, free-space reserves, supervisor/orphan recovery, and declared profile capabilities pass. Any unprotected remote endpoint or unproved credential/supervisor boundary fails readiness. ### Error and Status Mapping | Condition | A2A result | Required extension detail | |---|---|---| -| Authentication, malformed/unsupported extension, invalid source/profile, unauthorized policy, expired deadline, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | -| Identical invocation replay | Existing Task | No new Task, worker attempt, or quota reservation | -| Conflicting invocation key | Operation error; no new Task | Conflict code; existing Task unchanged | +| New-admission authentication, malformed/unsupported extension carrier, invalid source/profile, unauthorized policy, expired deadline, current-profile/readiness failure, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | +| Identical retained invocation replay | Existing Task | Returned from stored original request/profile/schema bindings before current deadline, quota, authorization, readiness, or profile checks; no new Task, worker attempt, or quota reservation | +| Conflicting invocation key or inconsistent stored binding | Operation error; no new Task | Conflict code; existing Task unchanged | | Worker capacity loss after acceptance | `TASK_STATE_FAILED` | `dispatch/capacity_exhausted`, retriable fact, no workspace created; gateway does not retry | | Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | | Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | | Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | -| Source, setup, provider, check, mandatory evidence, worker crash, or infrastructure failure | `TASK_STATE_FAILED` | Typed primary phase, safe message, retriable fact, structured result `not_produced` when requested, separate termination/cleanup/completeness | -| Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` or `structured_result/invalid`, no structured-result Artifact | -| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors, native abort, structured result `not_produced` unless already valid, termination, cleanup | +| Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary source/setup/provider/dispatch/crash/infrastructure phase, safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness | +| Check, mandatory-evidence, cleanup, crash, or infrastructure failure after result validation | `TASK_STATE_FAILED` | Preserve selected `valid` or `invalid`; preserve exactly one fixed structured-result Artifact for `valid`; later phase remains primary failure | +| Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` with `not_produced`, or `structured_result/invalid` with `invalid`; no structured-result Artifact | +| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors and native abort; use `not_produced` only before a candidate, otherwise preserve `valid`/`invalid` and the valid Artifact; record termination and cleanup | | Cancellation loses to terminal completion | Existing terminal Task / `TaskNotCancelableError` | No state mutation or second abort | -| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Output plus complete required evidence; a requested valid result uses the fixed-name Artifact with one A2A `Part` containing `data` and `mediaType: application/json` | +| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Required extension integrity Artifact plus complete evidence; a requested valid result uses the separate fixed-name Artifact with one A2A `Part` containing `data` and `mediaType: application/json` | | Successful action with allowed bounded optional-evidence gap | `TASK_STATE_COMPLETED` | Per-dimension incomplete flag, reason, original/captured size, digest and redaction/truncation flags | -| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid, structured result `not_produced` unless already committed, and cleanup unknown unless proven | +| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid; use `not_produced` only before a candidate, otherwise preserve selected state and valid Artifact; cleanup unknown unless proven | | Retention expiry | Not found | Aggregate logically hidden before physical deletion; Artifact URL also invalid | ### Phased Delivery -1. Create the private Node 22 service package and freeze the public extension, portable result-schema subset, structured-result Artifact, worker protocol, profiles, fixtures, and error vocabulary. -2. Build authenticated durable A2A Task handling and fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. -3. Build the supervised single-execution worker lifecycle, pre-readiness orphan reaper, and hardened source/evidence handling against a fake adapter. -4. Add the direct Codex SDK adapter and prove structured output, cancellation, provider-credential exclusion from model commands, environment isolation, and native evidence. -5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing confined tools plus the terminating result tool. -6. Package the services and run cross-backend, security, process, and A2A conformance before enabling a consumer. +1. Create the private Node 22 service package and freeze the public extension URI and standard carriers, integrity and structured-result Artifacts, portable result-schema subset, worker protocol including command revisions/tombstones, profiles, fixtures, and error vocabulary. +2. Build authenticated durable A2A Task handling, retained-claim-first replay, and trusted fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. +3. Build the supervised single-execution worker lifecycle, monotonic command record, failed-quiescence boundary recycling, pre-readiness orphan reaper, OS credential boundary, and hardened source/evidence handling against a fake adapter. +4. Add the direct Codex SDK adapter and prove structured output, cancellation, OS-enforced provider/tool credential separation, and native evidence. +5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing OS-confined tools plus the terminating result tool. +6. Package the services and run cross-backend, transport, security, process, and A2A conformance before enabling a consumer. ### System-Wide Impact - **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. - **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. - **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. -- **Security:** New review-critical surfaces are auth, owner-key derivation, source SSRF, admission/resource quotas, setup/check policy, provider-credential exclusion from model tools, Pi extension/tool replacement, phase-scoped secrets, internal fences, Artifact capture/serving, and reviewed-source trust enforcement. -- **Operations:** Gateway and worker health, readiness, quotas, low-space state, structured logs, traces, tombstone backlog, lease expiry, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. +- **Security:** New review-critical surfaces are trusted public/private transports, auth, owner-key derivation, retained-replay ordering, source SSRF, admission/resource quotas, setup/check policy, OS-enforced provider/tool credential separation, Pi extension/tool replacement, phase-scoped secrets, metadata-only telemetry filtering and operator boundaries, internal fences and monotonic command records, Artifact capture/serving, and reviewed-source trust enforcement. +- **Operations:** Gateway and worker health, readiness, transport/peer identity, quotas, low-space state, allowlisted metadata-only structured logs/traces, telemetry-specific access/retention, command tombstones, lease expiry, poisoned-worker exit, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. - **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. ### Risks and Mitigations - **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup, keep captured provider fixtures versioned, and use Promptfoo's Codex tests as characterization input rather than vendored implementation. -- **False idempotency or stale settlement:** Claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test duplicate delivery, cancellation races, restart, and late results. +- **False idempotency or stale settlement:** Resolve owner-scoped retained claims before mutable admission and compare stored original request/profile/schema bindings. For new work, claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test conflicts, cancellation races, restart, and late results. - **Task/store corruption:** Publish immutable blobs and generations before one manifest switch; tombstone before deletion; validate owner tuples/manifests at startup; garbage-collect unreachable generations; document the one-replica limit. - **Owner collision or path injection:** Hash a bounded canonical issuer/tenant/subject tuple, store and verify the tuple inside the owner aggregate, and use only server-generated opaque IDs in paths. -- **Orphan processes and roots:** Combine explicit cancel, native abort, process-group termination, one-execution supervisor/container death, lease expiry, pre-readiness orphan reaping or quarantine, and separate termination/filesystem proof before evidence or readiness. +- **Bearer interception or worker impersonation:** Require TLS at the named public ingress boundary and mTLS/equivalent authenticated encryption for remote worker routes, pin worker identity/capabilities, bind attempt capabilities to that identity and fence, and reject plaintext or wrong-peer readiness. +- **Orphan processes and roots:** Combine explicit cancel, native abort, process-set verification, one-execution supervisor/container death, lease expiry, and pre-readiness orphan reaping or quarantine. Failed quiescence poisons admission and exits the worker so the supervisor destroys the boundary; termination/filesystem outcomes remain separate. - **False recovery claims:** Persist Task and evidence truth only. Startup fails active Tasks, invalidates fences, and relies on lease expiry or supervisor-boundary proof instead of resuming provider sessions. -- **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in idempotency/provenance, pass the exact accepted schema through each adapter's supported mechanism, validate with one shared validator, publish only the fixed Artifact shape, and fail rather than publish missing or invalid JSON. -- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral, origin-bound, and absent from repository config, process arguments, model-initiated command/tool environments, retained workspaces, logs, and errors; Pi repository extensions and unrestricted built-in tools never load, and policy tools cannot access Pi config/data roots. -- **Resource exhaustion:** Reserve per-owner/global gateway quota before claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. +- **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in provenance and original claim bindings, pass the exact accepted schema through each adapter, validate with one shared validator, preserve an already selected result across later failures, and enforce the two fixed Artifact shapes. +- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral and origin-bound. Credentialed profiles also enforce the R13 OS provider/tool boundary or broker; environment filtering remains defense in depth. Pi repository extensions and unrestricted built-in tools never load. +- **Telemetry disclosure:** Apply KTD10's pre-export guard before every structured log/span processor and reject content or secret-bearing attributes rather than relying on exporter policy. Canary-secret and cross-owner-fragment tests cover agent, model, tool, stale-event, and error paths; telemetry operators receive only bounded metadata and opaque owner correlation under separate access and retention. +- **Resource exhaustion:** Reserve per-owner/global gateway quota only for new claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. - **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. - **Evidence overclaim:** Enforce KTD12's integrity kernel and per-dimension completeness. Truncation and redaction remain independent facts. - **Permission deadlock:** Initial profiles never prompt. Known requests resolve for one isolated invocation; unknown shapes fail closed as adapter incompatibility. -- **Trust-boundary overclaim:** Reject pooled hostile-source/cross-tenant profiles and state the reviewed mutual-trust boundary in config, readiness, Agent Card metadata, and docs. +- **Trust-boundary overclaim:** Enforce the narrow provider/tool credential boundary for credentialed profiles while rejecting pooled hostile-source/cross-tenant claims; state plainly that the former does not provide the latter. - **Cross-platform drift:** Keep gateway/store tests cross-platform. State that worker execution and hardened evidence/source controls are Linux-only. ### Assumptions @@ -435,7 +453,7 @@ docs/src/content/docs/ - The first production deployment runs one gateway replica with persistent storage. Multi-replica transactional storage is deferred. - Git over hardened HTTPS and exact commit object ID covers the initial consumer. Other source transports require a later extension version or capability. - Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. -- Initial repositories are reviewed inside one configured mutual-trust domain. Strong hostile-code or cross-tenant execution remains unavailable until a stronger sandbox driver exists. +- Initial repositories are reviewed inside one configured mutual-trust domain. Credentialed profiles still enforce provider/tool credential separation, but that narrower boundary does not make hostile-code or cross-tenant execution available; those claims require a stronger sandbox driver. - Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. --- @@ -444,34 +462,35 @@ docs/src/content/docs/ ### U1. Versioned public and worker contracts -- **Goal:** Freeze the extension, profile vocabulary, private worker protocol, canonical digest input, result envelope, typed failures, and conformance fixtures before either service endpoint. +- **Goal:** Freeze the standard public extension carriers, integrity and structured-result Artifacts, profile vocabulary, private worker protocol including monotonic command state, original idempotency bindings, typed failures, and conformance fixtures before either service endpoint. - **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. - **Dependencies:** None. -- **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. -- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas, one public extension URI, and one private protocol version. Define the exact `allagents.result-schema/v1` keyword allowlist and bounds, canonical schema digest, four structured-result states, fixed-name Artifact containing one A2A `Part` with `data` and `mediaType: application/json`, and shared schema/result validator. Include attempt/fence/lease identity, monotonic event sequence, accepted dispatch, renew/cancel, bounded terminal acknowledgement, public/private state separation, and integrity-kernel rules. Canonicalize caller input plus effective profile and result-schema digests for idempotency. Generate checked-in JSON Schemas and fixtures from the same source. +- **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/result-schema-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. +- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas and freeze `https://allagents.dev/a2a/extensions/coding-execution/v1`: required Agent Card advertisement, `A2A-Extensions` negotiation, `Message.extensions`, request data only at `Message.metadata[uri]`, and terminal integrity data only in the single Part of the fixed-name `allagents.execution-integrity` Artifact whose `extensions` contains the URI. Explicitly forbid `Task.extensions`. Define the portable result-schema subset, canonical caller/schema/profile digests, four result states, separate fixed `allagents.structured-result` Artifact, and shared validator. Define original claim bindings independently from mutable current policy. Add worker identity, attempt/fence/lease identity, monotonic command revision, unseen-attempt cancel tombstone, conditional effect revision, event sequence, terminal acknowledgement, and integrity rules. Generate checked-in schemas and fixtures from one source. - **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions, unsafe paths, invalid public states, stale fences, oversized records, and conflicting canonical inputs before implementing schemas. -- **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, and `src/core/native/types.ts` for safe error/provenance normalization. +- **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, `src/core/native/types.ts` for safe error/provenance normalization, and Buzz's structurally non-secret intent template for the narrow digest-input pattern. - **Test scenarios:** - - A minimal valid request with text prompt, invocation key, profile, exact commit, deadline, and optional `allagents.result-schema/v1` schema parses and produces a stable digest across object-key ordering. - - Changing prompt, source object ID, profile ID/digest, result schema, artifact selection, or deadline changes the digest; trace IDs and transport metadata do not. + - A minimal valid Message negotiates the exact URI in `A2A-Extensions`, includes it in `Message.extensions`, puts the bounded request only at `Message.metadata[uri]`, and produces a stable digest across object-key ordering; missing/mismatched carriers and any `Task.extensions` field are rejected. Every terminal fixture has exactly one `allagents.execution-integrity` Artifact with the URI in `Artifact.extensions` and the schema-defined envelope in its single `data` Part. + - Changing prompt, source object ID, profile ID, result schema, artifact selection, or deadline changes the canonical caller digest; trace IDs and transport metadata do not. The original effective-profile and result-schema digests are stored separately for retained replay. + - Rotating a resolved secret value, changing attempt/lease/trace identity, or changing a per-run path leaves the profile digest unchanged; changing a policy field or environment-variable name changes it, and the digest serializer cannot accept secret-bearing runtime state. - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - - Worker fixtures reject missing/mismatched attempt IDs, lease epochs, profile digests, event sequence, bounds, and terminal acknowledgements. - - `not_requested`, `not_produced`, `valid`, and `invalid` cover success, pre-output failure, cancellation, missing output, and invalid output without replacing the primary Task classification; only `valid` permits one `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data`, `mediaType: application/json`, and a matching schema digest in Artifact metadata. + - Worker fixtures reject missing/mismatched worker identities, attempt IDs, lease epochs, profile digests, command revisions, conditional-effect revisions, event sequences, bounds, and terminal acknowledgements. Cancel for an unseen attempt persists a tombstone; tombstoned or lower-revision dispatch is invalid before workspace creation. + - `not_requested`, `not_produced`, `valid`, and `invalid` cover success and failure without replacing the primary Task classification. `not_produced` is accepted only before candidate production; a selected `valid` or `invalid` survives later check/evidence/infrastructure failure, and only `valid` permits exactly one separate `allagents.structured-result` Artifact with the matching schema digest. - File evidence accepts create/edit/delete/rename and rejects unsafe paths, duplicate identities, oversized inline content, and inconsistent before/after forms. - **Verification:** Generated schemas are stable, public/private fixtures round-trip, digest vectors are cross-platform deterministic, and the private client/server fixture suite agrees before gateway or worker implementation. ### U2. Authentication and durable gateway repository -- **Goal:** Provide caller-scoped authentication, authorization, atomic Task/idempotency aggregates, Artifact storage, quota admission, pagination, restart fencing, logical expiry, and cleanup. +- **Goal:** Provide caller-scoped authentication, trusted-ingress configuration, retained-claim-first idempotency aggregates with original bindings, Artifact storage, new-claim quota admission, pagination, restart fencing, logical expiry, and cleanup. - **Requirements:** R4-R9, R13-R14, R17-R18, R22; AE2-AE4, AE6, AE8, AE10-AE13; KTD1, KTD3-KTD4, KTD12. - **Dependencies:** U1. - **Files:** `packages/execution-service/src/gateway/config.ts`, `packages/execution-service/src/gateway/auth.ts`, `packages/execution-service/src/gateway/store/gateway-repository.ts`, `packages/execution-service/src/gateway/store/file-gateway-repository.ts`, `packages/execution-service/tests/unit/gateway/auth.test.ts`, `packages/execution-service/tests/unit/gateway/file-gateway-repository.test.ts`. -- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Commit claim plus submitted Task in one manifest generation; publish immutable Artifact blobs before atomically switching the manifest to a new generation; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover and garbage-collect unreachable generations on startup. Startup recovery is a readiness barrier: invalidate every old fence and terminalize every nonterminal Task before admission, subscriptions, dispatch, or lease renewal begin. Reserve owner/global quotas before claims. Verify OIDC JWTs and constant-time static tokens before all repository access. +- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Resolve a retained claim after authentication and bounded parsing, and compare its stored canonical caller request plus original effective-profile/result-schema digests without consulting mutable current policy. For new work only, reserve owner/global quota and commit the claim, original bindings, and submitted Task in one manifest generation. Publish immutable Artifact blobs before one manifest switch; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover unreachable generations; and complete startup recovery before serving. Verify OIDC/static tokens before repository access and validate the configured named TLS ingress boundary before readiness. - **Execution note:** Implement concurrent-claim, transition-race, and crash-publication tests before request handling. Inject faults between blob, generation, manifest, tombstone, and cleanup operations. - **Patterns to follow:** `src/core/marketplace.ts` and `src/core/profile/files.ts` for atomic publication/recovery, `src/core/mcp-http-stdio-proxy.ts` for private files and loopback safety, and the official A2A `TaskStore` owner-scoping contract. - **Test scenarios:** - - Covers AE2-AE3. Concurrent identical claims create one aggregate; a conflicting digest returns conflict without dispatch permission. + - Covers AE2-AE3. Concurrent identical new claims create one aggregate; a conflicting original request/schema binding returns conflict without dispatch permission. Identical retained replay still returns the existing Task after its deadline, quota, authorization, readiness, or current profile changes, while an inconsistent stored binding fails closed. - Covers AE4. Load/list/cancel/subscribe/Artifact lookup scopes before path/database access and gives unknown, unauthorized, and expired IDs indistinguishable behavior. - Hostile/ambiguous issuer, tenant, subject, invocation key, Task ID, Artifact name, Unicode, case, delimiter, traversal, and Windows-reserved values cannot collide or become paths. - All standard list filters, `historyLength`, page size 1-100, omitted Artifacts, ordering, total size, and always-present next token match A2A semantics. Tokens are owner/query-bound and reject malformed, swapped, or stale filters. @@ -479,29 +498,31 @@ docs/src/content/docs/ - Restart, including repeated failure during startup recovery, completes the recovery barrier before serving: it fails every nonterminal Task once, invalidates fences, never renews an old lease or requests provider reattachment/replay, preserves terminal Tasks, and records cleanup unknown unless proven. - Covers AE13. Exact expiry tombstones the aggregate before cleanup; failed deletion never restores visibility; same-key replay before expiry returns the old Task and after expiry creates a new Task. - A crash between every aggregate publication step leaves either the prior or next valid manifest, never claim-without-Task or Task-with-missing-Artifact state. - - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. - - Quota-boundary races admit exactly the allowed count and preserve reserved capacity for cancel/terminal writes; low-space mode stops new claims without blocking settlement. - - Unauthenticated mode starts on loopback and refuses wildcard or non-loopback listeners. + - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. Production readiness rejects missing/mismatched named TLS termination, while unauthenticated plaintext remains loopback-only. + - Quota-boundary races admit exactly the allowed new claims and preserve reserved capacity for cancel/terminal writes; low-space mode stops new claims without blocking retained replay or settlement. + - Unauthenticated mode starts only on loopback and refuses wildcard or non-loopback listeners. - **Verification:** A fresh process retrieves prior records, fault recovery finds one valid aggregate generation, authorization cannot reveal neighboring owners, and expiry/quota behavior remains deterministic under concurrency. ### U3. A2A gateway server and fenced worker client -- **Goal:** Expose the accepted A2A profile while making admission, replay, streaming, lookup, worker fencing, failure, and cancellation use one durable state machine. +- **Goal:** Expose the accepted A2A profile while making extension negotiation, retained replay, new admission, streaming, lookup, authenticated worker routing, monotonic worker commands, failure, and cancellation use one durable state machine. - **Requirements:** R1-R9, R11, R14-R15, R17-R22; F1-F5; AE1-AE4, AE6, AE8-AE13; KTD1-KTD7, KTD9, KTD12. - **Dependencies:** U1, U2. - **Files:** `packages/execution-service/src/gateway/agent-card.ts`, `packages/execution-service/src/gateway/request-handler.ts`, `packages/execution-service/src/gateway/executor.ts`, `packages/execution-service/src/gateway/server.ts`, `packages/execution-service/src/gateway/worker-client.ts`, `packages/execution-service/tests/unit/gateway/agent-card.test.ts`, `packages/execution-service/tests/unit/gateway/request-handler.test.ts`, `packages/execution-service/tests/unit/gateway/executor.test.ts`, `packages/execution-service/tests/e2e/gateway-fake-worker.test.ts`. -- **Approach:** Mount the official HTTP+JSON and Agent Card handlers behind auth. Put an AllAgents `A2ARequestHandler` decorator above `DefaultRequestHandler` so admission and canonical Task reservation happen first, identical replay bypasses new SDK Task/bus allocation, and cancellation waits for worker terminal evidence. Persist each public state before emission. Dispatch one fenced attempt, validate sequence/fence on every worker event, renew its lease, and atomically publish Artifact blobs plus terminal manifest. +- **Approach:** Mount the official HTTP+JSON and Agent Card handlers behind trusted ingress and auth. Advertise the exact required URI; validate `A2A-Extensions`, `Message.extensions`, and `Message.metadata[uri]`; and publish the integrity envelope only through the standard Artifact carrier, never `Task.extensions`. The `A2ARequestHandler` decorator authenticates and bounded-canonicalizes, resolves the owner-scoped retained claim, and returns or conflicts against stored original bindings before current profile/deadline/quota/readiness checks. New requests then pass mutable admission and canonical Task reservation. Keep transition selection pure and execute fenced I/O outside it. Dispatch revisioned commands over mTLS/equivalent authenticated encryption or a same-host Unix socket, binding worker identity/capability/fence; atomically publish Artifact blobs plus the terminal manifest. - **Execution note:** Begin with an in-process fake worker and official A2A client. Prove operation errors versus accepted-Task failures, replay/subscribe behavior, fencing, cancellation races, and restart before adding providers. -- **Patterns to follow:** Official A2A sample `AgentExecutor`, `A2ARequestHandler`, `DefaultRequestHandler`, Express handlers, and cancellable-agent flow; `src/core/mcp-http-stdio-proxy.ts` for HTTP shutdown and loopback tests. +- **Patterns to follow:** Official A2A sample `AgentExecutor`, `A2ARequestHandler`, `DefaultRequestHandler`, Express handlers, and cancellable-agent flow; `src/core/mcp-http-stdio-proxy.ts` for HTTP shutdown and loopback tests; and Buzz's pure classifier/I/O reconciler split for transition selection without adopting its Kubernetes model. - **Test scenarios:** - - Covers AE1. `returnImmediately` true returns the submitted/working Task, false/unset waits for terminal state, and streaming starts with the same durable Task before ordered updates. - - Authentication, invalid extension/source/profile, expired deadline, and pre-claim quota failure return operation errors with no Task or worker request. + - Covers AE1. Agent Card negotiation, `A2A-Extensions`, `Message.extensions`, `Message.metadata[uri]`, and the fixed integrity Artifact pass through the official client; missing/mismatched carriers and `Task.extensions` fail. `returnImmediately` and streaming expose the same durable Task. + - New-admission authentication, invalid extension/source/profile, expired deadline, current-profile/readiness failure, and pre-claim quota failure return operation errors with no Task or worker request. - Covers AE11. Capacity loss after acceptance fails the retained Task at `dispatch/capacity_exhausted`; ambiguous dispatch fails `dispatch_unknown`; neither is retried. - - Covers AE2-AE3. Identical send/stream replay returns the existing Task and follows only future events if active; conflict returns the documented operation error. + - Covers AE2-AE3. Identical send/stream replay returns the existing Task before mutable checks even after the stored deadline or current profile changes; changed request/schema or inconsistent original binding conflicts. - Covers AE8. Active subscribe emits current snapshot then future events without missed-event replay; terminal subscribe errors and `GetTask` returns terminal truth. - Covers AE4. Get/list/subscribe/cancel/Artifact endpoints apply owner authorization consistently. - - Covers AE6. Cancel in submitted/working, cancel versus accept/completion, caller versus deadline, duplicate cancel, and terminal cancel each produce one linearized outcome and at most one worker abort. - - Covers AE12. Duplicate, out-of-order, malformed, wrong-fence, and late terminal events cannot overwrite Task state; stale facts go only to safe telemetry. + - Covers AE6. Cancel in submitted/working, cancel versus accept/completion, caller versus deadline, duplicate cancel, and terminal cancel each produce one linearized outcome and at most one worker abort. If dispatch send is paused after selection and a newer cancel completes first, releasing the stale dispatch cannot create a workspace or provider process. + - Covers AE12. Duplicate, out-of-order, malformed, wrong-identity, wrong-revision, wrong-fence, and late terminal events cannot overwrite Task state; stale facts go only to allowlisted metadata-only telemetry with opaque owner correlation. + - A failed fenced effect or changed observation causes a durable re-read and reclassification; the executor never substitutes a fresher fence or revision into an effect selected from stale state. + - Plaintext remote workers, wrong certificates, wrong configured worker identity/capability, and replayed attempt capabilities fail before dispatch; mTLS/equivalent protected routes and same-host Unix sockets succeed. - Known policy denial rejects only after stop/cleanup; unknown permission shape fails as adapter incompatibility. - Caller SSE disconnect and telemetry exporter failure leave execution and terminal lookup intact. - Graceful shutdown stops admission, claims cancellation for bounded active work, persists honest terminal state, and closes listeners. @@ -509,26 +530,28 @@ docs/src/content/docs/ ### U4. Worker protocol and safe workspace lifecycle -- **Goal:** Implement the supervised single-execution worker, hardened immutable Git acquisition, profile enforcement, leases, isolated backend roots, resource controls, race-resistant evidence, termination, and cleanup independent of any provider. +- **Goal:** Implement the supervised single-execution worker with authenticated transport, a minimal monotonic command record, hardened immutable Git acquisition, OS-enforced credential separation, leases, isolated roots, resource controls, race-resistant evidence, failed-quiescence recycling, and cleanup independent of any provider. - **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. - **Dependencies:** U1. - **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. -- **Approach:** Authenticate and fence the private protocol, reserve the one execution before workspace creation, validate profile/deployment capability, and emit sequenced NDJSON. Run the worker server inside a deployment-approved supervisor boundary that kills all invocation descendants if the server exits. Before readiness, inspect invocation manifests, require supervisor proof that prior descendants are dead, and delete or quarantine orphaned roots; an unprovable root blocks reuse and reports degraded readiness. Acquire source under KTD13. Create separate workspace and backend config/data roots with a scrubbed phase-specific environment. Run setup, baseline, adapter, and checks under enforced budgets. Stop and verify the process group before descriptor-based regular-file evidence staging, then clean in `finally`. Lease expiry self-cancels. +- **Approach:** Authenticate the configured worker identity and fence every private command. Persist one minimal monotonic command record scoped to worker identity/lease before workspace creation: unseen-attempt cancel writes a tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel effect conditionally rechecks the stored revision immediately before mutation. Reserve one execution only after that check. Validate profile/deployment and OS provider/tool credential-boundary capabilities, then emit sequenced NDJSON. Keep transition selection pure. Run inside a dedicated container process namespace under init/reaper or an equivalent systemd/cgroup boundary. After bounded termination escalation, prove the complete invocation process set empty; if proof fails, persist termination unknown/failed, poison admission, and exit so the supervisor destroys the boundary. Replacement readiness proves boundary termination and reaps/quarantines owned roots. Use KTD13 acquisition, separate roots, phase environments, budgets, and descriptor-safe evidence; clean in `finally`. - **Execution note:** Characterize every phase with a fake adapter, malicious fixtures, and disposable Git servers before real providers. Fault-inject dispatch acknowledgement, events, leases, acquisition, processes, evidence publication, and cleanup. -- **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, and `tests/helpers/env.ts` for isolated state. +- **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, `tests/helpers/env.ts` for isolated state, and Buzz's bounded process-group/job-object cancellation as a lifecycle characterization checklist rather than copied code. - **Test scenarios:** - - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, and fetch failure stop before adapter invocation. + - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, fetch failure, and setup failure stop before adapter invocation. After worker acceptance each such source/setup failure emits the selected `Submitted -> Working -> Failed` public trace. - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no secondary connection or helper execution. - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. - - Setup changes establish the baseline; setup, checks, and model-initiated tools receive no provider/control secrets; every backend gets disjoint invocation config/data roots with ambient selectors removed. + - Setup changes establish the baseline; setup and checks receive no provider/control secrets. Credentialed provider runtimes and model tools run across the declared OS UID/process/mount boundary or broker, with disjoint config/data roots and ambient selectors removed. - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected; worker control credentials are absent from child environment and configured filesystem roots. - - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation and preserve worker health. + - Block dispatch after effect selection, complete a newer cancel for the unseen attempt, then release dispatch: the command tombstone/revision check rejects it before workspace or provider creation. Duplicate commands remain idempotent and all effects stay fence-bound. + - Covers AE12. A descendant calls `setsid`, ignores graceful signals, and survives per-process-group escalation during cancellation and normal completion; the worker records termination unknown/failed, refuses another reservation, exits, and replacement readiness reaps or quarantines the orphaned root after supervisor boundary destruction. + - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected. Credentialed readiness fails without the narrow OS provider/tool boundary, and model tools cannot inspect provider/control process environments, procfs/process listings, or configured backend roots. + - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation; failed quiescence recycles the worker rather than claiming it remains healthy. - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. - - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps; capture I/O or malformed result that breaks the kernel fails the Task. + - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps. A valid or invalid result selected before a later check/evidence failure is preserved, including the valid Artifact; only a pre-candidate failure records `not_produced`. - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. - - Covers AE12. SIGKILL the worker before and after provider spawn with live descendants and a persistent invocation root; the supervisor proves descendant death, the replacement reaper deletes or quarantines the root before readiness, and the gateway retains one failed Task with separate termination and cleanup outcomes. -- **Verification:** A built supervised worker mutates a disposable exact-SHA repository through the fake adapter and proves fenced dispatch, source hardening, phase isolation, budgets, quiescence, evidence integrity, worker-crash containment, orphan-root handling, and cleanup from its emitted result plus supervisor proof. + - SIGKILL before and after provider spawn proves supervisor descendant death and replacement root recovery; the gateway retains one failed Task with separate termination and cleanup outcomes. +- **Verification:** A built supervised worker mutates a disposable exact-SHA repository through the fake adapter and proves authenticated revisioned dispatch, unseen-cancel tombstones, source hardening, OS credential separation, budgets, result preservation, quiescence or poisoned-boundary exit, evidence integrity, worker-crash containment, orphan-root handling, and cleanup. ### U5. Codex backend adapter @@ -536,19 +559,18 @@ docs/src/content/docs/ - **Requirements:** R10-R22; AE1, AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. - **Dependencies:** U4. - **Files:** `packages/execution-service/src/worker/adapters/codex.ts`, `packages/execution-service/tests/unit/worker/adapters/codex.test.ts`, `packages/execution-service/tests/fixtures/execution/codex-events.jsonl`. -- **Approach:** Depend directly on a pinned `@openai/codex-sdk` and fail worker readiness when it is unavailable or incompatible. Construct one fresh SDK thread in the invocation workspace with an isolated `CODEX_HOME` and a minimal allowlisted environment. Apply model, sandbox, network, approval, working-directory, writable-root, and a pinned `shell_environment_policy` only from the profile. The Codex runtime may receive its scoped provider credential, but model-initiated shell commands receive only named non-secret variables and never provider or worker-control credentials. Consume `runStreamed()`, pass the invocation `AbortSignal`, and pass the exact accepted result schema as per-turn `outputSchema`. Parse and validate the final JSON with the shared worker validator before publishing the canonical result Artifact. Normalize agent messages, items, usage, failures, and file-change events while preserving the bounded native stream. Never call `resumeThread`, pool threads, or reuse provider sessions after interruption. +- **Approach:** Depend directly on pinned `@openai/codex-sdk` and fail readiness when the runtime or configured credential boundary is unavailable. Create one fresh SDK thread with isolated `CODEX_HOME`. The credential-bearing Codex runtime runs on the provider side of the declared UID/process/mount boundary or obtains credentials through the configured broker; model-invoked commands run on the tool side and cannot inspect provider procfs/process entries or config/data roots. A minimal allowlisted environment and pinned `shell_environment_policy` remain defense in depth. Apply profile model/sandbox/network/approval/path policy, pass `AbortSignal` and exact `outputSchema`, validate final JSON with the shared validator, normalize bounded events/evidence, and never resume or pool threads. Once validation selects `valid` or `invalid`, later check/evidence/infrastructure failure preserves that state and the valid Artifact. - **Execution note:** Wrap the SDK behind an injectable factory and drive it through fixture events and its executable override before any credentialed smoke test. Use Promptfoo's provider and tests to enumerate observable edge cases, not as copied code or a runtime dependency. - **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, the SDK's `startThread`/`runStreamed`/`AbortSignal`/`outputSchema` and shell-environment policy contracts, and Promptfoo's Codex provider tests for characterization of option forwarding, environment isolation, cancellation, structured output, and cleanup. - **Test scenarios:** - A successful stream exposes thread ID, progress, final response, token usage, native file-change items, and terminal completion. - - A structured request forwards the exact accepted schema to `outputSchema`; valid JSON becomes the fixed-name Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`, while missing, malformed, or schema-invalid output fails with a typed structured-result error. + - A structured request forwards the exact accepted schema to `outputSchema`; valid JSON selects `valid` and produces the fixed-name `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; malformed or schema-invalid output selects `invalid` without the Artifact and reports the typed validation error; missing output selects `not_produced` and reports the typed missing-output error. - Empty final response, turn failure, malformed JSONL, non-zero exit, unavailable runtime, and usage omission map to typed result/completeness fields. - Covers AE6/AE12. A pre-aborted signal prevents start; in-flight cancellation aborts the SDK once; worker escalation proves descendant termination; completion after cancel or stale fence cannot alter the selected terminal outcome. - - Profile sandbox, network, model, approval, working directory, shell-environment policy, and environment settings reach the SDK; caller input cannot override them or supply raw Codex config. - - The Codex runtime receives only its scoped credential and minimal runtime environment. A model-initiated command that attempts to print provider/control credential names observes no values, and output, errors, and retained evidence contain none. + - The credential-bearing Codex runtime receives only its scoped credential and minimal environment. Adversarial model commands probing parent/sibling environments, `/proc` and process listings, known or discovered `CODEX_HOME`/backend roots, and outbound secret exfiltration cannot recover provider/control credentials; readiness fails when this OS boundary or broker is unavailable. - Two sequential invocations create fresh threads with disjoint `CODEX_HOME`, session state, and writable roots; no resume or thread-persistence API is called. - Native diffs and shared Git evidence coexist without claiming identical attribution. -- **Verification:** Fixture-driven tests cover every supported event and failure shape, SDK option/schema/signal forwarding, environment isolation, fresh-thread behavior, and cleanup escalation, followed by an isolated credentialed repository smoke test when Codex credentials are available. +- **Verification:** Fixture-driven tests cover every supported event/failure shape, SDK option/schema/signal forwarding, OS-enforced provider/tool separation plus environment defense in depth, fresh-thread behavior, result-state preservation, and cleanup escalation, followed by an isolated credentialed repository smoke test when prerequisites are available. ### U6. Pi backend adapter @@ -556,59 +578,66 @@ docs/src/content/docs/ - **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. - **Dependencies:** U4, U5. - **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/src/worker/adapters/pi-policy-extension.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-policy-extension.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. -- **Approach:** Spawn a supported Pi 0.85.x runtime with `--mode rpc --no-session --no-extensions --no-builtin-tools`, an invocation-local `PI_CODING_AGENT_DIR`, profile model/provider, and scrubbed environment. Materialize the scoped provider credential only in Pi's supported invocation-local credential store with private permissions, not in the process environment. Explicitly load one worker-owned policy extension from outside the repository and verify the loaded extension/tool inventory before accepting work. The extension registers workspace-confined read/write/edit/search and sandboxed command tools plus the terminating result tool; command children receive only the phase allowlist and cannot access the Pi config/data roots. Implement an LF-only JSONL parser rather than Node `readline`. Correlate responses, wait for `agent_settled`, read messages/stats, send RPC abort, and escalate process-group termination after the grace period. For a structured request, generate the terminating tool from the exact accepted schema. The first observed tool call atomically claims the result candidate before validation: valid arguments produce the canonical Artifact; malformed or schema-invalid arguments fail the Task; later calls cannot replace the candidate. A prior cancel, deadline, or stale fence suppresses the call. Settling without a call fails as missing structured output. +- **Approach:** Spawn supported Pi 0.85.x in strict RPC mode with an invocation-local `PI_CODING_AGENT_DIR`, no sessions/extensions/built-ins, and one explicit worker-owned policy extension. The credential store and provider runtime stay on the provider side of the configured UID/process/mount boundary or credential broker; policy/command tools run on the tool side and cannot inspect the provider process, procfs entries, or Pi config/data roots. Verify exact tool inventory before work. Implement bounded LF JSONL, correlation, settlement/stats, abort and escalation. For a structured request, generate the terminating tool from the accepted schema; the first call atomically claims and validates the candidate, later calls cannot replace it, and later checks/evidence failures preserve its `valid` or `invalid` state and valid Artifact. - **Execution note:** Build parser, extension/tool-inventory, terminating-tool, policy-tool, and state-machine tests from captured RPC fixtures before process integration. Reuse the contract established by U5 rather than adding Pi-shaped public fields. - **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing, `--no-extensions` plus explicit `--extension`, `--no-builtin-tools`, custom-tool, credential-store, and cancellation contracts. - **Test scenarios:** - Successful prompt acceptance streams message/tool events, stops on `agent_settled`, retrieves final messages/stats, and reports session ID, usage, and cost. - - A structured request exposes only the invocation-scoped terminating tool in addition to the policy tools. The first observed call claims the candidate; valid arguments produce the fixed-name Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; an invalid first call fails without replacement; a later duplicate cannot replace the result; a cancel/deadline/fence that wins first suppresses the call; and settled completion without a call fails as missing output. + - A structured request exposes only the invocation-scoped terminating tool in addition to the policy tools. The first observed call claims the candidate; valid arguments select `valid` and produce the fixed-name `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; an invalid first call selects `invalid` without replacement or an Artifact; a later duplicate cannot replace the result; later check/evidence/infrastructure failure preserves the selected state and valid Artifact; a cancel/deadline/fence that wins first suppresses the call; and settled completion without a call selects `not_produced` as missing output. - LF framing preserves `U+2028`/`U+2029` inside JSON strings, accepts CRLF by stripping trailing CR, handles partial/multiple chunks, and rejects oversized/malformed records. - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled or terminating-tool events cannot overwrite the terminal fence. - Prompt rejection, agent error, aborted stop reason, retry/compaction sequence, premature exit, stderr overflow, and stats failure map truthfully. - - Sequential invocations have disjoint `PI_CODING_AGENT_DIR`, tool registration, and session state; the provider credential exists only in the private invocation-local store; policy tools cannot read that root and their command children observe no provider/control credential; repository `.pi/extensions` and unrestricted built-in tools do not load; and caller input cannot send extension commands, steering/follow-up, arbitrary RPC commands, or override provider/model. -- **Verification:** Fixture and fake-process tests prove framing, correlation, deterministic terminating-tool selection, shared validation, exact extension/tool inventory, workspace confinement, command-environment and credential isolation, settled completion, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when Pi credentials are available. + - Sequential invocations have disjoint `PI_CODING_AGENT_DIR`, tool registration, and session state. Adversarial policy/command tools probing parent/sibling environments, procfs/process listings, known or discovered Pi/backend roots, and outbound secret exfiltration cannot recover provider/control credentials; readiness fails without the OS boundary or broker. Repository `.pi/extensions` and unrestricted built-ins do not load, and caller input cannot override provider/model or issue arbitrary RPC/extension commands. +- **Verification:** Fixture and fake-process tests prove framing, correlation, terminating-tool selection, shared validation, exact tool inventory, OS-enforced provider/tool separation plus environment defense in depth, result-state preservation, settlement, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when prerequisites are available. ### U7. Production registry, service packaging, and observability -- **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with safe startup, health, shutdown, tracing, and reproducible containers. -- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD11-KTD15. +- **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with trusted transports, peer identity, credential-boundary and supervisor readiness, safe startup/shutdown, tracing, and reproducible containers. +- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD15. - **Dependencies:** U3-U6. - **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. -- **Approach:** Register only Codex and Pi through an explicit capability/availability map. Add gateway and supervised worker entrypoints inside the private Node 22 workspace instead of the Node 18 CLI package. Pin both runtimes as direct service dependencies, validate config, store, workers, runtime availability, supervisor boundary, orphan roots, trust, quotas, and resource controls before readiness, and never discover a missing provider only after Task acceptance. Propagate `traceparent` and instrument every phase. Build a minimal gateway image with no provider runtimes and a one-execution worker image whose init/runtime kills the complete execution boundary when the worker server exits. +- **Approach:** Register only Codex and Pi. Add gateway and supervised worker entrypoints inside the private Node 22 workspace. Before readiness, validate named public TLS termination, every remote worker's mTLS/equivalent transport and pinned identity/capabilities, Unix-socket locality, store, runtimes, monotonic command storage, OS credential-boundary capability, supervisor boundary, orphan roots, trust, quotas, and resource controls. Propagate `traceparent`, then apply KTD10's small shared metadata allowlist and bounded filtering/redaction before any structured log/span processor or OTLP exporter; neither OpenInference nor backend-native attributes bypass it. Build a minimal gateway image with no provider runtime and a one-execution worker image whose init kills the complete boundary when the worker server exits, including poisoned failed-quiescence exit. - **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. - **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. - **Test scenarios:** - Registry exposes exactly Codex and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects OpenCode or unknown backend IDs before workspace creation. - - Gateway and supervised worker start from built service outputs, become ready only after dependencies and orphan recovery pass, and stop gracefully on SIGTERM. - - Gateway readiness fails for malformed auth, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. - - Killing the worker server while an adapter child and invocation root exist makes the supervisor kill the boundary; replacement readiness waits for root deletion or quarantine and never reuses it. - - Trace context enters through A2A, crosses the private call, and correlates result identities; exporter failure cannot change Task status. - - Gateway image contains no Codex, Pi, Git workspace, or provider credential material. - - Worker image pins both runtimes, confines one workspace/config root, excludes credentials from model tools, disables repository Pi extensions and unrestricted built-in tools, enforces deployment limits, and completes fake-provider health smoke tests. + - Gateway and supervised worker start from built outputs, become ready only after trusted transport/identity, credential and supervisor boundaries, dependencies, and orphan recovery pass, and stop gracefully on SIGTERM. + - Gateway readiness fails for malformed auth, missing/mismatched named TLS termination, plaintext production public ingress, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. + - Worker-route readiness fails for plaintext remote URL, wrong/untrusted certificate, worker identity/capability mismatch, or replayed capability; mTLS/equivalent authenticated encryption and same-host Unix sockets pass. + - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable OS credential/supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. + - Killing or poisoning the worker server while an adapter child and invocation root exist makes the supervisor destroy the boundary; replacement readiness waits for root deletion/quarantine and never reuses it. + - Trace context crosses the authenticated private call and correlates result identities using only opaque owner correlation. Exporter probes for agent, model, tool, stale-event, and error spans contain allowlisted bounded metadata but no canary secret, prompt/output, tool argument/result, file body/source fragment, raw caller identity, or cross-owner fragment; exporter failure cannot change Task status. + - Gateway image contains no Codex, Pi, Git workspace, provider credential material, or worker trust private keys. + - Worker image pins both runtimes, enforces provider/tool UID/process/mount separation or the credential broker, confines one workspace/config root, disables repository Pi extensions and unrestricted built-ins, enforces deployment limits, and completes fake-provider security probes. - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. -- **Verification:** The registry dispatches both adapters through the same worker contract; built services and images pass lifecycle/security smoke tests; CI and publication bind immutable image tags to the release commit. +- **Verification:** The registry dispatches both adapters through the same worker contract; built services and images pass lifecycle/security smoke tests; exporter-capture tests prove pre-processor metadata allowlisting, bounded redaction, opaque owner correlation, and canary/cross-owner exclusion across agent, model, tool, stale-event, and error spans; CI and publication bind immutable image tags to the release commit. ### U8. Cross-backend conformance, documentation, and release evidence -- **Goal:** Prove the public contract and operational workflow end to end and document deployment without leaking backend details into callers. +- **Goal:** Prove standard A2A extension carriers, retained replay, trusted transport, monotonic cancellation, truthful result preservation, credential separation, metadata-only telemetry, worker recycling, trace-order conformance, and the shared backend contract end to end without leaking backend details into callers. - **Requirements:** R1-R22; F1-F5; AE1-AE14. - **Dependencies:** U1-U7. - **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. -- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases. Exercise gateway and supervised worker as separate processes. Document extension/worker protocols, the portable result-schema subset, fixed structured-result Artifact, four result states, profiles, auth, trust boundary, storage/HA limits, explicit lack of execution resume, worker-crash supervision/orphan recovery, source hardening, quotas, runtime requirements, cancellation races, evidence integrity, Artifact access, retention, observability, and troubleshooting. +- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases, with gateway and supervised worker as separate processes. Each race fixture declares attempt/fence correlation, required durable transitions and observed effects, required happens-before edges, maximum occurrence counts, and effects forbidden after terminalization. A deliberately small test-side checker evaluates those constraints against durable records plus observed worker/process outcomes without calling the production selector. It remains coverage protection—not TLA+, a model checker, event sourcing, or a second lifecycle implementation. Document the exact extension URI and legal Agent Card/header/Message/Artifact carriers, both fixed Artifacts and four result states, retained-replay ordering, worker transport/identity, monotonic command tombstones, failed-quiescence recycling, the narrow OS credential boundary, reviewed-domain limitation, metadata-only telemetry and its separate operator access/retention, storage/HA limits, lack of execution resume, source hardening, quotas, retention, and operations. - **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. -- **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, and Starlight guide/reference organization under `docs/src/content/docs/docs/`. +- **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, Starlight guide/reference organization under `docs/src/content/docs/`, and Buzz's required-critical-action coverage rule without importing its TLA+ model or production implementation. - **Test scenarios:** - Covers AE1-AE14 through built services with a fake backend and official A2A client. - - The same accepted schema, valid result, invalid result, missing result, and pre-output failure fixtures pass through Codex and Pi adapters with identical validator decisions, integrity states, and Artifact presence/shape while retaining distinct native evidence. - - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; the one-execution worker serializes admitted work. - - Gateway restart, stream reconnect, lost dispatch acknowledgement, duplicate/out-of-order events, worker crash, lease expiry, cancellation race, provider failure, invalid structured output, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome without provider reattachment or replay. - - A worker SIGKILL with live descendants and an invocation root proves supervisor termination and pre-readiness deletion/quarantine; repeated gateway crashes cannot serve until startup terminalization completes. - - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, control-endpoint probing, provider-credential echo/read attempts, repository Pi extensions, unrestricted Pi built-in tools, and secret-exfiltration fixtures are blocked within the documented reviewed-source boundary. - - Examples validate with production schemas and reference secrets only through environment variable names. - - Docs state one gateway replica, one supervised execution per worker, reviewed mutual-trust sources, Node/runtime floors, ephemeral provider sessions, and no hostile-code isolation claim. - - Opt-in real-provider smoke tests record backend/runtime versions and skip only when the named credential/runtime prerequisite is absent. -- **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor, the full suites and docs pass, the official A2A client exercises every advertised operation, and release evidence records each available real backend plus explicit skipped prerequisites. + - Agent Card required-extension advertisement, `A2A-Extensions`, `Message.extensions`, request `Message.metadata[uri]`, and the single fixed integrity Artifact carrier interoperate; missing/mismatched carriers and `Task.extensions` fail. + - Identical retained replay after deadline expiry, quota exhaustion, readiness loss, authorization change, or profile replacement returns the original Task; changed request/schema or inconsistent original bindings conflict. + - The same accepted schema, valid result, invalid result, missing result, and pre-output failure pass through Codex and Pi with identical decisions. A valid-result-then-check-failure and invalid-result-then-evidence-failure preserve the selected state and only the valid Artifact; `not_produced` remains pre-candidate only. + - Source mismatch and setup failure after worker acceptance produce the selected `Submitted -> Working -> Failed` trace; provider invocation never begins, and durable snapshots, streams, and conformance records agree. + - Pause dispatch after selection, complete unseen-attempt cancel, then release dispatch; the stale command creates no workspace/process. The small independent checker enforces each fixture's attempt/fence correlation, happens-before edges, maximum counts, and forbidden post-terminal effects. Deliberately bad traces that still contain every required action name fail for wrong order, wrong fence, duplicate-over-maximum effects, and an extra stale dispatch after terminalization. + - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; one worker serializes admitted work. + - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order commands/events, worker crash, lease expiry, cancellation, provider failure, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome without provider reattachment or replay. + - A child that calls `setsid` and ignores graceful signals forces termination unknown/failed, poisoned-worker exit, supervisor boundary destruction, and replacement orphan recovery before readiness; no next reservation is accepted by the poisoned worker. + - Public plaintext, wrong TLS boundary, private plaintext, wrong certificate/worker identity, and capability replay fail readiness/dispatch; configured TLS, mTLS/equivalent overlay, and same-host Unix socket cases pass. + - Both adapters block model-tool probes of parent/sibling environments, procfs/process listings, known/discovered backend roots, and network secret exfiltration under the OS credential boundary. Environment filtering alone is never accepted as proof, and hostile-source/cross-tenant claims remain rejected. + - End-to-end exporter capture repeats the agent/model/tool/stale-event/error canary and cross-owner probes, proving only bounded allowlisted metadata and opaque owner correlation cross the telemetry boundary while Task/Artifact access and retention remain independent. + - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, repository Pi extensions, and unrestricted built-ins remain blocked within the documented reviewed-source boundary. + - Examples validate with production schemas and use only secret variable names. Docs state the two transport boundaries, one gateway replica, one execution per worker, reviewed trust domain, narrow credential isolation versus deferred hostile-code isolation, metadata-only telemetry with its fixed pre-processor allowlist and separate operator access/retention, runtime floors, and ephemeral provider sessions. + - Opt-in real-provider smoke tests record backend/runtime and credential-boundary prerequisites, skipping only when a named prerequisite is absent. +- **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor; full suites and docs pass; the official A2A client exercises every advertised operation including the `Submitted -> Working -> Failed` source/setup path; exporter capture proves the telemetry canary/cross-owner contract; the independent checker rejects all-name-present traces with wrong order/fence/multiplicity or forbidden stale dispatch; and release evidence records each available real backend plus explicit skipped prerequisites. --- @@ -616,18 +645,20 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| -| Contract generation | U1 | Public extension and private worker schema generation report no drift; positive and negative fixtures pass. | -| Focused unit tests | U1-U7 | Active-unit tests pass with fault injection, state races, limits, cancellation, and cleanup. | -| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on fenced dispatch, sequencing, leases, Task persistence, Artifacts, shutdown, and cleanup. | -| Backend conformance | U5-U8 | One shared suite passes against Codex and Pi adapters with fixture runtimes, including identical acceptance and validation of the versioned result-schema subset, four result states, and fixed Artifact shape. | -| Credentialed provider smoke | U5-U6, U8 | Each available provider mutates a disposable exact-SHA repository; missing credentials/runtime are recorded as skipped prerequisites, never passing coverage. | -| A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, replay, cancel races, expiry, and owner isolation. | -| Security and abuse | U2-U4, U7-U8 | Malicious identity/source/artifact/resource fixtures prove auth-before-lookup, opaque owner keys, Git SSRF controls, phase-scoped secrets, provider-credential exclusion from model tools, disabled repository Pi extensions/built-ins, policy-tool confinement, quotas, quiescence, race-resistant capture, and trust-topology rejection. | -| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, worker-crash containment/orphan recovery, and both container builds pass. | +| Contract generation | U1 | Exact extension URI/carriers, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | +| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, and cleanup. | +| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | +| Backend conformance | U5-U8 | One shared suite passes against Codex and Pi, including the versioned schema subset, four result states, valid/invalid preservation across later failure, integrity Artifact carrier, and structured-result Artifact rule. | +| Credentialed provider smoke | U5-U6, U8 | Each available provider mutates a disposable exact-SHA repository while adversarial tool probes cannot cross the OS credential boundary; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | +| A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes required-extension negotiation and legal carriers, immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, retained replay, cancel races, expiry, and owner isolation without `Task.extensions`. | +| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, Git SSRF controls, OS provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | +| Lifecycle trace conformance | U8 | The small test-side checker, independently of production selectors, validates attempt/fence correlation, required happens-before edges, maximum occurrence counts, and forbidden post-terminal effects against durable records plus observed worker/process outcomes; all-name-present bad traces fail for wrong order/fence/multiplicity and stale post-terminal dispatch. | +| Telemetry safety | U7-U8 | Exporter capture across agent, model, tool, stale-event, and error spans proves the pre-processor allowlist and bounded redaction exclude prompt/output/tool/source/file content, canary secrets, raw identities, and cross-owner fragments while retaining only bounded operational metadata and opaque owner correlation. | +| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both container builds pass. | | Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | | Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | -The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove protocol, durable aggregation, process isolation, fencing, cancellation, or cleanup integration. +The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove extension carriers, retained-replay ordering, trusted transport, durable aggregation, monotonic worker commands, credential/process isolation, cancellation, boundary recycling, cleanup integration, or telemetry export safety. The deliberately small independent trace checker supplements that path only by rejecting ordering, fence, multiplicity, and post-terminal-effect violations; it is not a production lifecycle model. --- @@ -636,23 +667,26 @@ The authoritative behavioral proof is the built-process E2E path with the offici ### Global - Every R1-R22 requirement is implemented or explicitly shown in a passing conformance scenario. -- Public Agent Card/extension and private worker schemas are stable, generated from one source, and consumable without importing root AllAgents CLI modules. -- Codex and Pi pass the same backend conformance suite, accept the same versioned result-schema subset, validate with the same shared validator, publish the same fixed structured-result Artifact shape and result states, and preserve bounded native evidence through the closed registry. +- The exact required extension is advertised and negotiated through standard Agent Card/header/Message/Artifact surfaces; requests live only at `Message.metadata[uri]`, terminal integrity lives only in the fixed integrity Artifact, and no `Task.extensions` exists. +- Codex and Pi pass the same backend conformance suite, schema subset, and validator. Every terminal Task publishes the integrity Artifact; selected `valid`/`invalid` states survive later failures, and only `valid` publishes the separate fixed structured-result Artifact. - Gateway and supervised worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. -- Authentication precedes lookup, quota precedes Task creation, aggregate commits cannot split claims/Tasks/Artifacts, startup recovery completes before serving, and terminal fences survive races and restart without claiming provider-session recovery. -- Cancellation/deadlines reach one native abort, process termination, quiescence, evidence, and cleanup for both backends; worker-process death triggers supervisor termination and pre-readiness orphan deletion or quarantine. -- Source hardening, phase-scoped secrets, provider-credential exclusion from model tools, disabled repository Pi extensions/unrestricted built-ins, Pi policy-tool confinement, one-execution trust policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end. +- Authentication and bounded parsing precede owner-scoped retained lookup; identical replay uses stored original bindings before mutable admission, while current authorization/profile/readiness/deadline and quota apply only to atomic new claims. +- Production public ingress uses its named TLS boundary, remote worker routes authenticate and encrypt peers with worker identity/capability binding, and same-host Unix sockets are the only non-network alternative; unprotected remote endpoints fail readiness. +- Cancellation/deadlines use monotonic worker command tombstones and one native abort. Stale dispatch cannot create work, and failed quiescence poisons and exits the worker so supervisor destruction and replacement orphan recovery precede new admission. +- Source hardening, the OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled repository Pi extensions/unrestricted built-ins, one-execution reviewed-domain policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end without claiming hostile-source/cross-tenant isolation. +- Metadata-only telemetry is filtered through the fixed allowlist and bounded redaction before processing/export; canary secrets, content, raw caller identities, and cross-owner fragments never reach exporters, and only opaque owner correlation crosses the separately governed operator boundary. +- Required source/setup and race traces satisfy attempt/fence, happens-before, maximum-count, and forbidden-post-terminal constraints in the independent test-side checker; all-name-present malformed traces fail without introducing a parallel lifecycle implementation. - Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. -- Public documentation states supported topology, configuration, security boundary, storage/HA limitation, runtime pins, structured-result contract, worker-crash recovery, and deferred capabilities. +- Public documentation states extension carriers, retained replay, trusted transports, credential versus hostile-code boundaries, metadata-only telemetry and its separate operator access/retention, topology, storage/HA limitation, runtime pins, result preservation, poisoned/crashed-worker recovery, and deferred capabilities. - Abandoned experiments, unused adapters, compatibility shims, generated scratch files, retained test workspaces, and stale documentation are removed. ### Per unit -- U1: Public/worker schemas, result-schema subset, structured-result Artifact/states, digest vectors, fence rules, typed failures, and fixtures are generated and stable. -- U2: Auth, opaque owner isolation, aggregate idempotency, CAS settlement, pagination, startup recovery barrier, quotas, Artifact access, tombstones, and cleanup pass fault injection. -- U3: Every advertised A2A operation agrees across stream and lookup while replay, fencing, and cancellation races preserve one Task. -- U4: Supervision, orphan recovery, worker dispatch/source/setup/action/check/quiescence/evidence/cleanup lifecycle pass malicious, crashed, and faulted disposable-repository scenarios. -- U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, usage, native evidence, minimal environment, model-tool credential exclusion, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. -- U6: Pi strict JSONL framing, deterministic terminating-tool output, exact policy-extension/tool inventory, disabled repository extensions and built-ins, credential-store and workspace confinement, settled completion, stats, isolated roots, abort, and process cleanup pass adapter and applicable smoke verification. -- U7: Closed registry, Node-version separation, runtime readiness, tracing, graceful shutdown, containers, and release artifacts work from built outputs. -- U8: Cross-backend E2E, A2A interoperability, abuse cases, examples, operator docs, changelog, and release evidence are complete. +- U1: Standard extension carriers, integrity/structured-result Artifact schemas, four result states, original claim digests, command revisions/tombstones, fence rules, typed failures, and fixtures are generated and stable. +- U2: Trusted ingress, auth, opaque owner isolation, retained-claim-first replay, original bindings, atomic new admission, CAS settlement, pagination, startup recovery, quotas, Artifact access, tombstones, and cleanup pass fault injection. +- U3: Every advertised A2A operation agrees across stream and lookup while extension negotiation, replay ordering, authenticated worker routes, fencing, monotonic cancellation, and races preserve one Task. +- U4: Worker command state, OS credential separation, supervision, poisoned-exit/orphan recovery, and dispatch/source/setup/action/check/quiescence/evidence/cleanup pass malicious, crashed, and faulted scenarios. +- U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, result preservation, OS credential separation, native evidence, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. +- U6: Pi strict RPC/framing, terminating result, exact policy tools, disabled repository extensions/built-ins, OS-isolated credential store/provider runtime, result preservation, settlement, stats, abort, and process cleanup pass verification. +- U7: Closed registry, trusted transport/identity readiness, credential/supervisor capability gating, poisoned-worker recycling, metadata-only pre-export telemetry controls, Node-version separation, tracing, shutdown, containers, and release artifacts work from built outputs. +- U8: Cross-backend E2E, standard A2A carriers, retained replay, selected source/setup transitions, independent race-trace constraints, telemetry canary/cross-owner probes, transport and credential abuse cases, command/quiescence races, examples, operator docs, changelog, and release evidence are complete. diff --git a/docs/research/agent-host-protocol-decision-inputs.md b/docs/research/agent-host-protocol-decision-inputs.md index a3ebebc9..113ea6fc 100644 --- a/docs/research/agent-host-protocol-decision-inputs.md +++ b/docs/research/agent-host-protocol-decision-inputs.md @@ -11,9 +11,9 @@ AHP does not replace ADR 0002's Task identity, caller-scoped idempotency, authorization, immutable source handling, cleanup, terminal evidence, or bounded result retention. -The initial backend set is Codex, OpenCode, and Pi. They are peer execution -adapters behind one conformance contract; provider-specific process, session, -permission, cancellation, and evidence behavior stays below that seam. +The initial backend set is Codex and Pi; OpenCode is deferred. They are peer +execution adapters behind one conformance contract; provider-specific process, +session, permission, cancellation, and evidence behavior stays below that seam. This note records the AllAgents-specific consequences. The reusable research, source inspection, and full protocol comparison live in the AI Research Wiki: From c8bb0740b4fc85c9af080557329b833027a30c48 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Fri, 18 Sep 2026 20:31:43 +1000 Subject: [PATCH 06/12] docs(architecture): define registered workspace materializers --- ...-agent-execution-through-an-a2a-gateway.md | 175 +++++- ...0837-feat-coding-execution-gateway-plan.md | 585 ++++++++++++++++-- .../harbor-repository-materialization.md | 179 ++++++ 3 files changed, 855 insertions(+), 84 deletions(-) create mode 100644 docs/research/harbor-repository-materialization.md diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 599d72e8..d8deb818 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -19,10 +19,12 @@ provenance. Future clients may need the same execution boundary without Promptfoo or evaluation semantics. A coding-agent execution is more than a model request. It includes immutable -source selection, repository acquisition, environment setup, credentials, -permissions, agent invocation, cancellation, evidence capture, process -termination, and cleanup. Those responsibilities need one public contract while -allowing materially different execution backends. +workspace selection, repository or snapshot acquisition, environment setup, +credentials, permissions, agent invocation, cancellation, evidence capture, +process termination, and cleanup. A workspace may contain multiple repositories +or be produced from a digest-pinned snapshot or organization-specific source. +Those responsibilities need one public contract while allowing materially +different execution backends and acquisition mechanisms. The contract must not turn AllAgents into an evaluation harness. Dataset expansion, repetition, assertions, scoring, experiment scheduling, and durable @@ -60,10 +62,10 @@ process, session, structured-output, cancellation, and evidence behavior remains behind its adapter. OpenCode and other coding agents remain possible follow-up adapters rather than part of the first delivery. -Execution backends own repository materialization, environment setup, agent +Execution workers own workspace materialization, environment setup, agent invocation, evidence collection, process termination, and cleanup. The gateway -must not execute evaluated agents or mount their writable repositories in the -gateway process. +must not execute evaluated agents, run materializer images, or mount writable +workspaces in the gateway process. When deployed on Kubernetes, the gateway runs as its own Deployment and ClusterIP Service, separate from consumers and execution workers. A backend @@ -75,6 +77,96 @@ A separate gateway Pod is a service and failure boundary, not per-invocation security isolation. Deployments requiring hostile-code or tenant isolation must create or select a stronger execution boundary behind the gateway. +### Make workspace materialization explicit and operator-registered + +The public extension represents one workspace as a closed discriminated union. +The allowed `kind` values and shapes are: + +1. `repositories`, with a bounded list of direct Git repositories, each with a + canonical credential-free HTTPS URL, full commit object ID, collision-free + relative destination, and optional repository-relative subdirectory; +2. `workspaceSnapshot`, with an OCI workspace snapshot referenced by manifest + digest and accompanied by the versioned AllAgents workspace manifest; or +3. `materializer`, with an operator-registered materializer ID, an expected + workspace-manifest digest, and bounded structured inputs. + +Fields from another union variant are invalid. + +The third mode supports organization-specific acquisition such as JFrog, +generated sources, or custom monorepo assembly without accepting executable +configuration from the caller. The request cannot supply a builder image, +Dockerfile, Compose file, shell command, credential, mutable image tag, network +policy, or output contract. + +Direct Git and OCI acquisition revalidate scheme, normalized host, resolved +address, port, and redirect policy for every connection. OCI foreign or +external layer URLs are rejected by default, and registry credentials are never +forwarded across origins. + +Each materializer ID is defined in an operator-owned deployment registry. The +gateway receives only its non-secret descriptor: ID, bounded input schema, +expected definition digest, expected output-manifest version, and required +worker capabilities. The worker receives the runtime definition, which +additionally pins an OCI image by digest and fixes credential handle names or +mount identities, allowed network destinations, resource and phase deadlines, +cache policy, and the OCI runner or sandbox capability. Credential values are +not part of either descriptor. + +The worker computes an algorithm-qualified definition digest over a versioned, +domain-separated canonical serialization of every non-secret, +behavior-affecting runtime field. The expected workspace-manifest digest +likewise identifies the canonical bytes of one declared manifest version. An +execution profile explicitly allows source modes and materializer IDs and +authorizes canonical Git repositories or namespaces, OCI namespaces, and +resource selectors inside structured materializer inputs. At readiness the +gateway matches its expected descriptor digest and profile against the +authenticated worker's computed digest and capabilities. At admission it +validates the selected ID, expected output digest, structured inputs, and +resource authorization; the worker resolves the same definition locally and +rejects missing, changed, or unsupported definitions before acquisition. + +Every source mode produces the same versioned workspace manifest. The manifest +separates worker-verified observations from materializer-attested claims and +records the verification method for each identity. It includes requested and +resolved commits or OCI digests, destinations, materializer identity and image +digest when applicable, normalized input and output digests, resulting tree +identities, and completeness. Materializer assertions are not described as +independently verified unless the worker or a configured trusted acquisition +service performs that verification. + +The worker materializes into a worker-owned staging directory under the same +filesystem publication root as the final workspace; readiness rejects a +cross-filesystem layout and publication never falls back to copy-then-delete. +After validating paths, file types, limits, identities, and the manifest, the +worker stops the materializer and removes its credential, process, mount, and +runner boundary. The validated host-owned staging tree remains. The worker then +atomically renames that tree into its final location before profile-owned setup +or any coding agent starts. + +The registered image is part of the deployment's trusted computing base. The +worker launches it through a configured OCI runner or sandbox in a boundary +separate from the agent runtime and never exposes that runner's control socket +to setup or model tools. Phase isolation prevents later code from receiving the +materializer's credentials or mounts, but it cannot make a malicious +operator-registered image safe from credentials intentionally given to it. +Operators must review and pin that image; deployments that do not trust it need +a credential broker or stronger acquisition service that never reveals reusable +credentials to the materializer. + +The canonical source request enters caller idempotency. The resolved +materializer definition digest enters the effective-profile binding, and both +the definition and output-manifest digests enter terminal provenance. New-claim +source authorization always runs before cache lookup. Cache metadata and keys +include the canonical source, materializer-definition digest, authorization +scope digest and revocation epoch, and configured trust domain. Reuse requires +manifest and content revalidation under the current authorization scope; +revocation advances the epoch and makes the old namespace unusable. + +This keeps Harbor's useful separation between content-addressed task acquisition +and environment execution without adopting task-owned opaque source. The +comparison is recorded in +[Harbor repository materialization lessons](../research/harbor-repository-materialization.md). + ### Persist Task truth, not live provider execution The gateway durably stores Task identity, idempotency claims, terminal status, @@ -265,30 +357,39 @@ adopted by this decision. ### Make execution provenance and cleanup explicit -The gateway and selected backend are collectively responsible for: - -1. resolving and verifying immutable source identity; -2. acquiring or restoring source through the selected transport; -3. creating a fresh working location for one execution attempt; -4. running setup before the evaluated agent action; -5. applying permissions and execution isolation; -6. invoking the agent and propagating cancellation and deadlines; -7. capturing bounded output, usage, cost, file changes, checks, and artifact - references; -8. returning terminal status, evidence completeness, and provenance; and -9. terminating processes and releasing or retaining resources according to the - documented lifecycle. - -Source transport and runtime transport are independent. A backend may use one -immutable runtime image plus a separately digest-addressed source artifact; the -contract does not require source code to be baked into the runtime image. +The gateway and selected worker are collectively responsible for: + +1. validating one canonical immutable workspace request and the selected + profile's exact source-resource or materializer authorization; +2. acquiring direct repositories, restoring a digest-pinned OCI snapshot, or + running the registered materializer in a phase-scoped boundary; +3. producing and validating the standard workspace manifest; +4. transferring the validated staging tree to worker ownership, destroying the + acquisition process/mount/credential boundary, and proving it gone; +5. atomically publishing the host-owned tree on the same filesystem; +6. running profile-owned setup before the evaluated agent action; +7. applying permissions and execution isolation; +8. invoking the agent and propagating cancellation and deadlines; +9. capturing bounded output, usage, cost, file changes, checks, artifact + references, workspace identity, and materializer provenance; +10. returning terminal status, evidence completeness, and provenance; and +11. terminating processes and releasing or retaining resources according to + the documented lifecycle. + +Source transport, materializer image, workspace snapshot, and harness runtime +are independent identities. A backend may use one immutable runtime image plus +a separately digest-addressed workspace artifact; the contract does not require +source code to be baked into the runtime image. Credentials remain deployment policy. Requests must not embed deployment -credentials. The gateway authenticates callers, and the selected backend scopes -source and model credentials to the execution boundary without returning -secret-bearing paths or values. Provider and worker-control credentials must -also be absent from model-initiated command environments, tool output, retained -evidence, and repository-visible configuration. +credentials. The gateway authenticates callers, and the selected worker scopes +source credentials to materialization and model credentials to provider +execution without returning secret-bearing paths or values. Materialization +credentials are absent from profile setup, the harness, model-initiated command +environments, tool output, retained evidence, and the published workspace. +Provider and worker-control credentials must likewise be absent from +model-initiated command environments, tool output, retained evidence, and +repository-visible configuration. Retries must not multiply non-idempotent agent execution. Every request carries a caller-scoped stable invocation key through the AllAgents extension. The @@ -322,6 +423,14 @@ but their product and ownership model requires a separate decision. - Gateway and execution workers scale and fail independently. - The gateway can remain lightweight; physical isolation and resource policy belong to the selected execution backend. +- Custom acquisition remains available without making caller-supplied code part + of the trust boundary: operators register digest-pinned materializers and + profiles decide which callers may select them. +- Direct Git, OCI snapshots, and registered materializers converge on one + validated workspace manifest and provenance contract. +- Deployments that enable external materializers must operate their image, + schema, credential, network, resource, and cache policies as worker + configuration. - A2A supplies discovery and lifecycle semantics. AllAgents supplies the coding-specific evidence contract. - W3C Trace Context, OpenTelemetry/OTLP, OpenInference, optional ATIF, and the @@ -362,6 +471,14 @@ Rejected because Harbor's formats own benchmark orchestration, verification, and persisted runner state. The AllAgents gateway executes one coding-agent request and does not become an evaluation harness. +### Let callers provide repository-acquisition code + +Rejected because a caller-selected image, Dockerfile, Compose file, or shell +script would turn request parsing into privileged code execution and would make +credential, network, provenance, and cache policy unreviewable. Callers may +select only source modes and materializer IDs explicitly registered and allowed +by the effective execution profile. + ### Replace A2A with the Agent Host Protocol Rejected because AHP explicitly targets synchronization of independent clients diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index d373b358..86c83707 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -13,11 +13,19 @@ execution: code ## Goal Capsule -- **Objective:** External systems can run Codex or Pi against an immutable repository revision through one authenticated, cancellable, evidence-preserving remote contract. +- **Objective:** External systems can run Codex or Pi against an immutable, + provenance-bearing workspace assembled from exact Git repositories, a + digest-pinned OCI snapshot, or an operator-registered materializer through + one authenticated, cancellable, evidence-preserving remote contract. - **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with two direct provider adapters (KTD1, KTD5, KTD7-KTD8). - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public boundary. The A2A 1.0 specification owns core wire semantics. The versioned AllAgents extension owns coding-execution semantics. - **Execution profile:** Build contract-first, then durable Task/evidence state, worker lifecycle, Codex, Pi, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. -- **Stop conditions:** Do not execute agents in the gateway process, accept mutable source identity, put deployment credentials in requests, treat streams or telemetry as terminal evidence, treat provider sessions as recovery checkpoints, vendor an evaluator's provider implementation, or add evaluation behavior. +- **Stop conditions:** Do not execute agents or materializers in the gateway + process, accept mutable source identity, accept caller-supplied acquisition + code or credentials, put deployment credentials in requests, treat streams + or telemetry as terminal evidence, treat provider sessions as recovery + checkpoints, vendor an evaluator's provider implementation, or add + evaluation behavior. - **Tail ownership:** The implementing workflow runs focused contract and lifecycle tests, the complete repository quality gates, isolated gateway/worker smoke tests, provider-specific credentialed smoke tests where credentials are available, and documentation validation. --- @@ -38,9 +46,15 @@ The two initial runtimes expose different programmatic contracts. Codex provides - A1. **Gateway caller:** An authenticated service such as AI Evals that creates, observes, lists, cancels, and retrieves coding-execution Tasks. - A2. **Execution gateway:** The A2A server that owns caller scope, Task identity, idempotency, routing, retention, and normalized results. -- A3. **Execution worker:** A separately deployed process that owns source materialization, one mutable workspace per invocation, provider execution, evidence capture, and cleanup. -- A4. **Backend adapter:** The Codex or Pi integration that translates native events, structured results, cancellation, usage, failures, and evidence into the worker contract. -- A5. **Operator:** The person or deployment system that defines profiles, credentials, limits, retention, worker endpoints, and observability policy. +- A3. **Execution worker:** A separately deployed process that owns registered + workspace materialization, one mutable workspace per invocation, provider + execution, evidence capture, and cleanup. +- A4. **Backend adapter:** The Codex or Pi integration that translates native + events, structured results, cancellation, usage, failures, and evidence into + the worker contract. +- A5. **Operator:** The person or deployment system that defines profiles, + materializer registrations, credentials, limits, retention, worker + endpoints, and observability policy. ### Key Decisions @@ -48,6 +62,10 @@ The two initial runtimes expose different programmatic contracts. Codex provides - **Keep execution outside the gateway process.** Mutable repositories and provider processes belong to workers. Governs R10-R16, R21-R22. - **Persist Task truth, not live executions.** Accepted Task identity and terminal evidence survive restart; provider sessions do not resume or replay. Governs R7, R14, R16-R18. - **Keep evaluation outside AllAgents.** Dataset expansion, repetitions, assertions, scoring, retries, and durable evaluation Runs remain caller concerns. Governs R20. +- **Make workspace acquisition explicit but extensible.** Requests select one + versioned workspace source mode; custom acquisition uses only + operator-registered, digest-pinned materializers allowed by the profile. + Governs R6, R11-R13, R16-R18, R21-R22. ### Requirements @@ -70,15 +88,78 @@ The two initial runtimes expose different programmatic contracts. Codex provides - R10. Codex and Pi are the complete initial backend set behind one conformance contract, delivered Codex first and Pi second. OpenCode is deferred. (session-settled: user-directed.) - R11. A request selects a server-defined execution profile and may include one `allagents.result-schema/v1` schema for the terminal result: a bounded JSON Schema Draft 2020-12 subset with an object root, every object schema setting `additionalProperties: false`, every declared property listed in `required`, optional values represented by `null` unions, and only `type`, `properties`, `required`, `additionalProperties` with the value `false`, `items`, `enum`, `const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`. The extension version fixes byte, depth, property, and enum limits; admission rejects remote references, format-dependent validation, and unknown keywords; one shared validator governs schema admission and returned values. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. Requests cannot supply raw provider configuration. -- R12. The only initial remote source form is a canonical credential-free HTTPS Git URL plus full commit object ID and optional repository-relative subdirectory. Acquisition revalidates destination policy for every connection, disables redirects and repository-controlled secondary fetch/exec features, uses hermetic Git configuration, and verifies that the fetched object is the requested commit before setup. +- R12. One request defines one workspace using exactly one closed source union: + `{ kind: "repositories", repositories: [...] }`, + `{ kind: "workspaceSnapshot", reference, workspaceManifestDigest }`, or + `{ kind: "materializer", materializerId, + expectedWorkspaceManifestDigest, inputs }`. Unknown kinds, fields from + another variant, and omitted variant fields fail admission. Repository + entries contain a canonical credential-free HTTPS Git URL, full commit object + ID, collision-free relative destination, and optional repository-relative + subdirectory. Snapshot references are digest-pinned OCI artifacts containing + the versioned workspace manifest. Materializer inputs are bounded by the + registered schema. Profiles explicitly allow source modes and materializer + IDs and authorize exact canonical Git repositories or namespaces, OCI + namespaces, and resource selectors inside materializer inputs. Callers cannot + supply builder images, Dockerfiles, Compose files, shell commands, + credentials, mutable image tags, network policy, or output contracts. Direct + Git revalidates destination policy for every connection, disables redirects + and repository-controlled secondary fetch/exec features, uses hermetic Git + configuration, fetches into an isolated object database from the approved + remote, and verifies that the checked-out commit equals the requested full + object ID. OCI acquisition rejects external or foreign layer URLs by default, + revalidates scheme, normalized host, resolved address, port, and redirects + for registry, authentication, manifest, and blob connections, never forwards + credentials across origins, and verifies every manifest and layer digest. A + materializer output must match the request's expected workspace-manifest + digest before publication. - R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, structured logs/spans before processing or export, and every model-initiated command or tool environment. Credentialed profiles additionally require an OS-enforced provider/tool credential boundary: the credential-bearing provider runtime and model-invoked tools use distinct UID/process/mount policy that prevents tool access to provider processes, procfs entries, and backend config/data roots, or an equivalent credential broker keeps reusable credentials out of the agent runtime. Worker readiness fails when the declared boundary cannot be proved; environment filtering alone is not credential isolation. + Materializer IDs are defined in an operator-owned deployment registry. The + gateway holds only the non-secret ID, bounded input schema, expected + definition digest, expected output-manifest version, and required worker + capabilities; the worker holds the runtime definition with the digest-pinned + image, credential handle names or mount identities, network destinations, + resource/deadline ceilings, cache policy, output-manifest version, and OCI + runner or sandbox capability. Credential values are excluded. The worker + derives an algorithm-qualified `sha256:<64 lowercase hex>` definition digest + from a versioned, domain-separated canonical serialization of every + non-secret behavior-affecting field and advertises it at readiness; the + gateway treats its copy only as the expected digest. The expected workspace + manifest and canonical materializer-input digests use the same + algorithm-qualified format with distinct domain separators and exact + versioned canonical JSON preimages. Readiness fails when computed and expected + descriptors differ across the authenticated route. Materialization + credentials exist only in that isolated phase and are not supplied to setup, + provider execution, or model tools. The registered image is operator-trusted + deployment code: phase isolation protects later phases but cannot make a + malicious registered image safe from credentials deliberately given to it. + Deployments requiring that stronger claim use a credential broker or + acquisition service that withholds reusable credentials. - R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. - R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. - R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves the complete invocation process set quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. If bounded termination escalation cannot prove quiescence, the worker persists termination as unknown/failed, poisons admission, and exits so the external supervisor destroys the complete process boundary; replacement readiness performs orphan recovery before accepting work. The same supervisor boundary handles a worker crash. + Every source mode materializes into a worker-owned staging directory under + the same filesystem publication root as the final workspace and produces the + same versioned workspace manifest. Readiness rejects cross-filesystem roots + and publication has no copy-then-delete fallback. The worker validates + repository or snapshot identities, destinations, paths, file types, limits, + materializer definition and image digests, output digest, provenance method, + and completeness. It then terminates the supervisor-owned acquisition + process, mount, runner, and credential boundary while preserving the + host-owned validated staging tree, proves that boundary gone, atomically + renames the tree into its final location, and only then starts setup. **Evidence and observability** - R17. Every terminal Task contains the required `allagents.execution-integrity` Artifact carrying an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one additional `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. `not_produced` is legal only before a result candidate is produced. Once validation selects `valid` or `invalid`, later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the fixed structured-result Artifact while the later phase remains the primary Task failure classification. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. + Workspace provenance includes the source mode, requested and resolved + repository commits or OCI digests, destination map, workspace-manifest + digest, and, when applicable, materializer ID, computed definition digest, + image digest, canonical input digest, and output digest. It labels each field + as a worker-verified observation, trusted-service verification, or + materializer-attested claim and records the verification method; a custom + image's assertion is never reported as independently verified merely because + its output digest matched. - R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. - R19. Gateway and worker calls propagate W3C Trace Context and export metadata-only OpenTelemetry data. One explicit pre-processor allowlist admits only bounded non-content operational metadata; OpenInference and backend-native attributes pass the same allowlist and bounded filtering/redaction before any structured log or span processor. Prompts, model outputs, tool arguments/results, file bodies, source fragments, and secret-bearing attributes are prohibited before export. Owner correlation uses only an opaque identifier appropriate to telemetry-operator access, never caller identity or Task/Artifact authorization. Telemetry access and retention are configured separately from Task and Artifact access and retention, and telemetry is neither durable result truth nor required for terminal lookup. @@ -87,6 +168,14 @@ The two initial runtimes expose different programmatic contracts. Codex provides - R20. The gateway executes one coding request. It does not own eval configuration, datasets, repetition, scoring, retry policy, experiment scheduling, or a durable evaluation Run ledger. - R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. R13's narrow OS-enforced provider/tool credential boundary is required for credentialed profiles but does not claim hostile-source or cross-tenant isolation. Profiles making either stronger claim are rejected until a full per-invocation UID, mount, PID, network, and credential isolation boundary is configured. - R22. Gateway admission and worker execution enforce profile limits for request rate, active/retained Tasks, subscriptions, stored bytes, source transfer/expansion, files/inodes, workspace bytes, CPU, memory, PIDs, network, phase deadlines, events, logs, and artifacts. Exhaustion is scoped to one invocation or owner and leaves capacity for terminalization and cleanup. + Materializer CPU, memory, PIDs, network, time, transfer, expansion, file, + inode, and workspace output count against the invocation's limits. New-claim + source authorization precedes every cache lookup. Cached outputs are reusable + only after manifest and content revalidation for the same canonical source, + materializer-definition digest, expected and actual output-manifest digests, + authorization-scope digest, source-authorization revocation epoch, and trust + domain. Revocation advances the epoch and makes the prior namespace + ineligible; the conservative default namespaces cache entries by owner. ### Key Flows @@ -123,26 +212,59 @@ The two initial runtimes expose different programmatic contracts. Codex provides ### Acceptance Examples -- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, an exact Git SHA, and an optional result schema, when the caller streams a request, then one Task moves from submitted to working to completed and later `GetTask` returns the same validated output and evidence Artifacts. +- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, one valid + immutable workspace source, and an optional result schema, when the caller + streams a request, then one Task moves from submitted to working to completed + and later `GetTask` returns the same validated output, workspace provenance, + and evidence Artifacts. - AE2. **Covers R6.** Given a retained Task whose original absolute deadline has passed or whose profile is now disabled, changed, or no longer authorized for new work, when its owner reuses the invocation key with the same canonical request and result schema, then the gateway returns the original Task from its stored original bindings before mutable admission checks and makes no second worker dispatch. - AE3. **Covers R6.** Given a retained Task, when its owner reuses the invocation key with a different prompt, source, profile ID, deadline, or result schema, or the stored original profile/schema binding is inconsistent, then the gateway rejects the request and leaves the original Task unchanged. - AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. -- AE5. **Covers R12, R16-R18.** Given a requested SHA that does not match the materialized repository or setup fails, when the worker has already accepted the current fence, then provider execution never starts, the selected public trace is `Submitted -> Working -> Failed`, and the Task retains source/setup-failure and cleanup evidence. +- AE5. **Covers R12, R16-R18.** Given a wrong or missing Git commit, + conflicting repository destination, OCI digest or manifest mismatch, unknown + or profile-disallowed materializer, materializer definition/image drift, + produced workspace-manifest digest that differs from the request, malformed + materializer output, direct known-secret disclosure, or setup failure, when + the worker has already accepted the current fence, then provider execution + never starts, the selected public trace is + `Submitted -> Working -> Failed`, and the Task retains bounded + materialization/setup-failure and cleanup evidence. - AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. Given cancel reaches a worker before its delayed dispatch, the worker tombstones the unseen attempt and the stale dispatch creates no workspace or provider process. - AE7. **Covers R3, R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both publish the required fixed-name integrity Artifact at the schema-defined extension carrier, validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. - AE8. **Covers R4, R7, R19.** Given canary secrets and cross-owner content fragments in prompts, model output, tool arguments/results, source files, stale events, and errors, when agent, model, tool, stale-event, and error telemetry is processed, then the exporter receives only allowlisted bounded metadata plus the correct opaque owner correlation and receives none of those canaries, fragments, or raw caller identities. Given a caller or exporter disconnects during work, reconnect still returns the current Task and future updates without duplicate dispatch, and telemetry loss does not affect terminal lookup. - AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. - AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. Given output validation has already selected `valid` or `invalid` and a later check or mandatory-evidence phase fails, the failed Task preserves that result state and a valid result preserves its one fixed structured-result Artifact; only a failure before candidate production records `not_produced`. -- AE11. **Covers R6, R22.** Given invalid input or exhausted admission quota, the gateway returns a request/resource error and creates no Task; given capacity disappears after durable acceptance, the retained Task fails at dispatch and replay returns it without retry. +- AE11. **Covers R6, R11-R12, R22.** Given invalid input, an unknown or + profile-disallowed source mode/materializer, or exhausted admission quota, + the gateway returns a request/resource error and creates no Task; given + materializer availability or worker capacity disappears after durable + acceptance, the retained Task fails at dispatch or materialization and replay + returns it without retry. - AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only allowlisted metadata-only operator telemetry. Given bounded escalation cannot stop a descendant that starts a new session and ignores graceful signals, the worker persists termination unknown/failed, refuses another reservation, exits, and its supervisor destroys the boundary; replacement readiness performs orphan recovery without changing the failed Task. - AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. -- AE14. **Covers R5, R13, R21-R22.** Given a production public listener or remote worker route lacks its configured trusted transport or authenticated peer identity, readiness fails; a same-host Unix worker socket is accepted. Given a credentialed reviewed-domain profile, model tools cannot inspect provider process environments, process listings, backend config/data roots, or exfiltrate provider/control credentials across the configured OS boundary. Hostile-source or cross-tenant claims remain rejected. +- AE14. **Covers R5, R13, R21-R22.** Given a production public listener or + remote worker route lacks its configured trusted transport or authenticated + peer identity, readiness fails; a same-host Unix worker socket is accepted. + Given a credentialed reviewed-domain profile, model tools cannot inspect + provider process environments, process listings, backend config/data roots, + or exfiltrate provider/control credentials across the configured OS + boundary. Given a registered materializer with source credentials, setup, + provider processes, and model tools have no access to its process, runner + control socket, credential environment/mounts, or staging root after + materialization, and direct known-secret canaries are absent from retained + logs, evidence, and published workspace files. The registered materializer + remains operator-trusted code; hostile-materializer, hostile-source, and + cross-tenant claims remain rejected without a stronger broker or sandbox. ### Success Criteria - The official A2A JavaScript client can discover the required extension, negotiate it through `A2A-Extensions`, use the standard Message and Artifact extension carriers, and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, retained replay, cancel, and expiry behavior against the built service without `Task.extensions`. - One conformance fixture passes unchanged through the Codex and Pi adapters. - Admission, retained replay, monotonic worker commands, fencing, acceptance-before-materialization source/setup failure, cancellation races, trace-order/fence/multiplicity constraints, failed-quiescence recycling, restart terminalization without resume, supervised worker-crash cleanup, trusted transports, authorization isolation, metadata-only telemetry export, OS-enforced provider/tool credential separation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. +- Direct multi-repository Git, digest-pinned OCI snapshots, and a fake + digest-pinned registered materializer all produce the same validated + workspace manifest and terminal provenance contract before either backend + starts. - The gateway image contains no coding-agent runtime and cannot access worker workspace roots. - The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools are OS-isolated from provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed, failed-quiescence, or crashed attempt. @@ -156,6 +278,9 @@ The two initial runtimes expose different programmatic contracts. Codex provides - Built-in bearer authentication with OIDC/JWT and static service-token modes behind the named production TLS boundary. - Single-replica durable file storage, authenticated Artifact retrieval, authenticated encrypted remote worker transport or same-host Unix sockets, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. - Reviewed repositories in one configured mutual-trust domain per worker deployment, with the narrow OS-enforced provider/tool credential boundary required for credentialed profiles. +- Direct multi-repository Git acquisition, digest-pinned OCI workspace + snapshots, and operator-registered digest-pinned materializers with + phase-scoped credentials and one standard workspace manifest. **Deferred to follow-up work** @@ -176,6 +301,7 @@ The two initial runtimes expose different programmatic contracts. Codex provides - [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) +- [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) - [AI Evals ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) - [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) @@ -204,15 +330,69 @@ The two initial runtimes expose different programmatic contracts. Codex provides - KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates a new invocation claim and submitted Task together after mutable admission, storing the canonical caller request and the original effective-profile and result-schema digests needed for retained replay. It stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. - KTD4. **Authenticate at a named trusted HTTP ingress before A2A storage or dispatch.** Production traffic reaches the gateway through TLS terminated by the configured gateway or named trusted reverse-proxy boundary; plaintext is allowed only for an unauthenticated loopback development listener. Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Readiness rejects a production public URL whose trusted TLS boundary is absent or inconsistent. Governs R5-R6, R13. - KTD5. **Use fenced, separately deployable gateway and worker services.** Remote gateway-worker routes use mTLS or an explicitly equivalent authenticated encrypted overlay; a same-host Unix socket is acceptable. The authenticated worker identity is pinned to the configured route/capability set, and every short-lived attempt capability is bound to that identity, attempt ID, lease ID/epoch, and fence. Each worker keeps one minimal durable monotonic command record scoped to its worker identity and lease: `Cancel(attempt, fence, revision)` tombstones even an unseen attempt, and `Dispatch` for a tombstoned or lower-revision attempt is rejected before workspace creation. Dispatch/cancel I/O conditionally verifies the persisted command/outbox revision immediately before any mutating or terminating effect. Duplicate delivery is idempotent; conflicting, stale, out-of-order, or identity-mismatched commands/events are rejected. Gateway and worker transition selectors remain pure and executors re-enter from persisted or freshly observed state. This record is worker-local fence state, not a new durable execution subsystem. Governs R5, R7, R10-R16, R21-R22. -- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Worker-process exit makes the external supervisor terminate the complete execution boundary. If bounded escalation cannot prove the complete invocation process set empty, the worker records termination unknown/failed, poisons admission, and exits rather than accepting another reservation; its supervisor destroys the boundary. Before readiness the replacement proves termination and reaps or quarantines orphaned invocation roots. Production readiness accepts a dedicated worker container process namespace under a minimal init/reaper as the baseline; a non-container deployment must prove an equivalent systemd/cgroup boundary. The gateway never reattaches to or resumes a provider session. Governs R7, R14, R16-R18. +- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Every external materializer launch creates a supervisor-owned runner resource labeled by worker, attempt, lease, and fence; worker-process exit makes the external supervisor terminate that resource and the complete execution boundary. If bounded escalation cannot prove the complete invocation process set empty, the worker records termination unknown/failed, poisons admission, and exits rather than accepting another reservation; its supervisor destroys the boundary. Before readiness the replacement proves termination and enumerates, destroys, or quarantines orphaned invocation roots, runner resources, credential mounts, and staging mounts; an unresolved resource keeps readiness false. Production readiness accepts a dedicated worker container process namespace under a minimal init/reaper as the baseline; a non-container deployment must prove an equivalent systemd/cgroup boundary. The gateway never reattaches to or resumes a provider session. Governs R7, R14, R16-R18. - KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, optional structured result, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, schema validation, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | pi` registry is the only production dispatch point. Governs R10-R11, R14-R18, R21-R22. - KTD8. **Use each provider's supported automation surface directly behind the credential boundary.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, and consumes streamed events. Pi uses strict RPC with an invocation-local credential store and one explicitly loaded worker-owned policy extension; repository extensions and unrestricted built-ins never load. For either adapter, a credentialed provider runtime is separated from every model-invoked tool by the R13 OS-enforced UID/process/mount boundary or an equivalent credential broker; shell-environment filtering is defense in depth, not the boundary. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. -- KTD9. **Make profiles the new-admission policy boundary.** Requests select a profile ID and may provide only an `allagents.result-schema/v1` schema. They cannot override backend credentials, executable paths, provider config, setup/check commands, environment allowlists, permission rules, trust class, resource limits, workspace retention, or evidence budgets. For a new claim, resolve a versioned canonical `EffectiveProfileIntent`, compute its digest without resolved secrets or per-attempt state, and persist it with the canonical caller request and result-schema digest. Retained replay compares those stored original bindings and never substitutes or re-resolves the current profile. Governs R6, R11-R16, R21-R22. +- KTD9. **Make profiles the new-admission policy boundary.** Requests select a + profile ID, one schema-defined workspace source mode, and optionally one + `allagents.result-schema/v1` schema. They cannot override backend or source + credentials, materializer definitions or images, executable paths, provider + config, setup/check commands, environment allowlists, permission rules, + trust class, resource limits, workspace retention, or evidence budgets. A + profile allowlists source modes and materializer IDs plus exact canonical Git + repository/namespace rules, OCI namespaces/signature rules, and resource + selectors for structured materializer inputs. New admission authorizes the + fully canonicalized resource and credential entitlement before cache lookup. + Resolve a versioned canonical `EffectiveProfileIntent` containing the + selected materializer definition digest, authorization-scope digest, and + source-authorization revocation epoch when applicable; compute its digest + without resolved secrets or per-attempt state and persist it with the + canonical caller request and result-schema digest. Retained replay compares + those stored original bindings and never substitutes or re-resolves current + policy. Governs R6, R11-R16, R21-R22. - KTD10. **Keep durable evidence and operational telemetry as separate bounded layers.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded evidence layer; neither Git nor provider evidence is promoted as exact causality when incomplete. Telemetry is a third, non-durable metadata-only channel: one small shared pre-export sanitizer applies an explicit operational-metadata allowlist plus bounded filtering/redaction before every structured log or span processor, and only opaque owner correlation may cross the separately governed operator boundary. OpenInference and backend-native attributes receive no bypass. This is an export guard, not a telemetry framework or alternate evidence store. Governs R13, R16-R19. - KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. - KTD12. **Separate terminal integrity from optional evidence bodies.** The fixed `allagents.execution-integrity` Artifact validates identity, action outcome, the four-state structured-result record, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance before terminal publication. `not_produced` applies only before result-candidate production. Once validation selects `valid` or `invalid`, a later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the separate fixed structured-result Artifact while retaining the later phase as the primary Task failure. Predictable optional-body truncation/redaction may preserve completion; failure that breaks the integrity kernel fails in the evidence phase. Governs R3-R4, R17-R18. -- KTD13. **Harden Git acquisition as a network security boundary.** Accept canonical HTTPS origins only. Use hermetic Git configuration, disable redirects, proxies, helpers, hooks, filters, LFS smudge, submodule recursion, alternates, and non-HTTPS protocols. Revalidate normalized host/address policy for every connection, never forward credentials across origins, and verify the full object ID resolves to a commit fetched from the approved remote. Governs R12-R13, R22. +- KTD13. **Standardize and harden workspace materialization.** Define one + closed `kind`-discriminated workspace-source union and one output manifest; + reject unknown kinds and cross-variant fields. The built-in Git path accepts + canonical HTTPS repository identities and full commit IDs only, uses + hermetic Git configuration, disables redirects, proxies, helpers, hooks, + filters, LFS smudge, submodule recursion, alternates, and non-HTTPS + protocols, revalidates normalized host/address policy for every connection, + fetches into an isolated object database from the authorized remote, and + verifies the checked-out commit and resulting tree. The OCI path accepts + manifest digests, not tags; rejects foreign/external URLs by default; + revalidates scheme, host, resolved address, port, redirect, and credential + origin for every registry/auth/manifest/blob request; and verifies every + manifest/layer plus the embedded workspace manifest. The custom path accepts + a registered ID, expected workspace-manifest digest, and schema-validated, + resource-authorized inputs. + + The operator-owned registry splits a non-secret gateway descriptor from the + worker-only runtime definition. The worker computes a + `sha256:<64 lowercase hex>` digest over the versioned, domain-separated + canonical non-secret runtime definition; readiness compares that value with + the gateway's expected digest and capabilities. Distinct domain-separated + canonical JSON preimages define materializer input and workspace-manifest + digests. All paths stage in a worker-owned directory on the final + publication filesystem, validate destinations, links, file types, bounds, + identities, content, and manifest, then terminate the supervisor-owned + acquisition process/mount/credential/runner boundary while retaining the + validated host-owned tree. Only after proving the boundary gone does the + worker atomically rename the tree; no copy fallback exists. Provenance + distinguishes worker-verified observations, trusted-service verification, + and materializer-attested claims. The caller digest covers source kind, + expected output identity, and inputs; the effective-profile digest covers + materializer and authorization bindings; terminal provenance covers both and + the validated output. Governs R6, R11-R13, R16-R18, R21-R22. - KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Credentialed profiles still require R13's narrower OS-enforced provider/tool separation: model tools cannot inspect provider processes, procfs entries, or backend config/data roots, and readiness fails without that capability. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load. This credential boundary does not imply hostile-source or cross-tenant isolation; that stronger sandbox-driver capability remains deferred. Governs R13, R16, R21-R22. + An external materializer image is reviewed operator code in the deployment's + trusted computing base, not hostile caller code. Its runner or sandbox + control plane is never mounted into the workspace or exposed to setup, + providers, or model tools. A deployment that does not trust the registered + image with source credentials must use a broker or stronger acquisition + service and advertise that capability explicitly. - KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. ### High-Level Technical Design @@ -225,7 +405,10 @@ flowchart TB Gateway --> Auth[Auth, retained replay, new admission] Gateway --> Store[Generation-based Task and Artifact store] Gateway -->|mTLS/authenticated overlay or same-host Unix socket| Worker[Single-execution worker] - Worker --> Source[Hardened Git acquisition] + Worker --> Materialization[Workspace materializer registry] + Materialization --> Git[Hardened multi-repository Git] + Materialization --> OCI[Digest-pinned OCI snapshot] + Materialization --> Custom[Registered materializer image] Worker --> Registry[Closed backend registry] Registry --> Codex[Codex SDK] Registry --> Pi[Pi RPC process] @@ -260,7 +443,8 @@ sequenceDiagram G->>W: Dispatch(attempt, fence, command revision) W->>W: Verify command record before workspace creation W-->>G: Accepted(attempt, fence) - W->>W: Materialize, verify, setup, baseline + W->>W: Materialize into staging and validate workspace manifest + W->>W: Destroy acquisition boundary, publish atomically, setup, baseline W->>B: Invoke with isolated roots and credential boundary B-->>W: Progress, usage, native evidence W-->>G: Sequenced fenced progress @@ -356,6 +540,12 @@ packages/execution-service/ reaper.ts lease.ts workspace.ts + materializers/ + types.ts + registry.ts + git.ts + oci.ts + external.ts evidence.ts adapters/ types.ts @@ -383,10 +573,51 @@ docs/src/content/docs/ ### Configuration Contract -- Gateway configuration defines the listener/public URL, a named trusted TLS termination boundary for production ingress, auth and canonical owner mapping, store/retention, admission and subscription quotas, low-space watermarks, Artifact limits, worker routes, internal capability secrets, and profiles. Each remote worker route declares mTLS or an explicitly equivalent authenticated encrypted overlay, pinned worker identity/capabilities, and trust material; a same-host route may declare a Unix socket. Plaintext remote URLs are invalid. -- Each profile defines backend, worker route, allowed Git origins/addresses, provider/model settings, phase-specific environment allowlists, deterministic permissions, setup/check commands, artifact globs, effective deadline ceiling, trust class, resource limits, cleanup policy, evidence budgets, and the required provider/tool credential-boundary capability for credentialed execution. -- Worker configuration fixes a private listener, worker identity, one-execution concurrency, workspace root, minimal worker-local command-record location, execution-supervisor mechanism, pre-readiness orphan policy, lease grace, backend runtime constraints, trust domain, resource-control and provider/tool credential-boundary capabilities, and request/result limits. -- Production worker readiness requires authenticated route identity, protected remote transport or a same-host Unix socket, an enforceable credential boundary for every credentialed profile, and a supervisor that proves complete descendant termination and root ownership. The supported supervisor baseline is a dedicated worker container process namespace under a minimal init/reaper; bare-host deployment requires an equivalent systemd/cgroup mechanism. +- Gateway configuration defines the listener/public URL, a named trusted TLS + termination boundary for production ingress, auth and canonical owner + mapping, store/retention, admission and subscription quotas, low-space + watermarks, Artifact limits, worker routes, internal capability secrets, + non-secret materializer descriptors, and profiles. A descriptor contains the + materializer ID, bounded input schema, expected definition digest, expected + output-manifest version, and required worker capabilities. Each remote worker + route declares mTLS or an explicitly equivalent authenticated encrypted + overlay, pinned worker identity/capabilities, source modes and matching + materializer definition digests, and trust material; a same-host route may + declare a Unix socket. Plaintext remote URLs are invalid. +- Each profile defines backend, worker route, allowed workspace source modes, + allowed materializer IDs, exact Git repository or namespace rules, allowed + Git origins/addresses, OCI namespace/registry/signature policy, structured + materializer-input resource selectors, authorization-scope derivation and + source-authorization revocation epoch, provider/model settings, + phase-specific environment allowlists, deterministic permissions, + setup/check commands, artifact globs, effective deadline ceiling, trust + class, resource limits, cleanup policy, evidence budgets, and required + acquisition/provider/tool isolation capabilities. +- Worker configuration fixes a private listener, worker identity, + one-execution concurrency, one same-filesystem publication root containing + private staging and final workspace directories, a closed materializer + runtime registry, minimal worker-local command-record location, + execution-supervisor mechanism, pre-readiness orphan policy, lease grace, + backend runtime constraints, trust domain, resource-control and + credential-boundary capabilities, and request/result limits. Each external + materializer runtime entry matches the gateway descriptor's ID and expected + definition digest and additionally fixes a digest-pinned image, credential + handle names or mount identities, network destinations, resource/deadline + limits, cache policy, output version, and OCI runner or sandbox; it contains + no credential values. The worker derives, rather than trusts, the definition + digest from that complete non-secret runtime entry. +- Production worker readiness requires authenticated route identity, protected + remote transport or a same-host Unix socket, exact agreement between the + gateway's expected descriptor digest and the worker's computed runtime + definition digest, a same-filesystem staging/publication root with atomic + rename and no copy fallback, and an OCI materializer runner or sandbox that + assigns supervisor-owned attempt/lease/fence labels without exposing its + control plane to the workspace. It also requires an enforceable credential + boundary for every credentialed phase and a supervisor that proves complete + descendant termination and enumerates or destroys orphan runner resources, + credential mounts, staging mounts, and roots before readiness. The supported + worker baseline is a dedicated process namespace under a minimal init/reaper; + bare-host deployment requires an equivalent systemd/cgroup mechanism. - Telemetry configuration defines the OTLP destination, filtering/redaction bounds, opaque owner-correlation derivation, and telemetry-specific operator access and retention. The service version fixes the metadata allowlist; configuration cannot extend it to prompt/output/tool/source/file-body attributes, secret-bearing fields, raw caller identity, or unfiltered backend-native/OpenInference attribute passthrough. - Configuration contains environment-variable names but never secret values. Startup resolves the complete graph and becomes ready only when trusted ingress, worker transports/identities, store, runtimes, quotas, free-space reserves, supervisor/orphan recovery, and declared profile capabilities pass. Any unprotected remote endpoint or unproved credential/supervisor boundary fails readiness. @@ -394,14 +625,14 @@ docs/src/content/docs/ | Condition | A2A result | Required extension detail | |---|---|---| -| New-admission authentication, malformed/unsupported extension carrier, invalid source/profile, unauthorized policy, expired deadline, current-profile/readiness failure, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | +| New-admission authentication, malformed/unsupported extension carrier, invalid workspace source/profile, unknown or profile-disallowed materializer, unauthorized policy, expired deadline, current-profile/readiness failure, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | | Identical retained invocation replay | Existing Task | Returned from stored original request/profile/schema bindings before current deadline, quota, authorization, readiness, or profile checks; no new Task, worker attempt, or quota reservation | | Conflicting invocation key or inconsistent stored binding | Operation error; no new Task | Conflict code; existing Task unchanged | | Worker capacity loss after acceptance | `TASK_STATE_FAILED` | `dispatch/capacity_exhausted`, retriable fact, no workspace created; gateway does not retry | | Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | | Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | | Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | -| Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary source/setup/provider/dispatch/crash/infrastructure phase, safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness | +| Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary dispatch/materialization/setup/provider/crash/infrastructure phase, including manifest or materializer failure; safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness, and bounded workspace provenance | | Check, mandatory-evidence, cleanup, crash, or infrastructure failure after result validation | `TASK_STATE_FAILED` | Preserve selected `valid` or `invalid`; preserve exactly one fixed structured-result Artifact for `valid`; later phase remains primary failure | | Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` with `not_produced`, or `structured_result/invalid` with `invalid`; no structured-result Artifact | | Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors and native abort; use `not_produced` only before a candidate, otherwise preserve `valid`/`invalid` and the valid Artifact; record termination and cleanup | @@ -415,7 +646,11 @@ docs/src/content/docs/ 1. Create the private Node 22 service package and freeze the public extension URI and standard carriers, integrity and structured-result Artifacts, portable result-schema subset, worker protocol including command revisions/tombstones, profiles, fixtures, and error vocabulary. 2. Build authenticated durable A2A Task handling, retained-claim-first replay, and trusted fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. -3. Build the supervised single-execution worker lifecycle, monotonic command record, failed-quiescence boundary recycling, pre-readiness orphan reaper, OS credential boundary, and hardened source/evidence handling against a fake adapter. +3. Build the supervised single-execution worker lifecycle, monotonic command + record, failed-quiescence boundary recycling, pre-readiness orphan reaper, + OS credential boundaries, the direct Git/OCI/registered-materializer + registry, and hardened workspace/evidence handling against fake + materializers and a fake backend. 4. Add the direct Codex SDK adapter and prove structured output, cancellation, OS-enforced provider/tool credential separation, and native evidence. 5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing OS-confined tools plus the terminating result tool. 6. Package the services and run cross-backend, transport, security, process, and A2A conformance before enabling a consumer. @@ -425,7 +660,15 @@ docs/src/content/docs/ - **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. - **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. - **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. -- **Security:** New review-critical surfaces are trusted public/private transports, auth, owner-key derivation, retained-replay ordering, source SSRF, admission/resource quotas, setup/check policy, OS-enforced provider/tool credential separation, Pi extension/tool replacement, phase-scoped secrets, metadata-only telemetry filtering and operator boundaries, internal fences and monotonic command records, Artifact capture/serving, and reviewed-source trust enforcement. +- **Security:** New review-critical surfaces are trusted public/private + transports, auth, owner-key derivation, retained-replay ordering, Git and OCI + source SSRF, materializer image supply chain, materializer input schemas, + phase-scoped source credentials and egress, workspace manifest validation, + admission/resource quotas, setup/check policy, OS-enforced provider/tool + credential separation, Pi extension/tool replacement, metadata-only + telemetry filtering and operator boundaries, internal fences and monotonic + command records, Artifact capture/serving, and reviewed-source trust + enforcement. - **Operations:** Gateway and worker health, readiness, transport/peer identity, quotas, low-space state, allowlisted metadata-only structured logs/traces, telemetry-specific access/retention, command tombstones, lease expiry, poisoned-worker exit, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. - **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. @@ -439,7 +682,15 @@ docs/src/content/docs/ - **Orphan processes and roots:** Combine explicit cancel, native abort, process-set verification, one-execution supervisor/container death, lease expiry, and pre-readiness orphan reaping or quarantine. Failed quiescence poisons admission and exits the worker so the supervisor destroys the boundary; termination/filesystem outcomes remain separate. - **False recovery claims:** Persist Task and evidence truth only. Startup fails active Tasks, invalidates fences, and relies on lease expiry or supervisor-boundary proof instead of resuming provider sessions. - **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in provenance and original claim bindings, pass the exact accepted schema through each adapter, validate with one shared validator, preserve an already selected result across later failures, and enforce the two fixed Artifact shapes. -- **Source SSRF or credential leakage:** Enforce KTD13 for every connection and phase. Credentials are ephemeral and origin-bound. Credentialed profiles also enforce the R13 OS provider/tool boundary or broker; environment filtering remains defense in depth. Pi repository extensions and unrestricted built-in tools never load. +- **Source SSRF, materializer compromise, or credential leakage:** Enforce + KTD13 for every source mode, connection, and phase. Pin external + materializer images and OCI snapshots by digest, validate their manifests, + isolate staging and acquisition processes, apply explicit egress and limits, + and atomically publish only validated outputs. Source credentials are + ephemeral, origin-bound, and removed before setup. Credentialed profiles also + enforce the R13 OS provider/tool boundary or broker; environment filtering + remains defense in depth. Pi repository extensions and unrestricted built-in + tools never load. - **Telemetry disclosure:** Apply KTD10's pre-export guard before every structured log/span processor and reject content or secret-bearing attributes rather than relying on exporter policy. Canary-secret and cross-owner-fragment tests cover agent, model, tool, stale-event, and error paths; telemetry operators receive only bounded metadata and opaque owner correlation under separate access and retention. - **Resource exhaustion:** Reserve per-owner/global gateway quota only for new claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. - **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. @@ -451,7 +702,10 @@ docs/src/content/docs/ ### Assumptions - The first production deployment runs one gateway replica with persistent storage. Multi-replica transactional storage is deferred. -- Git over hardened HTTPS and exact commit object ID covers the initial consumer. Other source transports require a later extension version or capability. +- The initial public extension supports direct multi-repository Git, + digest-pinned OCI workspace snapshots, and operator-registered materializers. + A deployment may enable only the source modes its worker route advertises; + direct hardened Git remains the required baseline. - Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. - Initial repositories are reviewed inside one configured mutual-trust domain. Credentialed profiles still enforce provider/tool credential separation, but that narrower boundary does not make hostile-code or cross-tenant execution available; those claims require a stronger sandbox driver. - Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. @@ -462,17 +716,54 @@ docs/src/content/docs/ ### U1. Versioned public and worker contracts -- **Goal:** Freeze the standard public extension carriers, integrity and structured-result Artifacts, profile vocabulary, private worker protocol including monotonic command state, original idempotency bindings, typed failures, and conformance fixtures before either service endpoint. +- **Goal:** Freeze the standard public extension carriers, versioned workspace + source and manifest contracts, materializer/profile vocabulary, integrity and + structured-result Artifacts, private worker protocol including monotonic + command state, original idempotency bindings, typed failures, and conformance + fixtures before either service endpoint. - **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. - **Dependencies:** None. - **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/result-schema-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. - **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas and freeze `https://allagents.dev/a2a/extensions/coding-execution/v1`: required Agent Card advertisement, `A2A-Extensions` negotiation, `Message.extensions`, request data only at `Message.metadata[uri]`, and terminal integrity data only in the single Part of the fixed-name `allagents.execution-integrity` Artifact whose `extensions` contains the URI. Explicitly forbid `Task.extensions`. Define the portable result-schema subset, canonical caller/schema/profile digests, four result states, separate fixed `allagents.structured-result` Artifact, and shared validator. Define original claim bindings independently from mutable current policy. Add worker identity, attempt/fence/lease identity, monotonic command revision, unseen-attempt cancel tombstone, conditional effect revision, event sequence, terminal acknowledgement, and integrity rules. Generate checked-in schemas and fixtures from one source. -- **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions, unsafe paths, invalid public states, stale fences, oversized records, and conflicting canonical inputs before implementing schemas. + The source contract is a strict `kind`-discriminated union for direct + repository lists, digest-pinned OCI snapshots, or a registered materializer + ID with an expected workspace-manifest digest and schema-validated structured + inputs; cross-variant fields are unrepresentable. Define the standard + workspace manifest, verification-method vocabulary, authorization scope and + revocation epoch, collision-safe destinations, and split gateway/worker + materializer descriptors. Define algorithm-qualified digest formats and + versioned, domain-separated canonical preimages for materializer definitions, + inputs, manifests, profiles, and caller requests; no public field can carry + acquisition code, image references, commands, credentials, or policy. +- **Execution note:** Start with fixture-driven schema, framing, and digest + tests. Observe failures for unknown versions, credential-bearing sources, + mutable revisions or image tags, duplicate/unsafe destinations, unknown or + disallowed materializers, invalid structured inputs or workspace manifests, + unsafe paths, invalid public states, stale fences, oversized records, and + conflicting canonical inputs before implementing schemas. - **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, `src/core/native/types.ts` for safe error/provenance normalization, and Buzz's structurally non-secret intent template for the narrow digest-input pattern. - **Test scenarios:** - A minimal valid Message negotiates the exact URI in `A2A-Extensions`, includes it in `Message.extensions`, puts the bounded request only at `Message.metadata[uri]`, and produces a stable digest across object-key ordering; missing/mismatched carriers and any `Task.extensions` field are rejected. Every terminal fixture has exactly one `allagents.execution-integrity` Artifact with the URI in `Artifact.extensions` and the schema-defined envelope in its single `data` Part. - Changing prompt, source object ID, profile ID, result schema, artifact selection, or deadline changes the canonical caller digest; trace IDs and transport metadata do not. The original effective-profile and result-schema digests are stored separately for retained replay. - - Rotating a resolved secret value, changing attempt/lease/trace identity, or changing a per-run path leaves the profile digest unchanged; changing a policy field or environment-variable name changes it, and the digest serializer cannot accept secret-bearing runtime state. + - Direct repositories are order-canonicalized without erasing destination + identity; duplicate destinations, mutable refs, unsafe subdirectories, and + ambiguous URL forms fail. The checked-out commit and tree match the + requested object from the authorized remote. OCI tags and external layer + URLs fail while allowed manifest digests pass. + - The source discriminator rejects unknown `kind` values, cross-variant + fields, and missing variant fields. Registered materializer inputs validate + against the operator schema and resource selectors, the request pins the + expected workspace-manifest digest, the worker-derived definition digest + changes the effective-profile digest, gateway and worker descriptors agree, + and caller-supplied image/command/credential fields are unrepresentable. + Fixed cross-language vectors prove algorithm-qualified, domain-separated + canonical digests and every non-secret runtime-field mutation changes the + definition digest while secret-value rotation does not. + - Rotating a resolved secret value, changing attempt/lease/trace identity, or + changing a per-run path leaves the profile digest unchanged; changing a + policy field, environment-variable name, authorization scope, or revocation + epoch changes it, and the digest serializer cannot accept secret-bearing + runtime state. - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - Worker fixtures reject missing/mismatched worker identities, attempt IDs, lease epochs, profile digests, command revisions, conditional-effect revisions, event sequences, bounds, and terminal acknowledgements. Cancel for an unseen attempt persists a tombstone; tombstoned or lower-revision dispatch is invalid before workspace creation. @@ -530,17 +821,89 @@ docs/src/content/docs/ ### U4. Worker protocol and safe workspace lifecycle -- **Goal:** Implement the supervised single-execution worker with authenticated transport, a minimal monotonic command record, hardened immutable Git acquisition, OS-enforced credential separation, leases, isolated roots, resource controls, race-resistant evidence, failed-quiescence recycling, and cleanup independent of any provider. +- **Goal:** Implement the supervised single-execution worker with authenticated + transport, a minimal monotonic command record, a closed workspace + materializer registry for hardened multi-repository Git, digest-pinned OCI, + and operator-registered images, standard manifest validation, OS-enforced + credential separation, leases, isolated roots, resource controls, + race-resistant evidence, failed-quiescence recycling, and cleanup independent + of any provider. - **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. - **Dependencies:** U1. -- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`. -- **Approach:** Authenticate the configured worker identity and fence every private command. Persist one minimal monotonic command record scoped to worker identity/lease before workspace creation: unseen-attempt cancel writes a tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel effect conditionally rechecks the stored revision immediately before mutation. Reserve one execution only after that check. Validate profile/deployment and OS provider/tool credential-boundary capabilities, then emit sequenced NDJSON. Keep transition selection pure. Run inside a dedicated container process namespace under init/reaper or an equivalent systemd/cgroup boundary. After bounded termination escalation, prove the complete invocation process set empty; if proof fails, persist termination unknown/failed, poison admission, and exit so the supervisor destroys the boundary. Replacement readiness proves boundary termination and reaps/quarantines owned roots. Use KTD13 acquisition, separate roots, phase environments, budgets, and descriptor-safe evidence; clean in `finally`. -- **Execution note:** Characterize every phase with a fake adapter, malicious fixtures, and disposable Git servers before real providers. Fault-inject dispatch acknowledgement, events, leases, acquisition, processes, evidence publication, and cleanup. +- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/materializers/types.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/worker/materializers/git.ts`, `packages/execution-service/src/worker/materializers/oci.ts`, `packages/execution-service/src/worker/materializers/external.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/materializers.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`, `packages/execution-service/tests/fixtures/execution/fake-materializer.ts`. +- **Approach:** Authenticate the configured worker identity and fence every + private command. Persist one minimal monotonic command record scoped to worker + identity/lease before workspace creation: unseen-attempt cancel writes a + tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel + effect conditionally rechecks the stored revision immediately before + mutation. Reserve one execution only after that check. Validate the fully + canonicalized source against profile resource policy before cache lookup; + bind cache entries to owner or authorization-scope digest, revocation epoch, + canonical source, definition digest, and expected/actual manifest digests. + Validate deployment, worker-computed materializer definition digest, and + acquisition plus provider/tool credential-boundary capabilities, then emit + sequenced NDJSON. Keep transition selection pure. Run inside a dedicated + container process namespace under init/reaper or an equivalent systemd/cgroup + boundary. Resolve only the closed KTD13 materializer registry. + + Launch an external materializer through the configured OCI runner or sandbox + as a supervisor-owned resource labeled by worker, attempt, lease, and fence, + with only schema-validated and resource-authorized inputs, its source + credentials, allowed egress, and a private host-owned staging mount under the + final publication root. Never expose gateway, provider, backend, + final-workspace roots, or the runner control socket. Treat the digest-pinned + image as operator-trusted deployment code. Verify the versioned output + manifest, request-pinned digest, content, immutable identities, and + verification-method labels. Terminate the acquisition process, credential + scope, mounts, and runner resource while retaining the validated host-owned + tree; prove that boundary gone before an atomic same-filesystem rename, + setup, and baseline. No copy fallback exists. After bounded termination + escalation, prove the complete invocation process set empty; if proof fails, + persist termination unknown/failed, poison admission, and exit so the + supervisor destroys the boundary. Replacement readiness enumerates and + destroys or quarantines orphan runner resources, credential/staging mounts, + and roots. Use separate phase environments, budgets, and descriptor-safe + evidence; clean in `finally`. +- **Execution note:** Characterize every phase with a fake adapter, fake + registered materializer, local OCI registry, malicious fixtures, and + disposable Git servers before real providers or private artifact systems. + Fault-inject dispatch acknowledgement, events, leases, every acquisition + mode, manifest publication, processes, evidence publication, and cleanup. - **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, `tests/helpers/env.ts` for isolated state, and Buzz's bounded process-group/job-object cancellation as a lifecycle characterization checklist rather than copied code. - **Test scenarios:** - - Covers AE5. Exact object ID verifies; wrong/missing object, disallowed URL/host/address/port, credential-bearing URL, redirect, DNS rebinding, unsafe subdirectory, fetch failure, and setup failure stop before adapter invocation. After worker acceptance each such source/setup failure emits the selected `Submitted -> Working -> Failed` public trace. - - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no secondary connection or helper execution. - - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. + - Covers AE5. Every repository checkout matches its requested full object ID + and tree and destinations are disjoint; wrong/missing objects, disallowed + repository/namespace/URL/host/address/port, credential-bearing URLs, + redirects, DNS rebinding, unsafe subdirectories, fetch failure, and setup + failure stop before adapter invocation. After worker acceptance each + materialization/setup failure emits the selected + `Submitted -> Working -> Failed` public trace. + - Repositories with LFS configuration/pointers, submodules, hooks, filters, + alternates, proxy/helper config, or non-HTTPS secondary protocols cause no + secondary connection or helper execution. + - Source credentials leave no repository config, process argument, child + phase environment, log, error, evidence, or retained workspace trace. + - OCI tags, foreign/external URLs, cross-origin credential forwarding, + disallowed registry/auth/blob host/address/port, redirects, DNS rebinding, + manifest/layer mismatches, unsafe layers, missing workspace manifests, and + expansion-limit violations fail before publication. A valid digest-pinned + snapshot produces the same manifest contract as direct Git. + - Unknown, profile-disallowed, unpinned, or definition-drifted materializers; + unauthorized structured-input resources; gateway/worker descriptor + mismatch; missing expected workspace-manifest digest; schema-invalid + inputs; undeclared egress; malformed output manifests; and mismatched + expected/reported repository or output identities fail before setup. The + request cannot select an image or command. + - Materializer credential environments/mounts, process state, runner control + plane, and staging mounts are inaccessible to later phases. Literal secret + canaries in output or retained logs fail publication. This verifies phase + teardown, not safety from a malicious operator-registered image that + intentionally transforms a credential. Provenance labels its unverified + source assertions as materializer-attested. A cache hit occurs only after + current authorization and revalidates content plus the same owner or + authorization scope, revocation epoch, canonical source, + materializer-definition, expected-output, and actual output-manifest + digests inside the same trust domain. - Setup changes establish the baseline; setup and checks receive no provider/control secrets. Credentialed provider runtimes and model tools run across the declared OS UID/process/mount boundary or broker, with disjoint config/data roots and ambient selectors removed. - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - Block dispatch after effect selection, complete a newer cancel for the unseen attempt, then release dispatch: the command tombstone/revision check rejects it before workspace or provider creation. Duplicate commands remain idempotent and all effects stay fence-bound. @@ -550,8 +913,22 @@ docs/src/content/docs/ - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps. A valid or invalid result selected before a later check/evidence failure is preserved, including the valid Artifact; only a pre-candidate failure records `not_produced`. - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. - - SIGKILL before and after provider spawn proves supervisor descendant death and replacement root recovery; the gateway retains one failed Task with separate termination and cleanup outcomes. -- **Verification:** A built supervised worker mutates a disposable exact-SHA repository through the fake adapter and proves authenticated revisioned dispatch, unseen-cancel tombstones, source hardening, OS credential separation, budgets, result preservation, quiescence or poisoned-boundary exit, evidence integrity, worker-crash containment, orphan-root handling, and cleanup. + - SIGKILL during materialization and before or after provider spawn proves + supervisor-owned runner/process death, credential/staging mount removal, + and replacement root recovery before readiness; unresolved resources keep + readiness false, and the gateway retains one failed Task with separate + termination and cleanup outcomes. + - Cross-filesystem staging/publication configuration fails readiness. Faults + around the final rename expose either no final workspace or the complete + validated tree, never a copy fallback or partial publication. +- **Verification:** A built supervised worker materializes equivalent + workspaces through disposable exact-SHA repositories, a local digest-pinned + OCI snapshot, and a fake registered materializer; validates one standard + manifest; mutates each through the fake adapter; and proves authenticated + revisioned dispatch, unseen-cancel tombstones, acquisition hardening and + credential teardown, budgets, result preservation, quiescence or + poisoned-boundary exit, evidence integrity, worker-crash containment, + orphan-root handling, and cleanup. ### U5. Codex backend adapter @@ -595,16 +972,42 @@ docs/src/content/docs/ - **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with trusted transports, peer identity, credential-boundary and supervisor readiness, safe startup/shutdown, tracing, and reproducible containers. - **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD15. - **Dependencies:** U3-U6. -- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. -- **Approach:** Register only Codex and Pi. Add gateway and supervised worker entrypoints inside the private Node 22 workspace. Before readiness, validate named public TLS termination, every remote worker's mTLS/equivalent transport and pinned identity/capabilities, Unix-socket locality, store, runtimes, monotonic command storage, OS credential-boundary capability, supervisor boundary, orphan roots, trust, quotas, and resource controls. Propagate `traceparent`, then apply KTD10's small shared metadata allowlist and bounded filtering/redaction before any structured log/span processor or OTLP exporter; neither OpenInference nor backend-native attributes bypass it. Build a minimal gateway image with no provider runtime and a one-execution worker image whose init kills the complete boundary when the worker server exits, including poisoned failed-quiescence exit. +- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/unit/worker/materializers/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. +- **Approach:** Register only Codex and Pi as backend adapters and register the + built-in Git/OCI materializers plus configured external materializers through + a separate closed registry. Add gateway and supervised worker entrypoints + inside the private Node 22 workspace. Before readiness, validate named public + TLS termination, every remote worker's mTLS/equivalent transport and pinned + identity/capabilities, Unix-socket locality, store, runtimes, matching + gateway/worker materializer definition digests, image digests, schemas, + credential names, egress, limits, and OCI runner/sandbox isolation, monotonic + command storage, acquisition and provider/tool credential-boundary + capabilities, + supervisor boundary, orphan roots, trust, quotas, and resource controls. + Propagate `traceparent`, then apply KTD10's small shared metadata allowlist and + bounded filtering/redaction before any structured log/span processor or OTLP + exporter; neither OpenInference nor backend-native attributes bypass it. + Build a minimal gateway image with no provider or materializer runtime and a + one-execution worker image whose init kills the complete boundary when the + worker server exits, including poisoned failed-quiescence exit. - **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. - **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. - **Test scenarios:** - Registry exposes exactly Codex and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects OpenCode or unknown backend IDs before workspace creation. + - Materializer registry exposes built-in Git and OCI plus only configured + external IDs, resolves every external image to the configured digest, + rejects duplicates/tags/unknown IDs, and cannot be influenced by request + image, command, credential, or policy fields. - Gateway and supervised worker start from built outputs, become ready only after trusted transport/identity, credential and supervisor boundaries, dependencies, and orphan recovery pass, and stop gracefully on SIGTERM. - Gateway readiness fails for malformed auth, missing/mismatched named TLS termination, plaintext production public ingress, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - Worker-route readiness fails for plaintext remote URL, wrong/untrusted certificate, worker identity/capability mismatch, or replayed capability; mTLS/equivalent authenticated encryption and same-host Unix sockets pass. - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable OS credential/supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. + - Worker readiness fails when a profile allows a materializer absent from its + route, gateway and worker definition digests differ, an external image is + not digest-pinned, an input schema or output manifest version is + unsupported, named credentials or the OCI runner/sandbox are unavailable, + the runner control plane would be visible to the workspace, or configured + egress and resource enforcement cannot be provided. - Killing or poisoning the worker server while an adapter child and invocation root exist makes the supervisor destroy the boundary; replacement readiness waits for root deletion/quarantine and never reuses it. - Trace context crosses the authenticated private call and correlates result identities using only opaque owner correlation. Exporter probes for agent, model, tool, stale-event, and error spans contain allowlisted bounded metadata but no canary secret, prompt/output, tool argument/result, file body/source fragment, raw caller identity, or cross-owner fragment; exporter failure cannot change Task status. - Gateway image contains no Codex, Pi, Git workspace, provider credential material, or worker trust private keys. @@ -619,6 +1022,14 @@ docs/src/content/docs/ - **Dependencies:** U1-U7. - **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. - **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases, with gateway and supervised worker as separate processes. Each race fixture declares attempt/fence correlation, required durable transitions and observed effects, required happens-before edges, maximum occurrence counts, and effects forbidden after terminalization. A deliberately small test-side checker evaluates those constraints against durable records plus observed worker/process outcomes without calling the production selector. It remains coverage protection—not TLA+, a model checker, event sourcing, or a second lifecycle implementation. Document the exact extension URI and legal Agent Card/header/Message/Artifact carriers, both fixed Artifacts and four result states, retained-replay ordering, worker transport/identity, monotonic command tombstones, failed-quiescence recycling, the narrow OS credential boundary, reviewed-domain limitation, metadata-only telemetry and its separate operator access/retention, storage/HA limits, lack of execution resume, source hardening, quotas, retention, and operations. + Document all three source modes, the exact source discriminator, canonical + repository/namespace and materializer-input authorization, multi-repository + destinations, the standard workspace manifest and verification-method + labels, materializer registration, worker-derived definition digests, + profile allowlisting, digest/profile/idempotency boundaries, same-filesystem + publication, supervisor-owned runner cleanup, acquisition credential + teardown, authorization-scoped cache reuse/revocation, source-mode + capabilities, and the prohibition on caller-supplied acquisition code. - **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. - **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, Starlight guide/reference organization under `docs/src/content/docs/`, and Buzz's required-critical-action coverage rule without importing its TLA+ model or production implementation. - **Test scenarios:** @@ -626,16 +1037,50 @@ docs/src/content/docs/ - Agent Card required-extension advertisement, `A2A-Extensions`, `Message.extensions`, request `Message.metadata[uri]`, and the single fixed integrity Artifact carrier interoperate; missing/mismatched carriers and `Task.extensions` fail. - Identical retained replay after deadline expiry, quota exhaustion, readiness loss, authorization change, or profile replacement returns the original Task; changed request/schema or inconsistent original bindings conflict. - The same accepted schema, valid result, invalid result, missing result, and pre-output failure pass through Codex and Pi with identical decisions. A valid-result-then-check-failure and invalid-result-then-evidence-failure preserve the selected state and only the valid Artifact; `not_produced` remains pre-candidate only. - - Source mismatch and setup failure after worker acceptance produce the selected `Submitted -> Working -> Failed` trace; provider invocation never begins, and durable snapshots, streams, and conformance records agree. + - Direct Git object/destination/authorization mismatch, OCI + digest/manifest/namespace mismatch, registered materializer + descriptor/input/resource/expected-output mismatch, direct known-secret + disclosure, and setup failure after worker acceptance produce the selected + `Submitted -> Working -> Failed` trace; provider invocation never begins, + and durable snapshots, streams, and conformance records agree. A policy + revocation before lookup cannot consume a previously populated cache entry. - Pause dispatch after selection, complete unseen-attempt cancel, then release dispatch; the stale command creates no workspace/process. The small independent checker enforces each fixture's attempt/fence correlation, happens-before edges, maximum counts, and forbidden post-terminal effects. Deliberately bad traces that still contain every required action name fail for wrong order, wrong fence, duplicate-over-maximum effects, and an extra stale dispatch after terminalization. - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; one worker serializes admitted work. - - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order commands/events, worker crash, lease expiry, cancellation, provider failure, evidence truncation, logical expiry, and cleanup failure preserve one truthful terminal outcome without provider reattachment or replay. - - A child that calls `setsid` and ignores graceful signals forces termination unknown/failed, poisoned-worker exit, supervisor boundary destruction, and replacement orphan recovery before readiness; no next reservation is accepted by the poisoned worker. + - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order + commands/events, worker crash, lease expiry, cancellation, provider failure, + evidence truncation, logical expiry, and cleanup failure preserve one + truthful terminal outcome without provider reattachment or replay. + - SIGKILL during external materialization and before or after provider spawn + forces supervisor-owned runner/process death, credential/staging mount + removal, and replacement orphan recovery before readiness. A child that + calls `setsid` and ignores graceful signals forces termination + unknown/failed, poisoned-worker exit, supervisor boundary destruction, and + replacement orphan recovery; no poisoned worker accepts a next reservation. - Public plaintext, wrong TLS boundary, private plaintext, wrong certificate/worker identity, and capability replay fail readiness/dispatch; configured TLS, mTLS/equivalent overlay, and same-host Unix socket cases pass. - Both adapters block model-tool probes of parent/sibling environments, procfs/process listings, known/discovered backend roots, and network secret exfiltration under the OS credential boundary. Environment filtering alone is never accepted as proof, and hostile-source/cross-tenant claims remain rejected. - End-to-end exporter capture repeats the agent/model/tool/stale-event/error canary and cross-owner probes, proving only bounded allowlisted metadata and opaque owner correlation cross the telemetry boundary while Task/Artifact access and retention remain independent. - - Redirect/DNS-rebinding, secondary Git fetch, resource exhaustion, malicious file types/link swaps, repository Pi extensions, and unrestricted built-ins remain blocked within the documented reviewed-source boundary. - - Examples validate with production schemas and use only secret variable names. Docs state the two transport boundaries, one gateway replica, one execution per worker, reviewed trust domain, narrow credential isolation versus deferred hostile-code isolation, metadata-only telemetry with its fixed pre-processor allowlist and separate operator access/retention, runtime floors, and ephemeral provider sessions. + - Redirect/DNS-rebinding and unauthorized-resource cases cover every Git and + OCI registry/auth/manifest/blob connection. Secondary Git fetch, OCI tag, + foreign/external layer URL, cross-origin credential forwarding, + layer/manifest mismatch, unregistered or unpinned materializer, undeclared + materializer egress, malicious output manifest, resource exhaustion, + malicious file types/link swaps, repository Pi extensions, and unrestricted + built-ins remain blocked within the documented reviewed-source boundary. + - Same-filesystem publication succeeds by atomic rename; a cross-filesystem + staging root fails readiness and fault injection never observes a partial + final tree or copy fallback. Provenance distinguishes worker-verified and + trusted-service identities from materializer-attested claims. + - Examples validate with production schemas and use only secret variable + names. Docs state the three source modes and exact discriminator, standard + workspace manifest and provenance labels, materializer registry/profile + boundary, worker-derived definition digest, resource authorization and + cache-revocation boundary, prohibition on caller-supplied acquisition code, + same-filesystem publication, supervisor-owned runner cleanup, acquisition + credential teardown, two transport boundaries, one gateway replica, one + execution per worker, reviewed trust domain, narrow credential isolation + versus deferred hostile-code isolation, metadata-only telemetry with its + fixed pre-processor allowlist and separate operator access/retention, + runtime floors, and ephemeral provider sessions. - Opt-in real-provider smoke tests record backend/runtime and credential-boundary prerequisites, skipping only when a named prerequisite is absent. - **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor; full suites and docs pass; the official A2A client exercises every advertised operation including the `Submitted -> Working -> Failed` source/setup path; exporter capture proves the telemetry canary/cross-owner contract; the independent checker rejects all-name-present traces with wrong order/fence/multiplicity or forbidden stale dispatch; and release evidence records each available real backend plus explicit skipped prerequisites. @@ -645,16 +1090,16 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| -| Contract generation | U1 | Exact extension URI/carriers, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | -| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, and cleanup. | -| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | +| Contract generation | U1 | Exact extension URI/carriers, closed workspace source discriminator and manifest, algorithm-qualified materializer/profile/input/output digest preimages and vectors, verification-method vocabulary, authorization-scope/revocation fields, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | +| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, source authorization/cache revocation, and cleanup. | +| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, direct Git/OCI/registered materialization, worker-derived registry digests, acquisition credential teardown, supervisor-owned materializer runners, same-filesystem atomic publication, workspace-manifest provenance labels, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | | Backend conformance | U5-U8 | One shared suite passes against Codex and Pi, including the versioned schema subset, four result states, valid/invalid preservation across later failure, integrity Artifact carrier, and structured-result Artifact rule. | -| Credentialed provider smoke | U5-U6, U8 | Each available provider mutates a disposable exact-SHA repository while adversarial tool probes cannot cross the OS credential boundary; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | +| Credentialed provider smoke | U4-U6, U8 | An available operator-trusted registered materializer and each available provider mutate a disposable immutable workspace while adversarial later-phase probes cannot directly access acquisition/provider credential environments, mounts, processes, roots, or runner control planes and literal canaries remain absent; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | | A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes required-extension negotiation and legal carriers, immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, retained replay, cancel races, expiry, and owner isolation without `Task.extensions`. | -| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, Git SSRF controls, OS provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | +| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, exact source-resource authorization, per-connection Git/OCI SSRF and credential-origin controls, authorization-scoped cache revocation, digest-pinned registered materializers, schema/manifest validation, truthful provenance labels, acquisition and provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | | Lifecycle trace conformance | U8 | The small test-side checker, independently of production selectors, validates attempt/fence correlation, required happens-before edges, maximum occurrence counts, and forbidden post-terminal effects against durable records plus observed worker/process outcomes; all-name-present bad traces fail for wrong order/fence/multiplicity and stale post-terminal dispatch. | | Telemetry safety | U7-U8 | Exporter capture across agent, model, tool, stale-event, and error spans proves the pre-processor allowlist and bounded redaction exclude prompt/output/tool/source/file content, canary secrets, raw identities, and cross-owner fragments while retaining only bounded operational metadata and opaque owner correlation. | -| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both container builds pass. | +| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, backend and materializer registry readiness, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both service container builds pass. | | Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | | Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | @@ -673,7 +1118,17 @@ The authoritative behavioral proof is the built-process E2E path with the offici - Authentication and bounded parsing precede owner-scoped retained lookup; identical replay uses stored original bindings before mutable admission, while current authorization/profile/readiness/deadline and quota apply only to atomic new claims. - Production public ingress uses its named TLS boundary, remote worker routes authenticate and encrypt peers with worker identity/capability binding, and same-host Unix sockets are the only non-network alternative; unprotected remote endpoints fail readiness. - Cancellation/deadlines use monotonic worker command tombstones and one native abort. Stale dispatch cannot create work, and failed quiescence poisons and exits the worker so supervisor destruction and replacement orphan recovery precede new admission. -- Source hardening, the OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled repository Pi extensions/unrestricted built-ins, one-execution reviewed-domain policy, resource limits, Artifact race defenses, completeness, provenance, and authenticated expiry are enforced end to end without claiming hostile-source/cross-tenant isolation. +- Workspace source validation and exact resource authorization, + per-connection direct Git/OCI controls, worker-derived registered-materializer + digests, the standard workspace manifest and truthful provenance labels, + authorization-scoped cache revocation, same-filesystem atomic publication, + supervisor-owned runner cleanup, acquisition credential teardown, + OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled + repository Pi extensions/unrestricted built-ins, one-execution + reviewed-domain policy, resource limits, Artifact race defenses, + completeness, provenance, and authenticated expiry are enforced end to end + without accepting caller acquisition code or claiming hostile-source or + cross-tenant isolation. - Metadata-only telemetry is filtered through the fixed allowlist and bounded redaction before processing/export; canary secrets, content, raw caller identities, and cross-owner fragments never reach exporters, and only opaque owner correlation crosses the separately governed operator boundary. - Required source/setup and race traces satisfy attempt/fence, happens-before, maximum-count, and forbidden-post-terminal constraints in the independent test-side checker; all-name-present malformed traces fail without introducing a parallel lifecycle implementation. - Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. @@ -682,11 +1137,31 @@ The authoritative behavioral proof is the built-process E2E path with the offici ### Per unit -- U1: Standard extension carriers, integrity/structured-result Artifact schemas, four result states, original claim digests, command revisions/tombstones, fence rules, typed failures, and fixtures are generated and stable. +- U1: Standard extension carriers, closed workspace source/manifest contracts, + algorithm-qualified materializer/profile/input/output digest preimages, + verification and authorization vocabulary, integrity/structured-result + Artifact schemas, four result states, original claim digests, command + revisions/tombstones, fence rules, typed failures, and fixtures are generated + and stable. - U2: Trusted ingress, auth, opaque owner isolation, retained-claim-first replay, original bindings, atomic new admission, CAS settlement, pagination, startup recovery, quotas, Artifact access, tombstones, and cleanup pass fault injection. - U3: Every advertised A2A operation agrees across stream and lookup while extension negotiation, replay ordering, authenticated worker routes, fencing, monotonic cancellation, and races preserve one Task. -- U4: Worker command state, OS credential separation, supervision, poisoned-exit/orphan recovery, and dispatch/source/setup/action/check/quiescence/evidence/cleanup pass malicious, crashed, and faulted scenarios. +- U4: Worker command state, exact source authorization, direct + Git/OCI/registered materialization, workspace-manifest validation and + provenance classification, authorization-scoped cache revocation, + same-filesystem atomic publication, acquisition and provider credential + separation, supervisor-owned runner cleanup, poisoned-exit/orphan recovery, + and dispatch/materialization/setup/action/check/quiescence/evidence/cleanup + pass malicious, crashed, and faulted scenarios. - U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, result preservation, OS credential separation, native evidence, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. - U6: Pi strict RPC/framing, terminating result, exact policy tools, disabled repository extensions/built-ins, OS-isolated credential store/provider runtime, result preservation, settlement, stats, abort, and process cleanup pass verification. -- U7: Closed registry, trusted transport/identity readiness, credential/supervisor capability gating, poisoned-worker recycling, metadata-only pre-export telemetry controls, Node-version separation, tracing, shutdown, containers, and release artifacts work from built outputs. -- U8: Cross-backend E2E, standard A2A carriers, retained replay, selected source/setup transitions, independent race-trace constraints, telemetry canary/cross-owner probes, transport and credential abuse cases, command/quiescence races, examples, operator docs, changelog, and release evidence are complete. +- U7: Closed backend and materializer registries, trusted + transport/identity/readiness, acquisition/provider credential and supervisor + capability gating, poisoned-worker recycling, metadata-only pre-export + telemetry controls, Node-version separation, tracing, shutdown, containers, + and release artifacts work from built outputs. +- U8: Cross-backend E2E, all three workspace source modes, standard manifest + provenance, acquisition credential teardown, standard A2A carriers, retained + replay, selected materialization/setup transitions, independent race-trace + constraints, telemetry canary/cross-owner probes, transport and credential + abuse cases, command/quiescence races, examples, operator docs, changelog, + and release evidence are complete. diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md new file mode 100644 index 00000000..d4708d4c --- /dev/null +++ b/docs/research/harbor-repository-materialization.md @@ -0,0 +1,179 @@ +# Harbor repository materialization lessons + +## Decision + +Borrow Harbor's content-addressed package cache, sparse Git reads, staged publication, +and prebuilt-environment option. Do not copy its task model as the execution gateway's +workspace contract. + +Harbor does not expose a first-class, general-purpose "repositories in a workspace" +layer. It first downloads a Harbor **task package**. The task then defines an execution +environment with a Dockerfile, Compose file, or prebuilt image. Acquisition of the +repository the agent edits is therefore benchmark- and task-owned: it may be baked into +an image, cloned by a Dockerfile, copied as task content, or otherwise prepared by the +task author. + +For AllAgents, repository and workspace provenance must remain explicit in the public +execution request and terminal evidence. Custom acquisition should be an +operator-registered, digest-pinned materializer behind the worker protocol, not an +arbitrary caller-supplied image or setup script. + +## What Harbor fetches + +### Task packages from Git + +Harbor's `GitRepoRegistryClient` resolves a dataset registry ref, inspects the selected +commit's tree without checking out blobs, and identifies task directories containing +`task.toml`. When task content is requested, `TaskClient` groups requested task paths by +Git URL and performs one shallow, no-checkout clone per URL. It uses a blobless partial +clone where supported, configures sparse checkout for only the selected task paths, +fetches each requested commit at depth one, checks it out, and records the resolved +commit. + +This is efficient for a large repository containing many independent Harbor tasks. It +is not a mechanism for assembling several application repositories into one agent +workspace. + +Harbor also accepts an omitted commit or a mutable ref and resolves it to a commit. +That is convenient for an interactive local benchmark CLI, but it is weaker than the +AllAgents gateway requirement that an accepted request already name immutable source. + +### Task packages from the package registry + +Package-registry tasks are downloaded as tar archives into a cache keyed by the task's +content hash. A direct `sha256:` reference can hit that cache without registry +resolution. Dataset manifests likewise refer to task packages by SHA-256 digest. This +is the closest Harbor analogue to an OCI workspace snapshot: a content-addressed, +reusable input bundle. + +Before publishing a Git task directory, Harbor stages it in a temporary directory, +rejects source paths containing symlinks, materializes only relative symlinks that stay +inside the task root, rejects cycles and special entries, then replaces the target. +Those containment and publish-after-validation properties are useful for any cached +workspace artifact. + +### The repository edited by the agent + +Once the task package is present, Harbor asks the selected environment provider to +start the task's `environment/` definition. For Docker this can be: + +- `[environment].docker_image`; +- `environment/Dockerfile`; or +- `environment/docker-compose.yaml`. + +The task format deliberately leaves the environment flexible. Harbor builds the +Dockerfile/Compose definition or uses the prebuilt image, then runs the agent in that +environment. There is no core repository-source schema carrying URL, exact commit, +destination, and per-repository provenance. + +The Multi-SWE-bench adapter makes the distinction concrete. Each generated task uses +an upstream `mswebench/...:pr-...` base image that already contains the repository at +`/home/{repo_name}`. Its Dockerfile creates `/workspace/{repo_name}` as a symlink and +sets that as `WORKDIR`; Harbor itself never clones that application repository. + +## Lessons for the AllAgents execution gateway + +### Adopt + +1. **Separate descriptor acquisition from execution.** Resolve and validate immutable + inputs before starting the coding-agent runtime. +2. **Use content-addressed caches.** Key reusable workspace snapshots by a digest of + normalized source identities, materializer version/digest, setup policy, current + authorization scope, and revocation epoch rather than a mutable name. Reauthorize + before lookup and make an old epoch ineligible after revocation. +3. **Avoid downloading irrelevant content.** For Git-backed descriptor catalogs, + Harbor's tree-only discovery and sparse checkout are sound optimizations. For an + application repository, use partial/shallow acquisition only when it preserves the + required commit and evidence semantics. +4. **Stage, validate, then publish.** Materialize into a temporary location, enforce + path/link/type/size limits, verify every requested identity, and atomically expose + the completed workspace to the worker. +5. **Support prebuilt immutable artifacts.** A digest-pinned OCI workspace snapshot is + the scalable path for very large repositories and expensive setup. + +### Adapt + +Keep a first-class workspace manifest instead of hiding source inside an environment +image. Each materialized repository should retain at least: + +- canonical source URL or snapshot identity; +- requested and resolved immutable commit or OCI digest; +- destination path and optional source subdirectory; +- materializer identity and version/digest; +- resulting tree/content identity; +- cache hit/miss and completeness facts. + +Use three explicit source modes: + +1. **Direct Git repositories** for the normal case, each with an exact commit and + collision-free destination. +2. **OCI workspace snapshots** for large, preassembled workspaces, referenced by digest + rather than tag and accompanied by a signed/validated workspace manifest. +3. **Operator-registered materializers** for JFrog, unusual monorepos, generated source, + or organization-specific setup. A request selects a configured materializer ID, + pins the expected workspace-manifest digest, and supplies validated, + resource-authorized structured inputs. The operator configuration pins the builder + image by digest, the worker derives the non-secret definition digest, credentials are + scoped only to materialization, and the builder must produce the standard workspace + manifest before the agent starts. The builder is operator-trusted deployment code; + deployments that cannot grant that trust need a broker or stronger acquisition + service. + +This retains Harbor's useful task-owned flexibility without allowing a caller to choose +an arbitrary executable image or shell script inside the trusted worker. + +### Do not copy + +- Mutable Git refs, `HEAD`, image tags, or package `latest` as accepted execution + identities. +- Harbor's broad Git transport set (`http`, `ssh`, and `git` as well as HTTPS) at a + remote service boundary. The gateway should keep canonical credential-free HTTPS, + destination-policy revalidation, disabled redirects/helpers/filters/hooks/submodules, + and exact commit verification. +- A non-fatal Git LFS miss. If declared workspace content cannot be materialized, + preparation must fail before provider execution. +- Hashing a prebuilt image reference string as environment identity. Resolve and pin + the OCI manifest digest. +- Arbitrary task-authored Dockerfiles, Compose files, or public-network setup as caller + input. Harbor runs benchmark definitions trusted by the evaluator; the gateway + accepts remote service requests and has a different threat boundary. +- Treating a container image alone as sufficient provenance. An image can carry the + correct files while obscuring which repositories, commits, generator, and setup + produced them. + +## Recommended boundary + +The worker should execute a dedicated materialization phase before any harness starts: + +1. Validate the normalized workspace request, exact source-resource authorization, + configured materializer, and current authorization scope before any cache lookup. +2. Resolve phase-scoped source credentials without exposing them to setup, the model, + or later evidence. +3. Populate a worker-owned staging directory on the final publication filesystem or + pull and unpack a digest-pinned workspace snapshot there. +4. Verify repository commits, paths, limits, content, the expected manifest digest, + and the standard workspace manifest; distinguish worker-verified identities from + materializer-attested claims. +5. Stop the acquisition process, revoke credentials, remove its mounts and runner + resource, and retain only the validated host-owned staging tree. +6. Atomically rename that tree into the final workspace, record provenance, run + operator-owned setup, record the post-setup baseline, and only then launch the + harness-specific worker runtime. + +The practical conclusion is narrow: Harbor is strong evidence for content-addressed +input bundles and environment-provider indirection. It is not evidence for making +repository acquisition opaque or task-defined in the AllAgents public contract. + +## Primary sources + +Inspected Harbor commit +[`b83e7686999a18ba90a8603794d7d18d42cab010`](https://github.com/harbor-framework/harbor/tree/b83e7686999a18ba90a8603794d7d18d42cab010): + +- [`src/harbor/registry/client/git_repo.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/registry/client/git_repo.py) — ref resolution, tree-only discovery, and sparse registry checkout. +- [`src/harbor/tasks/client.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/tasks/client.py) — Git/local/package task acquisition, content-hash cache, LFS behavior, safe staging, and resolved commits. +- [`src/harbor/models/task/id.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/models/task/id.py) — Git, local, and package task identities. +- [`src/harbor/models/dataset/manifest.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/models/dataset/manifest.py) — digest-addressed dataset task references. +- [`docs/content/docs/tasks/index.mdx`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/docs/content/docs/tasks/index.mdx) — task structure and Docker image/Dockerfile/Compose environment contract. +- [`src/harbor/environments/definition.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/environments/definition.py) — environment selection and content identity. +- [`src/harbor/environments/docker/docker.py`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/src/harbor/environments/docker/docker.py) — prebuilt-image versus build behavior and container startup. +- [`adapters/multi-swe-bench/README.md`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/adapters/multi-swe-bench/README.md) and its [`environment/Dockerfile`](https://github.com/harbor-framework/harbor/blob/b83e7686999a18ba90a8603794d7d18d42cab010/adapters/multi-swe-bench/src/multi_swe_bench_adapter/task-template/environment/Dockerfile) — application repository supplied by an upstream prebuilt image rather than cloned by Harbor core. From f395a04d0caac0c9a68709a14dccd7dc24c25e47 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Fri, 18 Sep 2026 21:57:48 +1000 Subject: [PATCH 07/12] docs(architecture): define GitHub source credentials --- ...-agent-execution-through-an-a2a-gateway.md | 109 ++- ...0837-feat-coding-execution-gateway-plan.md | 662 +++++++++++++++--- .../source-credential-broker-precedents.md | 260 +++++++ 3 files changed, 929 insertions(+), 102 deletions(-) create mode 100644 docs/research/source-credential-broker-precedents.md diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index d8deb818..f9357c13 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -149,24 +149,117 @@ separate from the agent runtime and never exposes that runner's control socket to setup or model tools. Phase isolation prevents later code from receiving the materializer's credentials or mounts, but it cannot make a malicious operator-registered image safe from credentials intentionally given to it. -Operators must review and pin that image; deployments that do not trust it need -a credential broker or stronger acquisition service that never reveals reusable -credentials to the materializer. +Operators must review and pin that image. A deployment that will not trust it +with credentials needs a separately versioned broker or central snapshot +protocol, which is deferred from the initial architecture. The canonical source request enters caller idempotency. The resolved materializer definition digest enters the effective-profile binding, and both the definition and output-manifest digests enter terminal provenance. New-claim source authorization always runs before cache lookup. Cache metadata and keys include the canonical source, materializer-definition digest, authorization -scope digest and revocation epoch, and configured trust domain. Reuse requires -manifest and content revalidation under the current authorization scope; -revocation advances the epoch and makes the old namespace unusable. +scope digest and revocation epoch, configured trust domain, and, for GitHub App +sources, the current installation-entitlement generation. Authenticated App +lifecycle webhooks and bounded control-plane reconciliation advance that +generation on uninstall, suspension, or repository-selection change. Unknown +or stale installation state fails cache authorization rather than reusing an +entry. Reuse also requires manifest and content revalidation under the current +authorization scope. Cache metadata preserves the original acquisition +provider metadata; terminal provenance distinguishes `cache_hit`, that original +provider, and the provider selected by current policy instead of claiming that +the current provider performed acquisition. Secret resolution and token minting +remain cache-miss-only. This keeps Harbor's useful separation between content-addressed task acquisition and environment execution without adopting task-owned opaque source. The comparison is recorded in [Harbor repository materialization lessons](../research/harbor-repository-materialization.md). +### Resolve GitHub source credentials from trusted deployment policy + +The public source request remains credential-free and does not select a +credential provider. The trusted acquisition boundary normalizes the repository +host and resolves a provider from operator configuration. `github.com` selects +the built-in GitHub source backend; GitHub Enterprise Server hosts require an +explicit host and API mapping because a custom hostname does not identify its +provider. The effective profile authorizes the canonical repository and provider +entitlement before cache lookup. + +For GitHub repositories, an ordered policy may prefer a GitHub App and permit a +local GitHub CLI fallback. The App provider is applicable only when trusted +operator configuration maps the requested repository to an installation ID; +`@octokit/auth-app` does not discover that mapping. It mints an installation +token scoped only to that repository, read-only contents permission, and its +GitHub expiry. A trusted-local CLI provider is pinned to one configured +non-secret account, which participates in its entitlement and effective-profile +digests. It may run only when no App installation mapping applies, as +`gh auth token --hostname --user `, with `GH_TOKEN`, +`GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` removed +from its environment. Failure to resolve the configured account fails that +provider. This is eligibility fallback, not authentication retry: after an App +provider is selected, configuration, authentication, minting, permission, +repository, rate-limit, or service failure terminates acquisition and never +falls through to the broader user identity. + +When AllAgents owns GitHub App token minting, a trusted control-plane +credential-provider component uses the focused `@octokit/auth-app` package +rather than implementing App JWT, clock-skew, expiry, and installation-token +renewal itself. Every cache-miss acquisition requests a fresh installation token +with auth-app cache bypass (`refresh: true`). Its remaining lifetime must be +strictly greater than the acquisition deadline plus the configured clock-skew +margin, and the delivery lease cannot outlive the token. Readiness rejects an +acquisition-phase ceiling that can exceed a fresh token's safe lifetime. Git +remains the repository transport; the full Octokit client is not required. + +The initial remote architecture is one central token-minter path. The +gateway/control-plane credential-lease controller is authoritative: an +authenticated worker requests credentials only for its active attempt and +fence; the controller rechecks the current command revision, lease epoch, +tombstone, and fence in durable dispatch state, then derives the +effective-profile digest, selected provider, host/API-mapping digest, +installation ID, canonical repository, operation, worker route and identity, +and expiry from durable dispatch and policy state. It issues a single-use, +non-durable grant/response and, when the minter is separate, requires its +configuration digest to agree with those selected bindings. Replay, worker +field substitution, stale command state, and configuration disagreement fail +closed. + +The authenticated delivery lease and channel bind that derived state to the +worker identity, attempt, lease epoch, command revision, fence, operation, and +expiry. Those bindings do not alter the bearer token: after delivery, the token +is enforceably scoped only by GitHub to the repository, read-only contents +permission, and token expiry. The remote worker never receives the App private +key, and only its one-shot acquisition child receives the token. A remote App +profile fails readiness when the central minter, authoritative lease controller, +fresh-token lifetime check, or authenticated non-durable delivery capability is +absent. A versioned central snapshot-delivery protocol is deferred and is not +an initial readiness alternative. + +The local GitHub CLI provider and remote lease path expose their resolved tokens +only to the one-shot acquisition process. Neither exposes credentials to setup, +the coding-agent runtime, model tools, repository configuration, process +arguments, logs, evidence, or the published workspace. Public failures use only +deterministic coarse source-auth code, safe reason, and retryability: +`source_auth_unavailable/no_eligible_provider`, +`source_auth_denied/installation_repository_denied`, and +`source_auth_failed` with `app_configuration_invalid`, +`app_authentication_failed`, `app_mint_failed`, or +`trusted_local_cli_failed` are not retryable; `source_auth_failed` with +`provider_rate_limited` or `provider_unavailable` is retryable. Provider, +installation, and account identifiers are non-secret but operator-only +provenance. Cache-hit provenance separately records `cache_hit`, the original +acquisition provider, and current policy selection. + +A standalone network broker is not required for trusted local execution: the +CLI provider may be a subprocess and a trusted co-located deployment may host +the App minter and lease controller inside its control plane. Remote routes +still use the same authenticated central-minter contract; the minter may be +split into a standalone service when private-key isolation, independent audit, +scaling, or blast-radius requirements demand it. + +The supporting precedents and trust-boundary analysis are recorded in +[Source credential broker precedents](../research/source-credential-broker-precedents.md). + ### Persist Task truth, not live provider execution The gateway durably stores Task identity, idempotency claims, terminal status, @@ -428,6 +521,10 @@ but their product and ownership model requires a separate decision. profiles decide which callers may select them. - Direct Git, OCI snapshots, and registered materializers converge on one validated workspace manifest and provenance contract. +- GitHub source credentials are selected by trusted host/profile policy rather + than caller input. GitHub App is preferred when applicable; GitHub CLI is a + local-only eligibility fallback and never masks an App authentication or + authorization failure. - Deployments that enable external materializers must operate their image, schema, credential, network, resource, and cache policies as worker configuration. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 86c83707..5f95d109 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -66,6 +66,13 @@ The two initial runtimes expose different programmatic contracts. Codex provides versioned workspace source mode; custom acquisition uses only operator-registered, digest-pinned materializers allowed by the profile. Governs R6, R11-R13, R16-R18, R21-R22. +- **Resolve source credentials from trusted host and profile policy.** Callers + never select a credential provider. For GitHub, prefer an applicable GitHub + App installation and permit GitHub CLI only as an explicit trusted-local + fallback when no installation applies; never fall back after a selected App + provider fails. When AllAgents owns App token minting, use + `@octokit/auth-app` rather than a custom minter. (session-settled: + user-directed.) Governs R6, R12-R13, R16, R21-R22. ### Requirements @@ -114,6 +121,60 @@ The two initial runtimes expose different programmatic contracts. Codex provides materializer output must match the request's expected workspace-manifest digest before publication. - R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, structured logs/spans before processing or export, and every model-initiated command or tool environment. Credentialed profiles additionally require an OS-enforced provider/tool credential boundary: the credential-bearing provider runtime and model-invoked tools use distinct UID/process/mount policy that prevents tool access to provider processes, procfs entries, and backend config/data roots, or an equivalent credential broker keeps reusable credentials out of the agent runtime. Worker readiness fails when the declared boundary cannot be proved; environment filtering alone is not credential isolation. + Source credential selection is server-side deployment policy, not caller + input. The acquisition boundary maps normalized repository hosts to source + backends; `github.com` selects the built-in GitHub backend, while GitHub + Enterprise Server hosts require explicit operator host/API mappings. Profiles + name an ordered provider policy and authorize its non-secret entitlement + before cache lookup. An App provider is applicable only when trusted operator + configuration maps the repository to an installation ID; auth-app does not + discover installations. Authenticated lifecycle webhooks plus bounded + reconciliation advance an installation-entitlement generation on uninstall, + suspension, or repository-selection change. Unknown or stale installation + state fails cache authorization. Cache metadata preserves the original + acquisition-provider metadata, while hit provenance separately records + `cache_hit`, that original provider, and current policy selection/entitlement. + Secret lookup and token minting remain cache-miss-only. + For a cache-miss GitHub acquisition, an applicable configured App installation + is preferred. Focused `@octokit/auth-app` minting uses `refresh: true` to + produce a fresh token scoped only to the authorized repository, read-only + contents permission, and GitHub expiry. Remaining lifetime must be strictly + greater than the acquisition deadline plus clock-skew margin, and the + credential lease cannot outlive the token. Readiness rejects an acquisition + ceiling that can exceed a fresh token's safe lifetime. A trusted-local + `github-cli` provider may run only when no App installation mapping applies; + it is pinned to a configured non-secret account included in entitlement and + effective-profile digests, invokes + `gh auth token --hostname --user ` without `GH_TOKEN`, + `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, or `GITHUB_ENTERPRISE_TOKEN`, and + fails if that account cannot be resolved. After App selection, no failure + falls through to `gh`. + The initial remote path requires a trusted central token minter and + authoritative gateway/control-plane credential-lease controller. The + authenticated worker requests only by active attempt and fence. From durable + dispatch and policy state, the controller rechecks active command revision, + lease epoch, tombstone, and fence, then derives the effective-profile digest, + selected provider, host/API-mapping digest, installation ID, canonical + repository, operation, worker route and identity, and expiry. It issues and + atomically consumes a single-use non-durable grant/response; a separately + deployed minter must agree with the selected configuration digest. Replay, + substitution, stale state, and controller/minter digest disagreement fail + closed. The authenticated lease/channel binds the derived repository and + provider state to worker identity, attempt, lease epoch, command revision, + fence, operation, and expiry; those are not token claims. The remote worker + never receives the App private key. Readiness fails without this complete + path. Versioned central snapshot delivery is deferred and is not an initial + readiness alternative. + Public failures expose only deterministic coarse code, safe reason, and + retryability; provider, installation, and account identifiers remain + operator-only. `source_auth_unavailable/no_eligible_provider`, + `source_auth_denied/installation_repository_denied`, + `source_auth_failed/app_configuration_invalid`, + `source_auth_failed/app_authentication_failed`, + `source_auth_failed/app_mint_failed`, and + `source_auth_failed/trusted_local_cli_failed` are not retryable. + `source_auth_failed/provider_rate_limited` and + `source_auth_failed/provider_unavailable` are retryable. Materializer IDs are defined in an operator-owned deployment registry. The gateway holds only the non-secret ID, bounded input schema, expected definition digest, expected output-manifest version, and required worker @@ -133,8 +194,8 @@ The two initial runtimes expose different programmatic contracts. Codex provides provider execution, or model tools. The registered image is operator-trusted deployment code: phase isolation protects later phases but cannot make a malicious registered image safe from credentials deliberately given to it. - Deployments requiring that stronger claim use a credential broker or - acquisition service that withholds reusable credentials. + Deployments requiring that stronger claim are deferred pending a separately + versioned broker or central snapshot-delivery protocol. - R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. - R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. - R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves the complete invocation process set quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. If bounded termination escalation cannot prove quiescence, the worker persists termination as unknown/failed, poisons admission, and exits so the external supervisor destroys the complete process boundary; replacement readiness performs orphan recovery before accepting work. The same supervisor boundary handles a worker crash. @@ -247,10 +308,30 @@ The two initial runtimes expose different programmatic contracts. Codex provides peer identity, readiness fails; a same-host Unix worker socket is accepted. Given a credentialed reviewed-domain profile, model tools cannot inspect provider process environments, process listings, backend config/data roots, - or exfiltrate provider/control credentials across the configured OS - boundary. Given a registered materializer with source credentials, setup, - provider processes, and model tools have no access to its process, runner - control socket, credential environment/mounts, or staging root after + or exfiltrate provider/control credentials across the configured OS boundary. + Given a GitHub repository, a trusted operator repository-to-installation + mapping wins over GitHub CLI; auth-app never discovers the installation. + Without a mapping, only an explicitly enabled trusted-local provider may + invoke the configured account through + `gh auth token --hostname --user ` with all four ambient + GitHub token variables absent. Any selected-App failure never invokes `gh`. + A remote worker requests a credential only by its active attempt/fence; the + controller derives all provider/repository/route bindings, rechecks current + command state, consumes one single-use grant, and rejects replay, + substitution, stale state, or minter configuration-digest disagreement. + The fresh token has repository/read-only/expiry scope only; the authenticated + lease carries worker, attempt, lease epoch, command revision, fence, + operation, and expiry bindings. Near-expiry cached auth-app output is bypassed + with `refresh: true`, lease expiry never exceeds token expiry, and an unsafe + acquisition ceiling fails readiness without CLI fallback. Authenticated App + lifecycle webhooks and bounded reconciliation invalidate old entitlement + generations; unknown or stale state cannot authorize a cache hit. Cache-hit + provenance distinguishes the original acquisition provider from current + policy selection. Every source-auth failure maps to the specified safe + code/reason/retryability, while provider, installation, and account identities + remain operator-only. Given a registered materializer with source credentials, + setup, provider processes, and model tools have no access to its process, + runner control socket, credential environment/mounts, or staging root after materialization, and direct known-secret canaries are absent from retained logs, evidence, and published workspace files. The registered materializer remains operator-trusted code; hostile-materializer, hostile-source, and @@ -291,6 +372,9 @@ The two initial runtimes expose different programmatic contracts. Codex provides - Push-notification configuration, gRPC, JSON-RPC transport, and A2A extended Agent Cards. - AHP server/client surfaces, long-lived interactive sessions, and client-contributed tools. - Optional ATIF conversion after the format and tooling mature. +- Versioned central source-snapshot acquisition and delivery; the initial + remote GitHub path is central token minting plus an authenticated non-durable + delivery lease. **Outside this product's identity** @@ -302,6 +386,9 @@ The two initial runtimes expose different programmatic contracts. Codex provides - [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) +- [GitHub source credential broker precedents](../research/source-credential-broker-precedents.md) +- [GitHub App installation access tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) +- [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) - [AI Evals ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) - [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) @@ -344,12 +431,16 @@ The two initial runtimes expose different programmatic contracts. Codex provides selectors for structured materializer inputs. New admission authorizes the fully canonicalized resource and credential entitlement before cache lookup. Resolve a versioned canonical `EffectiveProfileIntent` containing the - selected materializer definition digest, authorization-scope digest, and - source-authorization revocation epoch when applicable; compute its digest - without resolved secrets or per-attempt state and persist it with the - canonical caller request and result-schema digest. Retained replay compares - those stored original bindings and never substitutes or re-resolves current - policy. Governs R6, R11-R16, R21-R22. + selected materializer definition digest, authorization-scope digest, + source-authorization revocation epoch, provider policy and pinned CLI account, + and current GitHub App entitlement generation when applicable; compute its + digest without resolved secrets or per-attempt state and persist it with the + canonical caller request and result-schema digest. Unknown or stale App + entitlement state fails cache authorization. Cache metadata retains original + acquisition-provider metadata, while cache-hit provenance records `cache_hit` + plus current policy selection separately. Retained replay compares stored + original bindings and never substitutes or re-resolves current policy. + Governs R6, R11-R16, R21-R22. - KTD10. **Keep durable evidence and operational telemetry as separate bounded layers.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded evidence layer; neither Git nor provider evidence is promoted as exact causality when incomplete. Telemetry is a third, non-durable metadata-only channel: one small shared pre-export sanitizer applies an explicit operational-metadata allowlist plus bounded filtering/redaction before every structured log or span processor, and only opaque owner correlation may cross the separately governed operator boundary. OpenInference and backend-native attributes receive no bypass. This is an export guard, not a telemetry framework or alternate evidence store. Governs R13, R16-R19. - KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. - KTD12. **Separate terminal integrity from optional evidence bodies.** The fixed `allagents.execution-integrity` Artifact validates identity, action outcome, the four-state structured-result record, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance before terminal publication. `not_produced` applies only before result-candidate production. Once validation selects `valid` or `invalid`, a later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the separate fixed structured-result Artifact while retaining the later phase as the primary Task failure. Predictable optional-body truncation/redaction may preserve completion; failure that breaks the integrity kernel fails in the evidence phase. Governs R3-R4, R17-R18. @@ -357,11 +448,13 @@ The two initial runtimes expose different programmatic contracts. Codex provides closed `kind`-discriminated workspace-source union and one output manifest; reject unknown kinds and cross-variant fields. The built-in Git path accepts canonical HTTPS repository identities and full commit IDs only, uses - hermetic Git configuration, disables redirects, proxies, helpers, hooks, - filters, LFS smudge, submodule recursion, alternates, and non-HTTPS - protocols, revalidates normalized host/address policy for every connection, - fetches into an isolated object database from the authorized remote, and - verifies the checked-out commit and resulting tree. The OCI path accepts + hermetic Git configuration; disables inherited redirects, proxies, + credential helpers, hooks, filters, LFS smudge, submodule recursion, + alternates, and non-HTTPS protocols; injects only the KTD16-selected + one-shot credential channel; revalidates normalized host/address policy for + every connection; fetches into an isolated object database from the + authorized remote; and verifies the checked-out commit and resulting tree. + The OCI path accepts manifest digests, not tags; rejects foreign/external URLs by default; revalidates scheme, host, resolved address, port, redirect, and credential origin for every registry/auth/manifest/blob request; and verifies every @@ -391,9 +484,61 @@ The two initial runtimes expose different programmatic contracts. Codex provides trusted computing base, not hostile caller code. Its runner or sandbox control plane is never mounted into the workspace or exposed to setup, providers, or model tools. A deployment that does not trust the registered - image with source credentials must use a broker or stronger acquisition - service and advertise that capability explicitly. + image with source credentials is outside the initial trust model and must not + enable that registered materializer. Support requires the separately + versioned broker or central snapshot-delivery protocol deferred by R13. - KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. +- KTD16. **Resolve GitHub credentials through an authoritative ordered provider + registry and lease controller.** The caller supplies only a canonical + credential-free repository URL. The acquisition boundary maps `github.com` + to the built-in GitHub backend and requires explicit host/API mappings for + GitHub Enterprise Server. The profile supplies provider eligibility and + order, not secrets. A configured App provider is applicable only when trusted + operator configuration maps the authorized repository to an installation ID; + auth-app does not discover installations. A `github-cli` provider may follow + only in a trusted-local profile, only when no App mapping applies, and only + for its configured non-secret account. That account participates in + entitlement and effective-profile digests. Invoke + `gh auth token --hostname --user ` with `GH_TOKEN`, + `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` removed, + and fail when the configured account cannot be resolved. Selection is sticky: + App configuration, authentication, minting, authorization, rate-limit, or + service failure never falls through to the broader user identity. + + When AllAgents owns minting, its trusted central minter depends on focused + `@octokit/auth-app` rather than implementing App JWT, clock-skew, expiry, and + renewal; Git remains the transport and the full Octokit client is not added. + Every cache-miss acquisition uses `refresh: true` and accepts only a fresh + token whose remaining lifetime is strictly greater than the acquisition + deadline plus clock-skew margin. The token is scoped only to the repository, + read-only contents permission, and GitHub expiry. Lease expiry cannot exceed + token expiry, and readiness rejects an acquisition ceiling that can exceed a + fresh token's safe lifetime. + + The gateway/control-plane credential-lease controller, not the worker, is + authoritative. An authenticated worker request supplies only active attempt + and fence. The controller rechecks durable command revision, lease epoch, + tombstone, and fence and derives effective-profile digest, selected provider, + host/API-mapping digest, installation ID, canonical repository, operation, + worker route/identity, and expiry from durable dispatch and policy state. It + issues a single-use non-durable grant/response; a separate minter atomically + consumes the grant and must agree with the selected configuration digest. + Replay, substitution, stale command state, and digest disagreement fail + closed. The authenticated lease/channel binds the derived state to worker + identity, attempt, lease epoch, command revision, fence, operation, and + expiry. A remote worker never receives the App private key. Remote App + profiles fail readiness without that complete central path. A trusted + co-located deployment may keep the controller and minter in its control + plane. Versioned central snapshot delivery is deferred. + + Authenticated App lifecycle webhooks and bounded reconciliation advance an + installation-entitlement generation on uninstall, suspension, or + repository-selection change; unknown or stale state fails cache + authorization. Minting remains miss-only. Provenance distinguishes + `cache_hit`, original acquisition-provider metadata, and current policy + selection/entitlement. Public source-auth details contain only the specified + safe code, reason, and retryability; provider, installation, and account + identifiers are operator-only. Governs R6, R12-R13, R16, R21-R22. ### High-Level Technical Design @@ -405,8 +550,13 @@ flowchart TB Gateway --> Auth[Auth, retained replay, new admission] Gateway --> Store[Generation-based Task and Artifact store] Gateway -->|mTLS/authenticated overlay or same-host Unix socket| Worker[Single-execution worker] + Gateway --> LeaseController[Authoritative credential lease controller] + LeaseController --> AppMinter[Trusted GitHub App token minter] Worker --> Materialization[Workspace materializer registry] Materialization --> Git[Hardened multi-repository Git] + Git --> SourceCredentials[Source credential client] + SourceCredentials -->|Active attempt and fence| LeaseController + SourceCredentials --> GitHubCLI[Account-pinned trusted-local gh helper] Materialization --> OCI[Digest-pinned OCI snapshot] Materialization --> Custom[Registered materializer image] Worker --> Registry[Closed backend registry] @@ -427,6 +577,8 @@ sequenceDiagram participant S as Durable aggregate store participant W as Worker participant B as Backend adapter + participant L as Credential lease controller + participant M as App token minter C->>G: SendMessage + header/Message extension + metadata[uri] G->>G: Authenticate, check extension, canonicalize within bounds @@ -443,6 +595,14 @@ sequenceDiagram G->>W: Dispatch(attempt, fence, command revision) W->>W: Verify command record before workspace creation W-->>G: Accepted(attempt, fence) + opt private GitHub cache miss + W->>L: Request(active attempt, fence) + L->>S: Recheck command revision, lease epoch, tombstone, fence + L->>L: Derive profile/provider/mapping/install/repository/operation/route + L->>M: Single-use non-durable grant + configuration digest + M-->>L: Fresh repository/read-only token + GitHub expiry + L-->>W: Authenticated lease response bound to current command + end W->>W: Materialize into staging and validate workspace manifest W->>W: Destroy acquisition boundary, publish atomically, setup, baseline W->>B: Invoke with isolated roots and credential boundary @@ -520,6 +680,13 @@ packages/execution-service/ errors.ts profiles.ts telemetry.ts + source-credentials/ + contract.ts + registry.ts + github.ts + github-app-minter.ts + github-app-client.ts + github-cli.ts gateway/ index.ts config.ts @@ -559,6 +726,7 @@ packages/execution-service/ unit/execution/ unit/gateway/ unit/worker/ + unit/source-credentials/ e2e/execution-gateway.test.ts containers/ gateway.Dockerfile @@ -587,12 +755,39 @@ docs/src/content/docs/ - Each profile defines backend, worker route, allowed workspace source modes, allowed materializer IDs, exact Git repository or namespace rules, allowed Git origins/addresses, OCI namespace/registry/signature policy, structured - materializer-input resource selectors, authorization-scope derivation and - source-authorization revocation epoch, provider/model settings, - phase-specific environment allowlists, deterministic permissions, - setup/check commands, artifact globs, effective deadline ceiling, trust - class, resource limits, cleanup policy, evidence budgets, and required - acquisition/provider/tool isolation capabilities. + materializer-input resource selectors, authorization-scope derivation, + source-authorization revocation epoch, and applicable GitHub App + entitlement-generation authority, provider/model settings, phase-specific + environment allowlists, deterministic permissions, setup/check commands, + artifact globs, acquisition and effective deadline ceilings, clock-skew + margin, trust class, resource limits, cleanup policy, evidence budgets, and + required acquisition/provider/tool isolation capabilities. +- Source-credential configuration defines normalized-host backend mappings and + ordered provider entries. `github.com` has a built-in GitHub mapping; every + GitHub Enterprise Server hostname and API base URL is explicit. A + control-plane `github-app` entry references an App ID, private-key secret + handle, installation ID or deterministic repository-to-installation mapping, + requested read-only contents permission, entitlement-generation store, + authenticated lifecycle-webhook configuration, bounded reconciliation + interval and stale-state limit, fresh-token lifetime policy, and a + versioned non-secret provider/host/API-mapping configuration digest. The + worker receives no App private-key handle. A worker-local `github-cli` entry + contains no token, names one non-secret account/login included in entitlement + and effective-profile digests, and is valid only for an explicitly + trusted-local profile. It invokes the configured `gh` binary with + `auth token --hostname --user ` after removing `GH_TOKEN`, + `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN`. +- Every remote route using a GitHub App declares the authenticated central + token minter, authoritative credential-lease controller, and single-use + non-durable grant/response protocol. Startup rejects remote App profiles + without that complete path, `github-cli` on remote or multi-tenant routes, + missing central App secret handles, unsupported hosts, ambiguous + equal-priority providers, policies that allow runtime failure to trigger + identity fallback, or acquisition ceilings that can exceed a fresh token's + safe lifetime. Configuration and effective-profile digests include provider + IDs, order, host/API mappings, route capability, pinned CLI account, + non-secret entitlement policy, and mapping/configuration digest, but exclude + private keys, resolved tokens, lease payloads, and per-attempt state. - Worker configuration fixes a private listener, worker identity, one-execution concurrency, one same-filesystem publication root containing private staging and final workspace directories, a closed materializer @@ -618,6 +813,19 @@ docs/src/content/docs/ credential mounts, staging mounts, and roots before readiness. The supported worker baseline is a dedicated process namespace under a minimal init/reaper; bare-host deployment requires an equivalent systemd/cgroup mechanism. +- Remote App readiness additionally proves that the configured central minter + and gateway/control-plane lease controller authenticate the selected worker + route, agree on the selected provider/host/API-mapping configuration digest, + and support fresh `refresh: true` minting plus a single-use non-durable + grant/response. The controller must derive profile/provider/mapping/ + installation/repository/operation/route/identity/expiry from durable state, + recheck current command revision, lease epoch, tombstone, and fence, reject + replay or substitution, and bind delivery to worker identity, attempt, lease + epoch, command revision, fence, operation, and expiry. Readiness also proves + the acquisition ceiling plus clock-skew margin fits within a fresh token's + safe lifetime, lease expiry cannot exceed token expiry, token payloads never + persist in Task or command records, and delivery reaches only the acquisition + phase. The worker image and configuration contain no App private-key handle. - Telemetry configuration defines the OTLP destination, filtering/redaction bounds, opaque owner-correlation derivation, and telemetry-specific operator access and retention. The service version fixes the metadata allowlist; configuration cannot extend it to prompt/output/tool/source/file-body attributes, secret-bearing fields, raw caller identity, or unfiltered backend-native/OpenInference attribute passthrough. - Configuration contains environment-variable names but never secret values. Startup resolves the complete graph and becomes ready only when trusted ingress, worker transports/identities, store, runtimes, quotas, free-space reserves, supervisor/orphan recovery, and declared profile capabilities pass. Any unprotected remote endpoint or unproved credential/supervisor boundary fails readiness. @@ -632,6 +840,14 @@ docs/src/content/docs/ | Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | | Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | | Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | +| No eligible GitHub provider after acceptance | `TASK_STATE_FAILED` | `materialization/source_auth_unavailable`; safe reason `no_eligible_provider`; `retriable: false`; no provider identity in public detail | +| Selected installation does not cover the repository | `TASK_STATE_FAILED` | `materialization/source_auth_denied`; safe reason `installation_repository_denied`; `retriable: false`; installation identity is operator-only; no `gh` fallback | +| Selected App configuration is invalid | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_configuration_invalid`; `retriable: false`; operator-only provider detail; no `gh` fallback | +| Selected App authentication fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_authentication_failed`; `retriable: false`; operator-only provider detail; no `gh` fallback | +| Selected App token mint or fresh-lifetime validation fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_mint_failed`; `retriable: false`; operator-only provider detail; no `gh` fallback | +| Selected App provider is rate limited | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `provider_rate_limited`; `retriable: true`; no provider identity in public detail; no `gh` fallback | +| Selected App provider service is unavailable | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `provider_unavailable`; `retriable: true`; no provider identity in public detail; no `gh` fallback | +| Eligible trusted-local GitHub CLI provider fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `trusted_local_cli_failed`; `retriable: false`; configured account identity is operator-only | | Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary dispatch/materialization/setup/provider/crash/infrastructure phase, including manifest or materializer failure; safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness, and bounded workspace provenance | | Check, mandatory-evidence, cleanup, crash, or infrastructure failure after result validation | `TASK_STATE_FAILED` | Preserve selected `valid` or `invalid`; preserve exactly one fixed structured-result Artifact for `valid`; later phase remains primary failure | | Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` with `not_produced`, or `structured_result/invalid` with `invalid`; no structured-result Artifact | @@ -649,8 +865,9 @@ docs/src/content/docs/ 3. Build the supervised single-execution worker lifecycle, monotonic command record, failed-quiescence boundary recycling, pre-readiness orphan reaper, OS credential boundaries, the direct Git/OCI/registered-materializer - registry, and hardened workspace/evidence handling against fake - materializers and a fake backend. + registry, the central GitHub App minter/client and trusted-local GitHub CLI + source-credential registry using `@octokit/auth-app`, and hardened + workspace/evidence handling against fake materializers and a fake backend. 4. Add the direct Codex SDK adapter and prove structured output, cancellation, OS-enforced provider/tool credential separation, and native evidence. 5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing OS-confined tools plus the terminating result tool. 6. Package the services and run cross-backend, transport, security, process, and A2A conformance before enabling a consumer. @@ -658,6 +875,10 @@ docs/src/content/docs/ ### System-Wide Impact - **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. +- **Dependency surface:** `@octokit/auth-app` is private to the Node 22 + execution-service package and used only by the trusted control-plane GitHub + App minter. The root Node 18 CLI, remote worker, and acquisition subprocess + do not import the full Octokit client or hold App private-key material. - **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. - **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. - **Security:** New review-critical surfaces are trusted public/private @@ -691,6 +912,20 @@ docs/src/content/docs/ enforce the R13 OS provider/tool boundary or broker; environment filtering remains defense in depth. Pi repository extensions and unrestricted built-in tools never load. +- **Credential fallback, stale entitlement, or issuer-key escalation:** Treat + provider order as eligibility, not retry. Prefer only the operator-mapped + GitHub App installation, permit an account-pinned GitHub CLI provider only in + trusted-local profiles when no mapping applies, and fail closed after every + selected-App failure. Keep App private keys in the central minter. Make the + lease controller derive provider/repository/route state from the current + durable command, use one single-use grant, and reject replay, substitution, + stale fences, or controller/minter configuration-digest disagreement. Use a + fresh `refresh: true` token per cache-miss acquisition, bound its lifetime to + the acquisition deadline plus skew, and reject unsafe ceilings at readiness. + Authenticated lifecycle webhooks plus reconciliation advance entitlement + generations so stale/unknown App state cannot authorize cache reuse. Keep + tokens out of arguments, Git configuration, durable records, logs, evidence, + and later phases; public failures remain coarse and identities operator-only. - **Telemetry disclosure:** Apply KTD10's pre-export guard before every structured log/span processor and reject content or secret-bearing attributes rather than relying on exporter policy. Canary-secret and cross-owner-fragment tests cover agent, model, tool, stale-event, and error paths; telemetry operators receive only bounded metadata and opaque owner correlation under separate access and retention. - **Resource exhaustion:** Reserve per-owner/global gateway quota only for new claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. - **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. @@ -721,7 +956,7 @@ docs/src/content/docs/ structured-result Artifacts, private worker protocol including monotonic command state, original idempotency bindings, typed failures, and conformance fixtures before either service endpoint. -- **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12. +- **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12, KTD16. - **Dependencies:** None. - **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/result-schema-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. - **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas and freeze `https://allagents.dev/a2a/extensions/coding-execution/v1`: required Agent Card advertisement, `A2A-Extensions` negotiation, `Message.extensions`, request data only at `Message.metadata[uri]`, and terminal integrity data only in the single Part of the fixed-name `allagents.execution-integrity` Artifact whose `extensions` contains the URI. Explicitly forbid `Task.extensions`. Define the portable result-schema subset, canonical caller/schema/profile digests, four result states, separate fixed `allagents.structured-result` Artifact, and shared validator. Define original claim bindings independently from mutable current policy. Add worker identity, attempt/fence/lease identity, monotonic command revision, unseen-attempt cancel tombstone, conditional effect revision, event sequence, terminal acknowledgement, and integrity rules. Generate checked-in schemas and fixtures from one source. @@ -735,6 +970,28 @@ docs/src/content/docs/ versioned, domain-separated canonical preimages for materializer definitions, inputs, manifests, profiles, and caller requests; no public field can carry acquisition code, image references, commands, credentials, or policy. + Define normalized source-host/API mappings and ordered source-credential + provider policy as trusted profile/configuration fields. Public schemas cannot + select a provider. Effective-profile canonicalization includes provider IDs, + order, host/API mappings, mapping/configuration digest, pinned CLI account, + non-secret entitlement policy, and current App entitlement generation while + excluding App private keys, resolved tokens, local account tokens, and + per-attempt provider state. Define cache metadata that preserves original + acquisition-provider metadata and cache-hit provenance that separately names + `cache_hit` and current policy selection. + Define the private source-credential protocol separately from the durable + worker command record. A worker request contains only active attempt and + fence under its authenticated route. The controller response carries its + authoritative derivation of effective-profile digest, selected provider, + host/API-mapping digest, installation ID, repository, operation, worker + route/identity, lease epoch, command revision, and expiry, plus a single-use + non-durable grant/response state. The lease/channel binds all derived fields + and cannot outlive the token; the token schema expresses only repository, + read-only contents permission, and GitHub expiry. Define deterministic + source-auth code/reason/retryability enums and operator-only identity detail. + Token payloads are secret transport data: they are never part of public + schemas, canonical digests, Task storage, command records, events, logs, + errors, evidence, or provenance. - **Execution note:** Start with fixture-driven schema, framing, and digest tests. Observe failures for unknown versions, credential-bearing sources, mutable revisions or image tags, duplicate/unsafe destinations, unknown or @@ -761,9 +1018,34 @@ docs/src/content/docs/ definition digest while secret-value rotation does not. - Rotating a resolved secret value, changing attempt/lease/trace identity, or changing a per-run path leaves the profile digest unchanged; changing a - policy field, environment-variable name, authorization scope, or revocation - epoch changes it, and the digest serializer cannot accept secret-bearing - runtime state. + provider policy field, pinned CLI account, host/API mapping or mapping + digest, environment-variable name, authorization scope, revocation epoch, + or App entitlement generation changes it, and the digest serializer cannot + accept secret-bearing runtime state. + - Provider policy fixtures accept GitHub App followed by account-pinned + trusted-local GitHub CLI, reject CLI on remote or multi-tenant routes, + require explicit GitHub Enterprise Server host/API mappings, and reject + every public credential-provider field. Fixtures encode + `gh auth token --hostname --user ` and removal of all four + ambient GitHub token variables. Provider order, ID, host/API mapping, + pinned account, entitlement, or non-secret configuration digest changes + the effective-profile digest; private-key or token rotation does not. + - Private credential-request fixtures accept only active attempt and fence. + Controller-response fixtures carry worker identity/route, attempt, lease + epoch, command revision, fence, effective-profile/provider/mapping/ + installation/repository/operation bindings, and expiry; reject replay, + substitution, stale state, duplicate grant consumption, expiry after token + expiry, or controller/minter configuration-digest disagreement; and cannot + round-trip through durable command/Task serializers. Token fixtures contain + only repository/read-only/expiry scope. Remote App profile fixtures require + the complete central minter/controller capability, safe lifetime policy, + entitlement-generation authority, and no worker-side private-key handle. + - Cache metadata fixtures distinguish original acquisition-provider metadata, + `cache_hit`, and current policy selection/entitlement. Unknown or stale App + generations reject cache authorization. + - Exact source-auth fixtures cover every safe code/reason/retryability tuple + from the error table and prove provider, installation, and account + identifiers are absent from public detail but available to operators. - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - Worker fixtures reject missing/mismatched worker identities, attempt IDs, lease epochs, profile digests, command revisions, conditional-effect revisions, event sequences, bounds, and terminal acknowledgements. Cancel for an unseen attempt persists a tombstone; tombstoned or lower-revision dispatch is invalid before workspace creation. @@ -824,28 +1106,69 @@ docs/src/content/docs/ - **Goal:** Implement the supervised single-execution worker with authenticated transport, a minimal monotonic command record, a closed workspace materializer registry for hardened multi-repository Git, digest-pinned OCI, - and operator-registered images, standard manifest validation, OS-enforced - credential separation, leases, isolated roots, resource controls, - race-resistant evidence, failed-quiescence recycling, and cleanup independent - of any provider. -- **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14. + and operator-registered images, trusted source-credential resolution, + standard manifest validation, OS-enforced credential separation, leases, + isolated roots, resource controls, race-resistant evidence, + failed-quiescence recycling, and cleanup independent of any provider. +- **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14, KTD16. - **Dependencies:** U1. -- **Files:** `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/materializers/types.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/worker/materializers/git.ts`, `packages/execution-service/src/worker/materializers/oci.ts`, `packages/execution-service/src/worker/materializers/external.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/materializers.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`, `packages/execution-service/tests/fixtures/execution/fake-materializer.ts`. +- **Files:** `packages/execution-service/src/source-credentials/contract.ts`, `packages/execution-service/src/source-credentials/registry.ts`, `packages/execution-service/src/source-credentials/github.ts`, `packages/execution-service/src/source-credentials/github-app-minter.ts`, `packages/execution-service/src/source-credentials/github-app-client.ts`, `packages/execution-service/src/source-credentials/github-cli.ts`, `packages/execution-service/src/source-credentials/lease-controller.ts`, `packages/execution-service/src/source-credentials/github-app-entitlements.ts`, `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/materializers/types.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/worker/materializers/git.ts`, `packages/execution-service/src/worker/materializers/oci.ts`, `packages/execution-service/src/worker/materializers/external.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/source-credentials/registry.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-minter.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-client.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-cli.test.ts`, `packages/execution-service/tests/unit/source-credentials/lease-controller.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-entitlements.test.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/materializers.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`, `packages/execution-service/tests/fixtures/execution/fake-materializer.ts`. - **Approach:** Authenticate the configured worker identity and fence every private command. Persist one minimal monotonic command record scoped to worker identity/lease before workspace creation: unseen-attempt cancel writes a tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel effect conditionally rechecks the stored revision immediately before mutation. Reserve one execution only after that check. Validate the fully - canonicalized source against profile resource policy before cache lookup; - bind cache entries to owner or authorization-scope digest, revocation epoch, - canonical source, definition digest, and expected/actual manifest digests. + canonicalized source against profile resource policy before cache lookup. + Bind cache entries to owner or authorization-scope digest, revocation epoch, + current GitHub App entitlement generation when applicable, canonical source, + definition digest, expected/actual manifest digests, and original + acquisition-provider metadata. Authenticated App lifecycle webhooks plus a + bounded reconciler advance entitlement generation on uninstall, suspension, + and repository-selection changes; unknown or stale state fails cache + authorization. A hit revalidates current authorization and records + `cache_hit`, original acquisition provider, and current policy selection/ + entitlement separately. A hit never mints a token. Validate deployment, worker-computed materializer definition digest, and acquisition plus provider/tool credential-boundary capabilities, then emit sequenced NDJSON. Keep transition selection pure. Run inside a dedicated container process namespace under init/reaper or an equivalent systemd/cgroup boundary. Resolve only the closed KTD13 materializer registry. + Resolve direct-Git credentials through the closed source-credential registry + only after source authorization and a cache miss. Normalize the host, select + the configured backend, and evaluate providers in policy order. For GitHub, + trusted operator configuration resolves the installation ID; auth-app never + discovers it. The authenticated worker asks the gateway/control-plane + credential-lease controller only for its active attempt and fence. The + controller rechecks durable command revision, lease epoch, tombstone, and + fence; derives effective-profile digest, selected provider, host/API-mapping + digest, installation ID, canonical repository, operation, worker route/ + identity, and expiry; and issues a single-use non-durable grant/response. + A separate minter atomically consumes that grant and rejects a mismatched + configuration digest. Replay, field or provider substitution, and stale + state fail before minting. + + The control-plane minter uses focused `@octokit/auth-app` with + `refresh: true` for every acquisition. Accept only a fresh token scoped to the + authorized repository, read-only contents permission, and GitHub expiry, + with remaining lifetime strictly greater than the acquisition deadline plus + clock-skew margin; expire the lease no later than the token. The delivery + lease/channel—not the bearer token—binds worker identity, attempt, lease + epoch, command revision, fence, repository, operation, and expiry. The remote + worker never receives the App private key. + + Invoke `gh auth token --hostname --user ` only through the + account-pinned trusted-local provider when no App installation mapping + applies, after removing `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, + and `GITHUB_ENTERPRISE_TOKEN`; fail if the configured account cannot be + resolved. Once App selection begins, every configuration, minting, access, + rate-limit, or service failure is terminal and never retries as the user. + Deliver the selected token through an ephemeral helper channel to the + one-shot Git acquisition process, never a URL, argument, repository config, + durable Task or command record, or later-phase environment. Tear down the + helper and release token references before publishing the workspace. + Launch an external materializer through the configured OCI runner or sandbox as a supervisor-owned resource labeled by worker, attempt, lease, and fence, with only schema-validated and resource-authorized inputs, its source @@ -880,9 +1203,37 @@ docs/src/content/docs/ `Submitted -> Working -> Failed` public trace. - Repositories with LFS configuration/pointers, submodules, hooks, filters, alternates, proxy/helper config, or non-HTTPS secondary protocols cause no - secondary connection or helper execution. + secondary connection or execution of repository, user, or system helpers; + only the KTD16-selected one-shot credential channel can run. - Source credentials leave no repository config, process argument, child phase environment, log, error, evidence, or retained workspace trace. + - GitHub credential resolution uses only the trusted operator + repository-to-installation mapping and never auth-app discovery. A remote + worker request contains only active attempt/fence; the fake controller + derives profile/provider/mapping/installation/repository/operation/route, + rechecks command revision, lease epoch, tombstone, and fence, and returns + one authenticated single-use non-durable lease response. Wrong worker, + replay, duplicate consumption, substituted repository/provider/operation, + stale command state, or controller/minter configuration-digest disagreement + fails before token delivery. The token carries only repository, + read-only-contents, and GitHub-expiry scope; the lease carries worker, + attempt, lease epoch, command revision, fence, operation, and delivery + expiry. The worker never receives the App private key. + - Auth-app is called with `refresh: true` for each acquisition. A cached + near-expiry token is bypassed, remaining lifetime must exceed acquisition + deadline plus clock-skew margin, lease expiry is capped by token expiry, and + an acquisition ceiling that can exceed a fresh token's safe lifetime fails + readiness. Every boundary failure remains terminal without invoking `gh`. + - A trusted-local profile invokes its fake CLI only when no installation + mapping applies, pins `--hostname --user `, removes all four + ambient GitHub token variables, and fails on account mismatch. Remote + profiles cannot select it. GitHub Enterprise Server works only through an + explicit host/API mapping. Provider ID, host, installation/account, and + selection reason appear only in operator provenance, never public detail. + - Table-driven failures assert the exact public safe code, reason, and + retryability for no provider, installation/repository denial, App + configuration/authentication/mint failure, rate limit, service outage, and + trusted-local CLI failure. No selected-App case invokes the CLI. - OCI tags, foreign/external URLs, cross-origin credential forwarding, disallowed registry/auth/blob host/address/port, redirects, DNS rebinding, manifest/layer mismatches, unsafe layers, missing workspace manifests, and @@ -899,11 +1250,16 @@ docs/src/content/docs/ canaries in output or retained logs fail publication. This verifies phase teardown, not safety from a malicious operator-registered image that intentionally transforms a credential. Provenance labels its unverified - source assertions as materializer-attested. A cache hit occurs only after - current authorization and revalidates content plus the same owner or - authorization scope, revocation epoch, canonical source, - materializer-definition, expected-output, and actual output-manifest - digests inside the same trust domain. + source assertions as materializer-attested. + - A cache hit occurs only after current authorization and revalidates content + plus the same owner or authorization scope, revocation epoch, canonical + source, materializer-definition, expected-output, actual output-manifest, + trust domain, and current App entitlement generation. Authenticated webhook + events and bounded reconciliation for uninstall, suspension, and repository + selection advance the generation; unknown, stale, or mismatched state + rejects reuse. Hit provenance records `cache_hit`, original acquisition + provider metadata, and current policy selection/entitlement separately, + including a hit after provider-policy change, and performs no mint. - Setup changes establish the baseline; setup and checks receive no provider/control secrets. Credentialed provider runtimes and model tools run across the declared OS UID/process/mount boundary or broker, with disjoint config/data roots and ambient selectors removed. - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - Block dispatch after effect selection, complete a newer cancel for the unseen attempt, then release dispatch: the command tombstone/revision check rejects it before workspace or provider creation. Duplicate commands remain idempotent and all effects stay fence-bound. @@ -921,13 +1277,19 @@ docs/src/content/docs/ - Cross-filesystem staging/publication configuration fails readiness. Faults around the final rename expose either no final workspace or the complete validated tree, never a copy fallback or partial publication. -- **Verification:** A built supervised worker materializes equivalent - workspaces through disposable exact-SHA repositories, a local digest-pinned - OCI snapshot, and a fake registered materializer; validates one standard - manifest; mutates each through the fake adapter; and proves authenticated - revisioned dispatch, unseen-cancel tombstones, acquisition hardening and - credential teardown, budgets, result preservation, quiescence or - poisoned-boundary exit, evidence integrity, worker-crash containment, +- **Verification:** A built supervised worker, authoritative fake lease + controller, and fake central minter materialize equivalent workspaces through + disposable exact-SHA repositories, a local digest-pinned OCI snapshot, and a + fake registered materializer; validate one standard manifest; mutate each + through the fake adapter; and prove authenticated revisioned dispatch, + unseen-cancel tombstones, deterministic App-before-account-pinned-CLI + eligibility, remote App private-key exclusion, controller-derived single-use + lease delivery, repository/read/expiry-only token scope, replay/substitution/ + stale-state/config-digest rejection, fresh-token lifetime boundaries, + entitlement-generation cache revocation and truthful hit provenance, exact + source-auth mappings, no fallback after selected-App failure, acquisition + hardening and credential teardown, budgets, result preservation, quiescence + or poisoned-boundary exit, evidence integrity, worker-crash containment, orphan-root handling, and cleanup. ### U5. Codex backend adapter @@ -970,24 +1332,37 @@ docs/src/content/docs/ ### U7. Production registry, service packaging, and observability - **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with trusted transports, peer identity, credential-boundary and supervisor readiness, safe startup/shutdown, tracing, and reproducible containers. -- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD15. +- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD16. - **Dependencies:** U3-U6. -- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/unit/worker/materializers/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. +- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/gateway/github-app-webhook.ts`, `packages/execution-service/src/gateway/github-app-reconciler.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/gateway/github-app-webhook.test.ts`, `packages/execution-service/tests/unit/gateway/github-app-reconciler.test.ts`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/unit/worker/materializers/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. - **Approach:** Register only Codex and Pi as backend adapters and register the built-in Git/OCI materializers plus configured external materializers through a separate closed registry. Add gateway and supervised worker entrypoints - inside the private Node 22 workspace. Before readiness, validate named public - TLS termination, every remote worker's mTLS/equivalent transport and pinned - identity/capabilities, Unix-socket locality, store, runtimes, matching - gateway/worker materializer definition digests, image digests, schemas, - credential names, egress, limits, and OCI runner/sandbox isolation, monotonic - command storage, acquisition and provider/tool credential-boundary - capabilities, - supervisor boundary, orphan roots, trust, quotas, and resource controls. - Propagate `traceparent`, then apply KTD10's small shared metadata allowlist and - bounded filtering/redaction before any structured log/span processor or OTLP - exporter; neither OpenInference nor backend-native attributes bypass it. - Build a minimal gateway image with no provider or materializer runtime and a + inside the private Node 22 workspace. Register source credentials separately: + an authoritative gateway/control-plane lease controller, a trusted GitHub App + minter using focused `@octokit/auth-app`, its authenticated single-use + non-durable worker client, and an account-pinned GitHub CLI provider only on + trusted-local acquisition hosts. Wire authenticated GitHub App lifecycle + webhooks and bounded reconciliation to the durable entitlement-generation + store. Reject duplicate or ambiguous provider IDs, implicit enterprise host + detection, unpinned CLI accounts, ambient GitHub token variables, remote CLI + fallback, worker-side App private-key handles, fallback-on-error policy, and + provider/mapping configuration-digest disagreement. + Before readiness, validate named public TLS termination, every remote + worker's mTLS/equivalent transport and pinned identity/capabilities, the + complete central lease-controller/minter path for every remote App profile, + single-use grant consumption, fresh-token lifetime versus acquisition ceiling + and clock skew, current entitlement-generation authority, Unix-socket + locality, store, runtimes, matching gateway/worker materializer definition + digests, image digests, schemas, credential names, egress, limits, OCI + runner/sandbox isolation, monotonic command storage, acquisition and + provider/tool credential-boundary capabilities, supervisor boundary, orphan + roots, trust, quotas, and resource controls. Propagate `traceparent`, then + apply KTD10's small shared metadata allowlist and bounded filtering/redaction + before any structured log/span processor or OTLP exporter; neither + OpenInference nor backend-native attributes bypass it. Build a minimal + gateway/control-plane image with the lease controller and App minter but no + provider runtime, writable repository, or baked-in private key, and a one-execution worker image whose init kills the complete boundary when the worker server exits, including poisoned failed-quiescence exit. - **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. @@ -998,6 +1373,26 @@ docs/src/content/docs/ external IDs, resolves every external image to the configured digest, rejects duplicates/tags/unknown IDs, and cannot be influenced by request image, command, credential, or policy fields. + - Source-credential registry maps `github.com` and explicit enterprise + host/API pairs, selects only an operator-mapped App installation, and + permits GitHub CLI only for an account-pinned trusted-local no-mapping case. + The CLI invocation includes `--user` and no ambient GitHub token variables. + No request field can alter provider selection. Public output contains only + safe code/reason/retryability; operator provenance contains the non-secret + provider and installation/account identity. + - Remote App profiles fail readiness without the central minter and + authoritative lease controller, entitlement-generation webhook/ + reconciliation authority, safe fresh-token lifetime policy, single-use + grant support, matching provider/mapping configuration digest, or with an + App private-key handle in worker configuration. Runtime requests by active + attempt/fence derive every provider/repository/route field from current + durable state and reject replay, substitution, stale state, and duplicate + consumption. The same-host trusted case and a separately deployed minter + pass the same contract; neither uses snapshot delivery. + - Readiness rejects an acquisition ceiling that can exceed a fresh token's + safe lifetime. Near-expiry auth-app cache output is bypassed with + `refresh: true`, lease expiry is capped by token expiry, and failure never + falls through to the CLI. - Gateway and supervised worker start from built outputs, become ready only after trusted transport/identity, credential and supervisor boundaries, dependencies, and orphan recovery pass, and stop gracefully on SIGTERM. - Gateway readiness fails for malformed auth, missing/mismatched named TLS termination, plaintext production public ingress, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - Worker-route readiness fails for plaintext remote URL, wrong/untrusted certificate, worker identity/capability mismatch, or replayed capability; mTLS/equivalent authenticated encryption and same-host Unix sockets pass. @@ -1010,10 +1405,26 @@ docs/src/content/docs/ egress and resource enforcement cannot be provided. - Killing or poisoning the worker server while an adapter child and invocation root exist makes the supervisor destroy the boundary; replacement readiness waits for root deletion/quarantine and never reuses it. - Trace context crosses the authenticated private call and correlates result identities using only opaque owner correlation. Exporter probes for agent, model, tool, stale-event, and error spans contain allowlisted bounded metadata but no canary secret, prompt/output, tool argument/result, file body/source fragment, raw caller identity, or cross-owner fragment; exporter failure cannot change Task status. - - Gateway image contains no Codex, Pi, Git workspace, provider credential material, or worker trust private keys. - - Worker image pins both runtimes, enforces provider/tool UID/process/mount separation or the credential broker, confines one workspace/config root, disables repository Pi extensions and unrestricted built-ins, enforces deployment limits, and completes fake-provider security probes. + - Gateway/control-plane images contain no Codex, Pi, Git workspace, coding + provider credentials, or baked-in GitHub App private key; the App key enters + only through its configured secret handle. Worker images and configuration + contain neither App issuer material nor user credential stores, pin both + coding runtimes, enforce provider/tool UID/process/mount separation or the + credential broker, confine one workspace/config root, disable repository Pi + extensions and unrestricted built-ins, enforce deployment limits, and + complete fake-provider security probes. - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. -- **Verification:** The registry dispatches both adapters through the same worker contract; built services and images pass lifecycle/security smoke tests; exporter-capture tests prove pre-processor metadata allowlisting, bounded redaction, opaque owner correlation, and canary/cross-owner exclusion across agent, model, tool, stale-event, and error spans; CI and publication bind immutable image tags to the release commit. +- **Verification:** The registries dispatch both adapters and + source-credential providers through their respective contracts; built + services and images prove controller-authorized fresh App minting without + worker issuer material, single-use lease replay/staleness/config-digest + rejection, entitlement-generation webhook/reconciliation behavior, + account-pinned sanitized CLI eligibility, deterministic public failure + mapping with operator-only identities, and lifecycle/security behavior; + exporter-capture tests prove pre-processor metadata allowlisting, bounded + redaction, opaque owner correlation, and canary/cross-owner exclusion across + agent, model, tool, stale-event, and error spans; CI and publication bind + immutable image tags to the release commit. ### U8. Cross-backend conformance, documentation, and release evidence @@ -1030,6 +1441,16 @@ docs/src/content/docs/ publication, supervisor-owned runner cleanup, acquisition credential teardown, authorization-scoped cache reuse/revocation, source-mode capabilities, and the prohibition on caller-supplied acquisition code. + Document normalized host/API mapping, operator-owned + repository-to-installation mapping, GitHub App precedence and + repository/read/expiry token scope, account-pinned sanitized trusted-local + CLI eligibility, fail-closed selected-App behavior, focused + `@octokit/auth-app` ownership and `refresh: true`, central App private-key + custody, controller-derived single-use remote lease bindings, token/lease + lifetime rules, entitlement-generation webhooks/reconciliation and + cache-hit provenance, deterministic public source-auth mapping with + operator-only identities, remote readiness requirements, the one initial + token-minter path, and deferred versioned snapshot delivery. - **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. - **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, Starlight guide/reference organization under `docs/src/content/docs/`, and Buzz's required-critical-action coverage rule without importing its TLA+ model or production implementation. - **Test scenarios:** @@ -1043,7 +1464,24 @@ docs/src/content/docs/ disclosure, and setup failure after worker acceptance produce the selected `Submitted -> Working -> Failed` trace; provider invocation never begins, and durable snapshots, streams, and conformance records agree. A policy - revocation before lookup cannot consume a previously populated cache entry. + revocation, webhook-advanced entitlement generation, reconciliation result, + or unknown/stale App state before lookup cannot consume a previously + populated cache entry. A valid hit after provider-policy change records + `cache_hit`, original acquisition provider, and current selection separately + without minting. + - GitHub source cases prove operator-mapped App selection, account-pinned + sanitized trusted-local CLI selection only when no mapping applies, no CLI + invocation after any selected-App failure, explicit enterprise host/API + mapping, repository/read-only/expiry-only token narrowing, and an + authenticated controller-derived single-use lease carrying worker identity, + attempt, lease epoch, command revision, fence, repository, operation, and + expiry without worker issuer material. They reject replay, substitution, + stale state, duplicate grant consumption, and controller/minter + configuration-digest disagreement; bypass near-expiry auth-app cache output + with `refresh: true`; cap lease expiry by token expiry; fail readiness for an + unsafe acquisition ceiling; assert every safe source-auth + code/reason/retryability tuple and operator-only identity detail; and + publish a credential-free workspace. - Pause dispatch after selection, complete unseen-attempt cancel, then release dispatch; the stale command creates no workspace/process. The small independent checker enforces each fixture's attempt/fence correlation, happens-before edges, maximum counts, and forbidden post-terminal effects. Deliberately bad traces that still contain every required action name fail for wrong order, wrong fence, duplicate-over-maximum effects, and an extra stale dispatch after terminalization. - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; one worker serializes admitted work. - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order @@ -1074,7 +1512,11 @@ docs/src/content/docs/ names. Docs state the three source modes and exact discriminator, standard workspace manifest and provenance labels, materializer registry/profile boundary, worker-derived definition digest, resource authorization and - cache-revocation boundary, prohibition on caller-supplied acquisition code, + entitlement-generation cache-revocation boundary, cache-hit/original/current + provider provenance, prohibition on caller-supplied acquisition code, + one selected remote token-minter/lease path with deferred snapshot delivery, + account-pinned sanitized CLI invocation, token versus lease scope, fresh + token and readiness lifetime rules, deterministic source-auth mapping, same-filesystem publication, supervisor-owned runner cleanup, acquisition credential teardown, two transport boundaries, one gateway replica, one execution per worker, reviewed trust domain, narrow credential isolation @@ -1090,16 +1532,16 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| -| Contract generation | U1 | Exact extension URI/carriers, closed workspace source discriminator and manifest, algorithm-qualified materializer/profile/input/output digest preimages and vectors, verification-method vocabulary, authorization-scope/revocation fields, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | -| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, source authorization/cache revocation, and cleanup. | -| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, direct Git/OCI/registered materialization, worker-derived registry digests, acquisition credential teardown, supervisor-owned materializer runners, same-filesystem atomic publication, workspace-manifest provenance labels, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | +| Contract generation | U1 | Exact extension URI/carriers, closed workspace source discriminator and manifest, algorithm-qualified materializer/profile/input/output digest preimages and vectors, verification-method vocabulary, authorization-scope/revocation/App-entitlement fields, cache-hit/original/current-provider provenance, worker request limited to active attempt/fence, controller-derived single-use non-durable lease fields and token-versus-lease scope, exact source-auth code/reason/retryability tuples, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | +| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, account-pinned sanitized CLI execution, fresh-token lifetime boundaries, entitlement-generation authorization/cache revocation, provenance states, exact source-auth failures, and cleanup. | +| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, direct Git/OCI/registered materialization, deterministic operator-mapped-App-before-account-pinned-CLI eligibility, central fresh App minting without worker issuer material, controller-derived single-use non-durable lease delivery, repository/read/expiry-only token scope, replay/substitution/stale/config-digest rejection, entitlement-generation cache revocation and hit provenance, exact failure mapping, fail-closed selected-App behavior, worker-derived registry digests, acquisition credential teardown, supervisor-owned materializer runners, same-filesystem atomic publication, workspace-manifest provenance labels, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | | Backend conformance | U5-U8 | One shared suite passes against Codex and Pi, including the versioned schema subset, four result states, valid/invalid preservation across later failure, integrity Artifact carrier, and structured-result Artifact rule. | -| Credentialed provider smoke | U4-U6, U8 | An available operator-trusted registered materializer and each available provider mutate a disposable immutable workspace while adversarial later-phase probes cannot directly access acquisition/provider credential environments, mounts, processes, roots, or runner control planes and literal canaries remain absent; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | +| Credentialed provider smoke | U4-U6, U8 | An available centrally held GitHub App, account-pinned trusted-local `gh` login, operator-trusted registered materializer, and each available coding provider mutate disposable immutable workspaces while provider selection follows policy. App acquisition proves `refresh: true`, minimum remaining lifetime, lease-at-or-before-token expiry, repository/read-only/expiry token scope, and no worker issuer material; local CLI smoke proves `--user` and sanitized ambient token variables. Adversarial later-phase probes cannot directly access acquisition/provider credential environments, mounts, processes, roots, or runner control planes and literal canaries remain absent; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | | A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes required-extension negotiation and legal carriers, immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, retained replay, cancel races, expiry, and owner isolation without `Task.extensions`. | -| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, exact source-resource authorization, per-connection Git/OCI SSRF and credential-origin controls, authorization-scoped cache revocation, digest-pinned registered materializers, schema/manifest validation, truthful provenance labels, acquisition and provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | +| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, exact source-resource authorization, per-connection Git/OCI SSRF and credential-origin controls, operator-mapped GitHub provider eligibility without identity escalation, central issuer-key custody, worker request limited to active attempt/fence, authoritative current-state derivation, single-use lease replay/substitution/staleness/config-digest rejection, repository/read/expiry-only token scope, fresh-token lifetime safety, fail-closed selected-App errors, account-pinned sanitized CLI use, exact public failure mappings with operator-only identities, webhook/reconciliation-driven entitlement cache revocation, truthful cache-hit provenance, digest-pinned registered materializers, schema/manifest validation, acquisition and provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | | Lifecycle trace conformance | U8 | The small test-side checker, independently of production selectors, validates attempt/fence correlation, required happens-before edges, maximum occurrence counts, and forbidden post-terminal effects against durable records plus observed worker/process outcomes; all-name-present bad traces fail for wrong order/fence/multiplicity and stale post-terminal dispatch. | | Telemetry safety | U7-U8 | Exporter capture across agent, model, tool, stale-event, and error spans proves the pre-processor allowlist and bounded redaction exclude prompt/output/tool/source/file content, canary secrets, raw identities, and cross-owner fragments while retaining only bounded operational metadata and opaque owner correlation. | -| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build, gateway/supervised-worker smoke, backend and materializer registry readiness, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both service container builds pass. | +| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build with focused `@octokit/auth-app` and no full Octokit client, gateway/control-plane lease controller and minter plus supervised-worker smoke, entitlement webhook/reconciler, worker issuer-key exclusion, backend/materializer/source-credential registry readiness, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both service container builds pass. | | Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | | Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | @@ -1119,16 +1561,27 @@ The authoritative behavioral proof is the built-process E2E path with the offici - Production public ingress uses its named TLS boundary, remote worker routes authenticate and encrypt peers with worker identity/capability binding, and same-host Unix sockets are the only non-network alternative; unprotected remote endpoints fail readiness. - Cancellation/deadlines use monotonic worker command tombstones and one native abort. Stale dispatch cannot create work, and failed quiescence poisons and exits the worker so supervisor destruction and replacement orphan recovery precede new admission. - Workspace source validation and exact resource authorization, - per-connection direct Git/OCI controls, worker-derived registered-materializer - digests, the standard workspace manifest and truthful provenance labels, - authorization-scoped cache revocation, same-filesystem atomic publication, + per-connection direct Git/OCI controls, trusted-policy GitHub provider + resolution with operator-mapped App precedence, account-pinned sanitized + local-only CLI eligibility, no selected-App failure fallback, central App + private-key custody, controller-derived single-use authenticated lease + delivery, repository/read-only/expiry-only token scope, current-command + recheck and replay/substitution/stale/config-digest rejection, fresh-token and + readiness lifetime bounds, deterministic public source-auth mapping with + operator-only identities, authenticated webhook/reconciliation-driven App + entitlement generations, fail-closed unknown/stale cache authorization, and + separate cache-hit/original-acquisition/current-selection provenance are + enforced end to end. The initial remote path is central token minting and + non-durable lease delivery; versioned snapshot delivery remains deferred. + Worker-derived registered-materializer digests, the standard workspace + manifest and truthful provenance labels, same-filesystem atomic publication, supervisor-owned runner cleanup, acquisition credential teardown, OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled repository Pi extensions/unrestricted built-ins, one-execution reviewed-domain policy, resource limits, Artifact race defenses, - completeness, provenance, and authenticated expiry are enforced end to end - without accepting caller acquisition code or claiming hostile-source or - cross-tenant isolation. + completeness, provenance, and authenticated expiry are enforced without + accepting caller acquisition code or claiming hostile-source or cross-tenant + isolation. - Metadata-only telemetry is filtered through the fixed allowlist and bounded redaction before processing/export; canary secrets, content, raw caller identities, and cross-owner fragments never reach exporters, and only opaque owner correlation crosses the separately governed operator boundary. - Required source/setup and race traces satisfy attempt/fence, happens-before, maximum-count, and forbidden-post-terminal constraints in the independent test-side checker; all-name-present malformed traces fail without introducing a parallel lifecycle implementation. - Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. @@ -1139,27 +1592,44 @@ The authoritative behavioral proof is the built-process E2E path with the offici - U1: Standard extension carriers, closed workspace source/manifest contracts, algorithm-qualified materializer/profile/input/output digest preimages, - verification and authorization vocabulary, integrity/structured-result - Artifact schemas, four result states, original claim digests, command - revisions/tombstones, fence rules, typed failures, and fixtures are generated - and stable. + verification and authorization vocabulary, App entitlement generation and + cache provenance states, active-attempt/fence-only credential requests, + controller-derived single-use non-durable lease bindings, token-versus-lease + scope, exact source-auth code/reason/retryability tuples, + integrity/structured-result Artifact schemas, four result states, original + claim digests, command revisions/tombstones, fence rules, typed failures, and + fixtures are generated and stable. - U2: Trusted ingress, auth, opaque owner isolation, retained-claim-first replay, original bindings, atomic new admission, CAS settlement, pagination, startup recovery, quotas, Artifact access, tombstones, and cleanup pass fault injection. - U3: Every advertised A2A operation agrees across stream and lookup while extension negotiation, replay ordering, authenticated worker routes, fencing, monotonic cancellation, and races preserve one Task. - U4: Worker command state, exact source authorization, direct - Git/OCI/registered materialization, workspace-manifest validation and - provenance classification, authorization-scoped cache revocation, + Git/OCI/registered materialization, operator-mapped GitHub + App-before-account-pinned-CLI eligibility with sanitized invocation and + fail-closed selected-App errors, central fresh App minting without worker + issuer material, authoritative single-use lease delivery with + replay/substitution/stale/config-digest rejection, + repository/read-only/expiry-only token scope and safe lifetime boundaries, + deterministic public failure mapping with operator-only identities, + webhook/reconciliation-driven entitlement cache revocation and truthful hit + provenance, workspace-manifest validation and provenance classification, same-filesystem atomic publication, acquisition and provider credential separation, supervisor-owned runner cleanup, poisoned-exit/orphan recovery, and dispatch/materialization/setup/action/check/quiescence/evidence/cleanup pass malicious, crashed, and faulted scenarios. - U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, result preservation, OS credential separation, native evidence, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. - U6: Pi strict RPC/framing, terminating result, exact policy tools, disabled repository extensions/built-ins, OS-isolated credential store/provider runtime, result preservation, settlement, stats, abort, and process cleanup pass verification. -- U7: Closed backend and materializer registries, trusted - transport/identity/readiness, acquisition/provider credential and supervisor - capability gating, poisoned-worker recycling, metadata-only pre-export - telemetry controls, Node-version separation, tracing, shutdown, containers, - and release artifacts work from built outputs. -- U8: Cross-backend E2E, all three workspace source modes, standard manifest +- U7: Closed backend, materializer, and source-credential registries, focused + `@octokit/auth-app` packaging without a full Octokit or root-CLI dependency, + authoritative lease controller, central App private-key custody and fresh + minter readiness, entitlement webhook/reconciler, account-pinned sanitized + local CLI, trusted transport/identity/readiness, acquisition/provider + credential and supervisor capability gating, poisoned-worker recycling, + metadata-only pre-export telemetry controls, Node-version separation, + tracing, shutdown, containers, and release artifacts work from built outputs. +- U8: Cross-backend E2E, all three workspace source modes, central GitHub App + and account-pinned trusted-local CLI credential-selection cases, remote + issuer-key exclusion, controller-derived single-use lease delivery, + fresh-token lifetime, token-versus-lease scope, exact failure mappings, + entitlement-driven cache invalidation and hit provenance, standard manifest provenance, acquisition credential teardown, standard A2A carriers, retained replay, selected materialization/setup transitions, independent race-trace constraints, telemetry canary/cross-owner probes, transport and credential diff --git a/docs/research/source-credential-broker-precedents.md b/docs/research/source-credential-broker-precedents.md new file mode 100644 index 00000000..b1be9054 --- /dev/null +++ b/docs/research/source-credential-broker-precedents.md @@ -0,0 +1,260 @@ +# Source credential broker precedents + +## Decision + +The execution gateway does **not** need a mandatory standalone Git credential +broker for trusted local use. The settled local provider is an explicit, +account-pinned `gh auth token --hostname --user ` helper invoked +only when no App installation mapping applies. Its environment removes +`GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and +`GITHUB_ENTERPRISE_TOKEN`, and its token is exposed only to the one-shot +acquisition process. Git credential helpers and Git Credential Manager (GCM) +establish the process-boundary precedent, but arbitrary configured helpers are +not part of the selected implementation. A local helper is a broker in the +security sense; it is not a separately deployed network service. + +Remote or multi-tenant workers use one initial path: an authoritative +gateway/control-plane lease controller and trusted central token minter deliver +a fresh GitHub token over an authenticated, single-use, non-durable lease. The +GitHub bearer token is scoped only to the repository, read-only contents +permission, and GitHub expiry. Worker identity, attempt, lease epoch, command +revision, fence, operation, and delivery expiry are properties of the lease and +channel, not the token. Workers never inherit a person's credential helper, +credential store, SSH agent, or the App private key. A versioned central +snapshot-delivery protocol is deferred; it is not an alternative initial +readiness path. The minter may live inside the trusted control plane unless +private-key isolation, audit, scaling, or blast-radius requirements justify a +separate service process. + +## Precedents + +### Git credential helpers and Git Credential Manager + +**Trust boundary.** Git credential helpers are external programs. Git invokes a +configured helper through the shell, supplies an operation and credential context, +and stops consulting helpers after it has a username and a non-expired password +([Git `gitcredentials`](https://git-scm.com/docs/gitcredentials#Documentation/gitcredentials.txt-helper)). +The scriptable `git credential fill` interface sends the repository context on +standard input and returns the resolved username and password on standard output +([Git `git-credential`](https://git-scm.com/docs/git-credential#_typical_use_of_git_credential)). +Consequently, the Git/acquisition process receives the resulting bearer secret; +the helper is not a membrane that makes an untrusted caller safe. + +GCM is an implementation of this local contract, not a required remote service. +Its executable is a console application; on every invocation it reads Git's +request from standard input, retrieves or generates a credential, serializes the +credential to standard output, and terminates +([GCM architecture, “Command execution”](https://github.com/git-ecosystem/git-credential-manager/blob/main/docs/architecture.md#command-execution)). +Git calls it implicitly, and later Git commands reuse stored credentials or tokens +while they remain valid +([GCM README, “How to use”](https://github.com/git-ecosystem/git-credential-manager#how-to-use)). +GCM can put credentials in OS-controlled stores such as Windows Credential +Manager or macOS Keychain, use Secret Service or GPG-backed storage, use Git's +ephemeral cache, or disable its store entirely +([GCM credential stores](https://github.com/git-ecosystem/git-credential-manager/blob/main/docs/credstores.md)). + +**Lifetime.** Helper-process lifetime and credential lifetime are separate. GCM +exits after each request, while the selected store controls token persistence. +Git's built-in cache is an optional local daemon reachable over a Unix-domain +socket restricted to the current user; it forgets credentials after 900 seconds +by default or sooner if the daemon dies +([Git `git-credential-cache`](https://git-scm.com/docs/git-credential-cache#_description), +[options](https://git-scm.com/docs/git-credential-cache#_options)). This is a +local process/socket boundary, not a remotely reachable credential service. + +**Relevance.** Git helpers and GCM prove that a local credential provider can +be an on-demand process rather than a network service. AllAgents does not, +however, inherit or invoke an arbitrary configured helper chain. Its closed +provider registry permits only an explicit GitHub CLI provider pinned to a +configured non-secret account in a trusted-local profile, and only when no +configured GitHub App installation mapping applies. The helper invokes +`gh auth token --hostname --user ` without ambient GitHub token +variables. Its output reaches only the one-shot acquisition child; setup and +the coding harness inherit neither helper configuration nor the token. + +### SSH agent forwarding + +**Trust boundary.** `ssh-agent` holds private keys and exposes operations through +a Unix-domain socket. With forwarding, private keys and passphrases do not cross +the network; the SSH connection carries requests to the local agent and returns +the results +([OpenSSH `ssh-agent`](https://man.openbsd.org/ssh-agent#DESCRIPTION)). The +forwarded socket is nevertheless an authentication capability. OpenSSH warns +that anyone able to bypass the remote socket's permissions can use loaded +identities to authenticate even though they cannot extract the key material +([OpenSSH `ForwardAgent`](https://man.openbsd.org/ssh_config#ForwardAgent)). +GitHub gives the same operational warning: a trusted server can use the keys as +the user while the connection is established, so forwarding should be enabled +only for specifically trusted hosts +([GitHub, “Using SSH agent forwarding”](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/using-ssh-agent-forwarding#setting-up-ssh-agent-forwarding)). + +**Lifetime.** The remote forwarding capability lasts for the SSH connection. +The underlying identity may live longer: `ssh-agent` has no default maximum +identity lifetime unless configured, while `ssh-add -t` can impose one and +`ssh-add -c` can require confirmation for each use +([OpenSSH `ssh-agent -t`](https://man.openbsd.org/ssh-agent#t), +[OpenSSH `ssh-add`](https://man.openbsd.org/ssh-add#c)). + +**Relevance.** Agent forwarding is precedent for reusing a local identity +without copying the long-lived private key, but it is not selected for +AllAgents direct Git acquisition, which accepts canonical HTTPS repository URLs +only. A remote process with the forwarded socket could authenticate as the +user, so the socket must never reach a remote worker, setup code, or the +coding-agent runtime. + +### GitHub App installation tokens and Actions checkout + +**Trust boundary.** A GitHub App uses an RS256 JWT, created with the App private +key, to request an installation access token +([GitHub, “Generating a JSON Web Token”](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-a-json-web-token-jwt-for-a-github-app)). +The mint request can narrow the token to selected repositories and permissions, +and GitHub will not grant repositories or permissions beyond those already +granted to the installation +([GitHub, “Generating an installation access token”](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app#generating-an-installation-access-token)). +This separates high-value issuer material from the disposable credential handed +to a worker. + +GitHub Actions applies that model per job. GitHub creates a unique +`GITHUB_TOKEN` before each job; it is a GitHub App installation token limited to +the workflow repository, with permissions reducible through workflow policy +([GitHub Actions `GITHUB_TOKEN`](https://docs.github.com/en/actions/concepts/security/github_token#about-the-github_token)). +`actions/checkout` uses the token for Git commands, stores persisted credentials +in a separate file under `RUNNER_TEMP`, references that file from Git config, and +removes the references and file during post-job cleanup +([checkout README, v6 credential storage](https://github.com/actions/checkout#checkout-v6), +[checkout credential setup](https://github.com/actions/checkout/blob/main/src/git-auth-helper.ts#L329-L436), +[checkout credential cleanup](https://github.com/actions/checkout/blob/main/src/git-auth-helper.ts#L475-L510)). + +**Lifetime.** A normal GitHub App installation token expires after one hour +([GitHub installation token documentation](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app#generating-an-installation-access-token)). +The Actions token expires when its job finishes or at its effective maximum +lifetime; GitHub documents a six-hour maximum on GitHub-hosted runners and at +most 24 hours of refresh for longer self-hosted jobs +([GitHub Actions `GITHUB_TOKEN`](https://docs.github.com/en/actions/concepts/security/github_token#about-the-github_token)). +Checkout's credential file is a convenience capability inside that job, not a +long-term credential store, and its post-job deletion is defense in depth rather +than the token's revocation mechanism. + +**Relevance.** This is the closest production precedent for AllAgents: keep the +App private key at a trusted central minter, issue one fresh least-privilege +token for a particular repository acquisition, expose it only during that +phase, and remove its local material afterward. GitHub enforces repository, +read-only contents permission, and expiry; AllAgents separately enforces +attempt and operation bindings through its authenticated delivery lease. + +### BuildKit secret and SSH mounts + +**Trust boundary.** BuildKit distinguishes secret delivery from ordinary build +arguments and environment variables, which can persist in an image. A secret +mount makes a client-provided secret temporarily available only to a particular +build instruction; an SSH mount supplies an agent socket or key and is intended +for cases such as fetching private Git repositories +([Docker build secrets](https://docs.docker.com/build/building/secrets/#types-of-build-secrets)). +`RUN --mount=type=secret` makes the value available without baking it into the +image, while `RUN --mount=type=ssh` exposes SSH-agent access through a mounted +socket +([Dockerfile secret mount](https://docs.docker.com/reference/dockerfile/#run---mounttypesecret), +[Dockerfile SSH mount](https://docs.docker.com/reference/dockerfile/#run---mounttypessh)). + +The isolation guarantee is intentionally narrow. BuildKit states that secret +values must not be written to disk or included in cache checksums and that an +untrusted frontend cannot access forwarded SSH private keys; it also states that +a container explicitly run with a secret mount can read that secret +([BuildKit security boundary](https://github.com/moby/buildkit/blob/master/PROJECT.md#security-boundary)). +A mount therefore limits *where and when* a capability appears; it does not make +code within the mounted step trustworthy. + +**Lifetime.** The secret mount is available for the duration of its build +instruction, rather than becoming part of the resulting image +([Docker build secrets](https://docs.docker.com/build/building/secrets/#secret-mounts)). +When an agent socket is supplied, SSH access is available for the mounted +instruction without adding the private key to the image +([Dockerfile SSH mount](https://docs.docker.com/reference/dockerfile/#run---mounttypessh)). + +**Relevance.** AllAgents should copy the phase-scoping pattern, not necessarily +BuildKit itself: inject a token or agent capability only into the trusted source +acquisition operation, then tear down the mount/socket/environment before setup +or agent execution. Like BuildKit, this delivery mechanism does not mint +credentials and does not eliminate the need for a central issuer in production. + +## Recommendation for AllAgents + +### Local mode + +1. Resolve `github.com` through the built-in GitHub backend and require explicit + host/API mappings for GitHub Enterprise Server hostnames. +2. Prefer a configured GitHub App installation that trusted operator policy + maps to the authorized repository. Do not use `@octokit/auth-app` to discover + installations. If no installation mapping applies, a trusted-local profile + may invoke the explicit + `gh auth token --hostname --user ` provider pinned to a + configured non-secret account. Include that account in the entitlement and + effective-profile digests, remove `GH_TOKEN`, `GITHUB_TOKEN`, + `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` from the helper + environment, and fail if the configured account cannot be resolved. Do not + inherit an arbitrary Git helper/GCM chain or forward an SSH agent. +3. Treat provider order as eligibility, not retry. Once the App provider is + selected, configuration, authentication, minting, authorization, rate-limit, + or service failure terminates acquisition without falling through to the + user identity. +4. Give the resolved token only to the dedicated acquisition subprocess through + a temporary helper channel, remove that channel, terminate the child, and + publish only a credential-free verified workspace before setup or the coding + harness starts. +5. Do **not** require or auto-start an AllAgents network credential service for + trusted local execution. The explicit account-pinned provider subprocess is + sufficient. + +### Production remote or multi-tenant workers + +1. Put GitHub App issuer material in a trusted central token-minter component. + Trusted operator configuration, not auth-app discovery, maps the repository + to an installation ID. For every cache-miss acquisition, use focused + [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) with + `refresh: true` to bypass its installation-token cache and mint a fresh token + narrowed to that repository and read-only contents permission. Require + remaining lifetime strictly greater than the acquisition deadline plus + clock-skew margin, expire the delivery lease no later than the token, and + fail readiness when the configured acquisition ceiling can exceed a fresh + token's safe lifetime. +2. Make the gateway/control-plane credential-lease controller authoritative. + The authenticated worker requests only by active attempt and fence. From + durable dispatch and policy state, the controller derives the + effective-profile digest, selected provider, host/API-mapping digest, + installation ID, repository, operation, worker route and identity, lease + epoch, command revision, and expiry. Immediately before issuance it rechecks + active command revision, tombstone, fence, and lease state. +3. Deliver one single-use, non-durable grant/response over the authenticated + acquisition channel. A separate minter must agree with the controller's + configuration digest and consume the grant atomically. Reject replay, + substituted fields or providers, stale command state, and configuration + disagreement. The bearer token itself remains scoped only by GitHub to the + repository, read-only contents permission, and expiry; worker, attempt, + fence, and operation bindings belong to the lease. +4. Advance a GitHub App entitlement generation from authenticated lifecycle + webhooks plus bounded reconciliation whenever an installation is uninstalled, + suspended, or changes repository selection. Unknown or stale installation + state fails cache authorization. Mint only on a cache miss. On a miss, record + the acquiring provider in operator provenance; on a hit, record `cache_hit`, + the cached original acquisition-provider metadata, and current policy + selection/entitlement binding separately. +5. Publish deterministic coarse failures: `source_auth_unavailable` / + `no_eligible_provider` (not retryable); `source_auth_denied` / + `installation_repository_denied` (not retryable); + `source_auth_failed` with `app_configuration_invalid`, + `app_authentication_failed`, or `app_mint_failed` (not retryable), + `provider_rate_limited` or `provider_unavailable` (retryable), or + `trusted_local_cli_failed` (not retryable). Keep provider, installation, and + account identifiers in operator-only provenance. +6. Never forward an operator's general SSH agent or reuse their desktop GCM + store in a remote worker. Those capabilities represent the person, not the + individual execution request. +7. Keep minting logically central even if it initially lives inside the trusted + gateway process. Split the minter into a standalone network service when + remote trust boundaries, private-key isolation, audit, scaling, or + blast-radius controls require it. A versioned central snapshot-delivery + protocol may be designed later, but is not part of the initial architecture. + +The resulting rule is: **local reuse may be subprocess-mediated; production +issuance must be centrally policy-mediated.** A process boundary is required in +both cases, but a standalone credential service is not. From aec59d9b6d1f23e902aa169d8b14643004709dc4 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sat, 19 Sep 2026 12:39:27 +1000 Subject: [PATCH 08/12] docs(architecture): simplify execution gateway deployment --- ...-agent-execution-through-an-a2a-gateway.md | 957 +++---- ...0837-feat-coding-execution-gateway-plan.md | 2482 +++++++---------- .../agent-host-protocol-decision-inputs.md | 31 +- .../harbor-repository-materialization.md | 125 +- .../source-credential-broker-precedents.md | 180 +- 5 files changed, 1515 insertions(+), 2260 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index f9357c13..73afb298 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -2,606 +2,457 @@ - Status: Accepted; implementation pending - Date: 2026-09-17 +- Updated: 2026-09-19 ## Context -AllAgents already owns cross-client agent configuration, workspace knowledge, -plugins, hooks, MCP configuration, and launchers for Codex and other coding -agents. External systems also need to invoke those agents without importing -AllAgents internals or coupling to an interactive CLI process. - -The first planned consumer is AI Evals. Its -[ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) -removes AI Evals-owned coding workspaces in favor of a Promptfoo provider that -needs one remote coding-agent call to return output, usage, traces, file -changes, produced artifacts, failures, cleanup outcomes, and execution -provenance. Future clients may need the same execution boundary without -Promptfoo or evaluation semantics. - -A coding-agent execution is more than a model request. It includes immutable -workspace selection, repository or snapshot acquisition, environment setup, -credentials, permissions, agent invocation, cancellation, evidence capture, -process termination, and cleanup. A workspace may contain multiple repositories -or be produced from a digest-pinned snapshot or organization-specific source. -Those responsibilities need one public contract while allowing materially -different execution backends and acquisition mechanisms. +AllAgents already owns cross-client agent configuration, project workspace +knowledge, global profiles, plugins, hooks, MCP configuration, and generated +launchers. External systems also need to invoke those agents without importing +AllAgents internals or driving an interactive terminal. + +The first planned consumer is AI Evals. It needs one remote coding-agent call to +return terminal output, usage, traces, file changes, produced artifacts, +failures, cleanup outcomes, and execution provenance. Future trusted tools on a +private developer network may need the same execution boundary. + +The initial product is not a public multi-tenant control plane. Developers are +expected to run one gateway for one AllAgents project workspace and expose it on +loopback, a firewalled network, or a Tailscale network. Network reachability is +the trust and authorization boundary. + +A coding-agent execution still includes more than a model request. The gateway +must acquire an immutable workspace, select a configured agent target, contain +credentials to their required phases, propagate cancellation, collect evidence, +terminate descendants, and clean up. Those responsibilities need one public +contract even when the initial deployment remains a single trusted process. The contract must not turn AllAgents into an evaluation harness. Dataset -expansion, repetition, assertions, scoring, experiment scheduling, and durable +expansion, repetitions, assertions, scoring, experiment scheduling, and durable evaluation Runs remain consumer concerns. ## Decision -### Add a separately deployable execution gateway +### Add a trusted-network execution gateway -AllAgents will provide a separately testable and deployable execution-gateway -entry point. It will not be coupled to an interactive CLI command lifecycle. +AllAgents will provide an independently testable `allagents gateway serve` +entry point. It is separate from the interactive CLI command lifecycle but may +run as a single local service process that supervises acquisition and provider +child processes. -The gateway owns: +The gateway implements A2A 1.0 HTTP+JSON plus a required versioned AllAgents +coding-execution extension. It owns: -- authentication and authorization; - stable Task and idempotency identity; +- execution-target selection; +- workspace acquisition; - deadline and cancellation propagation; -- execution-profile and backend selection; -- normalization of terminal output and evidence; -- protocol-level Task status and bounded retention; and +- normalized terminal output and evidence; +- bounded Task and Artifact retention; and - enforcement of the coding-execution contract across every backend. The gateway is not an evaluator, grader, experiment scheduler, retry authority, -or durable evaluation Run ledger. It does not own a consumer's result store. - -### Keep the gateway separate from execution backends - -The initial design supports two execution backends, delivered in this order: - -1. Codex; and -2. Pi. - -Each backend implements the same conformance contract. Provider-specific -process, session, structured-output, cancellation, and evidence behavior -remains behind its adapter. OpenCode and other coding agents remain possible -follow-up adapters rather than part of the first delivery. - -Execution workers own workspace materialization, environment setup, agent -invocation, evidence collection, process termination, and cleanup. The gateway -must not execute evaluated agents, run materializer images, or mount writable -workspaces in the gateway process. - -When deployed on Kubernetes, the gateway runs as its own Deployment and -ClusterIP Service, separate from consumers and execution workers. A backend -dispatches to a worker pool, per-invocation Job, or stronger sandbox according -to the selected execution profile. The protocol does not require one worker -topology. - -A separate gateway Pod is a service and failure boundary, not per-invocation -security isolation. Deployments requiring hostile-code or tenant isolation -must create or select a stronger execution boundary behind the gateway. - -### Make workspace materialization explicit and operator-registered - -The public extension represents one workspace as a closed discriminated union. -The allowed `kind` values and shapes are: - -1. `repositories`, with a bounded list of direct Git repositories, each with a - canonical credential-free HTTPS URL, full commit object ID, collision-free - relative destination, and optional repository-relative subdirectory; -2. `workspaceSnapshot`, with an OCI workspace snapshot referenced by manifest - digest and accompanied by the versioned AllAgents workspace manifest; or -3. `materializer`, with an operator-registered materializer ID, an expected - workspace-manifest digest, and bounded structured inputs. - -Fields from another union variant are invalid. - -The third mode supports organization-specific acquisition such as JFrog, -generated sources, or custom monorepo assembly without accepting executable -configuration from the caller. The request cannot supply a builder image, -Dockerfile, Compose file, shell command, credential, mutable image tag, network -policy, or output contract. - -Direct Git and OCI acquisition revalidate scheme, normalized host, resolved -address, port, and redirect policy for every connection. OCI foreign or -external layer URLs are rejected by default, and registry credentials are never -forwarded across origins. - -Each materializer ID is defined in an operator-owned deployment registry. The -gateway receives only its non-secret descriptor: ID, bounded input schema, -expected definition digest, expected output-manifest version, and required -worker capabilities. The worker receives the runtime definition, which -additionally pins an OCI image by digest and fixes credential handle names or -mount identities, allowed network destinations, resource and phase deadlines, -cache policy, and the OCI runner or sandbox capability. Credential values are -not part of either descriptor. - -The worker computes an algorithm-qualified definition digest over a versioned, -domain-separated canonical serialization of every non-secret, -behavior-affecting runtime field. The expected workspace-manifest digest -likewise identifies the canonical bytes of one declared manifest version. An -execution profile explicitly allows source modes and materializer IDs and -authorizes canonical Git repositories or namespaces, OCI namespaces, and -resource selectors inside structured materializer inputs. At readiness the -gateway matches its expected descriptor digest and profile against the -authenticated worker's computed digest and capabilities. At admission it -validates the selected ID, expected output digest, structured inputs, and -resource authorization; the worker resolves the same definition locally and -rejects missing, changed, or unsupported definitions before acquisition. - -Every source mode produces the same versioned workspace manifest. The manifest -separates worker-verified observations from materializer-attested claims and -records the verification method for each identity. It includes requested and -resolved commits or OCI digests, destinations, materializer identity and image -digest when applicable, normalized input and output digests, resulting tree -identities, and completeness. Materializer assertions are not described as -independently verified unless the worker or a configured trusted acquisition -service performs that verification. - -The worker materializes into a worker-owned staging directory under the same -filesystem publication root as the final workspace; readiness rejects a -cross-filesystem layout and publication never falls back to copy-then-delete. -After validating paths, file types, limits, identities, and the manifest, the -worker stops the materializer and removes its credential, process, mount, and -runner boundary. The validated host-owned staging tree remains. The worker then -atomically renames that tree into its final location before profile-owned setup -or any coding agent starts. - -The registered image is part of the deployment's trusted computing base. The -worker launches it through a configured OCI runner or sandbox in a boundary -separate from the agent runtime and never exposes that runner's control socket -to setup or model tools. Phase isolation prevents later code from receiving the -materializer's credentials or mounts, but it cannot make a malicious -operator-registered image safe from credentials intentionally given to it. -Operators must review and pin that image. A deployment that will not trust it -with credentials needs a separately versioned broker or central snapshot -protocol, which is deferred from the initial architecture. - -The canonical source request enters caller idempotency. The resolved -materializer definition digest enters the effective-profile binding, and both -the definition and output-manifest digests enter terminal provenance. New-claim -source authorization always runs before cache lookup. Cache metadata and keys -include the canonical source, materializer-definition digest, authorization -scope digest and revocation epoch, configured trust domain, and, for GitHub App -sources, the current installation-entitlement generation. Authenticated App -lifecycle webhooks and bounded control-plane reconciliation advance that -generation on uninstall, suspension, or repository-selection change. Unknown -or stale installation state fails cache authorization rather than reusing an -entry. Reuse also requires manifest and content revalidation under the current -authorization scope. Cache metadata preserves the original acquisition -provider metadata; terminal provenance distinguishes `cache_hit`, that original -provider, and the provider selected by current policy instead of claiming that -the current provider performed acquisition. Secret resolution and token minting -remain cache-miss-only. - -This keeps Harbor's useful separation between content-addressed task acquisition -and environment execution without adopting task-owned opaque source. The -comparison is recorded in -[Harbor repository materialization lessons](../research/harbor-repository-materialization.md). - -### Resolve GitHub source credentials from trusted deployment policy - -The public source request remains credential-free and does not select a -credential provider. The trusted acquisition boundary normalizes the repository -host and resolves a provider from operator configuration. `github.com` selects -the built-in GitHub source backend; GitHub Enterprise Server hosts require an -explicit host and API mapping because a custom hostname does not identify its -provider. The effective profile authorizes the canonical repository and provider -entitlement before cache lookup. - -For GitHub repositories, an ordered policy may prefer a GitHub App and permit a -local GitHub CLI fallback. The App provider is applicable only when trusted -operator configuration maps the requested repository to an installation ID; -`@octokit/auth-app` does not discover that mapping. It mints an installation -token scoped only to that repository, read-only contents permission, and its -GitHub expiry. A trusted-local CLI provider is pinned to one configured -non-secret account, which participates in its entitlement and effective-profile -digests. It may run only when no App installation mapping applies, as -`gh auth token --hostname --user `, with `GH_TOKEN`, -`GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` removed -from its environment. Failure to resolve the configured account fails that -provider. This is eligibility fallback, not authentication retry: after an App -provider is selected, configuration, authentication, minting, permission, -repository, rate-limit, or service failure terminates acquisition and never -falls through to the broader user identity. - -When AllAgents owns GitHub App token minting, a trusted control-plane -credential-provider component uses the focused `@octokit/auth-app` package -rather than implementing App JWT, clock-skew, expiry, and installation-token -renewal itself. Every cache-miss acquisition requests a fresh installation token -with auth-app cache bypass (`refresh: true`). Its remaining lifetime must be -strictly greater than the acquisition deadline plus the configured clock-skew -margin, and the delivery lease cannot outlive the token. Readiness rejects an -acquisition-phase ceiling that can exceed a fresh token's safe lifetime. Git -remains the repository transport; the full Octokit client is not required. - -The initial remote architecture is one central token-minter path. The -gateway/control-plane credential-lease controller is authoritative: an -authenticated worker requests credentials only for its active attempt and -fence; the controller rechecks the current command revision, lease epoch, -tombstone, and fence in durable dispatch state, then derives the -effective-profile digest, selected provider, host/API-mapping digest, -installation ID, canonical repository, operation, worker route and identity, -and expiry from durable dispatch and policy state. It issues a single-use, -non-durable grant/response and, when the minter is separate, requires its -configuration digest to agree with those selected bindings. Replay, worker -field substitution, stale command state, and configuration disagreement fail -closed. - -The authenticated delivery lease and channel bind that derived state to the -worker identity, attempt, lease epoch, command revision, fence, operation, and -expiry. Those bindings do not alter the bearer token: after delivery, the token -is enforceably scoped only by GitHub to the repository, read-only contents -permission, and token expiry. The remote worker never receives the App private -key, and only its one-shot acquisition child receives the token. A remote App -profile fails readiness when the central minter, authoritative lease controller, -fresh-token lifetime check, or authenticated non-durable delivery capability is -absent. A versioned central snapshot-delivery protocol is deferred and is not -an initial readiness alternative. - -The local GitHub CLI provider and remote lease path expose their resolved tokens -only to the one-shot acquisition process. Neither exposes credentials to setup, -the coding-agent runtime, model tools, repository configuration, process -arguments, logs, evidence, or the published workspace. Public failures use only -deterministic coarse source-auth code, safe reason, and retryability: -`source_auth_unavailable/no_eligible_provider`, -`source_auth_denied/installation_repository_denied`, and -`source_auth_failed` with `app_configuration_invalid`, -`app_authentication_failed`, `app_mint_failed`, or -`trusted_local_cli_failed` are not retryable; `source_auth_failed` with -`provider_rate_limited` or `provider_unavailable` is retryable. Provider, -installation, and account identifiers are non-secret but operator-only -provenance. Cache-hit provenance separately records `cache_hit`, the original -acquisition provider, and current policy selection. - -A standalone network broker is not required for trusted local execution: the -CLI provider may be a subprocess and a trusted co-located deployment may host -the App minter and lease controller inside its control plane. Remote routes -still use the same authenticated central-minter contract; the minter may be -split into a standalone service when private-key isolation, independent audit, -scaling, or blast-radius requirements demand it. - -The supporting precedents and trust-boundary analysis are recorded in -[Source credential broker precedents](../research/source-credential-broker-precedents.md). +or durable evaluation Run ledger. + +### Trust the network boundary instead of adding application authentication + +The initial gateway has no application-level authentication or per-caller +authorization. It may bind to loopback, a specific interface, or `0.0.0.0`. +Loopback remains the default when no listen address is supplied, but an explicit +`0.0.0.0` binding is valid and requires no unsafe-mode flag. + +Every host able to reach the listener is equally trusted. Any reachable caller +may invoke every exposed target, list and retrieve retained Tasks and Artifacts, +and request cancellation. Task lookup and idempotency are deployment-wide, not +scoped to a caller identity. Operators must use Tailscale ACLs, host firewalls, +container networking, or equivalent network controls when the listener is not +loopback-only. + +TLS termination, OIDC, static bearer tokens, per-tenant ownership, and +multi-tenant information-hiding are deferred. They require a separate decision +when the service is exposed outside one trusted network boundary. + +### Use existing workspace files as the configuration authority + +The initial gateway has no `gateway.yaml` or `worker.yaml`. + +One gateway process serves one project workspace selected by `--workspace` or +the current directory. The project `.allagents/workspace.yaml` remains +canonical for repository identities, remote sources, destination paths, +default revisions, workspace files, plugins, and named OCI snapshot sources. + +The user `~/.allagents/workspace.yaml` remains canonical for global profiles and +launcher-backed execution targets. A launcher-bearing profile client is exposed +only when it explicitly declares: + +```yaml +profiles: + review: + clients: + - name: codex + launcher: codex-review + gateway: + expose: true +``` + +The public target ID is the launcher basename. Launcher names are already +portable and collision-checked across every user profile, while one profile may +contain several clients and therefore several launchers. Internally the target +resolves to exactly one `(profile, client)` pair. The gateway reserves built-in +target IDs, initially `codex` and `pi`; an exposed launcher whose portable +collision key matches a built-in ID is invalid. + +The built-in `codex` and `pi` targets remain available when their adapters are +ready. Explicit launcher-backed targets add configured variants such as +`codex-review` and `pi-tools`. Initially only Codex and Pi profile clients are +gateway-executable; other launcher-bearing clients become eligible only after a +reviewed adapter implements the common execution contract. + +The generated launcher file is a local UX artifact, not the remote execution +boundary. The gateway never discovers launchers from `PATH`, accepts a command, +executable path, arbitrary arguments, or environment overrides from a request, +or appends request data to a generated launcher. It resolves the profile through +its typed adapter and invokes the provider's supported automation surface. + +Process-level options use exact flags and environment variables for: + +- listener and workspace selection; +- a project-specific state-directory override; +- terminal Task retention and bounded Artifact/event storage; +- GitHub App identifiers and private-key file references; +- the configured GitHub CLI account; +- an OCI credential file or fixed credential-helper executable; and +- Codex and Pi auth-file handles. + +By default the state root is a deterministic child of +`~/.allagents/gateway/` keyed by the canonical project-workspace identity. The +store persists and verifies that identity and holds an exclusive lock for the +process lifetime. The root is current-user owned, private, symlink- and hard- +link-resistant, and disjoint from project, profile, and invocation roots. + +Secret values never belong in either workspace file. + +### Support direct repositories and OCI workspace snapshots + +Each request selects exactly one closed workspace source variant: + +1. `repositories`, which materializes the repositories declared by name in the + project workspace and accepts only optional revision overrides; or +2. `workspaceSnapshot`, which selects a named OCI snapshot repository declared + in the project workspace and supplies an immutable OCI manifest digest plus + the expected AllAgents workspace-manifest digest. + +Fields from another variant are invalid. The gateway does not fall back from an +OCI snapshot to Git repositories, or from Git repositories to a snapshot, after +a Task selects its source mode. + +For direct repositories, callers cannot override repository URLs or destination +paths. A revision override is keyed by a declared repository name. Branches and +tags may be accepted for developer convenience, but the gateway resolves and +records the full commit object ID before provider execution. Reproducibility- +sensitive callers should supply full commit IDs. + +For OCI snapshots, the project workspace declares the registry repository: + +```yaml +workspaceSnapshots: + evaluation: + repository: ghcr.io/entityprocess/allagents-workspaces +``` + +The request supplies the name `evaluation`, a `sha256:` OCI manifest digest, and +a `sha256:` workspace-manifest digest. The gateway constructs the full OCI +reference server-side. Callers cannot supply a registry host, repository name, +mutable tag, extraction destination, credential, or external-layer policy. + +Both modes produce the same versioned workspace manifest. It records requested +and resolved repository identities, destinations, acquisition kind, relevant +OCI manifest and layer digests, the workspace-manifest digest, completeness, +and whether each fact was independently verified or snapshot-attested. A commit +listed inside an OCI snapshot is not described as independently verified unless +the gateway separately verifies it against its Git remote. + +Acquisition occurs in a gateway-owned staging directory. The gateway validates +paths, collisions, file types, symlinks, layer and file counts, individual and +total sizes, digests, and the workspace manifest before atomically publishing +the invocation workspace. Absolute paths, traversal, device files, sockets, +escaping links, foreign or external OCI layers, and cross-origin credential +forwarding are rejected. + +### Resolve GitHub credentials with App-first eligibility fallback + +The source request is credential-free and never selects a credential provider. +For `github.com`, the gateway supports two trusted providers: + +1. a configured GitHub App; and +2. a configured GitHub CLI account. + +The App is preferred when it has an installation covering the configured +repository. Installation applicability has three outcomes: `eligible`, +`ineligible`, and `unknown`. The gateway discovers applicability through an +App-authenticated GitHub API client, or verifies an explicitly configured +installation ID against the repository. `@octokit/auth-app` handles App JWT and +installation-token authentication; it is not treated as the repository- +discovery policy by itself. + +For an eligible installation, the gateway requests a fresh repository-scoped +installation token for each acquisition and grants only required read +permissions. Acquisition receives at most 900 seconds or the shorter remaining +Task deadline. The token must remain valid beyond that sub-budget plus a +60-second clock-skew margin. + +GitHub CLI is an eligibility fallback only when the App is not configured or +applicability is positively `ineligible`. An `unknown` result caused by +configuration, authentication, rate-limit, permission, or service failure +terminates acquisition. The CLI provider invokes: + +```text +gh auth token --hostname github.com --user +``` + +with `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and +`GITHUB_ENTERPRISE_TOKEN` removed from its environment. The configured account +is part of the acquisition-policy digest. + +After an App installation is selected, App configuration, authentication, +token minting, permission, repository-coverage, rate-limit, or service failure +terminates acquisition. The gateway never retries the same Task through the +broader GitHub CLI identity. + +Git receives credentials only through an invocation-scoped helper under +hermetic Git configuration. The gateway excludes system, global, and repository +credential helpers, Git Credential Manager, askpass, SSH agents, repository- +controlled secondary fetches, and executable Git configuration. Tokens never +appear in clone URLs, command arguments, Git configuration, logs, Tasks, +Artifacts, retained workspaces, profile setup, MCP processes, agent processes, +or model-invoked tools. The helper and token are destroyed before provider +execution. + +OCI credentials come from a configured auth-file or standard credential helper, +are scoped to snapshot acquisition, and are removed before publication. Public +registries require no credential configuration. + +### Integrate providers through typed adapters + +The initial backend registry contains Codex and Pi, delivered in that order. +Each adapter implements one behavior-focused contract for availability, +capabilities, invocation, progress, deterministic permission handling, abort, +terminal output, optional structured result, usage, native evidence, and +disposal. + +The Codex adapter depends directly on `@openai/codex-sdk`, creates one fresh +thread per Task, passes cancellation and optional output schema through the SDK, +and consumes structured events. + +The Pi adapter uses strict RPC mode with invocation-owned configuration and a +restricted policy extension. Repository extensions and unrestricted built-ins +are not loaded merely because they exist in acquired source. + +CLI-backed compatibility adapters may be added later when a client has a stable +machine protocol. Missing controls are reported honestly as capability gaps. +The gateway never scrapes a TUI or exposes arbitrary installed executables. +OMP is Pi-derived and is added only for demonstrated OMP-specific value beyond +direct Pi. + +Provider preparation is adapter-owned and typed. The gateway never executes +project or user `setup` shell entries as part of acquisition or invocation. +Validated profile settings, plugins, MCP declarations, and deterministic +workspace projections are applied through existing typed transforms. + +Provider control processes, MCP children, and model-invoked tools receive +distinct allowlisted environments and filesystem views. The provider control +process sees only its invocation-private auth channel; each MCP child sees only +its own resolved secrets; shell and other model-invoked tools see neither +provider nor MCP credentials. Every view excludes gateway state, operator home, +App keys, GitHub/OCI stores, acquisition helpers, unrelated adapter auth, and +the parent environment. A target is not ready unless its adapter can enforce +these separations. This credential/state isolation is required even though +general hostile-code sandboxing remains deferred. ### Persist Task truth, not live provider execution -The gateway durably stores Task identity, idempotency claims, terminal status, -Artifact metadata, and retained evidence. A provider execution itself is -ephemeral. The initial service does not checkpoint, reattach, resume, or -automatically replay an interrupted provider session. - -Gateway restart invalidates the active attempt fence and settles each -nonterminal Task failed once. A live worker that loses its lease aborts the -provider and cleans its invocation. If the worker process crashes, an external -supervisor terminates the complete execution boundary and the replacement -worker reaps or quarantines orphaned invocation roots before readiness. -Termination and filesystem cleanup are recorded separately and become complete -only when the responsible boundary proves them; otherwise the terminal record -says unknown. Durable execution and provider-session restoration require a -later decision backed by public provider guarantees. - -### Integrate providers directly - -The Codex adapter depends directly on `@openai/codex-sdk`; AllAgents does not -vendor or depend on Promptfoo's provider. Promptfoo's -[Codex provider](https://github.com/promptfoo/promptfoo/blob/main/src/providers/openai/codex-sdk.ts) -and -[tests](https://github.com/promptfoo/promptfoo/blob/main/test/providers/openai-codex-sdk.test.ts) -are characterization references for strict option mapping, minimal child -environment, working-directory validation, `AbortSignal`, structured output, -event normalization, and cleanup edge cases. - -AllAgents keeps only the gateway-owned subset: one fresh provider session per -Task, server-owned profile settings, bounded native evidence, typed failures, -and worker-proven process cleanup. It does not inherit Promptfoo configuration -layering, caching, pricing, eval retries, thread pools, or `ProviderResponse`. - -The extension defines `allagents.result-schema/v1` as a closed, bounded JSON -Schema Draft 2020-12 subset shared by admission, Codex, Pi, and terminal -validation. It requires an object root, requires every object schema to set -`additionalProperties: false`, lists every declared property in `required`, and -uses `null` unions for optional values. It allows only `type`, `properties`, -`required`, `additionalProperties` with the value `false`, `items`, `enum`, -`const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`, and rejects -remote references, format-dependent validation, and unknown keywords. The -extension version fixes byte, nesting, property, and enum limits. One shared -validator checks both the schema and the returned value, and the accepted schema -digest enters idempotency and provenance. Adapters cannot widen or narrow this -contract. - -Codex receives that schema through the SDK's per-turn `outputSchema`; Pi -implements the same terminal contract with an invocation-scoped terminating -tool. A successful structured request publishes exactly one Artifact named -`allagents.structured-result` with one A2A `Part` whose `data` field contains -the validated result object and whose `mediaType` is `application/json`. -Artifact metadata contains the result-schema version and digest. The Artifact -exists only for a valid result. The integrity -kernel always records `not_requested`, `not_produced`, `valid`, or `invalid`; -an earlier source, setup, provider, cancellation, or deadline outcome remains -the primary Task classification when no result could be produced. - -### Profile A2A 1.0 instead of inventing an invocation API - -The external contract profiles the Linux Foundation -[Agent2Agent protocol](https://a2a-protocol.org/latest/specification/). The -initial profile requires the A2A 1.0 HTTP+JSON binding and retains Agent Card, -Message, Part, Task, Artifact, status, streaming, cancellation, security, and -error semantics. - -The profile narrows A2A for deterministic coding execution: - -- every accepted execution request creates exactly one addressable A2A Task; - direct-Message completion is not supported; -- the gateway implements all mandatory A2A core operations, including - `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; when its Agent Card - advertises streaming, it also implements `SendStreamingMessage` and - `SubscribeToTask`; capability-gated operations retain their standard A2A - behavior instead of being replaced by bespoke `/v1/invocations`, `/v1/runs`, - or `/v1/trials` resources; -- terminal Task results use Artifacts for output and evidence rather than - relying on transient messages or stream events; and -- each versioned Agent Card advertises one mandatory AllAgents extension version - for source and runtime identity, traces, usage and cost, file changes, - produced artifacts, typed failures, cancellation and cleanup outcomes, - evidence completeness, and provenance. - -Generic A2A conformance is insufficient. The AllAgents extension and its -conformance fixtures define the coding-execution guarantees every backend must -satisfy. - -Breaking extension changes use a new extension URI and a versioned Agent Card -or service endpoint. During migration, the gateway keeps the old card, endpoint, -and required extension serviceable while consumers move to the new profile. -Each card requires exactly one extension version. Clients pin the card they -support; the gateway never silently falls back across incompatible versions. -Retiring an old profile is a separate coordinated compatibility decision, not a -lockstep deployment requirement. - -The AAIF -[agentgateway](https://github.com/agentgateway/agentgateway) project may be used -as traffic-policy infrastructure for A2A, MCP, or model calls. It is not the -AllAgents execution service or evidence schema. Documentation uses **AllAgents -execution gateway** where the distinction matters. - -### Keep adjacent protocols at their proper boundaries - -The [Agent Client Protocol](https://agentclientprotocol.com/) may be used behind -a backend adapter when a coding agent supports it. Its session, progress, tool, -permission, terminal, diff, usage, and cancellation semantics are useful -internally, but its stdio editor-to-agent protocol is not the external gateway -API. - -The [Agent Host Protocol](https://microsoft.github.io/agent-host-protocol/) -may be used behind a backend adapter when a host exposes it, or beside the -gateway if AllAgents later adds a collaborative multi-client session surface. -Its host-authoritative snapshots, actions, reconnection, tools, permissions, -and changesets solve live session synchronization; they do not replace A2A -Task identity, idempotency, authorization, terminal evidence, or retention. -The supporting research and implementation consequences are captured in the -[AHP decision inputs](../research/agent-host-protocol-decision-inputs.md). - -[Model Context Protocol](https://modelcontextprotocol.io/) remains a tool and -resource protocol inside an execution backend. It does not represent the whole -coding-agent execution. - -[Agent Format](https://agentformat.org/) may provide an optional static agent -manifest and vocabulary. It does not define the execution transport or prove -observed execution evidence. - -The archived IBM/BeeAI Agent Communication Protocol is superseded by A2A and -will not be adopted. - -### Separate trace propagation, span semantics, and durable evidence - -Gateway calls propagate -[W3C Trace Context](https://www.w3.org/TR/trace-context/) across HTTP and process -boundaries. AllAgents uses OpenTelemetry and OTLP for metadata-only operational -telemetry by default. An explicit allowlist limits structured logs and spans to -non-content operational metadata. Prompts and model outputs, tool arguments and -results, file bodies and source fragments, and secret-bearing attributes are -prohibited before export. A bounded filtering and redaction step must run before -any structured log or span processor so disallowed content cannot enter the -telemetry pipeline. - -AllAgents-managed agent, model, and tool spans use -[OpenInference](https://arize-ai.github.io/openinference/) semantic conventions -only for attributes that pass this allowlist. Backend-native attributes must -pass the same allowlist. Owner correlation is limited to an opaque identifier -appropriate for the telemetry operators' access; it does not expose caller -identity or grant access to a Task or Artifact. Telemetry access and retention -are governed separately from Task and Artifact access and retention. -Consumer-owned evaluator spans may join the propagated trace without becoming -gateway-owned. - -These standards are complementary: - -- W3C Trace Context propagates causal trace identity; -- OpenTelemetry and OTLP represent and transport live operational telemetry; -- OpenInference describes AI operations on OpenTelemetry spans; and -- the AllAgents A2A extension returns durable coding evidence and provenance. - -An external trace backend is not the sole durable result. Sampling, redaction, -transport loss, or retention policy must not erase the terminal facts needed by -a consumer. - -### Trial ATIF only as an optional trajectory Artifact - -The Harbor -[Agent Trajectory Interchange Format](https://github.com/harbor-framework/harbor/blob/main/rfcs/0001-trajectory-format.md) -may be returned as an optional, explicitly versioned A2A Artifact when a backend -can produce or truthfully normalize an ordered agent trajectory. It is not the -A2A transport, the OpenTelemetry trace, or the AllAgents evidence envelope. -Backend-native trajectories remain available when conversion would lose -information. - -An ATIF Artifact must declare its exact schema version and correlate its A2A -Task, OpenTelemetry trace, AllAgents invocation, and backend session identities -through the versioned AllAgents extension. Reasoning content is excluded by -default. Tool arguments, observations, and media follow explicit redaction, -size, and disclosure policy. Truncation or conversion loss is reported rather -than hidden. - -ATIF remains optional until its compatibility policy, specification, tooling, -and non-Harbor conformance mature enough for a required public-contract -capability. - -Harbor's task package, Job configuration, Job/Trial result models, hosted API, -artifact manifest, registry formats, and trial-directory layout will not become -the gateway contract. They remain Harbor-native formats that a future adapter -may preserve. Harbor's ASP `.asp.json` is a draft v0 sandbox proposal and is not -adopted by this decision. - -### Make execution provenance and cleanup explicit - -The gateway and selected worker are collectively responsible for: - -1. validating one canonical immutable workspace request and the selected - profile's exact source-resource or materializer authorization; -2. acquiring direct repositories, restoring a digest-pinned OCI snapshot, or - running the registered materializer in a phase-scoped boundary; -3. producing and validating the standard workspace manifest; -4. transferring the validated staging tree to worker ownership, destroying the - acquisition process/mount/credential boundary, and proving it gone; -5. atomically publishing the host-owned tree on the same filesystem; -6. running profile-owned setup before the evaluated agent action; -7. applying permissions and execution isolation; -8. invoking the agent and propagating cancellation and deadlines; -9. capturing bounded output, usage, cost, file changes, checks, artifact - references, workspace identity, and materializer provenance; -10. returning terminal status, evidence completeness, and provenance; and -11. terminating processes and releasing or retaining resources according to - the documented lifecycle. - -Source transport, materializer image, workspace snapshot, and harness runtime -are independent identities. A backend may use one immutable runtime image plus -a separately digest-addressed workspace artifact; the contract does not require -source code to be baked into the runtime image. - -Credentials remain deployment policy. Requests must not embed deployment -credentials. The gateway authenticates callers, and the selected worker scopes -source credentials to materialization and model credentials to provider -execution without returning secret-bearing paths or values. Materialization -credentials are absent from profile setup, the harness, model-initiated command -environments, tool output, retained evidence, and the published workspace. -Provider and worker-control credentials must likewise be absent from -model-initiated command environments, tool output, retained evidence, and -repository-visible configuration. - -Retries must not multiply non-idempotent agent execution. Every request carries -a caller-scoped stable invocation key through the AllAgents extension. The -gateway binds the authenticated caller, invocation key, effective execution -profile, and request digest to the created Task for a documented retry-retention -window. An identical replay returns the original Task. Reusing the key with a -different request is rejected. Backend retry suppression remains an additional -safeguard; it does not replace gateway deduplication. +The gateway durably stores Task identity, the canonical request, idempotency +claim, selected target and source, effective configuration digest, terminal +status, Artifact metadata, and retained evidence under the configured state +directory. A provider session is not a durable recovery checkpoint. + +An identical idempotency replay returns the existing Task. Reusing the key with +a different canonical request conflicts. Because the initial service has no +caller identity, the idempotency namespace and Task visibility are gateway-wide. + +Terminal Task records, Artifacts, events, and invocation claims expire +atomically after the configured TTL. The retained-count limit never evicts an +unexpired Task; the gateway rejects new admission until expiry frees capacity. +State-store integrity or durability failure stops admission and prevents the +gateway from acknowledging creation or reporting terminal success. + +On gateway restart, interrupted nonterminal Tasks settle failed; provider work +is not resumed or automatically replayed. A new invocation may start fresh. + +### Make cancellation, evidence, and cleanup explicit + +The gateway supervises every acquisition and provider process set. Cancellation +first invokes the provider's native abort or protocol cancellation, then applies +bounded forced termination to the complete descendant set. + +Terminal cleanup evidence is recorded only after the supervisor proves the +complete invocation process set quiescent through an enforceable, invocation- +owned containment primitive. If the platform cannot provide that guarantee, the +gateway fails readiness rather than relying on best-effort process enumeration. +If termination or proof fails, the Task records termination as unknown or +failed and the gateway rejects new work. An unmanaged foreground gateway stays +alive with poisoned readiness and continues reaping while printing the stable +containment identifier and platform recovery command. It may exit with a +nonempty set only after a validated external manager accepts cleanup ownership. + +On startup the gateway identifies every interrupted invocation's containment +set and proves it empty before binding or advertising readiness. It may +quarantine a stale filesystem root only after process quiescence is proven. A +reaping or proof failure terminalizes the Task with unknown/failed termination, +keeps readiness false, and enters the same managed or unmanaged recovery path. + +Terminal evidence distinguishes: + +- agent output; +- optional validated structured result; +- requested and resolved repository or OCI identities; +- pre- and post-execution Git state where applicable; +- produced artifacts; +- usage and bounded provider-native evidence; +- cancellation and termination outcomes; and +- workspace cleanup outcome. + +Credentials, raw secret-bearing paths, and unrestricted prompt, output, tool, +source, or file contents are excluded from operational logs. + +### Profile A2A instead of inventing an invocation API + +The gateway uses A2A Agent Cards, Messages, Tasks, Artifacts, operations, errors, +streaming, and cancellation. The Agent Card declares the AllAgents coding- +execution extension as required. Every operation that creates, returns, lists, +subscribes to, or mutates profiled Tasks or Artifacts activates +`https://allagents.dev/a2a/extensions/coding-execution/v1` through the +`A2A-Extensions` header. Unsupported calls receive the standard A2A extension- +support error, and responses echo the activated URI. + +The versioned extension carries the invocation key, execution target, closed +workspace source, bounded deadline, and optional bounded result schema in its +own strict `Message.metadata` member without rejecting unrelated A2A metadata. +Every terminal Task has one fixed-name, versioned integrity Artifact plus zero +or more produced Artifacts. Breaking extension versions receive versioned cards +and endpoints rather than silent fallback. + +The Agent Card advertises built-in and explicitly exposed launcher-backed +targets through an allowlisted capability projection. It does not publish local +paths, commands, arguments, environment selectors, credentials, exact source +authorization details, or transient worker state. + +ACP, app-server, SDK, and RPC protocols remain backend implementation details. +W3C Trace Context may propagate correlation through HTTP and child-process +boundaries. OpenTelemetry and provider-native evidence remain optional, +separate layers; neither replaces durable Task evidence. ### Keep evaluation commands out of scope This decision does not add `allagents eval`, benchmark authoring, assertions, -scoring, datasets, or experiment scheduling. A community evaluation wrapper and -an enterprise AI Evals wrapper may share this execution service in the future, -but their product and ownership model requires a separate decision. +scoring, datasets, repetitions, experiment scheduling, or automatic execution +retry. Consumers own those concerns. ## Consequences -- AllAgents becomes a service boundary in addition to a local CLI, but retains a - narrow coding-execution responsibility. -- Consumers depend on A2A 1.0 plus a versioned AllAgents extension, not - AllAgents TypeScript modules, CLI behavior, or workspace internals. -- Codex and Pi are the initial execution backends behind one conformance suite; - Codex lands first and OpenCode is deferred. -- Durable Task and evidence records do not imply durable provider execution; - interrupted attempts fail rather than resume or replay. -- The result-schema subset, structured-result Artifact, and non-success result - states are public compatibility surface rather than adapter conventions. -- Reliable worker-crash cleanup requires an external execution supervisor and a - pre-readiness orphan-root reaper in addition to leases. -- Gateway and execution workers scale and fail independently. -- The gateway can remain lightweight; physical isolation and resource policy - belong to the selected execution backend. -- Custom acquisition remains available without making caller-supplied code part - of the trust boundary: operators register digest-pinned materializers and - profiles decide which callers may select them. -- Direct Git, OCI snapshots, and registered materializers converge on one - validated workspace manifest and provenance contract. -- GitHub source credentials are selected by trusted host/profile policy rather - than caller input. GitHub App is preferred when applicable; GitHub CLI is a - local-only eligibility fallback and never masks an App authentication or - authorization failure. -- Deployments that enable external materializers must operate their image, - schema, credential, network, resource, and cache policies as worker - configuration. -- A2A supplies discovery and lifecycle semantics. AllAgents supplies the - coding-specific evidence contract. -- W3C Trace Context, OpenTelemetry/OTLP, OpenInference, optional ATIF, and the - terminal evidence extension remain distinct layers rather than competing - universal formats. -- Implementations must preserve bounded native evidence whenever normalization - would lose information. +- Developers can start one endpoint with `allagents gateway serve` and use + loopback, `0.0.0.0`, a specific interface, Tailscale, or firewall policy. +- There is no application authentication, per-caller authorization, tenant + isolation, `gateway.yaml`, `worker.yaml`, remote worker protocol, or required + Kubernetes deployment in the initial product. +- Project and user workspace files remain the sole declaration authority for + source identities and exposed profile launchers. +- Network reachability grants access to every exposed target and retained Task. + Operators must treat network policy as the authorization boundary. +- GitHub App credentials support private repositories without forcing every + developer to use one identity; GitHub CLI remains a local eligibility + fallback when no App installation applies. +- Direct repositories and digest-pinned OCI snapshots converge on one validated + workspace manifest and evidence contract. +- The gateway process remains a meaningful API and lifecycle boundary, but not + a hostile-code sandbox. Strong multi-tenant isolation remains future work. +- Codex and Pi share one conformance suite while retaining bounded native + evidence and honest capability differences. +- A future deployment configuration becomes justified only when the product + needs multiple worker routes, tenants, credential policies, custom + materializers, centralized storage, or other operator-selected variants. ## Rejected alternatives -### Invent a bespoke invocation, run, or trial API +### Define a second profile registry in `gateway.yaml` -Rejected because A2A already defines remote-agent discovery, Task lifecycle, -streaming, artifacts, cancellation, errors, and web security. Coding-specific -evidence belongs in a versioned A2A extension rather than a parallel transport. +Rejected because global profiles and launcher identities already belong to +`~/.allagents/workspace.yaml`. A second profile map would drift in client, +model, plugin, MCP, and launcher configuration. -### Run agents in the gateway Pod +### Require application authentication for every deployment -Rejected because it couples control-plane availability and credentials to -mutable repository execution, prevents independent scaling, and mistakes a -service boundary for per-invocation isolation. +Rejected for the initial trusted-network product. It would add caller identity, +tenant scoping, token lifecycle, and ingress configuration before the expected +users need those boundaries. Tailscale ACLs and firewalls are the initial access +control. -### Use OpenInference instead of W3C Trace Context +### Restrict the listener to loopback -Rejected as a category error. W3C Trace Context propagates trace identity; -OpenInference supplies AI semantic conventions on OpenTelemetry spans. The -gateway uses both. +Rejected because developers need to expose the endpoint through Tailscale, +containers, VMs, and private networks. Explicit `0.0.0.0` binding is supported; +the operator owns the surrounding network policy. -### Use ATIF as the complete gateway result +### Execute generated launcher files as the remote protocol -Rejected because ATIF represents an ordered agent trajectory, not remote Task -lifecycle, repository provenance, workspace changes, produced artifacts, -cleanup, authorization, or evidence completeness. +Rejected because local launchers intentionally preserve cwd and append local +caller arguments. Remote requests must resolve a typed profile adapter and can +never control commands or argv. -### Adopt Harbor's Job or Trial API +### Let callers provide repository URLs or OCI repositories -Rejected because Harbor's formats own benchmark orchestration, verification, -and persisted runner state. The AllAgents gateway executes one coding-agent -request and does not become an evaluation harness. +Rejected because workspace configuration already defines trusted source +identities and destinations. Requests may select declared names and immutable +revisions or digests, not introduce new origins. -### Let callers provide repository-acquisition code +### Fall back from a selected GitHub App after runtime failure -Rejected because a caller-selected image, Dockerfile, Compose file, or shell -script would turn request parsing into privileged code execution and would make -credential, network, provenance, and cache policy unreviewable. Callers may -select only source modes and materializer IDs explicitly registered and allowed -by the effective execution profile. +Rejected because it would silently change identity and authorization scope after +selection. GitHub CLI fallback applies only when the App is ineligible. -### Replace A2A with the Agent Host Protocol +### Use mutable OCI tags -Rejected because AHP explicitly targets synchronization of independent clients -around host-owned sessions, not agent-to-agent Task execution. Its reconnect -and changeset models do not supply caller-scoped idempotency, immutable source -handling, cleanup, complete terminal evidence, or bounded Task retention. +Rejected because the same request could produce different workspaces. Snapshot +selection requires an OCI manifest digest and expected workspace-manifest +digest. -### Vendor Promptfoo's Codex provider +### Treat provider sessions as durable execution -Rejected because that provider includes Promptfoo-specific configuration -layering, caching, pricing, tracing, retry metadata, thread pooling, and result -mapping. AllAgents needs a smaller worker adapter against the Codex SDK and can -reuse Promptfoo's observable behavior as characterization evidence without -copying its implementation. +Rejected because a resumable provider thread does not prove workspace, +process, cancellation, evidence, or cleanup continuity across gateway restart. -### Treat provider session persistence as durable execution +### Invent a bespoke invocation API -Rejected because a resumable provider thread does not prove workspace, -process, cancellation, evidence, or cleanup continuity across gateway or worker -failure. The initial service durably records failure and cleanup truth but does -not resume interrupted work. +Rejected because A2A already supplies discovery, Task lifecycle, streaming, +Artifacts, cancellation, and errors. Coding-specific evidence belongs in a +versioned extension. + +### Adopt an evaluator's Job or Trial API + +Rejected because benchmark orchestration, verification, and persisted evaluation +state remain consumer concerns. The gateway executes one coding-agent Task. ## Reconsider when -Revisit this decision if A2A standardizes the required coding-execution evidence -without an extension, if a stable cross-vendor execution protocol subsumes the -same lifecycle and provenance guarantees, or if operational evidence shows that -the gateway and backend boundary prevents required cancellation, isolation, or -result integrity. +Revisit this decision when any of these become requirements: + +- callers outside one trusted network must share the endpoint; +- per-caller Task privacy, authorization, or audit identity is required; +- multiple gateway replicas need transactional shared storage; +- execution must route among remote worker pools or hostile-code sandboxes; +- custom materializers are needed beyond direct Git and OCI snapshots; +- multiple GitHub hosts, Apps, CLI accounts, or ordered credential policies need + declarative configuration; +- A2A standardizes the required coding-execution evidence without an extension; + or +- a stable cross-vendor automation protocol subsumes the backend adapter seam. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 5f95d109..4793ba0c 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,7 +1,7 @@ --- title: "Coding-Agent Execution Gateway - Plan" date: 2026-09-18 -deepened: 2026-09-18 +updated: 2026-09-19 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready @@ -13,20 +13,29 @@ execution: code ## Goal Capsule -- **Objective:** External systems can run Codex or Pi against an immutable, - provenance-bearing workspace assembled from exact Git repositories, a - digest-pinned OCI snapshot, or an operator-registered materializer through - one authenticated, cancellable, evidence-preserving remote contract. -- **Means:** Add a separately deployable A2A 1.0 gateway, a private worker protocol, and backend-neutral workers with two direct provider adapters (KTD1, KTD5, KTD7-KTD8). -- **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public boundary. The A2A 1.0 specification owns core wire semantics. The versioned AllAgents extension owns coding-execution semantics. -- **Execution profile:** Build contract-first, then durable Task/evidence state, worker lifecycle, Codex, Pi, packaging, and cross-backend conformance. Preserve the existing local CLI and Node 18 package compatibility. -- **Stop conditions:** Do not execute agents or materializers in the gateway - process, accept mutable source identity, accept caller-supplied acquisition - code or credentials, put deployment credentials in requests, treat streams - or telemetry as terminal evidence, treat provider sessions as recovery - checkpoints, vendor an evaluator's provider implementation, or add - evaluation behavior. -- **Tail ownership:** The implementing workflow runs focused contract and lifecycle tests, the complete repository quality gates, isolated gateway/worker smoke tests, provider-specific credentialed smoke tests where credentials are available, and documentation validation. +- **Objective:** A developer can run one trusted-network A2A endpoint for one + AllAgents workspace and invoke built-in or explicitly exposed profile targets + against either declared Git repositories or a digest-pinned OCI workspace + snapshot. +- **Means:** Add `allagents gateway serve`, a private execution-service package, + a bounded durable Task store, direct Codex and Pi adapters, GitHub App and + GitHub CLI acquisition providers, OCI snapshot acquisition, and one supervised + invocation lifecycle. +- **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) + owns the public and trust boundaries. Project and user `workspace.yaml` files + own source and profile declarations. A2A 1.0 owns core wire semantics. +- **Execution order:** Capture a red built-CLI E2E for the missing gateway; + freeze schemas and configuration projection; implement the Task store, A2A + server, acquisition, supervisor, Codex, and Pi; run a final implementation + review and fix important findings; then run the green built-CLI E2E, + repository gates, and documentation validation. +- **Stop conditions:** Do not add application authentication, `gateway.yaml`, + `worker.yaml`, remote worker routing, caller-supplied URLs or commands, + mutable OCI tags, selected-provider failure fallback, evaluation behavior, or + automatic execution retry. +- **Tail ownership:** The implementing workflow runs focused contract and + lifecycle tests, provider fixture tests, the repository quality gates, a + built-CLI trusted-network smoke test, and documentation validation. --- @@ -34,377 +43,519 @@ execution: code ### Summary -AllAgents gains a remote coding-execution service without becoming an evaluation framework. Callers use A2A Tasks and one required AllAgents extension. The gateway owns caller identity, idempotency, routing, status, cancellation, evidence normalization, and bounded retention. Separate workers own repository materialization, provider processes, mutable workspaces, evidence capture, termination, and cleanup. +AllAgents gains a single-workspace coding-execution service without becoming an +evaluation framework or multi-tenant platform. Callers use A2A Tasks and one +required AllAgents extension. Network reachability is authorization. The +service resolves configured targets and sources from existing workspace files, +acquires a fresh invocation workspace, invokes Codex or Pi through a typed +adapter, and retains bounded terminal evidence. ### Problem Frame -AllAgents currently configures and launches coding clients but has no service boundary for external callers. AI Evals and future consumers would otherwise need to import AllAgents internals, drive interactive CLIs, or independently reimplement repository acquisition, permissions, cancellation, evidence, and cleanup. +AllAgents configures and launches coding clients but has no service boundary for +trusted tools such as AI Evals. Those tools would otherwise import AllAgents +internals, drive interactive CLIs, or duplicate profile resolution, repository +acquisition, credential handling, cancellation, evidence capture, and cleanup. -The two initial runtimes expose different programmatic contracts. Codex provides a TypeScript SDK over structured JSONL events and native per-turn `outputSchema`; Pi provides a strict JSONL RPC mode and invocation-scoped custom tools. The public service must preserve one stable lifecycle and structured-result contract without flattening provider-specific facts into false equivalence. +Developers expect a process they can start in a workspace and expose on +loopback, `0.0.0.0`, a private interface, or Tailscale. They do not need an +application authentication stack, Kubernetes control plane, remote worker +registry, or another profile configuration file for the initial use case. ### Actors -- A1. **Gateway caller:** An authenticated service such as AI Evals that creates, observes, lists, cancels, and retrieves coding-execution Tasks. -- A2. **Execution gateway:** The A2A server that owns caller scope, Task identity, idempotency, routing, retention, and normalized results. -- A3. **Execution worker:** A separately deployed process that owns registered - workspace materialization, one mutable workspace per invocation, provider - execution, evidence capture, and cleanup. -- A4. **Backend adapter:** The Codex or Pi integration that translates native - events, structured results, cancellation, usage, failures, and evidence into - the worker contract. -- A5. **Operator:** The person or deployment system that defines profiles, - materializer registrations, credentials, limits, retention, worker - endpoints, and observability policy. +- A1. **Trusted-network caller:** Any process able to reach the endpoint. All + callers have the same authority and Task visibility. +- A2. **Execution gateway:** The A2A server and invocation supervisor. It owns + deployment-wide Task identity, acquisition, routing, status, cancellation, + evidence, retention, and cleanup. +- A3. **Backend adapter:** The Codex or Pi implementation translating native + automation events and cancellation into the common contract. +- A4. **Operator/developer:** The person who selects the project workspace, + exposes profile launchers, supplies process flags and credential handles, and + controls network access. +- A5. **GitHub/OCI source:** The remote content service used only during the + acquisition phase. ### Key Decisions -- **Profile A2A rather than creating a public invocation API.** The service keeps standard Agent Cards, Tasks, Artifacts, operations, errors, and capability negotiation. Governs R1-R4. -- **Keep execution outside the gateway process.** Mutable repositories and provider processes belong to workers. Governs R10-R16, R21-R22. -- **Persist Task truth, not live executions.** Accepted Task identity and terminal evidence survive restart; provider sessions do not resume or replay. Governs R7, R14, R16-R18. -- **Keep evaluation outside AllAgents.** Dataset expansion, repetitions, assertions, scoring, retries, and durable evaluation Runs remain caller concerns. Governs R20. -- **Make workspace acquisition explicit but extensible.** Requests select one - versioned workspace source mode; custom acquisition uses only - operator-registered, digest-pinned materializers allowed by the profile. - Governs R6, R11-R13, R16-R18, R21-R22. -- **Resolve source credentials from trusted host and profile policy.** Callers - never select a credential provider. For GitHub, prefer an applicable GitHub - App installation and permit GitHub CLI only as an explicit trusted-local - fallback when no installation applies; never fall back after a selected App - provider fails. When AllAgents owns App token minting, use - `@octokit/auth-app` rather than a custom minter. (session-settled: - user-directed.) Governs R6, R12-R13, R16, R21-R22. +- **Use A2A rather than inventing an invocation API.** Standard Agent Cards, + Tasks, Artifacts, errors, streaming, and cancellation remain the public + lifecycle. Governs R1-R3. +- **Treat the network as the trust boundary.** The initial service has no + application authentication or caller ownership. Explicit `0.0.0.0` binding is + valid. (session-settled: user-directed.) Governs R4-R5. +- **Reuse workspace configuration.** Project `workspace.yaml` owns sources; + user `workspace.yaml` owns profiles, launchers, and exposure. There is no + `gateway.yaml`. (session-settled: user-directed.) Governs R6-R8, R18. +- **Support two acquisition modes.** Direct declared repositories and named, + digest-pinned OCI workspace snapshots converge on one manifest and evidence + contract. (session-settled: user-directed.) Governs R9-R11. +- **Use App-first GitHub credential eligibility.** Prefer an applicable GitHub + App; use a configured `gh` account only when no App installation applies; + never fall back after selected-App failure. (session-settled: user-directed.) + Governs R10-R11. +- **Keep a typed backend seam.** Codex SDK and Pi RPC are the complete initial + backend set. Launcher-backed profiles resolve through those adapters rather + than executing generated wrapper files. Governs R7-R8, R12-R15. +- **Persist Task truth, not provider sessions.** Restart settles interrupted + work failed; it never resumes or automatically replays provider execution. + Governs R5, R13-R16. +- **Keep evaluation outside AllAgents.** Consumers own datasets, repetitions, + scoring, assertions, and evaluation Runs. Governs R17. ### Requirements -**Public protocol and compatibility** - -- R1. The gateway implements A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, and `CancelTask`; it implements streaming send and task subscription when the card advertises streaming. -- R2. Every valid new request returns exactly one addressable Task. Direct-Message completion and follow-up messages to an existing Task are unsupported. Non-streaming send honors A2A `returnImmediately`; streaming always emits the durable Task first. -- R3. The public extension URI is `https://allagents.dev/a2a/extensions/coding-execution/v1`. The Agent Card advertises it as required; HTTP clients opt in with `A2A-Extensions`; each request sets `Message.extensions` to include the URI and puts the schema-defined request only at `Message.metadata[uri]`. Every terminal Task contains exactly one fixed-name `allagents.execution-integrity` Artifact whose `extensions` includes the URI and whose single `Part` contains the schema-defined integrity envelope in `data` with `mediaType: application/json`. The Task uses only the standard A2A fields and never adds `Task.extensions`. Unsupported or missing required extension versions fail without fallback. -- R4. Terminal output and execution evidence are retrievable as Task Artifacts for the configured retention window even when the original stream disconnects. Active subscription emits the current Task snapshot then future events without promising replay of missed progress; terminal subscription returns the standard unsupported-operation error and callers use `GetTask`. - -**Caller identity, Task identity, and retention** - -- R5. Every protocol operation authenticates the caller and scopes Task lookup, listing, subscription, cancellation, and artifact retrieval to that caller's tenant and principal before storage access can reveal resource existence. Production public ingress reaches the gateway through TLS terminated at the configured named trusted boundary; an unauthenticated loopback-only development listener is the sole plaintext exception. Remote gateway-worker links use mTLS or an explicitly configured equivalent authenticated encrypted overlay, while a same-host Unix socket is acceptable. The authenticated worker identity is bound to its route, capabilities, and attempt fence, and readiness fails for plaintext or identity-mismatched remote endpoints. -- R6. After authentication, required-extension checks, and bounded canonical parsing, the gateway first resolves the owner-scoped invocation claim. A retained claim compares the canonical caller request and result-schema digest against the originals and returns its existing Task only while its stored original effective-profile and schema bindings remain intact; a mismatch conflicts without dispatch. Current source/profile authorization, profile resolution/readiness, quota, and deadline checks apply only when atomically creating a new claim that binds the authenticated owner, canonical caller request digest, original effective-profile digest, original result-schema digest, and submitted Task. -- R7. Public Task state uses only A2A states and each Task has one immutable terminal transition. Acceptance of the current worker fence moves a submitted Task to working before source materialization or setup, so a subsequent source/setup failure transitions from working to failed. Task state and terminal Artifact metadata survive gateway restart. Every nonterminal Task present at startup settles failed once, its old attempt fence is invalidated, and stale worker events cannot overwrite it; the initial service never resumes or automatically replays interrupted provider work. -- R8. List operations implement all A2A filters, history bounds, page-size bounds, owner/query-bound cursor pagination, and descending status-update time. One immutable expiry logically hides the Task, claim, events, and artifacts before best-effort physical deletion; expired and unauthorized IDs are indistinguishable. -- R9. Small deployments work without an external database. The built-in durable store supports one gateway replica, enforces per-owner/global admission and storage quotas, and reserves capacity for cancellation and terminal settlement; multi-replica storage is outside this delivery. - -**Execution and policy** - -- R10. Codex and Pi are the complete initial backend set behind one conformance contract, delivered Codex first and Pi second. OpenCode is deferred. (session-settled: user-directed.) -- R11. A request selects a server-defined execution profile and may include one `allagents.result-schema/v1` schema for the terminal result: a bounded JSON Schema Draft 2020-12 subset with an object root, every object schema setting `additionalProperties: false`, every declared property listed in `required`, optional values represented by `null` unions, and only `type`, `properties`, `required`, `additionalProperties` with the value `false`, `items`, `enum`, `const`, `anyOf`, `$defs`, local `$ref`, `title`, and `description`. The extension version fixes byte, depth, property, and enum limits; admission rejects remote references, format-dependent validation, and unknown keywords; one shared validator governs schema admission and returned values. The profile fixes backend, model/runtime settings, source policy, setup and check commands, permissions, environment allowlists, artifact paths, resource budgets, deadline ceiling, trust class, and evidence limits. Requests cannot supply raw provider configuration. -- R12. One request defines one workspace using exactly one closed source union: - `{ kind: "repositories", repositories: [...] }`, - `{ kind: "workspaceSnapshot", reference, workspaceManifestDigest }`, or - `{ kind: "materializer", materializerId, - expectedWorkspaceManifestDigest, inputs }`. Unknown kinds, fields from - another variant, and omitted variant fields fail admission. Repository - entries contain a canonical credential-free HTTPS Git URL, full commit object - ID, collision-free relative destination, and optional repository-relative - subdirectory. Snapshot references are digest-pinned OCI artifacts containing - the versioned workspace manifest. Materializer inputs are bounded by the - registered schema. Profiles explicitly allow source modes and materializer - IDs and authorize exact canonical Git repositories or namespaces, OCI - namespaces, and resource selectors inside materializer inputs. Callers cannot - supply builder images, Dockerfiles, Compose files, shell commands, - credentials, mutable image tags, network policy, or output contracts. Direct - Git revalidates destination policy for every connection, disables redirects - and repository-controlled secondary fetch/exec features, uses hermetic Git - configuration, fetches into an isolated object database from the approved - remote, and verifies that the checked-out commit equals the requested full - object ID. OCI acquisition rejects external or foreign layer URLs by default, - revalidates scheme, normalized host, resolved address, port, and redirects - for registry, authentication, manifest, and blob connections, never forwards - credentials across origins, and verifies every manifest and layer digest. A - materializer output must match the request's expected workspace-manifest - digest before publication. -- R13. Requests never contain deployment credentials or arbitrary secret values. Profiles name environment variables whose values are scoped to the required worker phase and excluded from repository configuration, process arguments, logs, errors, evidence, retained workspaces, structured logs/spans before processing or export, and every model-initiated command or tool environment. Credentialed profiles additionally require an OS-enforced provider/tool credential boundary: the credential-bearing provider runtime and model-invoked tools use distinct UID/process/mount policy that prevents tool access to provider processes, procfs entries, and backend config/data roots, or an equivalent credential broker keeps reusable credentials out of the agent runtime. Worker readiness fails when the declared boundary cannot be proved; environment filtering alone is not credential isolation. - Source credential selection is server-side deployment policy, not caller - input. The acquisition boundary maps normalized repository hosts to source - backends; `github.com` selects the built-in GitHub backend, while GitHub - Enterprise Server hosts require explicit operator host/API mappings. Profiles - name an ordered provider policy and authorize its non-secret entitlement - before cache lookup. An App provider is applicable only when trusted operator - configuration maps the repository to an installation ID; auth-app does not - discover installations. Authenticated lifecycle webhooks plus bounded - reconciliation advance an installation-entitlement generation on uninstall, - suspension, or repository-selection change. Unknown or stale installation - state fails cache authorization. Cache metadata preserves the original - acquisition-provider metadata, while hit provenance separately records - `cache_hit`, that original provider, and current policy selection/entitlement. - Secret lookup and token minting remain cache-miss-only. - For a cache-miss GitHub acquisition, an applicable configured App installation - is preferred. Focused `@octokit/auth-app` minting uses `refresh: true` to - produce a fresh token scoped only to the authorized repository, read-only - contents permission, and GitHub expiry. Remaining lifetime must be strictly - greater than the acquisition deadline plus clock-skew margin, and the - credential lease cannot outlive the token. Readiness rejects an acquisition - ceiling that can exceed a fresh token's safe lifetime. A trusted-local - `github-cli` provider may run only when no App installation mapping applies; - it is pinned to a configured non-secret account included in entitlement and - effective-profile digests, invokes - `gh auth token --hostname --user ` without `GH_TOKEN`, - `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, or `GITHUB_ENTERPRISE_TOKEN`, and - fails if that account cannot be resolved. After App selection, no failure - falls through to `gh`. - The initial remote path requires a trusted central token minter and - authoritative gateway/control-plane credential-lease controller. The - authenticated worker requests only by active attempt and fence. From durable - dispatch and policy state, the controller rechecks active command revision, - lease epoch, tombstone, and fence, then derives the effective-profile digest, - selected provider, host/API-mapping digest, installation ID, canonical - repository, operation, worker route and identity, and expiry. It issues and - atomically consumes a single-use non-durable grant/response; a separately - deployed minter must agree with the selected configuration digest. Replay, - substitution, stale state, and controller/minter digest disagreement fail - closed. The authenticated lease/channel binds the derived repository and - provider state to worker identity, attempt, lease epoch, command revision, - fence, operation, and expiry; those are not token claims. The remote worker - never receives the App private key. Readiness fails without this complete - path. Versioned central snapshot delivery is deferred and is not an initial - readiness alternative. - Public failures expose only deterministic coarse code, safe reason, and - retryability; provider, installation, and account identifiers remain - operator-only. `source_auth_unavailable/no_eligible_provider`, - `source_auth_denied/installation_repository_denied`, - `source_auth_failed/app_configuration_invalid`, - `source_auth_failed/app_authentication_failed`, - `source_auth_failed/app_mint_failed`, and - `source_auth_failed/trusted_local_cli_failed` are not retryable. - `source_auth_failed/provider_rate_limited` and - `source_auth_failed/provider_unavailable` are retryable. - Materializer IDs are defined in an operator-owned deployment registry. The - gateway holds only the non-secret ID, bounded input schema, expected - definition digest, expected output-manifest version, and required worker - capabilities; the worker holds the runtime definition with the digest-pinned - image, credential handle names or mount identities, network destinations, - resource/deadline ceilings, cache policy, output-manifest version, and OCI - runner or sandbox capability. Credential values are excluded. The worker - derives an algorithm-qualified `sha256:<64 lowercase hex>` definition digest - from a versioned, domain-separated canonical serialization of every - non-secret behavior-affecting field and advertises it at readiness; the - gateway treats its copy only as the expected digest. The expected workspace - manifest and canonical materializer-input digests use the same - algorithm-qualified format with distinct domain separators and exact - versioned canonical JSON preimages. Readiness fails when computed and expected - descriptors differ across the authenticated route. Materialization - credentials exist only in that isolated phase and are not supplied to setup, - provider execution, or model tools. The registered image is operator-trusted - deployment code: phase isolation protects later phases but cannot make a - malicious registered image safe from credentials deliberately given to it. - Deployments requiring that stronger claim are deferred pending a separately - versioned broker or central snapshot-delivery protocol. -- R14. The effective deadline is the earlier of the caller deadline and profile ceiling and is persisted before dispatch. The first durable terminal-or-cancel-intent write wins; cancellation is idempotent, reaches the worker and provider once, suppresses late success, and records termination and cleanup before publishing canceled. Stream or HTTP disconnect alone does not cancel a Task. -- R15. Initial profiles are unattended. Known provider permission requests are deterministically approved or denied by profile policy for one invocation; unknown permission types fail as adapter incompatibility. The gateway never emits `INPUT_REQUIRED` or `AUTH_REQUIRED` for these profiles and never depends on a live client. -- R16. A worker creates a fresh invocation directory, fresh provider session, and isolated backend configuration/data roots, runs setup, captures a post-setup baseline, invokes the provider, validates any requested structured result, and runs configured checks. It then proves the complete invocation process set quiescent before final evidence/artifact capture and cleanup or explicit retention. No workspace or provider session is reused after interruption. If bounded termination escalation cannot prove quiescence, the worker persists termination as unknown/failed, poisons admission, and exits so the external supervisor destroys the complete process boundary; replacement readiness performs orphan recovery before accepting work. The same supervisor boundary handles a worker crash. - Every source mode materializes into a worker-owned staging directory under - the same filesystem publication root as the final workspace and produces the - same versioned workspace manifest. Readiness rejects cross-filesystem roots - and publication has no copy-then-delete fallback. The worker validates - repository or snapshot identities, destinations, paths, file types, limits, - materializer definition and image digests, output digest, provenance method, - and completeness. It then terminates the supervisor-owned acquisition - process, mount, runner, and credential boundary while preserving the - host-owned validated staging tree, proves that boundary gone, atomically - renames the tree into its final location, and only then starts setup. - -**Evidence and observability** - -- R17. Every terminal Task contains the required `allagents.execution-integrity` Artifact carrying an integrity kernel: Task/source/profile/backend identities, action outcome, a structured-result state of `not_requested`, `not_produced`, `valid`, or `invalid` plus reason and schema digest when requested, cancellation or failure classification, separate termination and filesystem-cleanup outcomes including explicit unknown, Artifact index metadata, per-dimension completeness, and provenance. A valid structured result is exactly one additional `allagents.structured-result` Artifact with one A2A `Part` whose `data` field contains the validated result object and whose `mediaType` is `application/json`; missing or invalid result data never publishes that Artifact. `not_produced` is legal only before a result candidate is produced. Once validation selects `valid` or `invalid`, later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the fixed structured-result Artifact while the later phase remains the primary Task failure classification. Missing or invalid integrity data fails the Task; predictable bounded omission of optional evidence may complete with an explicit gap. - Workspace provenance includes the source mode, requested and resolved - repository commits or OCI digests, destination map, workspace-manifest - digest, and, when applicable, materializer ID, computed definition digest, - image digest, canonical input digest, and output digest. It labels each field - as a worker-verified observation, trusted-service verification, or - materializer-attested claim and records the verification method; a custom - image's assertion is never reported as independently verified merely because - its output digest matched. -- R18. Normalized file evidence distinguishes create, edit, delete, and rename where truthful. It preserves bounded provider-native diffs, events, or trajectories when normalization loses information and separately records truncation, redaction, attribution, original/captured size, and digest semantics. -- R19. Gateway and worker calls propagate W3C Trace Context and export metadata-only OpenTelemetry data. One explicit pre-processor allowlist admits only bounded non-content operational metadata; OpenInference and backend-native attributes pass the same allowlist and bounded filtering/redaction before any structured log or span processor. Prompts, model outputs, tool arguments/results, file bodies, source fragments, and secret-bearing attributes are prohibited before export. Owner correlation uses only an opaque identifier appropriate to telemetry-operator access, never caller identity or Task/Artifact authorization. Telemetry access and retention are configured separately from Task and Artifact access and retention, and telemetry is neither durable result truth nor required for terminal lookup. - -**Ownership and safety boundary** - -- R20. The gateway executes one coding request. It does not own eval configuration, datasets, repetition, scoring, retry policy, experiment scheduling, or a durable evaluation Run ledger. -- R21. The initial worker topology is one execution at a time for reviewed repositories inside one configured mutual-trust domain. R13's narrow OS-enforced provider/tool credential boundary is required for credentialed profiles but does not claim hostile-source or cross-tenant isolation. Profiles making either stronger claim are rejected until a full per-invocation UID, mount, PID, network, and credential isolation boundary is configured. -- R22. Gateway admission and worker execution enforce profile limits for request rate, active/retained Tasks, subscriptions, stored bytes, source transfer/expansion, files/inodes, workspace bytes, CPU, memory, PIDs, network, phase deadlines, events, logs, and artifacts. Exhaustion is scoped to one invocation or owner and leaves capacity for terminalization and cleanup. - Materializer CPU, memory, PIDs, network, time, transfer, expansion, file, - inode, and workspace output count against the invocation's limits. New-claim - source authorization precedes every cache lookup. Cached outputs are reusable - only after manifest and content revalidation for the same canonical source, - materializer-definition digest, expected and actual output-manifest digests, - authorization-scope digest, source-authorization revocation epoch, and trust - domain. Revocation advances the epoch and makes the prior namespace - ineligible; the conservative default namespaces cache entries by owner. +**Public protocol** + +- R1. Implement A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, + `GetTask`, `ListTasks`, `CancelTask`, streaming send, and active Task + subscription when advertised. The Agent Card declares + `https://allagents.dev/a2a/extensions/coding-execution/v1` with + `required: true`. Every operation that creates, returns, lists, subscribes to, + or mutates profiled Tasks or Artifacts must include + `A2A-Extensions: https://allagents.dev/a2a/extensions/coding-execution/v1`; + responses echo the activated URI, and unsupported calls receive A2A + `ExtensionSupportRequiredError`. +- R2. Generate a strict versioned request schema from Zod and place it only at + `Message.metadata[extensionUri]`. Strict objects reject every unlisted member. + V1 uses these wire scalars: + - `InvocationKey` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. + - `ConfigName` and `TargetId` match + `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`. + - `RevisionText` is NFC UTF-8, 1-255 bytes, with no U+0000-U+001F or U+007F. + - `Digest` matches `^sha256:[0-9a-f]{64}$`. + The request object is exactly: + `version: "1"`; `invocationKey: InvocationKey`; `target: TargetId`; `source`, + one of `{ kind: "repositories", revisions?: Record }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, + digest: Digest, workspaceManifestDigest: Digest }`; optional + `deadlineSeconds` (integer 1-3600, default 1800); and optional + `resultSchema: { version: "1", schema: SchemaNode }`. + + A `SchemaNode` is exactly one branch below. `description` is optional NFC + UTF-8 of at most 1024 bytes. Scalar `enum` arrays contain 1-128 canonically + distinct values of the node's type; string enum values are at most 4096 UTF-8 + bytes, numbers are finite, integers are JSON safe integers, and null permits + only `[null]`. + - null or boolean: `{ type, description?, enum? }`; + - string: `{ type: "string", description?, enum?, minLength?, maxLength? }`, + where lengths are integers 0-1,048,576 Unicode scalar values and minimum + does not exceed maximum; + - number or integer: + `{ type, description?, enum?, minimum?, maximum? }`, where number bounds + are finite, integer bounds are JSON safe integers, and minimum does not + exceed maximum; + - array: + `{ type: "array", description?, items: SchemaNode, minItems?, maxItems? }`, + where item bounds are integers 0-4096 and minimum does not exceed maximum; + - object: + `{ type: "object", description?, properties, required?, + additionalProperties: false, minProperties?, maxProperties? }`, where + `properties` is a strict record of 0-256 `ConfigName` keys, + `required` is a unique subset of those keys, property bounds are integers + 0-256, and minimum does not exceed maximum. + No pattern dialect exists in v1. References, unions/combinators, conditionals, + formats, defaults, coercion, non-finite numbers, duplicate canonical enum + values, and unknown keywords are rejected. The canonical result schema is at + most 64 KiB, 256 nodes, and 32 levels deep. Repository revision count cannot + exceed declared repositories. The Message contains exactly one `TextPart` + whose UTF-8 prompt is 1 byte to 1 MiB; other Part kinds are rejected. Only the + extension-owned metadata object is strict; unrelated A2A metadata and other + activated-extension keys are preserved or ignored according to A2A. + Canonicalization materializes defaults, normalizes extension strings to UTF-8 + NFC, sorts record keys, and hashes RFC 8785 extension JSON plus prompt bytes. + Do not add `Task.extensions` or backend-specific public fields. +- R3. One valid new request creates one addressable Task. Follow-up messages to + an existing Task are unsupported. Every terminal Task has exactly one + integrity Artifact plus zero or more produced Artifacts. The integrity + Artifact has `artifactId` and `name` equal to + `allagents.execution-integrity` and one `DataPart` whose strict + `allagents.execution-integrity/v1` object has the following normative wire + shape. `SafeUInt` is an integer 0-9,007,199,254,740,991; `ShortText` is valid + UTF-8 of at most 4096 bytes; `ArtifactId` matches + `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`; and `MediaType` is a valid RFC 6838 + media type of at most 255 ASCII bytes. + - `version` is the literal `"1"`; `taskId` is a lowercase canonical UUIDv7; + and `target` is `TargetId`. + - `sourceIdentity` is either + `{ kind: "repositories", repositories }` or + `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, + workspaceManifestDigest: Digest, repositories }`. + `repositories` contains 1-64 unique strict entries + `{ name: ConfigName, canonicalUrl: string, requestedRevision?: + RevisionText, resolvedCommit: string, verification: + "independentlyVerified" | "snapshotAttested" }`; `canonicalUrl` is a + canonical HTTPS URL of at most 2048 bytes, and `resolvedCommit` matches + `^[0-9a-f]{40}$`. + - optional `workspaceManifestDigest` is `Digest`. + - `terminalOutput` is `{ text, truncated }`, where `text` is valid UTF-8 of at + most 1 MiB and `truncated` is boolean. + - optional `usage` is a strict object with optional `inputTokens`, + `outputTokens`, `cachedInputTokens`, and `totalTokens` `SafeUInt` fields, + plus optional `provider` containing 0-64 `ConfigName: SafeUInt` counters. + - `producedArtifacts` contains 0-128 strict entries + `{ artifactId: ArtifactId, name?: ShortText, mediaType?: MediaType, + size: SafeUInt, digest: Digest }`; each references one additional A2A + Artifact. + - `evidence` is `{ items, complete, truncated }`, where the booleans have + their literal JSON meaning and `items` contains 0-256 strict entries + `{ kind, artifactId?, digest?, summary? }`. `kind` is one of + `gitState | providerTrace | fileChanges | usage | cancellation | + termination | cleanup`; `artifactId` and `digest` use the aliases above, + `summary` is `ShortText`, and at least one of those three optional members + is present. + - `termination` is `{ status: "clean" | "failed" | "unknown", + reason?: ShortText }`; `cleanup` is + `{ workspace: "removed" | "retained" | "failed", reason?: ShortText }`. + - optional `failure` is `{ code, message, retryable, cause }`, where `code` + is one stable code from the error table below, `cause` is one member of the + closed cause union defined below that table, `message` is `ShortText`, and + `retryable` is that row's fresh-invocation value. + - `result` is exactly one of + `{ status: "valid", value }`, + `{ status: "invalid", errors }`, or + `{ status: "notProduced", reason }`. `value` is JSON that validates against + the requested `SchemaNode`, serializes to at most 1 MiB, and is used only + when a result schema was requested. `errors` contains 1-64 strict + `{ path, keyword, message }` entries: `path` is an RFC 6901 JSON Pointer of + at most 1024 bytes, `keyword` is one of the v1 `SchemaNode` member names, + and `message` is `ShortText`. `reason` is one of + `notRequested | providerDidNotReturn | providerFailed | cancelled | + deadlineExceeded | invalidProviderPayload`. + + Failure, rejection, and cancellation retain every available field without + implying a valid result. Task records, Artifacts, events, and claims expire + atomically after the configured TTL; expired keys may create new Tasks. The + gateway never evicts unexpired Tasks to satisfy the retained-count limit: it + rejects new admission until expiry frees capacity. + +**Trust, identity, and Task storage** + +- R4. Do not authenticate application callers. Allow loopback, specific-address, + and explicit `0.0.0.0` listeners. Every reachable caller may create, list, + retrieve, subscribe to, cancel, and fetch Artifacts for every Task. Document + Tailscale ACLs, firewalls, or equivalent network controls as the authorization + boundary. +- R5. Idempotency and Task visibility are deployment-wide. Atomically and + durably bind an invocation key to the canonical request, selected target, + source identity, optional result-schema digest, deadline, and effective + configuration digest before acknowledging Task creation. Identical replay + returns the existing Task; a changed request conflicts. The project-specific + state root persists the canonical workspace identity and holds an exclusive + process lock. The root and all state files must be current-user owned, use + `0700`/`0600`-equivalent permissions, be disjoint from project, profile, and + invocation roots, and be opened descriptor-relatively without following + symlinks or accepting hard-linked files. Startup verifies those invariants, + store integrity, and workspace identity; terminalizes interrupted Tasks + failed; and never resumes provider work. Store open, corruption, write, + rename, or fsync failure stops admission, aborts and contains active work, + prevents terminal success, and exits only after quiescence or a validated + external manager accepts cleanup ownership. + +**Workspace and target configuration** + +- R6. One gateway process serves one project workspace selected by `--workspace` + or cwd. Parse its `.allagents/workspace.yaml` through the authoritative project + schema. Repository names, URLs, destinations, default revisions, workspace + projection, plugins, and named OCI snapshot repositories come only from that + declaration. +- R7. Parse `~/.allagents/workspace.yaml` through the authoritative user schema. + Built-in `codex` and `pi` targets are available when ready. A launcher-bearing + profile client adds a target only when `gateway.expose: true`. Its public ID is + the globally collision-checked launcher basename and resolves to exactly one + `(profile, client)` pair. Built-in IDs are reserved under the same portable + collision key; colliding exposure is a configuration error. Initially only + Codex and Pi profile clients are executable. +- R8. A request selects a declared target, may set the bounded + `deadlineSeconds`, and may provide one bounded result schema. The overall + deadline covers acquisition, publication, typed preparation, provider + execution, and evidence collection. Acquisition receives + `min(900 seconds, remaining overall deadline)`; exceeding that sub-budget + fails before provider execution. Overall expiry initiates abort and bounded + forced termination. Cleanup then uses its own fixed bounded budget and the + R16 fail-closed quiescence rule. A request cannot provide or override backend, + executable path, command, argv, environment, profile settings, plugins, MCP + servers, repository URLs, destination paths, credential provider, setup + behavior, or permission policy. Readiness rejects missing, partial, drifted, + unsupported, or declaration-missing exposed profiles. + +**Workspace acquisition** + +- R9. Use exactly one closed source union: + - `{ kind: "repositories", revisions?: Record }`; or + - `{ kind: "workspaceSnapshot", snapshot, digest, workspaceManifestDigest }`. + Unknown variants, cross-variant fields, undeclared names, mutable snapshot + references, malformed digests, and destination overrides fail admission. + Source-mode failure never falls through to the other mode. +- R10. Repository mode materializes every configured repository required by the + selected project workspace. Caller revisions may override only a declared + repository's default revision. Canonicalize HTTPS GitHub origins, resolve and + record full commits before provider execution, use hermetic Git configuration, + disable redirects and repository-controlled secondary fetch/exec features, + verify checkout identities, and reject path collisions or escapes. +- R11. Snapshot mode maps `snapshot` to a declared OCI repository and constructs + `@` server-side. Validate registry origin, OCI manifest + and layer digests, expected workspace-manifest digest, paths, symlinks, file + types, file/layer counts, individual and total sizes, and manifest + completeness in staging before atomic publication. Reject external or foreign + layers and cross-origin credential forwarding. The common workspace manifest + distinguishes independently verified Git facts from snapshot-attested facts. + +**Credential selection and containment** + +- R12. Repository requests never carry credentials or select providers. For + `github.com`, determine configured-App applicability as + `eligible | ineligible | unknown` through an App-authenticated GitHub API + client, or verify an explicit installation ID against the repository. For + `eligible`, mint a fresh repository-scoped, read-only installation token + through `@octokit/auth-app` and require remaining lifetime greater than the + R8 acquisition sub-budget plus a 60-second clock-skew margin. Use the + configured GitHub CLI account only when the App is absent or applicability is + positively `ineligible`. An `unknown` result or any selected-App + configuration, authentication, minting, permission, repository, rate-limit, + or service failure terminates acquisition without `gh` fallback. Run + `gh auth token --hostname github.com --user ` with ambient token + variables removed. Deliver either token only through an invocation-scoped Git + credential helper and destroy it before typed preparation or provider + execution. OCI credentials likewise exist only during snapshot acquisition. + +**Execution, evidence, and cleanup** + +- R13. Keep one closed `codex | pi` backend registry and one behavior-focused + interface covering availability, capabilities, invocation, progress, + deterministic permission handling, abort, terminal output, optional structured + result, usage, bounded native evidence, and disposal. Profile targets resolve + adapter-owned configuration directly; never execute generated launchers, + discover executables as targets from `PATH`, scrape a TUI, or append public + input to argv. +- R14. Codex uses pinned `@openai/codex-sdk`, one fresh thread per Task, + `AbortSignal`, streamed events, optional native `outputSchema`, and an + operator-selected Codex auth-file handle. Pi uses strict RPC, invocation-owned + configuration, an operator-selected Pi auth-file handle, and one restricted + policy extension; repository extensions and unrestricted built-ins do not + auto-load. The gateway copies only the selected adapter's required auth + material into an invocation-private, read-only control-process view and + removes it during cleanup. +- R15. Acquire into a private staging root and atomically publish the invocation + workspace. Run only adapter-owned typed preparation that projects validated + project/profile settings, plugins, and MCP declarations through existing + deterministic transforms; never execute project or user `setup` entries or + other configured shell commands. Enforce distinct process views: + - the provider control process receives only its invocation workspace, + minimum non-secret profile configuration, and adapter auth channel; + - each MCP child receives only its own resolved secret references; and + - model-invoked shell/tools receive the workspace and no provider or MCP + credentials. + All views exclude gateway state, operator home, App keys, GitHub/OCI stores, + source helpers, unrelated adapter credentials, and the parent environment. + Fail readiness for a target when its adapter cannot enforce those separations. + Acquisition credentials and mounts are absent first. Treat the mutated + workspace as untrusted during evidence collection: use descriptor-relative + no-follow reads; reject hard links, special/sparse files, path replacement, + out-of-root targets, and `.git` gitdir/core.worktree/alternates escapes; and + run Git inspection with hermetic configuration that disables hooks, filters, + drivers, fsmonitor, pagers, helpers, and external commands. +- R16. Supervise the complete acquisition/provider descendant set inside an + invocation-owned OS containment primitive whose membership children cannot + escape. Fail readiness when the platform cannot enforce and inspect that + boundary. Cancellation persists intent with an atomic state transition before + native abort, then applies bounded forced termination. A terminal commit that + wins first makes later cancellation not cancelable; cancellation intent that + wins settles cancelled after quiescence. Terminal cleanup evidence requires + proof that the containment set is empty. Failure records termination + unknown/failed and rejects admission. An unmanaged foreground gateway remains + alive with poisoned readiness and continues reaping; it prints the stable + containment identifier and platform recovery command. It may exit with a + nonempty set only after a validated external manager accepts ownership. Startup + proves every interrupted set empty before it may quarantine stale roots or + advertise readiness. Graceful shutdown stops admission atomically, persists + shutdown/cancellation intent, drains or aborts active work within a bounded + grace period, proves quiescence, settles once, and only then exits. + +**Scope and configuration** + +- R17. Do not add evaluation commands, datasets, assertions, scoring, + repetitions, experiment scheduling, or automatic Task retry. +- R18. Do not add `gateway.yaml` or `worker.yaml`. Process configuration uses + the exact CLI flags and environment variables in the Configuration Contract + for listener, workspace, state/retention, GitHub, OCI, and Codex/Pi auth-file + handles. Secret values never enter workspace files, requests, logs, Tasks, + Artifacts, retained workspaces, or model-invoked tool environments. ### Key Flows -- F1. **Admit, create, and stream an execution** - - **Actors:** A1, A2, A3, A4. - - **Trigger:** A caller opts into `https://allagents.dev/a2a/extensions/coding-execution/v1` and sends a text Message whose `extensions` includes that URI and whose `metadata[uri]` contains the immutable source, profile, invocation key, deadline, and optional bounded result schema. - - **Steps:** Authenticate, check extension negotiation, and bounded-canonicalize the request; resolve an owner-scoped retained claim and return or conflict against its original request/profile/schema bindings before mutable admission checks. For a new claim only, validate current source/profile authorization, profile/readiness, quota, and deadline; atomically create the claim and submitted Task; dispatch a fenced worker attempt; accept the current fence and move the Task to working; materialize and verify source; execute the selected backend; validate structured output with the shared validator when requested; persist progress before emission; and terminalize with the required integrity Artifact plus the fixed-name structured-result Artifact only for a valid result after quiescence and cleanup. - - **Outcome:** `returnImmediately: true` returns the durable current Task, false/unset waits for terminal state, and streaming starts with that Task before ordered updates. - - **Covered by:** R1-R22. -- F2. **Replay or reconnect to an invocation** - - **Actors:** A1, A2. - - **Trigger:** The owner repeats an invocation key or subscribes after a stream disconnect. - - **Steps:** After authentication and bounded canonical parsing, resolve the owner-scoped claim; compare the request and schema digest with the stored originals and verify the retained Task's original effective-profile/schema bindings without resolving the current profile. Reject a mismatch; otherwise return the existing Task before current authorization, quota, readiness, profile, or deadline checks. For active streaming replay/subscription emit its current snapshot then future events; for a terminal Task return it through send replay or `GetTask` without dispatch. - - **Outcome:** Retries do not multiply agent work, and reconnect never promises transient event replay. - - **Covered by:** R4, R6-R8. -- F3. **Cancel or time out an execution** - - **Actors:** A1, A2, A3, A4. - - **Trigger:** The caller invokes `CancelTask`, the effective deadline expires, or gateway shutdown claims cancellation. - - **Steps:** Atomically record the first cancellation source; send one revisioned fenced worker cancel even when dispatch delivery is unconfirmed, so an unseen attempt is tombstoned before any delayed dispatch can create a workspace. If work exists, invoke native abort, terminate descendants, capture termination-safe evidence, clean, and publish canceled only after verification. - - **Outcome:** Completion that wins first remains terminal and later cancel returns `TaskNotCancelableError`; cancellation that wins suppresses stale dispatch and late provider success. If bounded escalation cannot prove the complete invocation process set empty, the Task fails rather than claiming canceled, the worker poisons admission and exits, and its supervisor destroys the boundary. - - **Covered by:** R7, R14, R16-R18. -- F4. **Settle after gateway or worker loss** - - **Actors:** A2, A3. - - **Trigger:** The gateway restarts with nonterminal Tasks, an acknowledgement is lost, a live worker loses its lease, or a worker process crashes. - - **Steps:** Invalidate the attempt fence and settle every affected Task failed once without provider-session reattachment or automatic replay. A live worker that loses its lease self-aborts and cleans. On worker-process crash or unproved quiescence after bounded escalation, poison admission and exit the worker so the external supervisor terminates the complete execution boundary; the replacement worker proves termination, then reaps or quarantines orphaned roots before readiness. Reject late events/results and record termination and filesystem cleanup separately as complete only when the responsible boundary proves each outcome. - - **Outcome:** One Task has one terminal result, interrupted work is never presented as resumed, no stale worker can overwrite durable truth, and a failed quiescence proof cannot leave the poisoned worker available for another reservation. - - **Covered by:** R7, R9, R14, R16-R18, R21-R22. -- F5. **Expire retained execution data** - - **Actors:** A1, A2. - - **Trigger:** The immutable Task expiry is reached. - - **Steps:** Atomically tombstone the complete ownership aggregate; stop authorizing Task and Artifact access; retry physical cleanup independently; permit the old invocation key to create a new Task only after logical expiry. - - **Outcome:** Expired, unknown, and unauthorized identifiers are indistinguishable and no Artifact outlives Task authorization. - - **Covered by:** R5-R9. +- F1. **Start and advertise** + 1. Resolve cwd or `--workspace`, user workspace, project-specific state root, + retention limits, listen address, source credentials, and provider auth + handles. + 2. Validate state-root ownership, permissions, links, disjointness, and + workspace identity; acquire the exclusive lock; validate repositories, + snapshots, target namespace, backend availability, profile state, + containment, separate provider/MCP/tool views, and credential handles. + 3. Reconcile interrupted Tasks and prove every stale containment set empty + before quarantining filesystem roots. + 4. Bind the requested address, including `0.0.0.0` when explicit, and publish + one Agent Card whose required extension and allowlisted targets match the + validated configuration. + +- F2. **Acquire repositories and execute** + 1. Negotiate the required extension and validate the strict request, one text + prompt, target, repository-name/revision map, result schema, deadline, and + deployment-wide idempotency claim. + 2. Durably commit the claim and Task before acknowledgment; create the + invocation containment and staging root. + 3. For each declared repository, classify App applicability, select App or + `gh` only by eligibility, resolve the revision, fetch hermetically, verify + the commit, and remove credentials. + 4. Publish the complete workspace, run typed preparation, invoke the isolated + adapter, validate any structured result, collect evidence through safe + reads, terminate descendants, clean up, and settle the Task once. + +- F3. **Acquire an OCI snapshot and execute** + 1. Resolve the named snapshot repository and digest-pinned reference. + 2. Authenticate if required, pull and verify the OCI manifest and layers, + extract safely, and validate the workspace-manifest digest. + 3. Remove registry credentials, publish atomically, run typed preparation, + invoke the isolated adapter, collect safe evidence, clean up, and settle. + +- F4. **Cancel** + 1. Atomically persist cancellation intent if the Task remains cancelable. + 2. Abort acquisition or provider work, escalate within the bounded termination + budget, prove containment quiescence, preserve partial evidence, clean up, + and settle cancelled. + 3. Repeated cancellation while intent is pending does not re-signal work. + Cancellation after any terminal state returns A2A + `TaskNotCancelableError`. + +- F5. **Shut down** + 1. Stop new admission before signaling active work. + 2. Persist shutdown cancellation intent, abort and escalate, drain evidence, + prove quiescence, and settle the accepted Task once. + 3. Exit only after durable settlement and empty containment. If proof fails, + unmanaged mode remains alive, not ready, and continues reaping while + printing the platform recovery command. Managed mode may exit only after + its validated external manager accepts containment ownership. ### Acceptance Examples -- AE1. **Covers R1-R4, R10-R18.** Given an authorized Codex profile, one valid - immutable workspace source, and an optional result schema, when the caller - streams a request, then one Task moves from submitted to working to completed - and later `GetTask` returns the same validated output, workspace provenance, - and evidence Artifacts. -- AE2. **Covers R6.** Given a retained Task whose original absolute deadline has passed or whose profile is now disabled, changed, or no longer authorized for new work, when its owner reuses the invocation key with the same canonical request and result schema, then the gateway returns the original Task from its stored original bindings before mutable admission checks and makes no second worker dispatch. -- AE3. **Covers R6.** Given a retained Task, when its owner reuses the invocation key with a different prompt, source, profile ID, deadline, or result schema, or the stored original profile/schema binding is inconsistent, then the gateway rejects the request and leaves the original Task unchanged. -- AE4. **Covers R5.** Given a Task owned by caller A, when caller B lists Tasks, gets the Task, cancels it, subscribes, or requests an Artifact, then the gateway reveals no resource existence or content. -- AE5. **Covers R12, R16-R18.** Given a wrong or missing Git commit, - conflicting repository destination, OCI digest or manifest mismatch, unknown - or profile-disallowed materializer, materializer definition/image drift, - produced workspace-manifest digest that differs from the request, malformed - materializer output, direct known-secret disclosure, or setup failure, when - the worker has already accepted the current fence, then provider execution - never starts, the selected public trace is - `Submitted -> Working -> Failed`, and the Task retains bounded - materialization/setup-failure and cleanup evidence. -- AE6. **Covers R7, R14.** Given cancellation races worker acceptance or completion, when the first durable outcome is chosen, then exactly one abort occurs when needed, late success cannot overwrite cancellation, and terminal cancellation appears only after termination and cleanup are verified. Given cancel reaches a worker before its delayed dispatch, the worker tombstones the unseen attempt and the stale dispatch creates no workspace or provider process. -- AE7. **Covers R3, R10-R11, R17.** Given equivalent profiles, one accepted `allagents.result-schema/v1` schema, and fixture runtime events for Codex and Pi, when each completes the same repository mutation, then both publish the required fixed-name integrity Artifact at the schema-defined extension carrier, validate with the same schema and validator, publish the same fixed-name structured-result Artifact containing one A2A `Part` with the validated `data` and `mediaType: application/json`, record the same integrity state, and produce the required normalized evidence fields while retaining distinct native evidence. -- AE8. **Covers R4, R7, R19.** Given canary secrets and cross-owner content fragments in prompts, model output, tool arguments/results, source files, stale events, and errors, when agent, model, tool, stale-event, and error telemetry is processed, then the exporter receives only allowlisted bounded metadata plus the correct opaque owner correlation and receives none of those canaries, fragments, or raw caller identities. Given a caller or exporter disconnects during work, reconnect still returns the current Task and future updates without duplicate dispatch, and telemetry loss does not affect terminal lookup. -- AE9. **Covers R15.** Given a known capability denied by profile, the accepted Task becomes rejected after stop and cleanup; given an unknown permission type, it becomes failed as an adapter incompatibility without waiting for a client. -- AE10. **Covers R17-R18.** Given optional logs/diffs/native events exceed configured budgets, the Task may complete with explicit truncation metadata; given capture cannot establish the integrity kernel, it fails in the evidence phase. Given output validation has already selected `valid` or `invalid` and a later check or mandatory-evidence phase fails, the failed Task preserves that result state and a valid result preserves its one fixed structured-result Artifact; only a failure before candidate production records `not_produced`. -- AE11. **Covers R6, R11-R12, R22.** Given invalid input, an unknown or - profile-disallowed source mode/materializer, or exhausted admission quota, - the gateway returns a request/resource error and creates no Task; given - materializer availability or worker capacity disappears after durable - acceptance, the retained Task fails at dispatch or materialization and replay - returns it without retry. -- AE12. **Covers R7, R14, R16.** Given a duplicate, out-of-order, or stale-fence worker event arrives after restart or terminal settlement, the gateway ignores it for Task state and records only allowlisted metadata-only operator telemetry. Given bounded escalation cannot stop a descendant that starts a new session and ignores graceful signals, the worker persists termination unknown/failed, refuses another reservation, exits, and its supervisor destroys the boundary; replacement readiness performs orphan recovery without changing the failed Task. -- AE13. **Covers R8.** Given a Task reaches expiry while physical deletion fails, all Task and Artifact operations return the same not-found response and the invocation key can create a new Task. -- AE14. **Covers R5, R13, R21-R22.** Given a production public listener or - remote worker route lacks its configured trusted transport or authenticated - peer identity, readiness fails; a same-host Unix worker socket is accepted. - Given a credentialed reviewed-domain profile, model tools cannot inspect - provider process environments, process listings, backend config/data roots, - or exfiltrate provider/control credentials across the configured OS boundary. - Given a GitHub repository, a trusted operator repository-to-installation - mapping wins over GitHub CLI; auth-app never discovers the installation. - Without a mapping, only an explicitly enabled trusted-local provider may - invoke the configured account through - `gh auth token --hostname --user ` with all four ambient - GitHub token variables absent. Any selected-App failure never invokes `gh`. - A remote worker requests a credential only by its active attempt/fence; the - controller derives all provider/repository/route bindings, rechecks current - command state, consumes one single-use grant, and rejects replay, - substitution, stale state, or minter configuration-digest disagreement. - The fresh token has repository/read-only/expiry scope only; the authenticated - lease carries worker, attempt, lease epoch, command revision, fence, - operation, and expiry bindings. Near-expiry cached auth-app output is bypassed - with `refresh: true`, lease expiry never exceeds token expiry, and an unsafe - acquisition ceiling fails readiness without CLI fallback. Authenticated App - lifecycle webhooks and bounded reconciliation invalidate old entitlement - generations; unknown or stale state cannot authorize a cache hit. Cache-hit - provenance distinguishes the original acquisition provider from current - policy selection. Every source-auth failure maps to the specified safe - code/reason/retryability, while provider, installation, and account identities - remain operator-only. Given a registered materializer with source credentials, - setup, provider processes, and model tools have no access to its process, - runner control socket, credential environment/mounts, or staging root after - materialization, and direct known-secret canaries are absent from retained - logs, evidence, and published workspace files. The registered materializer - remains operator-trusted code; hostile-materializer, hostile-source, and - cross-tenant claims remain rejected without a stronger broker or sandbox. +- AE1. A caller on a permitted Tailscale or firewalled network discovers the + gateway bound to `0.0.0.0`, selects `codex-review`, and receives one durable + Task without presenting an application credential. +- AE2. Any reachable caller can list, retrieve, cancel, and fetch Artifacts for + a Task created by another reachable caller; documentation states this shared + trust model without implying tenant privacy. +- AE3. A launcher-bearing Codex profile without `gateway.expose: true` is absent + from discovery and rejected when selected. An exposed but drifted profile + fails readiness/new admission. +- AE4. A multi-client profile exposes `codex-review` and `pi-review` as distinct + targets. Both resolve through adapters; neither generated wrapper is executed. +- AE5. Repository mode accepts declared names and revision overrides, rejects an + undeclared name or URL override, and records the resolved full commits. +- AE6. An applicable GitHub App mints a fresh repository-scoped token whose + lifetime exceeds the acquisition sub-budget plus skew. A repository with no + applicable installation uses the configured `gh` account. Unknown App + applicability, auth, or mint failure does not fall through to `gh`. +- AE7. Snapshot mode accepts a declared snapshot name and matching OCI/workspace + digests, rejects mutable tags, traversal, foreign layers, digest mismatch, or + undeclared registry repositories, and publishes only after full validation. +- AE8. Repository and snapshot modes produce the same workspace-manifest shape, + while OCI-contained commit identities remain marked snapshot-attested unless + independently verified. +- AE9. Identical invocation-key replay returns the original Task. Reusing the key + with a changed target, source, prompt, or result schema conflicts. +- AE10. Cancellation during Git, OCI pull, Codex, or Pi terminates the complete + process set and records cleanup. Unproved quiescence poisons readiness; an + unmanaged foreground process stays alive and reaps, while managed exit + requires accepted external cleanup ownership. +- AE11. Restart turns interrupted Tasks into one terminal failure and never + resumes a provider session. Terminal Tasks and Artifacts remain retrievable + until expiry. +- AE12. A valid structured result survives later check or evidence failure as a + valid result with an overall failed Task; invalid or absent results are never + published as valid. +- AE13. An exposed launcher named `codex`, `pi`, or a portable case-equivalent + fails configuration compilation instead of shadowing a built-in target. +- AE14. Two gateways for different workspaces use distinct private state roots; + a second process for the same root fails the exclusive lock. Wrong-owner, + permissive, linked, hard-linked, or overlapping roots fail startup. Store + fault injection cannot acknowledge an uncommitted Task or false success. +- AE15. Deadline expiry during Git, OCI, preparation, Codex, Pi, or evidence + initiates one abort/termination path and retains truthful partial evidence. +- AE16. Repeated cancel while cancellation is pending is idempotent; cancel + after cancelled, completed, failed, or rejected returns + `TaskNotCancelableError`. +- AE17. A workspace containing `setup` shell entries never executes them through + gateway acquisition or startup. Built-in Codex/Pi authenticate through their + selected private control-process auth views; model-invoked tools cannot read + provider or MCP secrets, operator stores, or gateway state. +- AE18. Evidence collection rejects a provider-created escaping link, hard link, + special file, sparse-file abuse, or `.git` indirection and runs Git inspection + without repository-controlled execution hooks. +- AE19. The 1001st unexpired retained Task is rejected with + `retention_capacity_exhausted`; no retained Task is evicted before TTL. +- AE20. Unrelated Message metadata survives request processing. Every profiled + A2A operation requires activation, and a terminal Task may contain the single + integrity Artifact plus referenced produced Artifacts. ### Success Criteria -- The official A2A JavaScript client can discover the required extension, negotiate it through `A2A-Extensions`, use the standard Message and Artifact extension carriers, and exercise create, immediate/waiting send, stream, reconnect, get, list, subscribe, retained replay, cancel, and expiry behavior against the built service without `Task.extensions`. -- One conformance fixture passes unchanged through the Codex and Pi adapters. -- Admission, retained replay, monotonic worker commands, fencing, acceptance-before-materialization source/setup failure, cancellation races, trace-order/fence/multiplicity constraints, failed-quiescence recycling, restart terminalization without resume, supervised worker-crash cleanup, trusted transports, authorization isolation, metadata-only telemetry export, OS-enforced provider/tool credential separation, source hardening, portable structured-result validation, quotas, and evidence integrity have deterministic integration coverage. -- Direct multi-repository Git, digest-pinned OCI snapshots, and a fake - digest-pinned registered materializer all produce the same validated - workspace manifest and terminal provenance contract before either backend - starts. -- The gateway image contains no coding-agent runtime and cannot access worker workspace roots. -- The initial worker runs one reviewed-trust-domain execution at a time, model-initiated tools are OS-isolated from provider/control credentials, repository Pi extensions cannot auto-load, and no live descendant or reusable workspace survives a completed, failed-quiescence, or crashed attempt. +- `allagents gateway serve` starts from a real workspace with no deployment YAML. +- Explicit loopback, private-interface, and `0.0.0.0` listeners work. +- The official A2A client exercises required-extension negotiation, send, + stream, get, list, subscribe, replay, cancel, terminal cancel errors, Artifact + retrieval, and expiry. +- Built-in Codex/Pi and exposed profile targets pass one conformance suite, + including reserved-ID collisions. +- Direct Git and OCI snapshot fixtures produce equivalent validated workspace + manifests and truthful provenance. +- GitHub App eligibility, unknown failure, no-installation `gh` fallback, + selected-App failure, token lifetime, containment, and OCI credential cleanup + are proven end to end. +- No request can supply a command, executable, URL, destination, credential, + mutable OCI tag, backend override, or arbitrary environment value. +- State-store fault, deadline, cancellation-race, shutdown, descendant escape, + unsafe evidence, and stale-root scenarios fail closed. +- The built CLI passes a trusted-network smoke test against project and user + workspaces created under `/tmp/`. ### Scope Boundaries **In scope** -- A2A 1.0 HTTP+JSON and SSE streaming. -- One versioned AllAgents coding-execution extension and one versioned private worker protocol. -- Codex and Pi backends. -- Built-in bearer authentication with OIDC/JWT and static service-token modes behind the named production TLS boundary. -- Single-replica durable file storage, authenticated Artifact retrieval, authenticated encrypted remote worker transport or same-host Unix sockets, OpenTelemetry, admission/resource limits, container images, configuration examples, and operator documentation. -- Reviewed repositories in one configured mutual-trust domain per worker deployment, with the narrow OS-enforced provider/tool credential boundary required for credentialed profiles. -- Direct multi-repository Git acquisition, digest-pinned OCI workspace - snapshots, and operator-registered digest-pinned materializers with - phase-scoped credentials and one standard workspace manifest. - -**Deferred to follow-up work** - -- Multi-replica database-backed Task and idempotency storage. -- Durable provider execution, checkpointing, provider-session restoration, and automatic replay after gateway or worker restart. -- OpenCode and additional coding backends. -- Kubernetes Job dispatch, queue brokers, autoscaling controllers, and stronger hostile-source or cross-tenant sandbox providers. -- Push-notification configuration, gRPC, JSON-RPC transport, and A2A extended Agent Cards. -- AHP server/client surfaces, long-lived interactive sessions, and client-contributed tools. -- Optional ATIF conversion after the format and tooling mature. -- Versioned central source-snapshot acquisition and delivery; the initial - remote GitHub path is central token minting plus an authenticated non-durable - delivery lease. - -**Outside this product's identity** - -- Evaluation authoring, datasets, assertions, grading, repetitions, experiment scheduling, and durable evaluation Runs. -- Caller-specific result projections such as Promptfoo `ProviderResponse` mapping. +- A2A 1.0 HTTP+JSON and the required AllAgents extension. +- One process and one active invocation at a time initially. +- Built-in and exposed profile-backed Codex/Pi targets. +- Direct declared Git repositories and named OCI workspace snapshots. +- GitHub App and configured GitHub CLI acquisition credentials. +- Local durable Task/evidence storage, cancellation, cleanup, and provenance. +- Listen addresses including `0.0.0.0`. + +**Out of scope** + +- Application authentication, tenant isolation, caller-private Tasks, and public + Internet hardening. +- `gateway.yaml`, `worker.yaml`, remote workers, mTLS worker links, Kubernetes + routing, autoscaling, and multiple gateway replicas. +- Caller-provided repository or registry origins, mutable OCI tags, custom + materializers, Dockerfiles, Compose files, or acquisition commands. +- GitHub Enterprise Server and multiple ordered Apps/accounts in the initial + delivery. +- OpenCode, Claude, Copilot, OMP, arbitrary CLI, and TUI adapters. +- Evaluation orchestration and automatic retries. ### Sources - [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) -- [GitHub source credential broker precedents](../research/source-credential-broker-precedents.md) -- [GitHub App installation access tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) -- [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) -- [AI Evals ADR 0036](https://github.com/WiseTechGlobal/ai-evals/blob/main/docs/adr/0036-remove-the-ai-evals-workspace-runtime.md) -- [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) -- [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) -- [Codex TypeScript SDK](https://github.com/openai/codex/tree/main/sdk/typescript) -- [Codex configuration reference](https://developers.openai.com/codex/config-reference) -- [Promptfoo Codex provider documentation](https://github.com/promptfoo/promptfoo/blob/main/site/docs/providers/openai-codex-sdk.md) -- [Promptfoo Codex provider implementation](https://github.com/promptfoo/promptfoo/blob/main/src/providers/openai/codex-sdk.ts) -- [Promptfoo Codex provider tests](https://github.com/promptfoo/promptfoo/blob/main/test/providers/openai-codex-sdk.test.ts) -- [Pi RPC protocol](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/rpc.md) -- [Pi CLI reference](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/README.md#cli-reference) -- [Pi extension API](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/extensions.md) -- [Pi provider credentials](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/providers.md) -- [Buzz pure Kubernetes state classifier](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-backend-kubernetes/src/classify.rs) -- [Buzz non-secret intent fingerprint](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-backend-kubernetes/src/intent.rs) -- [Buzz conformance coverage checker](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-conformance/src/checker.rs) -- [Buzz bounded process-tree cancellation](https://github.com/block/buzz/blob/779af8886caae1317b4de962082429867ab61503/crates/buzz-dev-mcp/src/shell.rs) +- [Source credential broker precedents](../research/source-credential-broker-precedents.md) +- [A2A 1.0 specification](https://a2a-protocol.org/latest/specification/) +- [OpenAI Codex SDK](https://developers.openai.com/codex/sdk/) +- [OpenAI Codex app-server](https://developers.openai.com/codex/app-server/) +- [GitHub App installation tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) +- [Git credential helpers](https://git-scm.com/docs/gitcredentials) +- [OCI Image Specification](https://github.com/opencontainers/image-spec) --- @@ -412,1119 +563,447 @@ The two initial runtimes expose different programmatic contracts. Codex provides ### Key Technical Decisions -- KTD1. **Use the official A2A JavaScript SDK behind an AllAgents request-handler decorator.** Pin a compatible A2A 1.x SDK. After authentication, required-extension checks, and bounded canonical parsing, the decorator resolves an owner-scoped retained claim before mutable admission; identical replay bypasses current profile/deadline/quota/readiness checks and any new SDK Task/bus allocation. New requests then pass mutable admission and canonical Task reservation. The decorator also owns stream snapshot selection and cancellation routing before `DefaultRequestHandler` can allocate another Task or terminalize cancellation prematurely; the SDK retains standard transport/event mechanics. Governs R1-R8, R14. -- KTD2. **Define the public extension and private worker protocol from canonical Zod schemas.** U1 freezes `https://allagents.dev/a2a/extensions/coding-execution/v1`, its standard Agent Card/header/Message/Artifact negotiation, `Message.metadata[uri]` request location, and the single-Part `allagents.execution-integrity` Artifact data location; no schema or implementation adds `Task.extensions`. The public contract also carries the `allagents.result-schema/v1` closed subset, its canonical digest, four structured-result states, and the separate fixed `allagents.structured-result` Artifact. The worker protocol carries worker identity, attempt identity, profile digest, monotonic command revision and tombstone state, dispatch acceptance, event sequence, lease fence/expiry, renew/cancel, terminal acknowledgement, and error mapping. Governs R3, R6-R7, R11-R18, R22. -- KTD3. **Commit each Task ownership aggregate through generations and one manifest.** The built-in repository creates a new invocation claim and submitted Task together after mutable admission, storing the canonical caller request and the original effective-profile and result-schema digests needed for retained replay. It stores immutable Artifact blobs before atomically switching the manifest to a new generation, tombstones the aggregate before physical retention cleanup, and garbage-collects unreachable generations on startup. A revision/fence compare-and-swap makes terminal settlement immutable. Governs R4-R9, R14, R17-R18. -- KTD4. **Authenticate at a named trusted HTTP ingress before A2A storage or dispatch.** Production traffic reaches the gateway through TLS terminated by the configured gateway or named trusted reverse-proxy boundary; plaintext is allowed only for an unauthenticated loopback development listener. Production OIDC mode verifies JWT issuer, audience, signature, expiry, and required execution scope. Static token mode uses constant-time comparison for local or service deployments. A canonical length-delimited issuer/tenant/subject tuple is hashed into an opaque owner key; raw claims and caller IDs never become paths. Readiness rejects a production public URL whose trusted TLS boundary is absent or inconsistent. Governs R5-R6, R13. -- KTD5. **Use fenced, separately deployable gateway and worker services.** Remote gateway-worker routes use mTLS or an explicitly equivalent authenticated encrypted overlay; a same-host Unix socket is acceptable. The authenticated worker identity is pinned to the configured route/capability set, and every short-lived attempt capability is bound to that identity, attempt ID, lease ID/epoch, and fence. Each worker keeps one minimal durable monotonic command record scoped to its worker identity and lease: `Cancel(attempt, fence, revision)` tombstones even an unseen attempt, and `Dispatch` for a tombstoned or lower-revision attempt is rejected before workspace creation. Dispatch/cancel I/O conditionally verifies the persisted command/outbox revision immediately before any mutating or terminating effect. Duplicate delivery is idempotent; conflicting, stale, out-of-order, or identity-mismatched commands/events are rejected. Gateway and worker transition selectors remain pure and executors re-enter from persisted or freshly observed state. This record is worker-local fence state, not a new durable execution subsystem. Governs R5, R7, R10-R16, R21-R22. -- KTD6. **Make worker leases and the execution supervisor orphan fail-safes, not replay mechanisms.** Gateway cancellation is explicit. Lost acknowledgement or ambiguous dispatch settles `dispatch_unknown` without automatic redelivery; lease expiry makes a live worker abort and clean. Gateway restart terminalizes every nonterminal Task and invalidates old fences. Every external materializer launch creates a supervisor-owned runner resource labeled by worker, attempt, lease, and fence; worker-process exit makes the external supervisor terminate that resource and the complete execution boundary. If bounded escalation cannot prove the complete invocation process set empty, the worker records termination unknown/failed, poisons admission, and exits rather than accepting another reservation; its supervisor destroys the boundary. Before readiness the replacement proves termination and enumerates, destroys, or quarantines orphaned invocation roots, runner resources, credential mounts, and staging mounts; an unresolved resource keeps readiness false. Production readiness accepts a dedicated worker container process namespace under a minimal init/reaper as the baseline; a non-container deployment must prove an equivalent systemd/cgroup boundary. The gateway never reattaches to or resumes a provider session. Governs R7, R14, R16-R18. -- KTD7. **Keep one behavior-focused backend interface and explicit registry.** Adapters implement availability/capabilities, invoke, progress, deterministic permission response, abort, terminal output, optional structured result, usage, native evidence, and disposal. Shared worker code owns source, setup, checks, schema validation, Git evidence, artifacts, process-tree cleanup, limits, and isolated backend roots. A closed `codex | pi` registry is the only production dispatch point. Governs R10-R11, R14-R18, R21-R22. -- KTD8. **Use each provider's supported automation surface directly behind the credential boundary.** Codex depends directly on pinned `@openai/codex-sdk`, creates one fresh thread per Task, passes `AbortSignal` and optional per-turn `outputSchema`, and consumes streamed events. Pi uses strict RPC with an invocation-local credential store and one explicitly loaded worker-owned policy extension; repository extensions and unrestricted built-ins never load. For either adapter, a credentialed provider runtime is separated from every model-invoked tool by the R13 OS-enforced UID/process/mount boundary or an equivalent credential broker; shell-environment filtering is defense in depth, not the boundary. Promptfoo's Codex provider and tests are characterization references only; AllAgents neither vendors them nor inherits their config, cache, pricing, retry, thread-pool, or `ProviderResponse` concerns. Governs R10-R18. -- KTD9. **Make profiles the new-admission policy boundary.** Requests select a - profile ID, one schema-defined workspace source mode, and optionally one - `allagents.result-schema/v1` schema. They cannot override backend or source - credentials, materializer definitions or images, executable paths, provider - config, setup/check commands, environment allowlists, permission rules, - trust class, resource limits, workspace retention, or evidence budgets. A - profile allowlists source modes and materializer IDs plus exact canonical Git - repository/namespace rules, OCI namespaces/signature rules, and resource - selectors for structured materializer inputs. New admission authorizes the - fully canonicalized resource and credential entitlement before cache lookup. - Resolve a versioned canonical `EffectiveProfileIntent` containing the - selected materializer definition digest, authorization-scope digest, - source-authorization revocation epoch, provider policy and pinned CLI account, - and current GitHub App entitlement generation when applicable; compute its - digest without resolved secrets or per-attempt state and persist it with the - canonical caller request and result-schema digest. Unknown or stale App - entitlement state fails cache authorization. Cache metadata retains original - acquisition-provider metadata, while cache-hit provenance records `cache_hit` - plus current policy selection separately. Retained replay compares stored - original bindings and never substitutes or re-resolves current policy. - Governs R6, R11-R16, R21-R22. -- KTD10. **Keep durable evidence and operational telemetry as separate bounded layers.** The worker verifies source, runs setup, records a post-setup Git tree, invokes the adapter, runs checks, and stops every invocation process before final Git/artifact capture. Provider-native events remain a distinct bounded evidence layer; neither Git nor provider evidence is promoted as exact causality when incomplete. Telemetry is a third, non-durable metadata-only channel: one small shared pre-export sanitizer applies an explicit operational-metadata allowlist plus bounded filtering/redaction before every structured log or span processor, and only opaque owner correlation may cross the separately governed operator boundary. OpenInference and backend-native attributes receive no bypass. This is an export guard, not a telemetry framework or alternate evidence store. Governs R13, R16-R19. -- KTD11. **Treat Codex and Pi as the complete initial backend set.** Codex lands first; Pi lands second against the established contract; OpenCode is deferred. (session-settled: user-directed.) Governs R10. -- KTD12. **Separate terminal integrity from optional evidence bodies.** The fixed `allagents.execution-integrity` Artifact validates identity, action outcome, the four-state structured-result record, failure/cancellation, separate termination and filesystem cleanup, Artifact index, completeness, and provenance before terminal publication. `not_produced` applies only before result-candidate production. Once validation selects `valid` or `invalid`, a later check, evidence, cleanup, infrastructure, or crash failure preserves that state and, for `valid`, the separate fixed structured-result Artifact while retaining the later phase as the primary Task failure. Predictable optional-body truncation/redaction may preserve completion; failure that breaks the integrity kernel fails in the evidence phase. Governs R3-R4, R17-R18. -- KTD13. **Standardize and harden workspace materialization.** Define one - closed `kind`-discriminated workspace-source union and one output manifest; - reject unknown kinds and cross-variant fields. The built-in Git path accepts - canonical HTTPS repository identities and full commit IDs only, uses - hermetic Git configuration; disables inherited redirects, proxies, - credential helpers, hooks, filters, LFS smudge, submodule recursion, - alternates, and non-HTTPS protocols; injects only the KTD16-selected - one-shot credential channel; revalidates normalized host/address policy for - every connection; fetches into an isolated object database from the - authorized remote; and verifies the checked-out commit and resulting tree. - The OCI path accepts - manifest digests, not tags; rejects foreign/external URLs by default; - revalidates scheme, host, resolved address, port, redirect, and credential - origin for every registry/auth/manifest/blob request; and verifies every - manifest/layer plus the embedded workspace manifest. The custom path accepts - a registered ID, expected workspace-manifest digest, and schema-validated, - resource-authorized inputs. - - The operator-owned registry splits a non-secret gateway descriptor from the - worker-only runtime definition. The worker computes a - `sha256:<64 lowercase hex>` digest over the versioned, domain-separated - canonical non-secret runtime definition; readiness compares that value with - the gateway's expected digest and capabilities. Distinct domain-separated - canonical JSON preimages define materializer input and workspace-manifest - digests. All paths stage in a worker-owned directory on the final - publication filesystem, validate destinations, links, file types, bounds, - identities, content, and manifest, then terminate the supervisor-owned - acquisition process/mount/credential/runner boundary while retaining the - validated host-owned tree. Only after proving the boundary gone does the - worker atomically rename the tree; no copy fallback exists. Provenance - distinguishes worker-verified observations, trusted-service verification, - and materializer-attested claims. The caller digest covers source kind, - expected output identity, and inputs; the effective-profile digest covers - materializer and authorization bindings; terminal provenance covers both and - the validated output. Governs R6, R11-R13, R16-R18, R21-R22. -- KTD14. **Limit the initial worker to one reviewed trust domain and one execution.** The worker rejects hostile-source or cross-tenant claims and runs with concurrency one. Deployment-level CPU/memory/PID/network/filesystem limits become per-invocation limits. Credentialed profiles still require R13's narrower OS-enforced provider/tool separation: model tools cannot inspect provider processes, procfs entries, or backend config/data roots, and readiness fails without that capability. Provider/source credentials are absent from setup/check phases and child-visible worker control state. Pi disables repository extensions and built-in tools; only the worker-owned policy extension may load. This credential boundary does not imply hostile-source or cross-tenant isolation; that stronger sandbox-driver capability remains deferred. Governs R13, R16, R21-R22. - An external materializer image is reviewed operator code in the deployment's - trusted computing base, not hostile caller code. Its runner or sandbox - control plane is never mounted into the workspace or exposed to setup, - providers, or model tools. A deployment that does not trust the registered - image with source credentials is outside the initial trust model and must not - enable that registered materializer. Support requires the separately - versioned broker or central snapshot-delivery protocol deferred by R13. -- KTD15. **Keep service dependencies out of the Node 18 CLI package.** Add a private `packages/execution-service` workspace requiring Node 22.19+ for the A2A SDK, Codex SDK, current Pi, gateway, and worker. The published root `allagents` CLI keeps its Node 18 engine and does not import service-only dependencies. Governs R1, R10, R16. -- KTD16. **Resolve GitHub credentials through an authoritative ordered provider - registry and lease controller.** The caller supplies only a canonical - credential-free repository URL. The acquisition boundary maps `github.com` - to the built-in GitHub backend and requires explicit host/API mappings for - GitHub Enterprise Server. The profile supplies provider eligibility and - order, not secrets. A configured App provider is applicable only when trusted - operator configuration maps the authorized repository to an installation ID; - auth-app does not discover installations. A `github-cli` provider may follow - only in a trusted-local profile, only when no App mapping applies, and only - for its configured non-secret account. That account participates in - entitlement and effective-profile digests. Invoke - `gh auth token --hostname --user ` with `GH_TOKEN`, - `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` removed, - and fail when the configured account cannot be resolved. Selection is sticky: - App configuration, authentication, minting, authorization, rate-limit, or - service failure never falls through to the broader user identity. - - When AllAgents owns minting, its trusted central minter depends on focused - `@octokit/auth-app` rather than implementing App JWT, clock-skew, expiry, and - renewal; Git remains the transport and the full Octokit client is not added. - Every cache-miss acquisition uses `refresh: true` and accepts only a fresh - token whose remaining lifetime is strictly greater than the acquisition - deadline plus clock-skew margin. The token is scoped only to the repository, - read-only contents permission, and GitHub expiry. Lease expiry cannot exceed - token expiry, and readiness rejects an acquisition ceiling that can exceed a - fresh token's safe lifetime. - - The gateway/control-plane credential-lease controller, not the worker, is - authoritative. An authenticated worker request supplies only active attempt - and fence. The controller rechecks durable command revision, lease epoch, - tombstone, and fence and derives effective-profile digest, selected provider, - host/API-mapping digest, installation ID, canonical repository, operation, - worker route/identity, and expiry from durable dispatch and policy state. It - issues a single-use non-durable grant/response; a separate minter atomically - consumes the grant and must agree with the selected configuration digest. - Replay, substitution, stale command state, and digest disagreement fail - closed. The authenticated lease/channel binds the derived state to worker - identity, attempt, lease epoch, command revision, fence, operation, and - expiry. A remote worker never receives the App private key. Remote App - profiles fail readiness without that complete central path. A trusted - co-located deployment may keep the controller and minter in its control - plane. Versioned central snapshot delivery is deferred. - - Authenticated App lifecycle webhooks and bounded reconciliation advance an - installation-entitlement generation on uninstall, suspension, or - repository-selection change; unknown or stale state fails cache - authorization. Minting remains miss-only. Provenance distinguishes - `cache_hit`, original acquisition-provider metadata, and current policy - selection/entitlement. Public source-auth details contain only the specified - safe code, reason, and retryability; provider, installation, and account - identifiers are operator-only. Governs R6, R12-R13, R16, R21-R22. +- KTD1. **Use the official A2A JavaScript SDK behind a small AllAgents request + decorator.** The decorator validates the required extension and canonical + request, resolves the deployment-wide retained claim, performs new-admission + checks, and preserves standard Task/Artifact carriers. +- KTD2. **Generate public, Task-store, workspace-manifest, and adapter contracts + from canonical Zod schemas.** Keep the result-schema subset shared across + Codex and Pi and forbid backend-specific public fields. +- KTD3. **Use a single-process supervisor, not a remote worker protocol.** One + service owns Task state, staging, publication, backend child processes, + evidence, termination, and cleanup. Child processes remain contained behind + an invocation lifecycle boundary. +- KTD4. **Make application authentication intentionally absent.** All Tasks and + Artifacts share one deployment namespace. The listener accepts explicit + `0.0.0.0`; network controls are external. (session-settled: user-directed.) +- KTD5. **Compile configuration from existing workspace files.** Add + `workspaceSnapshots` to the project schema and `gateway.expose` to strict + profile-client schemas. Resolve the public launcher ID to one profile/client. + Add no deployment YAML. (session-settled: user-directed.) +- KTD6. **Keep source input name-based and closed.** Repository requests carry + only declared-name revisions; snapshot requests carry only a declared snapshot + name and immutable digests. Compute one canonical source identity for + idempotency and provenance. +- KTD7. **Use direct acquisition implementations.** Git runs with hermetic config + and an invocation credential helper. OCI pulls through a library or fixed + non-shell client interface that validates registry redirects and digests and + extracts without trusting archive paths. +- KTD8. **Select GitHub credentials by three-way eligibility.** Discover + repository coverage with an App-authenticated GitHub API client or verify an + explicit installation ID. Only positive `ineligible` permits the configured + `gh` account; `unknown` and selected-provider failure are terminal. + (session-settled: user-directed.) +- KTD9. **Keep one behavior-focused `codex | pi` adapter registry.** Direct + targets and exposed profile targets resolve to the same adapter types and + conformance tests; profile context modifies server-owned configuration, never + the public command line. Provider control, each MCP child, and model-invoked + tools receive separate secret scopes and filesystem/environment views. +- KTD10. **Store one immutable terminal Task generation.** A private, + project-specific locked local store supports one process, durable atomic + idempotency claim plus Task creation, monotonic status, bounded + events/Artifacts, no eviction before TTL, atomic expiry, and startup + terminalization. State paths are ownership/mode/link/disjointness checked. + Integrity or durability failure stops admission and prevents false success. +- KTD11. **Use enforceable invocation containment.** The platform implementation + owns a non-escapable descendant set, drains stdout/stderr, and covers + credential helpers, Git/OCI, MCP, and provider processes. Failure to inspect + or prove an empty set fails or poisons readiness. An unmanaged gateway remains + alive to reap and expose recovery instructions; managed exit requires an + external manager that already accepted containment ownership. +- KTD12. **Keep durable evidence separate and collect it defensively.** Evidence + retains bounded source, Git, provider, result, artifact, and cleanup facts. + Post-execution workspace reads are descriptor-relative and no-follow; Git + metadata indirections and repository-controlled execution are rejected. + Structured logs remain metadata-only and never retain secrets or unrestricted + request/output/file bodies. ### High-Level Technical Design -#### Component topology - ```mermaid flowchart TB - Caller[Authenticated A2A caller] -->|TLS at named trusted ingress| Gateway[execution-service gateway] - Gateway --> Auth[Auth, retained replay, new admission] - Gateway --> Store[Generation-based Task and Artifact store] - Gateway -->|mTLS/authenticated overlay or same-host Unix socket| Worker[Single-execution worker] - Gateway --> LeaseController[Authoritative credential lease controller] - LeaseController --> AppMinter[Trusted GitHub App token minter] - Worker --> Materialization[Workspace materializer registry] - Materialization --> Git[Hardened multi-repository Git] - Git --> SourceCredentials[Source credential client] - SourceCredentials -->|Active attempt and fence| LeaseController - SourceCredentials --> GitHubCLI[Account-pinned trusted-local gh helper] - Materialization --> OCI[Digest-pinned OCI snapshot] - Materialization --> Custom[Registered materializer image] - Worker --> Registry[Closed backend registry] - Registry --> Codex[Codex SDK] - Registry --> Pi[Pi RPC process] - Worker --> Evidence[Quiesced checks, Git and native evidence] - Evidence -->|Bounded terminal result| Gateway - Gateway --> Telemetry[OpenTelemetry exporter] - Worker --> Telemetry -``` - -#### Admission, dispatch, and settlement sequence - -```mermaid -sequenceDiagram - participant C as Caller - participant G as Gateway decorator - participant S as Durable aggregate store - participant W as Worker - participant B as Backend adapter - participant L as Credential lease controller - participant M as App token minter - - C->>G: SendMessage + header/Message extension + metadata[uri] - G->>G: Authenticate, check extension, canonicalize within bounds - G->>S: Resolve owner-scoped invocation claim - alt retained identical replay - S-->>G: Existing Task + original request/profile/schema bindings - G-->>C: Existing Task before current admission checks - else conflicting retained claim - G-->>C: Conflict; existing Task unchanged - else no retained claim - G->>G: Current authorization, profile/readiness, quota, deadline - G->>S: Atomic new claim + submitted Task + original digests - S-->>G: Task + attempt/lease fence - G->>W: Dispatch(attempt, fence, command revision) - W->>W: Verify command record before workspace creation - W-->>G: Accepted(attempt, fence) - opt private GitHub cache miss - W->>L: Request(active attempt, fence) - L->>S: Recheck command revision, lease epoch, tombstone, fence - L->>L: Derive profile/provider/mapping/install/repository/operation/route - L->>M: Single-use non-durable grant + configuration digest - M-->>L: Fresh repository/read-only token + GitHub expiry - L-->>W: Authenticated lease response bound to current command - end - W->>W: Materialize into staging and validate workspace manifest - W->>W: Destroy acquisition boundary, publish atomically, setup, baseline - W->>B: Invoke with isolated roots and credential boundary - B-->>W: Progress, usage, native evidence - W-->>G: Sequenced fenced progress - G->>S: Compare-and-swap Task generation - opt cancellation or deadline wins - C->>G: CancelTask - G->>S: Persist cancellation intent once - G->>W: Cancel(attempt, fence, newer command revision) - W->>W: Persist tombstone before effects - W->>B: Native abort - end - W->>W: Stop descendants, capture evidence, cleanup - W-->>G: Fenced terminal result - G->>S: Store blobs then atomically commit terminal manifest - G-->>C: Terminal status and extension Artifacts - end -``` - -#### Public A2A Task state - -```mermaid -stateDiagram-v2 - [*] --> Submitted: claim and Task committed - Submitted --> Working: worker accepts current fence - Submitted --> Canceled: cancellation proves no workspace exists - Submitted --> Failed: dispatch or restart failure - Submitted --> Rejected: accepted policy refusal before work - Working --> Completed: integrity kernel and cleanup validate - Working --> Failed: source, setup, provider, check, evidence, cleanup, crash, or restart failure - Working --> Rejected: known profile permission denial after stop and cleanup - Working --> Canceled: cancellation wins and stop/cleanup verify - Completed --> [*] - Failed --> [*] - Rejected --> [*] - Canceled --> [*] + C[Trusted-network A2A caller] --> G[Gateway server] + G --> S[Local Task store] + G --> W[Workspace compiler] + W --> PW[Project workspace.yaml] + W --> UW[User workspace.yaml] + G --> A[Acquisition supervisor] + A --> Git[Declared Git repositories] + A --> OCI[Named OCI snapshot] + A --> P[Atomically published invocation workspace] + G --> R[Closed adapter registry] + R --> Codex[Codex SDK] + R --> Pi[Pi RPC] + Codex --> E[Evidence and cleanup] + Pi --> E + E --> S ``` -Terminal states are immutable. Cancellation intent, termination, evidence capture, cleanup, and retention expiry are private record phases, not A2A Task states. - -#### Private execution-record phases +### Configuration Contract -```mermaid -stateDiagram-v2 - [*] --> Admitted - Admitted --> Dispatching - Dispatching --> Running: current command revision accepted - Dispatching --> Terminalizing: dispatch rejected, tombstoned, or unknown - Running --> CancelRequested: caller, deadline, shutdown, or lease expiry - Running --> Quiescing: provider and checks finish - CancelRequested --> Quiescing - Quiescing --> CapturingEvidence: complete process set verified empty - Quiescing --> Poisoned: bounded escalation cannot prove empty - Poisoned --> [*]: persist unknown/failed and exit boundary - CapturingEvidence --> Cleaning - Cleaning --> Terminalizing - Terminalizing --> Retained - Retained --> Tombstoned: expiry - Tombstoned --> [*]: physical cleanup +No `gateway.yaml` or `worker.yaml` is introduced. + +**CLI flags and environment** + +| Concern | CLI | Environment | Default | +|---|---|---|---| +| Listener | `--listen` | `ALLAGENTS_GATEWAY_LISTEN` | `127.0.0.1:4732` | +| Project workspace | `--workspace` | `ALLAGENTS_GATEWAY_WORKSPACE` | cwd | +| State directory | `--state-dir` | `ALLAGENTS_GATEWAY_STATE_DIR` | `~/.allagents/gateway/` | +| Terminal Task TTL | `--task-ttl` | `ALLAGENTS_GATEWAY_TASK_TTL` | `24h` | +| Retained Task limit | `--max-retained-tasks` | `ALLAGENTS_GATEWAY_MAX_RETAINED_TASKS` | `1000` | +| Per-Task retained bytes | `--max-task-bytes` | `ALLAGENTS_GATEWAY_MAX_TASK_BYTES` | `64MiB` | +| GitHub App ID | `--github-app-id` | `ALLAGENTS_GITHUB_APP_ID` | unset | +| App private key file | `--github-app-private-key-file` | `ALLAGENTS_GITHUB_APP_PRIVATE_KEY_FILE` | unset | +| App installation ID | `--github-app-installation-id` | `ALLAGENTS_GITHUB_APP_INSTALLATION_ID` | discovered/unset | +| GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GITHUB_CLI_ACCOUNT` | unset | +| OCI auth file | `--oci-auth-file` | `ALLAGENTS_OCI_AUTH_FILE` | unset | +| OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_OCI_CREDENTIAL_HELPER` | unset | +| Codex auth file | `--codex-auth-file` | `ALLAGENTS_CODEX_AUTH_FILE` | supported Codex default if safe | +| Pi auth file | `--pi-auth-file` | `ALLAGENTS_PI_AUTH_FILE` | supported Pi default if safe | + +Precedence is CLI over environment over default. Credential options name file +handles, accounts, or IDs, never secret values. Auth files must be regular, +current-user/root-owned, non-hard-linked, and no broader than `0600`. Setting +both OCI options is a startup error. The OCI helper value is one absolute +executable path with no arguments; it must be current-user/root-owned and not +group/world-writable. The gateway implements Docker credential-helper `get` +directly, without a shell: argv is exactly `[helperPath, "get"]`; stdin is the +canonical registry origin `https://[:nondefault-port]` plus one +newline; and an exit-zero stdout must be one UTF-8 JSON object with exactly +nonempty string fields `Username` and `Secret`, each at most 64 KiB. Stdout over +128 KiB, a timeout, nonzero exit, signal, malformed UTF-8/JSON, an unknown +member, or an empty credential fails acquisition with +`source_auth_oci_failed`; stderr is bounded, treated as secret-bearing, and not +placed in logs or evidence. The helper is invoked once per registry origin and +its credential is scoped to that origin and destroyed after acquisition. +Provider defaults are eligible only when their resolved auth files pass the +same checks; otherwise the target is not ready. The gateway projects only the +selected provider auth into its control-process view. + +The derived workspace ID is a stable digest of the canonical project-workspace +path and is verified against store metadata. Retention includes Task records, +Artifacts, events, and invocation-key claims; expiry is atomic. When the +unexpired Task-count limit is reached, new admission fails rather than evicting +retained Tasks. + +**Project workspace additions** + +```yaml +repositories: + - name: allagents + source: https://github.com/EntityProcess/allagents.git + path: allagents + branch: main + +workspaceSnapshots: + evaluation: + repository: ghcr.io/entityprocess/allagents-workspaces ``` -### Output Structure - -```text -packages/execution-service/ - package.json - tsconfig.json - src/ - execution/ - contract.ts - extension-v1.ts - result-schema-v1.ts - worker-protocol-v1.ts - errors.ts - profiles.ts - telemetry.ts - source-credentials/ - contract.ts - registry.ts - github.ts - github-app-minter.ts - github-app-client.ts - github-cli.ts - gateway/ - index.ts - config.ts - auth.ts - agent-card.ts - request-handler.ts - executor.ts - server.ts - worker-client.ts - store/ - gateway-repository.ts - file-gateway-repository.ts - worker/ - index.ts - config.ts - server.ts - supervisor.ts - reaper.ts - lease.ts - workspace.ts - materializers/ - types.ts - registry.ts - git.ts - oci.ts - external.ts - evidence.ts - adapters/ - types.ts - registry.ts - codex.ts - pi.ts - pi-rpc.ts - pi-policy-extension.ts - tests/ - fixtures/execution/ - unit/execution/ - unit/gateway/ - unit/worker/ - unit/source-credentials/ - e2e/execution-gateway.test.ts -containers/ - gateway.Dockerfile - worker.Dockerfile -examples/gateway/ - gateway.yaml - worker.yaml -docs/src/content/docs/ - guides/execution-gateway.mdx - reference/execution-gateway-configuration.mdx +Snapshot names use the portable profile-name vocabulary. Repositories must have +unique stable names for remote acquisition. Snapshot repository values contain +only scheme/host/repository identity and never tags, digests, credentials, or +extraction paths. + +**User workspace additions** + +```yaml +profiles: + review: + clients: + - name: codex + launcher: codex-review + gateway: + expose: true ``` -### Configuration Contract - -- Gateway configuration defines the listener/public URL, a named trusted TLS - termination boundary for production ingress, auth and canonical owner - mapping, store/retention, admission and subscription quotas, low-space - watermarks, Artifact limits, worker routes, internal capability secrets, - non-secret materializer descriptors, and profiles. A descriptor contains the - materializer ID, bounded input schema, expected definition digest, expected - output-manifest version, and required worker capabilities. Each remote worker - route declares mTLS or an explicitly equivalent authenticated encrypted - overlay, pinned worker identity/capabilities, source modes and matching - materializer definition digests, and trust material; a same-host route may - declare a Unix socket. Plaintext remote URLs are invalid. -- Each profile defines backend, worker route, allowed workspace source modes, - allowed materializer IDs, exact Git repository or namespace rules, allowed - Git origins/addresses, OCI namespace/registry/signature policy, structured - materializer-input resource selectors, authorization-scope derivation, - source-authorization revocation epoch, and applicable GitHub App - entitlement-generation authority, provider/model settings, phase-specific - environment allowlists, deterministic permissions, setup/check commands, - artifact globs, acquisition and effective deadline ceilings, clock-skew - margin, trust class, resource limits, cleanup policy, evidence budgets, and - required acquisition/provider/tool isolation capabilities. -- Source-credential configuration defines normalized-host backend mappings and - ordered provider entries. `github.com` has a built-in GitHub mapping; every - GitHub Enterprise Server hostname and API base URL is explicit. A - control-plane `github-app` entry references an App ID, private-key secret - handle, installation ID or deterministic repository-to-installation mapping, - requested read-only contents permission, entitlement-generation store, - authenticated lifecycle-webhook configuration, bounded reconciliation - interval and stale-state limit, fresh-token lifetime policy, and a - versioned non-secret provider/host/API-mapping configuration digest. The - worker receives no App private-key handle. A worker-local `github-cli` entry - contains no token, names one non-secret account/login included in entitlement - and effective-profile digests, and is valid only for an explicitly - trusted-local profile. It invokes the configured `gh` binary with - `auth token --hostname --user ` after removing `GH_TOKEN`, - `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN`. -- Every remote route using a GitHub App declares the authenticated central - token minter, authoritative credential-lease controller, and single-use - non-durable grant/response protocol. Startup rejects remote App profiles - without that complete path, `github-cli` on remote or multi-tenant routes, - missing central App secret handles, unsupported hosts, ambiguous - equal-priority providers, policies that allow runtime failure to trigger - identity fallback, or acquisition ceilings that can exceed a fresh token's - safe lifetime. Configuration and effective-profile digests include provider - IDs, order, host/API mappings, route capability, pinned CLI account, - non-secret entitlement policy, and mapping/configuration digest, but exclude - private keys, resolved tokens, lease payloads, and per-attempt state. -- Worker configuration fixes a private listener, worker identity, - one-execution concurrency, one same-filesystem publication root containing - private staging and final workspace directories, a closed materializer - runtime registry, minimal worker-local command-record location, - execution-supervisor mechanism, pre-readiness orphan policy, lease grace, - backend runtime constraints, trust domain, resource-control and - credential-boundary capabilities, and request/result limits. Each external - materializer runtime entry matches the gateway descriptor's ID and expected - definition digest and additionally fixes a digest-pinned image, credential - handle names or mount identities, network destinations, resource/deadline - limits, cache policy, output version, and OCI runner or sandbox; it contains - no credential values. The worker derives, rather than trusts, the definition - digest from that complete non-secret runtime entry. -- Production worker readiness requires authenticated route identity, protected - remote transport or a same-host Unix socket, exact agreement between the - gateway's expected descriptor digest and the worker's computed runtime - definition digest, a same-filesystem staging/publication root with atomic - rename and no copy fallback, and an OCI materializer runner or sandbox that - assigns supervisor-owned attempt/lease/fence labels without exposing its - control plane to the workspace. It also requires an enforceable credential - boundary for every credentialed phase and a supervisor that proves complete - descendant termination and enumerates or destroys orphan runner resources, - credential mounts, staging mounts, and roots before readiness. The supported - worker baseline is a dedicated process namespace under a minimal init/reaper; - bare-host deployment requires an equivalent systemd/cgroup mechanism. -- Remote App readiness additionally proves that the configured central minter - and gateway/control-plane lease controller authenticate the selected worker - route, agree on the selected provider/host/API-mapping configuration digest, - and support fresh `refresh: true` minting plus a single-use non-durable - grant/response. The controller must derive profile/provider/mapping/ - installation/repository/operation/route/identity/expiry from durable state, - recheck current command revision, lease epoch, tombstone, and fence, reject - replay or substitution, and bind delivery to worker identity, attempt, lease - epoch, command revision, fence, operation, and expiry. Readiness also proves - the acquisition ceiling plus clock-skew margin fits within a fresh token's - safe lifetime, lease expiry cannot exceed token expiry, token payloads never - persist in Task or command records, and delivery reaches only the acquisition - phase. The worker image and configuration contain no App private-key handle. -- Telemetry configuration defines the OTLP destination, filtering/redaction bounds, opaque owner-correlation derivation, and telemetry-specific operator access and retention. The service version fixes the metadata allowlist; configuration cannot extend it to prompt/output/tool/source/file-body attributes, secret-bearing fields, raw caller identity, or unfiltered backend-native/OpenInference attribute passthrough. -- Configuration contains environment-variable names but never secret values. Startup resolves the complete graph and becomes ready only when trusted ingress, worker transports/identities, store, runtimes, quotas, free-space reserves, supervisor/orphan recovery, and declared profile capabilities pass. Any unprotected remote endpoint or unproved credential/supervisor boundary fails readiness. +The nested object is strict and initially contains only `expose: true`. Absence +means not exposed. Exposure requires a launcher, an initial supported backend, +and a healthy installed profile with matching declaration digest. ### Error and Status Mapping -| Condition | A2A result | Required extension detail | +| Condition | Stable code and A2A outcome | Fresh-invocation retryable | |---|---|---| -| New-admission authentication, malformed/unsupported extension carrier, invalid workspace source/profile, unknown or profile-disallowed materializer, unauthorized policy, expired deadline, current-profile/readiness failure, or pre-claim quota failure | Operation error; no Task | Safe standard/extension code and field; no invocation claim | -| Identical retained invocation replay | Existing Task | Returned from stored original request/profile/schema bindings before current deadline, quota, authorization, readiness, or profile checks; no new Task, worker attempt, or quota reservation | -| Conflicting invocation key or inconsistent stored binding | Operation error; no new Task | Conflict code; existing Task unchanged | -| Worker capacity loss after acceptance | `TASK_STATE_FAILED` | `dispatch/capacity_exhausted`, retriable fact, no workspace created; gateway does not retry | -| Lost acknowledgement or ambiguous dispatch | `TASK_STATE_FAILED` | `dispatch/dispatch_unknown`; old fence invalidated and cleanup unknown until proven | -| Known profile permission denial after acceptance | `TASK_STATE_REJECTED` | Policy decision plus provider stop and cleanup outcomes | -| Unknown permission or provider protocol shape | `TASK_STATE_FAILED` | Adapter incompatibility, never mislabeled as policy | -| No eligible GitHub provider after acceptance | `TASK_STATE_FAILED` | `materialization/source_auth_unavailable`; safe reason `no_eligible_provider`; `retriable: false`; no provider identity in public detail | -| Selected installation does not cover the repository | `TASK_STATE_FAILED` | `materialization/source_auth_denied`; safe reason `installation_repository_denied`; `retriable: false`; installation identity is operator-only; no `gh` fallback | -| Selected App configuration is invalid | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_configuration_invalid`; `retriable: false`; operator-only provider detail; no `gh` fallback | -| Selected App authentication fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_authentication_failed`; `retriable: false`; operator-only provider detail; no `gh` fallback | -| Selected App token mint or fresh-lifetime validation fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `app_mint_failed`; `retriable: false`; operator-only provider detail; no `gh` fallback | -| Selected App provider is rate limited | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `provider_rate_limited`; `retriable: true`; no provider identity in public detail; no `gh` fallback | -| Selected App provider service is unavailable | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `provider_unavailable`; `retriable: true`; no provider identity in public detail; no `gh` fallback | -| Eligible trusted-local GitHub CLI provider fails | `TASK_STATE_FAILED` | `materialization/source_auth_failed`; safe reason `trusted_local_cli_failed`; `retriable: false`; configured account identity is operator-only | -| Failure before result-candidate production | `TASK_STATE_FAILED` | Typed primary dispatch/materialization/setup/provider/crash/infrastructure phase, including manifest or materializer failure; safe message, retriable fact, requested structured result `not_produced`, separate termination/cleanup/completeness, and bounded workspace provenance | -| Check, mandatory-evidence, cleanup, crash, or infrastructure failure after result validation | `TASK_STATE_FAILED` | Preserve selected `valid` or `invalid`; preserve exactly one fixed structured-result Artifact for `valid`; later phase remains primary failure | -| Requested structured result is missing or invalid after an otherwise successful action | `TASK_STATE_FAILED` | Typed `structured_result/missing` with `not_produced`, or `structured_result/invalid` with `invalid`; no structured-result Artifact | -| Cancellation/deadline wins and stop/cleanup verify | `TASK_STATE_CANCELED` | First source plus contributors and native abort; use `not_produced` only before a candidate, otherwise preserve `valid`/`invalid` and the valid Artifact; record termination and cleanup | -| Cancellation loses to terminal completion | Existing terminal Task / `TaskNotCancelableError` | No state mutation or second abort | -| Successful action with valid integrity kernel and complete evidence | `TASK_STATE_COMPLETED` | Required extension integrity Artifact plus complete evidence; a requested valid result uses the separate fixed-name Artifact with one A2A `Part` containing `data` and `mediaType: application/json` | -| Successful action with allowed bounded optional-evidence gap | `TASK_STATE_COMPLETED` | Per-dimension incomplete flag, reason, original/captured size, digest and redaction/truncation flags | -| Restart cannot reattach active work | `TASK_STATE_FAILED` | `gateway_restart`; old fence invalid; use `not_produced` only before a candidate, otherwise preserve selected state and valid Artifact; cleanup unknown unless proven | -| Retention expiry | Not found | Aggregate logically hidden before physical deletion; Artifact URL also invalid | +| Missing required extension | A2A `ExtensionSupportRequiredError`; no Task | No | +| Malformed request, source, digest, schema, prompt, or unknown target/source | `invalid_execution_request` in A2A `InvalidParamsError.data`; no Task | No | +| Invocation-key conflict | `invocation_key_conflict` in A2A `InvalidParamsError.data`; no new Task | No | +| Identical retained invocation replay | Existing Task and Artifacts | N/A | +| Cancel after terminal state | A2A `TaskNotCancelableError` | No | +| Retained Task capacity exhausted | `retention_capacity_exhausted` in A2A `InternalError.data`; no Task | Yes, after expiry | +| Runtime capacity unavailable after acceptance | `execution_capacity_unavailable`; failed Task | Yes | +| App absent/ineligible and configured `gh` succeeds | Continue with recorded provider class | N/A | +| App applicability unknown | `source_auth_applicability_unknown`; failed Task; no fallback | Yes for rate-limit/service causes only | +| Selected App config/auth/mint failure | `source_auth_failed`; failed Task; no fallback | No | +| Selected App permission/repository denial | `source_auth_denied`; failed Task; no fallback | No | +| Selected App rate limit | `source_auth_rate_limited`; failed Task; no fallback | Yes | +| Selected App service failure | `source_auth_unavailable`; failed Task; no fallback | Yes | +| `gh` account missing or token resolution fails | `source_auth_unavailable`; failed Task | No | +| Git revision/identity failure | `source_git_identity_invalid`; failed Task | No | +| Git transport failure | `source_git_unavailable`; failed Task | Yes | +| OCI helper timeout, process, protocol, or credential failure | `source_auth_oci_failed`; failed Task; no fallback | No | +| OCI auth/digest/manifest/extraction validation failure | `source_snapshot_invalid`; failed Task; no Git fallback | No | +| OCI registry service failure | `source_snapshot_unavailable`; failed Task; no Git fallback | Yes | +| Deadline expires | `execution_deadline_exceeded`; abort/terminate; failed Task | Yes | +| Known provider permission denial | `execution_permission_denied`; rejected Task | No | +| Unknown provider protocol or result shape | `provider_protocol_invalid`; failed Task | No | +| Cancellation with proven quiescence | `execution_cancelled`; cancelled Task | No | +| Termination or cleanup cannot be proven | `execution_quiescence_unknown`; failed Task; readiness poisoned | No | +| State store durability/integrity failure | `task_store_failed`; stop admission; abort/contain; no success | No | +| Restart finds interrupted Task | `gateway_restarted`; failed Task; no provider resume | Yes as a new invocation | +| Retention expiry | A2A `TaskNotFoundError` | Yes as a new invocation | + +Accepted-Task failures use the integrity Artifact's strict `failure` object with +`code`, safe `message`, table-defined `retryable`, and one closed cause from +`validation | capacity | sourceAuth | sourceGit | sourceSnapshot | deadline | +permission | providerProtocol | cancellation | termination | stateStore | +restart`. Admission failures use the exact A2A error type in the table with the +same stable code and retryability in safe `data`. Retryability describes whether +a caller may create a fresh invocation; it never enables automatic Task retry +or fallback. Provider identifiers, credentials, paths, and raw upstream +messages never enter either carrier. ### Phased Delivery -1. Create the private Node 22 service package and freeze the public extension URI and standard carriers, integrity and structured-result Artifacts, portable result-schema subset, worker protocol including command revisions/tombstones, profiles, fixtures, and error vocabulary. -2. Build authenticated durable A2A Task handling, retained-claim-first replay, and trusted fenced worker dispatch against a fake worker; startup terminalizes interrupted Tasks without attempting provider reattachment. -3. Build the supervised single-execution worker lifecycle, monotonic command - record, failed-quiescence boundary recycling, pre-readiness orphan reaper, - OS credential boundaries, the direct Git/OCI/registered-materializer - registry, the central GitHub App minter/client and trusted-local GitHub CLI - source-credential registry using `@octokit/auth-app`, and hardened - workspace/evidence handling against fake materializers and a fake backend. -4. Add the direct Codex SDK adapter and prove structured output, cancellation, OS-enforced provider/tool credential separation, and native evidence. -5. Add the Pi RPC adapter against the same contract, with repository extensions and built-in tools disabled and one worker-owned policy extension providing OS-confined tools plus the terminating result tool. -6. Package the services and run cross-backend, transport, security, process, and A2A conformance before enabling a consumer. +1. Build the current CLI and record the red E2E showing that + `allagents gateway serve` is unavailable. Record the exact `/tmp/` workspace + setup, command, and observed failure. +2. Freeze workspace additions, public extension, common manifests, result + schema, errors, and fixtures. +3. Build the deployment-wide Task store and unauthenticated A2A server against + a fake adapter. +4. Add repository and OCI acquisition with credential containment and manifest + validation. +5. Add the invocation supervisor, execution containment, safe evidence, and + shared backend contract. +6. Add Codex, then Pi, against the same conformance suite. +7. Run final implementation review and fix important correctness, security, + contract, reliability, DRY, and coverage findings. +8. Run the green built-CLI `/tmp/` E2E, repository quality gates, user + documentation, and release evidence. ### System-Wide Impact -- **Package surface:** A private Node 22 execution-service workspace and two container entrypoints are added. The published root `allagents` CLI package, Node 18 engine, command surface, and imports remain unchanged. -- **Dependency surface:** `@octokit/auth-app` is private to the Node 22 - execution-service package and used only by the trusted control-plane GitHub - App minter. The root Node 18 CLI, remote worker, and acquisition subprocess - do not import the full Octokit client or hold App private-key material. -- **Runtime support:** Gateway and worker require Node 22.19+; startup checks SDK/CLI versions. The Linux worker is one execution per instance and scales by adding instances, not concurrent work inside one trust domain. -- **Filesystem:** The gateway owns a generation-based private Task/Artifact store. Workers own isolated invocation and backend roots. Existing workspace/profile paths are never execution workspaces. -- **Security:** New review-critical surfaces are trusted public/private - transports, auth, owner-key derivation, retained-replay ordering, Git and OCI - source SSRF, materializer image supply chain, materializer input schemas, - phase-scoped source credentials and egress, workspace manifest validation, - admission/resource quotas, setup/check policy, OS-enforced provider/tool - credential separation, Pi extension/tool replacement, metadata-only - telemetry filtering and operator boundaries, internal fences and monotonic - command records, Artifact capture/serving, and reviewed-source trust - enforcement. -- **Operations:** Gateway and worker health, readiness, transport/peer identity, quotas, low-space state, allowlisted metadata-only structured logs/traces, telemetry-specific access/retention, command tombstones, lease expiry, poisoned-worker exit, supervisor boundary health, orphan-root quarantine/reaping, stale event rejection, and graceful shutdown need independent signals. -- **Consumers:** AI Evals can build its runner provider only after the Agent Card, extension schemas, and conformance fixtures are versioned and published. +- **Package surface:** Add a private execution-service package and the public + `allagents gateway serve` command. Preserve existing profile and sync commands. +- **Schema surface:** Extend project workspace schemas with named snapshots and + user profile-client schemas with explicit exposure. Regenerate versioned JSON + Schemas and update configuration docs. +- **Dependency surface:** Add the official A2A SDK, pinned Codex SDK, + `@octokit/auth-app`, and a focused OCI client/extraction implementation to the + private execution package. +- **State surface:** Add a bounded gateway state root and per-invocation staging, + publication, evidence, and cleanup roots. Do not alter existing profile state. +- **Security surface:** The network is the authorization boundary. Source + credentials are phase-scoped; acquired code and agent tools never receive App, + `gh`, or OCI credentials. +- **Compatibility:** Existing workspace files remain valid because new fields are + optional. Older binaries reject the new strict nested profile field, so docs + must state the minimum supporting version. ### Risks and Mitigations -- **Provider API churn:** Pin exact compatible SDK/CLI versions in the service lockfile and worker image. Gate capabilities at startup, keep captured provider fixtures versioned, and use Promptfoo's Codex tests as characterization input rather than vendored implementation. -- **False idempotency or stale settlement:** Resolve owner-scoped retained claims before mutable admission and compare stored original request/profile/schema bindings. For new work, claim Task/idempotency in one aggregate, use revision/fence compare-and-swap, sequence events, and fault-test conflicts, cancellation races, restart, and late results. -- **Task/store corruption:** Publish immutable blobs and generations before one manifest switch; tombstone before deletion; validate owner tuples/manifests at startup; garbage-collect unreachable generations; document the one-replica limit. -- **Owner collision or path injection:** Hash a bounded canonical issuer/tenant/subject tuple, store and verify the tuple inside the owner aggregate, and use only server-generated opaque IDs in paths. -- **Bearer interception or worker impersonation:** Require TLS at the named public ingress boundary and mTLS/equivalent authenticated encryption for remote worker routes, pin worker identity/capabilities, bind attempt capabilities to that identity and fence, and reject plaintext or wrong-peer readiness. -- **Orphan processes and roots:** Combine explicit cancel, native abort, process-set verification, one-execution supervisor/container death, lease expiry, and pre-readiness orphan reaping or quarantine. Failed quiescence poisons admission and exits the worker so the supervisor destroys the boundary; termination/filesystem outcomes remain separate. -- **False recovery claims:** Persist Task and evidence truth only. Startup fails active Tasks, invalidates fences, and relies on lease expiry or supervisor-boundary proof instead of resuming provider sessions. -- **Structured-output drift:** Admit only the versioned closed schema subset, include its canonical digest in provenance and original claim bindings, pass the exact accepted schema through each adapter, validate with one shared validator, preserve an already selected result across later failures, and enforce the two fixed Artifact shapes. -- **Source SSRF, materializer compromise, or credential leakage:** Enforce - KTD13 for every source mode, connection, and phase. Pin external - materializer images and OCI snapshots by digest, validate their manifests, - isolate staging and acquisition processes, apply explicit egress and limits, - and atomically publish only validated outputs. Source credentials are - ephemeral, origin-bound, and removed before setup. Credentialed profiles also - enforce the R13 OS provider/tool boundary or broker; environment filtering - remains defense in depth. Pi repository extensions and unrestricted built-in - tools never load. -- **Credential fallback, stale entitlement, or issuer-key escalation:** Treat - provider order as eligibility, not retry. Prefer only the operator-mapped - GitHub App installation, permit an account-pinned GitHub CLI provider only in - trusted-local profiles when no mapping applies, and fail closed after every - selected-App failure. Keep App private keys in the central minter. Make the - lease controller derive provider/repository/route state from the current - durable command, use one single-use grant, and reject replay, substitution, - stale fences, or controller/minter configuration-digest disagreement. Use a - fresh `refresh: true` token per cache-miss acquisition, bound its lifetime to - the acquisition deadline plus skew, and reject unsafe ceilings at readiness. - Authenticated lifecycle webhooks plus reconciliation advance entitlement - generations so stale/unknown App state cannot authorize cache reuse. Keep - tokens out of arguments, Git configuration, durable records, logs, evidence, - and later phases; public failures remain coarse and identities operator-only. -- **Telemetry disclosure:** Apply KTD10's pre-export guard before every structured log/span processor and reject content or secret-bearing attributes rather than relying on exporter policy. Canary-secret and cross-owner-fragment tests cover agent, model, tool, stale-event, and error paths; telemetry operators receive only bounded metadata and opaque owner correlation under separate access and retention. -- **Resource exhaustion:** Reserve per-owner/global gateway quota only for new claims, enforce store watermarks and stream limits, and require one-execution deployment CPU/memory/PID/network/filesystem controls before accepting a profile. -- **Artifact race or disclosure:** Stop all invocation processes first; accept only stable regular files under the repository subdirectory; reject links, special files, mount crossings, unstable metadata, and unsafe sparse files; stage bounded bytes privately, hash once, and verify size/digest at gateway publication. -- **Evidence overclaim:** Enforce KTD12's integrity kernel and per-dimension completeness. Truncation and redaction remain independent facts. -- **Permission deadlock:** Initial profiles never prompt. Known requests resolve for one isolated invocation; unknown shapes fail closed as adapter incompatibility. -- **Trust-boundary overclaim:** Enforce the narrow provider/tool credential boundary for credentialed profiles while rejecting pooled hostile-source/cross-tenant claims; state plainly that the former does not provide the latter. -- **Cross-platform drift:** Keep gateway/store tests cross-platform. State that worker execution and hardened evidence/source controls are Linux-only. +- **Accidental network exposure:** Binding `0.0.0.0` is intentional and allowed; + startup output and docs state that every reachable host has full authority. +- **Profile identity drift:** Derive targets only from current validated user + declarations and matching installed state; never resurrect declaration-missing + launchers from retained profile state. +- **Credential leakage:** Use fresh App tokens or one configured `gh` account, + invocation-only helpers, hermetic Git, and credential teardown before + publication. Separate provider-control, per-MCP, and model-tool views prevent + one secret scope from reading another. +- **Identity-changing fallback:** Classify App applicability as eligible, + ineligible, or unknown; only positive ineligibility permits `gh`. +- **OCI archive abuse:** Require immutable digests, configured repositories, + bounded extraction, path/type/link validation, and manifest verification. +- **Untrusted acquired code:** General hostile-code sandboxing is not claimed, + but model-invoked tools cannot reach provider/MCP/operator credentials or + gateway state. Project/user setup shell commands are never automatic. +- **Evidence-time attacks:** Treat the mutated workspace as untrusted, use + descriptor-relative no-follow reads, reject Git metadata indirection, and + disable repository-controlled Git execution features. +- **Provider/API churn:** Pin compatible SDK/CLI versions and retain versioned + native fixtures plus one adapter conformance suite. +- **Orphaned processes:** Require a platform containment primitive whose + descendants cannot escape; fail readiness when unavailable. On uncertain + quiescence, unmanaged mode stays alive to reap and managed mode exits only + after cleanup ownership transfer. +- **Store corruption or disclosure:** Validate ownership, modes, links, root + disjointness, lock, and workspace identity. Integrity/durability failure stops + admission and prevents terminal success. ### Assumptions -- The first production deployment runs one gateway replica with persistent storage. Multi-replica transactional storage is deferred. -- The initial public extension supports direct multi-repository Git, - digest-pinned OCI workspace snapshots, and operator-registered materializers. - A deployment may enable only the source modes its worker route advertises; - direct hardened Git remains the required baseline. -- Setup and check commands are operator-controlled profile policy, not caller-supplied shell text. -- Initial repositories are reviewed inside one configured mutual-trust domain. Credentialed profiles still enforce provider/tool credential separation, but that narrower boundary does not make hostile-code or cross-tenant execution available; those claims require a stronger sandbox driver. -- Current implementation baselines are A2A SDK 1.x on Node 20+, Codex SDK 0.154.x, and Pi 0.85.x on Node 22.19+. The private service standardizes on Node 22.19+ and rechecks exact pins before lockfile changes. +- The initial deployment is one gateway process and one active invocation. +- Every network peer able to connect is trusted with all exposed targets and + retained Tasks. +- The selected project workspace is operator-controlled and uses supported + repository/snapshot declarations. +- GitHub.com is the only authenticated Git host in the initial delivery. +- OCI snapshots use registries reachable through HTTPS and immutable manifests. +- Codex and Pi automation surfaces remain compatible with the pinned versions. +- Supported platforms provide a non-escapable invocation containment strategy; + the gateway fails readiness where that invariant cannot be met. --- ## Implementation Units -### U1. Versioned public and worker contracts - -- **Goal:** Freeze the standard public extension carriers, versioned workspace - source and manifest contracts, materializer/profile vocabulary, integrity and - structured-result Artifacts, private worker protocol including monotonic - command state, original idempotency bindings, typed failures, and conformance - fixtures before either service endpoint. -- **Requirements:** R2-R3, R6-R7, R10-R22; AE2-AE3, AE6-AE12, AE14; KTD2, KTD5-KTD12, KTD16. -- **Dependencies:** None. -- **Files:** `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `packages/execution-service/src/execution/contract.ts`, `packages/execution-service/src/execution/extension-v1.ts`, `packages/execution-service/src/execution/result-schema-v1.ts`, `packages/execution-service/src/execution/worker-protocol-v1.ts`, `packages/execution-service/src/execution/errors.ts`, `packages/execution-service/src/execution/profiles.ts`, `packages/execution-service/tests/unit/execution/contracts.test.ts`, `packages/execution-service/tests/fixtures/execution/*.json`, `scripts/generate-execution-schemas.ts`, `package.json`, `bun.lock`. -- **Approach:** Create the private Node 22 workspace package. Define strict Zod request/result/profile schemas and freeze `https://allagents.dev/a2a/extensions/coding-execution/v1`: required Agent Card advertisement, `A2A-Extensions` negotiation, `Message.extensions`, request data only at `Message.metadata[uri]`, and terminal integrity data only in the single Part of the fixed-name `allagents.execution-integrity` Artifact whose `extensions` contains the URI. Explicitly forbid `Task.extensions`. Define the portable result-schema subset, canonical caller/schema/profile digests, four result states, separate fixed `allagents.structured-result` Artifact, and shared validator. Define original claim bindings independently from mutable current policy. Add worker identity, attempt/fence/lease identity, monotonic command revision, unseen-attempt cancel tombstone, conditional effect revision, event sequence, terminal acknowledgement, and integrity rules. Generate checked-in schemas and fixtures from one source. - The source contract is a strict `kind`-discriminated union for direct - repository lists, digest-pinned OCI snapshots, or a registered materializer - ID with an expected workspace-manifest digest and schema-validated structured - inputs; cross-variant fields are unrepresentable. Define the standard - workspace manifest, verification-method vocabulary, authorization scope and - revocation epoch, collision-safe destinations, and split gateway/worker - materializer descriptors. Define algorithm-qualified digest formats and - versioned, domain-separated canonical preimages for materializer definitions, - inputs, manifests, profiles, and caller requests; no public field can carry - acquisition code, image references, commands, credentials, or policy. - Define normalized source-host/API mappings and ordered source-credential - provider policy as trusted profile/configuration fields. Public schemas cannot - select a provider. Effective-profile canonicalization includes provider IDs, - order, host/API mappings, mapping/configuration digest, pinned CLI account, - non-secret entitlement policy, and current App entitlement generation while - excluding App private keys, resolved tokens, local account tokens, and - per-attempt provider state. Define cache metadata that preserves original - acquisition-provider metadata and cache-hit provenance that separately names - `cache_hit` and current policy selection. - Define the private source-credential protocol separately from the durable - worker command record. A worker request contains only active attempt and - fence under its authenticated route. The controller response carries its - authoritative derivation of effective-profile digest, selected provider, - host/API-mapping digest, installation ID, repository, operation, worker - route/identity, lease epoch, command revision, and expiry, plus a single-use - non-durable grant/response state. The lease/channel binds all derived fields - and cannot outlive the token; the token schema expresses only repository, - read-only contents permission, and GitHub expiry. Define deterministic - source-auth code/reason/retryability enums and operator-only identity detail. - Token payloads are secret transport data: they are never part of public - schemas, canonical digests, Task storage, command records, events, logs, - errors, evidence, or provenance. -- **Execution note:** Start with fixture-driven schema, framing, and digest - tests. Observe failures for unknown versions, credential-bearing sources, - mutable revisions or image tags, duplicate/unsafe destinations, unknown or - disallowed materializers, invalid structured inputs or workspace manifests, - unsafe paths, invalid public states, stale fences, oversized records, and - conflicting canonical inputs before implementing schemas. -- **Patterns to follow:** `src/models/workspace-config.ts` for strict schemas, `scripts/generate-workspace-schemas.ts` for generated-schema drift checks, `src/core/native/types.ts` for safe error/provenance normalization, and Buzz's structurally non-secret intent template for the narrow digest-input pattern. -- **Test scenarios:** - - A minimal valid Message negotiates the exact URI in `A2A-Extensions`, includes it in `Message.extensions`, puts the bounded request only at `Message.metadata[uri]`, and produces a stable digest across object-key ordering; missing/mismatched carriers and any `Task.extensions` field are rejected. Every terminal fixture has exactly one `allagents.execution-integrity` Artifact with the URI in `Artifact.extensions` and the schema-defined envelope in its single `data` Part. - - Changing prompt, source object ID, profile ID, result schema, artifact selection, or deadline changes the canonical caller digest; trace IDs and transport metadata do not. The original effective-profile and result-schema digests are stored separately for retained replay. - - Direct repositories are order-canonicalized without erasing destination - identity; duplicate destinations, mutable refs, unsafe subdirectories, and - ambiguous URL forms fail. The checked-out commit and tree match the - requested object from the authorized remote. OCI tags and external layer - URLs fail while allowed manifest digests pass. - - The source discriminator rejects unknown `kind` values, cross-variant - fields, and missing variant fields. Registered materializer inputs validate - against the operator schema and resource selectors, the request pins the - expected workspace-manifest digest, the worker-derived definition digest - changes the effective-profile digest, gateway and worker descriptors agree, - and caller-supplied image/command/credential fields are unrepresentable. - Fixed cross-language vectors prove algorithm-qualified, domain-separated - canonical digests and every non-secret runtime-field mutation changes the - definition digest while secret-value rotation does not. - - Rotating a resolved secret value, changing attempt/lease/trace identity, or - changing a per-run path leaves the profile digest unchanged; changing a - provider policy field, pinned CLI account, host/API mapping or mapping - digest, environment-variable name, authorization scope, revocation epoch, - or App entitlement generation changes it, and the digest serializer cannot - accept secret-bearing runtime state. - - Provider policy fixtures accept GitHub App followed by account-pinned - trusted-local GitHub CLI, reject CLI on remote or multi-tenant routes, - require explicit GitHub Enterprise Server host/API mappings, and reject - every public credential-provider field. Fixtures encode - `gh auth token --hostname --user ` and removal of all four - ambient GitHub token variables. Provider order, ID, host/API mapping, - pinned account, entitlement, or non-secret configuration digest changes - the effective-profile digest; private-key or token rotation does not. - - Private credential-request fixtures accept only active attempt and fence. - Controller-response fixtures carry worker identity/route, attempt, lease - epoch, command revision, fence, effective-profile/provider/mapping/ - installation/repository/operation bindings, and expiry; reject replay, - substitution, stale state, duplicate grant consumption, expiry after token - expiry, or controller/minter configuration-digest disagreement; and cannot - round-trip through durable command/Task serializers. Token fixtures contain - only repository/read-only/expiry scope. Remote App profile fixtures require - the complete central minter/controller capability, safe lifetime policy, - entitlement-generation authority, and no worker-side private-key handle. - - Cache metadata fixtures distinguish original acquisition-provider metadata, - `cache_hit`, and current policy selection/entitlement. Unknown or stale App - generations reject cache authorization. - - Exact source-auth fixtures cover every safe code/reason/retryability tuple - from the error table and prove provider, installation, and account - identifiers are absent from public detail but available to operators. - - Unsupported keywords, remote references, non-object roots, object schemas that omit `additionalProperties: false`, undeclared optional properties, format-dependent validation, or schemas over byte/depth/property/enum limits are rejected before Task creation; every accepted schema validates identically in admission, worker, Codex forwarding, and Pi tool generation. - - Public Task fixtures accept only A2A states; cancellation, cleanup, evidence, and tombstone phases exist only in private records. - - Worker fixtures reject missing/mismatched worker identities, attempt IDs, lease epochs, profile digests, command revisions, conditional-effect revisions, event sequences, bounds, and terminal acknowledgements. Cancel for an unseen attempt persists a tombstone; tombstoned or lower-revision dispatch is invalid before workspace creation. - - `not_requested`, `not_produced`, `valid`, and `invalid` cover success and failure without replacing the primary Task classification. `not_produced` is accepted only before candidate production; a selected `valid` or `invalid` survives later check/evidence/infrastructure failure, and only `valid` permits exactly one separate `allagents.structured-result` Artifact with the matching schema digest. - - File evidence accepts create/edit/delete/rename and rejects unsafe paths, duplicate identities, oversized inline content, and inconsistent before/after forms. -- **Verification:** Generated schemas are stable, public/private fixtures round-trip, digest vectors are cross-platform deterministic, and the private client/server fixture suite agrees before gateway or worker implementation. - -### U2. Authentication and durable gateway repository - -- **Goal:** Provide caller-scoped authentication, trusted-ingress configuration, retained-claim-first idempotency aggregates with original bindings, Artifact storage, new-claim quota admission, pagination, restart fencing, logical expiry, and cleanup. -- **Requirements:** R4-R9, R13-R14, R17-R18, R22; AE2-AE4, AE6, AE8, AE10-AE13; KTD1, KTD3-KTD4, KTD12. -- **Dependencies:** U1. -- **Files:** `packages/execution-service/src/gateway/config.ts`, `packages/execution-service/src/gateway/auth.ts`, `packages/execution-service/src/gateway/store/gateway-repository.ts`, `packages/execution-service/src/gateway/store/file-gateway-repository.ts`, `packages/execution-service/tests/unit/gateway/auth.test.ts`, `packages/execution-service/tests/unit/gateway/file-gateway-repository.test.ts`. -- **Approach:** Adapt one owner-scoped repository to the A2A SDK `TaskStore`. Derive an opaque owner key from a bounded canonical issuer/tenant/subject tuple. Resolve a retained claim after authentication and bounded parsing, and compare its stored canonical caller request plus original effective-profile/result-schema digests without consulting mutable current policy. For new work only, reserve owner/global quota and commit the claim, original bindings, and submitted Task in one manifest generation. Publish immutable Artifact blobs before one manifest switch; compare-and-swap revisions/fences; tombstone before physical expiry cleanup; recover unreachable generations; and complete startup recovery before serving. Verify OIDC/static tokens before repository access and validate the configured named TLS ingress boundary before readiness. -- **Execution note:** Implement concurrent-claim, transition-race, and crash-publication tests before request handling. Inject faults between blob, generation, manifest, tombstone, and cleanup operations. -- **Patterns to follow:** `src/core/marketplace.ts` and `src/core/profile/files.ts` for atomic publication/recovery, `src/core/mcp-http-stdio-proxy.ts` for private files and loopback safety, and the official A2A `TaskStore` owner-scoping contract. -- **Test scenarios:** - - Covers AE2-AE3. Concurrent identical new claims create one aggregate; a conflicting original request/schema binding returns conflict without dispatch permission. Identical retained replay still returns the existing Task after its deadline, quota, authorization, readiness, or current profile changes, while an inconsistent stored binding fails closed. - - Covers AE4. Load/list/cancel/subscribe/Artifact lookup scopes before path/database access and gives unknown, unauthorized, and expired IDs indistinguishable behavior. - - Hostile/ambiguous issuer, tenant, subject, invocation key, Task ID, Artifact name, Unicode, case, delimiter, traversal, and Windows-reserved values cannot collide or become paths. - - All standard list filters, `historyLength`, page size 1-100, omitted Artifacts, ordering, total size, and always-present next token match A2A semantics. Tokens are owner/query-bound and reject malformed, swapped, or stale filters. - - Covers AE12. Terminal compare-and-swap wins once; stale fence, duplicate, and out-of-order updates cannot mutate the Task. - - Restart, including repeated failure during startup recovery, completes the recovery barrier before serving: it fails every nonterminal Task once, invalidates fences, never renews an old lease or requests provider reattachment/replay, preserves terminal Tasks, and records cleanup unknown unless proven. - - Covers AE13. Exact expiry tombstones the aggregate before cleanup; failed deletion never restores visibility; same-key replay before expiry returns the old Task and after expiry creates a new Task. - - A crash between every aggregate publication step leaves either the prior or next valid manifest, never claim-without-Task or Task-with-missing-Artifact state. - - OIDC rejects wrong issuer, audience, signature, expiry, scope, tenant, and subject; static tokens and internal capabilities never appear in logs/errors. Production readiness rejects missing/mismatched named TLS termination, while unauthenticated plaintext remains loopback-only. - - Quota-boundary races admit exactly the allowed new claims and preserve reserved capacity for cancel/terminal writes; low-space mode stops new claims without blocking retained replay or settlement. - - Unauthenticated mode starts only on loopback and refuses wildcard or non-loopback listeners. -- **Verification:** A fresh process retrieves prior records, fault recovery finds one valid aggregate generation, authorization cannot reveal neighboring owners, and expiry/quota behavior remains deterministic under concurrency. - -### U3. A2A gateway server and fenced worker client - -- **Goal:** Expose the accepted A2A profile while making extension negotiation, retained replay, new admission, streaming, lookup, authenticated worker routing, monotonic worker commands, failure, and cancellation use one durable state machine. -- **Requirements:** R1-R9, R11, R14-R15, R17-R22; F1-F5; AE1-AE4, AE6, AE8-AE13; KTD1-KTD7, KTD9, KTD12. -- **Dependencies:** U1, U2. -- **Files:** `packages/execution-service/src/gateway/agent-card.ts`, `packages/execution-service/src/gateway/request-handler.ts`, `packages/execution-service/src/gateway/executor.ts`, `packages/execution-service/src/gateway/server.ts`, `packages/execution-service/src/gateway/worker-client.ts`, `packages/execution-service/tests/unit/gateway/agent-card.test.ts`, `packages/execution-service/tests/unit/gateway/request-handler.test.ts`, `packages/execution-service/tests/unit/gateway/executor.test.ts`, `packages/execution-service/tests/e2e/gateway-fake-worker.test.ts`. -- **Approach:** Mount the official HTTP+JSON and Agent Card handlers behind trusted ingress and auth. Advertise the exact required URI; validate `A2A-Extensions`, `Message.extensions`, and `Message.metadata[uri]`; and publish the integrity envelope only through the standard Artifact carrier, never `Task.extensions`. The `A2ARequestHandler` decorator authenticates and bounded-canonicalizes, resolves the owner-scoped retained claim, and returns or conflicts against stored original bindings before current profile/deadline/quota/readiness checks. New requests then pass mutable admission and canonical Task reservation. Keep transition selection pure and execute fenced I/O outside it. Dispatch revisioned commands over mTLS/equivalent authenticated encryption or a same-host Unix socket, binding worker identity/capability/fence; atomically publish Artifact blobs plus the terminal manifest. -- **Execution note:** Begin with an in-process fake worker and official A2A client. Prove operation errors versus accepted-Task failures, replay/subscribe behavior, fencing, cancellation races, and restart before adding providers. -- **Patterns to follow:** Official A2A sample `AgentExecutor`, `A2ARequestHandler`, `DefaultRequestHandler`, Express handlers, and cancellable-agent flow; `src/core/mcp-http-stdio-proxy.ts` for HTTP shutdown and loopback tests; and Buzz's pure classifier/I/O reconciler split for transition selection without adopting its Kubernetes model. -- **Test scenarios:** - - Covers AE1. Agent Card negotiation, `A2A-Extensions`, `Message.extensions`, `Message.metadata[uri]`, and the fixed integrity Artifact pass through the official client; missing/mismatched carriers and `Task.extensions` fail. `returnImmediately` and streaming expose the same durable Task. - - New-admission authentication, invalid extension/source/profile, expired deadline, current-profile/readiness failure, and pre-claim quota failure return operation errors with no Task or worker request. - - Covers AE11. Capacity loss after acceptance fails the retained Task at `dispatch/capacity_exhausted`; ambiguous dispatch fails `dispatch_unknown`; neither is retried. - - Covers AE2-AE3. Identical send/stream replay returns the existing Task before mutable checks even after the stored deadline or current profile changes; changed request/schema or inconsistent original binding conflicts. - - Covers AE8. Active subscribe emits current snapshot then future events without missed-event replay; terminal subscribe errors and `GetTask` returns terminal truth. - - Covers AE4. Get/list/subscribe/cancel/Artifact endpoints apply owner authorization consistently. - - Covers AE6. Cancel in submitted/working, cancel versus accept/completion, caller versus deadline, duplicate cancel, and terminal cancel each produce one linearized outcome and at most one worker abort. If dispatch send is paused after selection and a newer cancel completes first, releasing the stale dispatch cannot create a workspace or provider process. - - Covers AE12. Duplicate, out-of-order, malformed, wrong-identity, wrong-revision, wrong-fence, and late terminal events cannot overwrite Task state; stale facts go only to allowlisted metadata-only telemetry with opaque owner correlation. - - A failed fenced effect or changed observation causes a durable re-read and reclassification; the executor never substitutes a fresher fence or revision into an effect selected from stale state. - - Plaintext remote workers, wrong certificates, wrong configured worker identity/capability, and replayed attempt capabilities fail before dispatch; mTLS/equivalent protected routes and same-host Unix sockets succeed. - - Known policy denial rejects only after stop/cleanup; unknown permission shape fails as adapter incompatibility. - - Caller SSE disconnect and telemetry exporter failure leave execution and terminal lookup intact. - - Graceful shutdown stops admission, claims cancellation for bounded active work, persists honest terminal state, and closes listeners. -- **Verification:** The official SDK client exercises every advertised operation against the built gateway and fake worker; persisted snapshots match streams while aggregate/fence invariants remain intact under races. - -### U4. Worker protocol and safe workspace lifecycle - -- **Goal:** Implement the supervised single-execution worker with authenticated - transport, a minimal monotonic command record, a closed workspace - materializer registry for hardened multi-repository Git, digest-pinned OCI, - and operator-registered images, trusted source-credential resolution, - standard manifest validation, OS-enforced credential separation, leases, - isolated roots, resource controls, race-resistant evidence, - failed-quiescence recycling, and cleanup independent of any provider. -- **Requirements:** R10-R22; F1, F3-F4; AE5-AE6, AE8-AE10, AE12, AE14; KTD2, KTD5-KTD7, KTD9-KTD10, KTD12-KTD14, KTD16. -- **Dependencies:** U1. -- **Files:** `packages/execution-service/src/source-credentials/contract.ts`, `packages/execution-service/src/source-credentials/registry.ts`, `packages/execution-service/src/source-credentials/github.ts`, `packages/execution-service/src/source-credentials/github-app-minter.ts`, `packages/execution-service/src/source-credentials/github-app-client.ts`, `packages/execution-service/src/source-credentials/github-cli.ts`, `packages/execution-service/src/source-credentials/lease-controller.ts`, `packages/execution-service/src/source-credentials/github-app-entitlements.ts`, `packages/execution-service/src/worker/config.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/worker/server.ts`, `packages/execution-service/src/worker/lease.ts`, `packages/execution-service/src/worker/workspace.ts`, `packages/execution-service/src/worker/materializers/types.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/worker/materializers/git.ts`, `packages/execution-service/src/worker/materializers/oci.ts`, `packages/execution-service/src/worker/materializers/external.ts`, `packages/execution-service/src/worker/evidence.ts`, `packages/execution-service/src/worker/adapters/types.ts`, `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/tests/unit/source-credentials/registry.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-minter.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-client.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-cli.test.ts`, `packages/execution-service/tests/unit/source-credentials/lease-controller.test.ts`, `packages/execution-service/tests/unit/source-credentials/github-app-entitlements.test.ts`, `packages/execution-service/tests/unit/worker/supervisor.test.ts`, `packages/execution-service/tests/unit/worker/reaper.test.ts`, `packages/execution-service/tests/unit/worker/server.test.ts`, `packages/execution-service/tests/unit/worker/lease.test.ts`, `packages/execution-service/tests/unit/worker/workspace.test.ts`, `packages/execution-service/tests/unit/worker/materializers.test.ts`, `packages/execution-service/tests/unit/worker/evidence.test.ts`, `packages/execution-service/tests/fixtures/execution/fake-backend.ts`, `packages/execution-service/tests/fixtures/execution/fake-materializer.ts`. -- **Approach:** Authenticate the configured worker identity and fence every - private command. Persist one minimal monotonic command record scoped to worker - identity/lease before workspace creation: unseen-attempt cancel writes a - tombstone, stale/lower-revision dispatch is rejected, and each dispatch/cancel - effect conditionally rechecks the stored revision immediately before - mutation. Reserve one execution only after that check. Validate the fully - canonicalized source against profile resource policy before cache lookup. - Bind cache entries to owner or authorization-scope digest, revocation epoch, - current GitHub App entitlement generation when applicable, canonical source, - definition digest, expected/actual manifest digests, and original - acquisition-provider metadata. Authenticated App lifecycle webhooks plus a - bounded reconciler advance entitlement generation on uninstall, suspension, - and repository-selection changes; unknown or stale state fails cache - authorization. A hit revalidates current authorization and records - `cache_hit`, original acquisition provider, and current policy selection/ - entitlement separately. A hit never mints a token. - Validate deployment, worker-computed materializer definition digest, and - acquisition plus provider/tool credential-boundary capabilities, then emit - sequenced NDJSON. Keep transition selection pure. Run inside a dedicated - container process namespace under init/reaper or an equivalent systemd/cgroup - boundary. Resolve only the closed KTD13 materializer registry. - - Resolve direct-Git credentials through the closed source-credential registry - only after source authorization and a cache miss. Normalize the host, select - the configured backend, and evaluate providers in policy order. For GitHub, - trusted operator configuration resolves the installation ID; auth-app never - discovers it. The authenticated worker asks the gateway/control-plane - credential-lease controller only for its active attempt and fence. The - controller rechecks durable command revision, lease epoch, tombstone, and - fence; derives effective-profile digest, selected provider, host/API-mapping - digest, installation ID, canonical repository, operation, worker route/ - identity, and expiry; and issues a single-use non-durable grant/response. - A separate minter atomically consumes that grant and rejects a mismatched - configuration digest. Replay, field or provider substitution, and stale - state fail before minting. - - The control-plane minter uses focused `@octokit/auth-app` with - `refresh: true` for every acquisition. Accept only a fresh token scoped to the - authorized repository, read-only contents permission, and GitHub expiry, - with remaining lifetime strictly greater than the acquisition deadline plus - clock-skew margin; expire the lease no later than the token. The delivery - lease/channel—not the bearer token—binds worker identity, attempt, lease - epoch, command revision, fence, repository, operation, and expiry. The remote - worker never receives the App private key. - - Invoke `gh auth token --hostname --user ` only through the - account-pinned trusted-local provider when no App installation mapping - applies, after removing `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, - and `GITHUB_ENTERPRISE_TOKEN`; fail if the configured account cannot be - resolved. Once App selection begins, every configuration, minting, access, - rate-limit, or service failure is terminal and never retries as the user. - Deliver the selected token through an ephemeral helper channel to the - one-shot Git acquisition process, never a URL, argument, repository config, - durable Task or command record, or later-phase environment. Tear down the - helper and release token references before publishing the workspace. - - Launch an external materializer through the configured OCI runner or sandbox - as a supervisor-owned resource labeled by worker, attempt, lease, and fence, - with only schema-validated and resource-authorized inputs, its source - credentials, allowed egress, and a private host-owned staging mount under the - final publication root. Never expose gateway, provider, backend, - final-workspace roots, or the runner control socket. Treat the digest-pinned - image as operator-trusted deployment code. Verify the versioned output - manifest, request-pinned digest, content, immutable identities, and - verification-method labels. Terminate the acquisition process, credential - scope, mounts, and runner resource while retaining the validated host-owned - tree; prove that boundary gone before an atomic same-filesystem rename, - setup, and baseline. No copy fallback exists. After bounded termination - escalation, prove the complete invocation process set empty; if proof fails, - persist termination unknown/failed, poison admission, and exit so the - supervisor destroys the boundary. Replacement readiness enumerates and - destroys or quarantines orphan runner resources, credential/staging mounts, - and roots. Use separate phase environments, budgets, and descriptor-safe - evidence; clean in `finally`. -- **Execution note:** Characterize every phase with a fake adapter, fake - registered materializer, local OCI registry, malicious fixtures, and - disposable Git servers before real providers or private artifact systems. - Fault-inject dispatch acknowledgement, events, leases, every acquisition - mode, manifest publication, processes, evidence publication, and cleanup. -- **Patterns to follow:** `src/core/managed-repos.ts` and `src/core/git.ts` for Git execution shape, `src/core/native/types.ts` for child-process results and redaction, `src/core/profile/files.ts` for filesystem ownership, profile adapter context isolation under `src/core/profile/adapters/`, `tests/helpers/env.ts` for isolated state, and Buzz's bounded process-group/job-object cancellation as a lifecycle characterization checklist rather than copied code. -- **Test scenarios:** - - Covers AE5. Every repository checkout matches its requested full object ID - and tree and destinations are disjoint; wrong/missing objects, disallowed - repository/namespace/URL/host/address/port, credential-bearing URLs, - redirects, DNS rebinding, unsafe subdirectories, fetch failure, and setup - failure stop before adapter invocation. After worker acceptance each - materialization/setup failure emits the selected - `Submitted -> Working -> Failed` public trace. - - Repositories with LFS configuration/pointers, submodules, hooks, filters, - alternates, proxy/helper config, or non-HTTPS secondary protocols cause no - secondary connection or execution of repository, user, or system helpers; - only the KTD16-selected one-shot credential channel can run. - - Source credentials leave no repository config, process argument, child - phase environment, log, error, evidence, or retained workspace trace. - - GitHub credential resolution uses only the trusted operator - repository-to-installation mapping and never auth-app discovery. A remote - worker request contains only active attempt/fence; the fake controller - derives profile/provider/mapping/installation/repository/operation/route, - rechecks command revision, lease epoch, tombstone, and fence, and returns - one authenticated single-use non-durable lease response. Wrong worker, - replay, duplicate consumption, substituted repository/provider/operation, - stale command state, or controller/minter configuration-digest disagreement - fails before token delivery. The token carries only repository, - read-only-contents, and GitHub-expiry scope; the lease carries worker, - attempt, lease epoch, command revision, fence, operation, and delivery - expiry. The worker never receives the App private key. - - Auth-app is called with `refresh: true` for each acquisition. A cached - near-expiry token is bypassed, remaining lifetime must exceed acquisition - deadline plus clock-skew margin, lease expiry is capped by token expiry, and - an acquisition ceiling that can exceed a fresh token's safe lifetime fails - readiness. Every boundary failure remains terminal without invoking `gh`. - - A trusted-local profile invokes its fake CLI only when no installation - mapping applies, pins `--hostname --user `, removes all four - ambient GitHub token variables, and fails on account mismatch. Remote - profiles cannot select it. GitHub Enterprise Server works only through an - explicit host/API mapping. Provider ID, host, installation/account, and - selection reason appear only in operator provenance, never public detail. - - Table-driven failures assert the exact public safe code, reason, and - retryability for no provider, installation/repository denial, App - configuration/authentication/mint failure, rate limit, service outage, and - trusted-local CLI failure. No selected-App case invokes the CLI. - - OCI tags, foreign/external URLs, cross-origin credential forwarding, - disallowed registry/auth/blob host/address/port, redirects, DNS rebinding, - manifest/layer mismatches, unsafe layers, missing workspace manifests, and - expansion-limit violations fail before publication. A valid digest-pinned - snapshot produces the same manifest contract as direct Git. - - Unknown, profile-disallowed, unpinned, or definition-drifted materializers; - unauthorized structured-input resources; gateway/worker descriptor - mismatch; missing expected workspace-manifest digest; schema-invalid - inputs; undeclared egress; malformed output manifests; and mismatched - expected/reported repository or output identities fail before setup. The - request cannot select an image or command. - - Materializer credential environments/mounts, process state, runner control - plane, and staging mounts are inaccessible to later phases. Literal secret - canaries in output or retained logs fail publication. This verifies phase - teardown, not safety from a malicious operator-registered image that - intentionally transforms a credential. Provenance labels its unverified - source assertions as materializer-attested. - - A cache hit occurs only after current authorization and revalidates content - plus the same owner or authorization scope, revocation epoch, canonical - source, materializer-definition, expected-output, actual output-manifest, - trust domain, and current App entitlement generation. Authenticated webhook - events and bounded reconciliation for uninstall, suspension, and repository - selection advance the generation; unknown, stale, or mismatched state - rejects reuse. Hit provenance records `cache_hit`, original acquisition - provider metadata, and current policy selection/entitlement separately, - including a hit after provider-policy change, and performs no mint. - - Setup changes establish the baseline; setup and checks receive no provider/control secrets. Credentialed provider runtimes and model tools run across the declared OS UID/process/mount boundary or broker, with disjoint config/data roots and ambient selectors removed. - - Covers AE6. Cancel, deadline in every phase, lease expiry, worker shutdown, and adapter failure terminate/clean once; late adapter completion cannot change the result. - - Block dispatch after effect selection, complete a newer cancel for the unseen attempt, then release dispatch: the command tombstone/revision check rejects it before workspace or provider creation. Duplicate commands remain idempotent and all effects stay fence-bound. - - Covers AE12. A descendant calls `setsid`, ignores graceful signals, and survives per-process-group escalation during cancellation and normal completion; the worker records termination unknown/failed, refuses another reservation, exits, and replacement readiness reaps or quarantines the orphaned root after supervisor boundary destruction. - - Covers AE14. Concurrency above one and hostile/cross-tenant trust claims are rejected. Credentialed readiness fails without the narrow OS provider/tool boundary, and model tools cannot inspect provider/control process environments, procfs/process listings, or configured backend roots. - - Source pack/tree/file/inode/path/sparse-file/disk limits and setup/provider/check CPU, memory, PID, network, phase-time, and workspace limits stop only the invocation; failed quiescence recycles the worker rather than claiming it remains healthy. - - Covers AE9. Known permissions receive one-invocation decisions; prompt-required profiles fail startup; unknown permission types fail the adapter. - - Covers AE10. Predictable evidence limits retain the integrity kernel and explicit gaps. A valid or invalid result selected before a later check/evidence failure is preserved, including the valid Artifact; only a pre-candidate failure records `not_produced`. - - Background swap attacks, links, mount crossings, FIFOs/devices/sockets, unstable files, and tampering between worker staging and gateway publication never expose external bytes or partial Artifacts. - - SIGKILL during materialization and before or after provider spawn proves - supervisor-owned runner/process death, credential/staging mount removal, - and replacement root recovery before readiness; unresolved resources keep - readiness false, and the gateway retains one failed Task with separate - termination and cleanup outcomes. - - Cross-filesystem staging/publication configuration fails readiness. Faults - around the final rename expose either no final workspace or the complete - validated tree, never a copy fallback or partial publication. -- **Verification:** A built supervised worker, authoritative fake lease - controller, and fake central minter materialize equivalent workspaces through - disposable exact-SHA repositories, a local digest-pinned OCI snapshot, and a - fake registered materializer; validate one standard manifest; mutate each - through the fake adapter; and prove authenticated revisioned dispatch, - unseen-cancel tombstones, deterministic App-before-account-pinned-CLI - eligibility, remote App private-key exclusion, controller-derived single-use - lease delivery, repository/read/expiry-only token scope, replay/substitution/ - stale-state/config-digest rejection, fresh-token lifetime boundaries, - entitlement-generation cache revocation and truthful hit provenance, exact - source-auth mappings, no fallback after selected-App failure, acquisition - hardening and credential teardown, budgets, result preservation, quiescence - or poisoned-boundary exit, evidence integrity, worker-crash containment, - orphan-root handling, and cleanup. +### U1. Workspace, extension, and manifest contracts + +- **Goal:** Freeze configuration additions and all versioned public/private data + contracts before runtime implementation. +- **Requirements:** R1, R2, R3, R6, R7, R8, R9, R18; AE3, AE4, AE5, AE7, + AE8, AE9, AE13, AE16, AE20; KTD1, KTD2, KTD5, KTD6. +- **Files:** `src/models/workspace-config.ts`, schema generation tests and + generated public schemas, `packages/execution-service/src/contracts/*`, + `packages/execution-service/tests/unit/contracts/*`, configuration docs. +- **Approach:** Add strict named `workspaceSnapshots` and nested profile-client + `gateway.expose`; preserve project/user scope and reserve built-in IDs. Define + activation on every profiled operation, the extension-owned request envelope, + other-metadata behavior, exact result-schema grammar, source union, deadline, + integrity and produced Artifacts, stable failures, workspace manifest, and + canonical digest preimages from Zod. +- **Execution note:** Start with fixtures that reject missing activation, cross- + variant/unknown extension fields, extra Message Parts, undeclared names, + mutable snapshot references, malformed digests, invalid deadlines, exposure + without launcher, built-in collisions, and unsupported clients while + preserving unrelated metadata. +- **Verification:** Focused workspace-schema and contract tests; generated schema + drift check; representative YAML and wire examples parse through runtime + schemas; canonicalization and Artifact-cardinality fixtures pass. + +### U2. Deployment-wide Task store and A2A server + +- **Goal:** Serve the A2A lifecycle without application authentication and keep + durable deployment-wide Task/idempotency truth. +- **Requirements:** R1, R2, R3, R4, R5, R8, R16, R17, R18; AE1, AE2, AE9, + AE11, AE14, AE15, AE16, AE19, AE20; KTD1, KTD2, KTD3, KTD4, KTD10. +- **Files:** execution-service Task repository, Agent Card, request handler, + server, pagination/retention, CLI gateway command, and focused tests. +- **Approach:** Implement flags/env precedence, private link-safe project state + and lock, loopback default, explicit `0.0.0.0`, startup integrity/ + reconciliation, per-operation extension negotiation, durable atomic + claim+Task creation, monotonic terminal settlement, bounded events/Artifacts, + no early eviction, atomic expiry, global listing/cancellation, deadline + handling, and fail-closed graceful shutdown. +- **Execution note:** Prove with the official A2A client that one caller can read + and cancel another caller's Task; this is expected behavior. Fault-inject + unsafe state paths plus open/write/rename/fsync boundaries before + acknowledgment, cancellation intent, Artifact, and terminal settlement. +- **Verification:** A2A discovery/send/stream/get/list/subscribe/cancel/replay/ + expiry integration tests on loopback and `0.0.0.0`; state-path, retained- + capacity, store-fault, competing-lock, deadline, shutdown, and restart tests. + +### U3. Git and OCI workspace acquisition + +- **Goal:** Materialize declared repository sets and named OCI snapshots into the + same validated invocation workspace contract. +- **Requirements:** R6, R9, R10, R11, R12, R15, R16, R18; AE5, AE6, AE7, + AE8, AE10, AE15, AE17, AE18; KTD6, KTD7, KTD8, KTD11. +- **Files:** acquisition coordinator, Git transport, GitHub provider selection, + OCI client/extractor, workspace-manifest validator, staging/publication helper, + fixtures and tests. +- **Approach:** Resolve name-based source requests from project workspace. + Implement hermetic Git and full-commit verification. Classify App + applicability as eligible/ineligible/unknown, mint a fresh token with adequate + lifetime, permit `gh` only for positive ineligibility, and use temporary + credential helpers. Pull digest-pinned OCI manifests from declared + repositories, validate every layer and extraction boundary, validate the + expected workspace-manifest digest, and publish atomically. Tear down every + acquisition credential before typed preparation. +- **Execution note:** Use local Git remotes and a local OCI test registry/fixture; + prove unknown/selected-App failures do not call `gh`, token lifetime is + enforced, and snapshot failures never invoke Git fallback. +- **Verification:** Focused three-way provider-selection tests, Git integration + tests including branch/tag resolution, OCI digest/path/limit tests, credential + leak scans, and equivalent manifest output across both acquisition modes. + +### U4. Invocation supervisor and backend contract + +- **Goal:** Run one invocation through acquisition, adapter execution, evidence, + cancellation, descendant termination, and cleanup with truthful terminal + outcomes. +- **Requirements:** R3, R5, R8, R13, R14, R15, R16; AE9, AE10, AE11, AE12, + AE14, AE15, AE16, AE17, AE18; KTD3, KTD9, KTD10, KTD11, KTD12. +- **Files:** backend interface/registry, typed preparation, provider/MCP/tool + secret-view compiler, invocation state machine, platform containment, evidence + collector, result validator, cleanup/reaper, fake adapter, and lifecycle tests. +- **Approach:** Resolve targets to typed adapter context; never use generated + launchers or workspace setup commands. Project validated configuration through + deterministic transforms. Give provider control, each MCP child, and model + tools separate minimal views; enforce deadlines; track the non-escapable + containment set; preserve result states; collect evidence through safe reads; + and require quiescence before cleanup success. Startup reconciles Tasks and + containment before roots. Unmanaged poisoned mode continues reaping; managed + exit proves cleanup ownership transfer. +- **Execution note:** Build the fake adapter first and fault-inject every + boundary: cancellation/terminal races, deadline, shutdown, child escape, + cross-scope provider/MCP/tool secret reads, unsafe state reads, output + truncation, malicious evidence, valid-result-then-evidence-failure, unknown + cleanup, and manager handoff. +- **Verification:** Deterministic lifecycle, containment, separate-secret-view, + preparation, and evidence tests plus one real child-process smoke fixture per + supported platform strategy. ### U5. Codex backend adapter -- **Goal:** Run Codex directly through its supported TypeScript SDK while preserving structured progress, validated output, usage, file-change evidence, cancellation, and runtime identity. -- **Requirements:** R10-R22; AE1, AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. -- **Dependencies:** U4. -- **Files:** `packages/execution-service/src/worker/adapters/codex.ts`, `packages/execution-service/tests/unit/worker/adapters/codex.test.ts`, `packages/execution-service/tests/fixtures/execution/codex-events.jsonl`. -- **Approach:** Depend directly on pinned `@openai/codex-sdk` and fail readiness when the runtime or configured credential boundary is unavailable. Create one fresh SDK thread with isolated `CODEX_HOME`. The credential-bearing Codex runtime runs on the provider side of the declared UID/process/mount boundary or obtains credentials through the configured broker; model-invoked commands run on the tool side and cannot inspect provider procfs/process entries or config/data roots. A minimal allowlisted environment and pinned `shell_environment_policy` remain defense in depth. Apply profile model/sandbox/network/approval/path policy, pass `AbortSignal` and exact `outputSchema`, validate final JSON with the shared validator, normalize bounded events/evidence, and never resume or pool threads. Once validation selects `valid` or `invalid`, later check/evidence/infrastructure failure preserves that state and the valid Artifact. -- **Execution note:** Wrap the SDK behind an injectable factory and drive it through fixture events and its executable override before any credentialed smoke test. Use Promptfoo's provider and tests to enumerate observable edge cases, not as copied code or a runtime dependency. -- **Patterns to follow:** `src/core/profile/adapters/codex.ts` for root/environment isolation, `src/core/native/codex.ts` for version checks, the SDK's `startThread`/`runStreamed`/`AbortSignal`/`outputSchema` and shell-environment policy contracts, and Promptfoo's Codex provider tests for characterization of option forwarding, environment isolation, cancellation, structured output, and cleanup. -- **Test scenarios:** - - A successful stream exposes thread ID, progress, final response, token usage, native file-change items, and terminal completion. - - A structured request forwards the exact accepted schema to `outputSchema`; valid JSON selects `valid` and produces the fixed-name `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; malformed or schema-invalid output selects `invalid` without the Artifact and reports the typed validation error; missing output selects `not_produced` and reports the typed missing-output error. - - Empty final response, turn failure, malformed JSONL, non-zero exit, unavailable runtime, and usage omission map to typed result/completeness fields. - - Covers AE6/AE12. A pre-aborted signal prevents start; in-flight cancellation aborts the SDK once; worker escalation proves descendant termination; completion after cancel or stale fence cannot alter the selected terminal outcome. - - The credential-bearing Codex runtime receives only its scoped credential and minimal environment. Adversarial model commands probing parent/sibling environments, `/proc` and process listings, known or discovered `CODEX_HOME`/backend roots, and outbound secret exfiltration cannot recover provider/control credentials; readiness fails when this OS boundary or broker is unavailable. - - Two sequential invocations create fresh threads with disjoint `CODEX_HOME`, session state, and writable roots; no resume or thread-persistence API is called. - - Native diffs and shared Git evidence coexist without claiming identical attribution. -- **Verification:** Fixture-driven tests cover every supported event/failure shape, SDK option/schema/signal forwarding, OS-enforced provider/tool separation plus environment defense in depth, fresh-thread behavior, result-state preservation, and cleanup escalation, followed by an isolated credentialed repository smoke test when prerequisites are available. +- **Goal:** Run built-in and profile-backed Codex targets through the supported + SDK while preserving structured progress, result, usage, cancellation, and + native evidence. +- **Requirements:** R7, R8, R13, R14, R15, R16; AE1, AE3, AE4, AE10, AE12, + AE15, AE17, AE18; KTD9, KTD11, KTD12. +- **Files:** Codex adapter, profile-context and auth bridge, fixtures, + conformance and optional credentialed smoke tests. +- **Approach:** Pin the SDK; create one fresh thread per Task; pass cwd, typed + profile configuration, abort signal, optional output schema, and the private + Codex control-process auth view inside containment. Keep Codex-invoked tools + outside that auth view; normalize events/usage; bound evidence; dispose fully. +- **Execution note:** Characterize the pinned SDK and its tool-sandbox/auth + separation with captured fixtures before implementing normalization. Do not + import Promptfoo provider code. +- **Verification:** Shared adapter conformance, deadline, auth-isolation, and + tool-secret-denial fixtures plus an opt-in credentialed smoke case. ### U6. Pi backend adapter -- **Goal:** Run Pi through strict RPC mode while preserving settled completion, schema-backed terminal output, usage/cost, tool progress, cancellation, and process cleanup. -- **Requirements:** R10-R22; AE6-AE10, AE12, AE14; KTD7-KTD12, KTD14-KTD15. -- **Dependencies:** U4, U5. -- **Files:** `packages/execution-service/src/worker/adapters/pi.ts`, `packages/execution-service/src/worker/adapters/pi-rpc.ts`, `packages/execution-service/src/worker/adapters/pi-policy-extension.ts`, `packages/execution-service/tests/unit/worker/adapters/pi.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-rpc.test.ts`, `packages/execution-service/tests/unit/worker/adapters/pi-policy-extension.test.ts`, `packages/execution-service/tests/fixtures/execution/pi-events.jsonl`. -- **Approach:** Spawn supported Pi 0.85.x in strict RPC mode with an invocation-local `PI_CODING_AGENT_DIR`, no sessions/extensions/built-ins, and one explicit worker-owned policy extension. The credential store and provider runtime stay on the provider side of the configured UID/process/mount boundary or credential broker; policy/command tools run on the tool side and cannot inspect the provider process, procfs entries, or Pi config/data roots. Verify exact tool inventory before work. Implement bounded LF JSONL, correlation, settlement/stats, abort and escalation. For a structured request, generate the terminating tool from the accepted schema; the first call atomically claims and validates the candidate, later calls cannot replace it, and later checks/evidence failures preserve its `valid` or `invalid` state and valid Artifact. -- **Execution note:** Build parser, extension/tool-inventory, terminating-tool, policy-tool, and state-machine tests from captured RPC fixtures before process integration. Reuse the contract established by U5 rather than adding Pi-shaped public fields. -- **Patterns to follow:** `src/core/native/pi.ts` for version/trust checks, `src/core/profile/adapters/pi.ts` for root isolation, and the official Pi RPC framing, `--no-extensions` plus explicit `--extension`, `--no-builtin-tools`, custom-tool, credential-store, and cancellation contracts. -- **Test scenarios:** - - Successful prompt acceptance streams message/tool events, stops on `agent_settled`, retrieves final messages/stats, and reports session ID, usage, and cost. - - A structured request exposes only the invocation-scoped terminating tool in addition to the policy tools. The first observed call claims the candidate; valid arguments select `valid` and produce the fixed-name `allagents.structured-result` Artifact with one A2A `Part` containing the validated object in `data` and `mediaType: application/json`; an invalid first call selects `invalid` without replacement or an Artifact; a later duplicate cannot replace the result; later check/evidence/infrastructure failure preserves the selected state and valid Artifact; a cancel/deadline/fence that wins first suppresses the call; and settled completion without a call selects `not_produced` as missing output. - - LF framing preserves `U+2028`/`U+2029` inside JSON strings, accepts CRLF by stripping trailing CR, handles partial/multiple chunks, and rejects oversized/malformed records. - - Covers AE6/AE12. Cancellation sends RPC abort once, waits for idle, then terminates the process group only after grace; late settled or terminating-tool events cannot overwrite the terminal fence. - - Prompt rejection, agent error, aborted stop reason, retry/compaction sequence, premature exit, stderr overflow, and stats failure map truthfully. - - Sequential invocations have disjoint `PI_CODING_AGENT_DIR`, tool registration, and session state. Adversarial policy/command tools probing parent/sibling environments, procfs/process listings, known or discovered Pi/backend roots, and outbound secret exfiltration cannot recover provider/control credentials; readiness fails without the OS boundary or broker. Repository `.pi/extensions` and unrestricted built-ins do not load, and caller input cannot override provider/model or issue arbitrary RPC/extension commands. -- **Verification:** Fixture and fake-process tests prove framing, correlation, terminating-tool selection, shared validation, exact tool inventory, OS-enforced provider/tool separation plus environment defense in depth, result-state preservation, settlement, stats, isolation, and abort, followed by an isolated credentialed repository smoke test when prerequisites are available. - -### U7. Production registry, service packaging, and observability - -- **Goal:** Compose exactly two production adapters and package independently runnable gateway and supervised worker services with trusted transports, peer identity, credential-boundary and supervisor readiness, safe startup/shutdown, tracing, and reproducible containers. -- **Requirements:** R1, R5, R7-R22; AE7-AE8, AE12, AE14; KTD4-KTD8, KTD10-KTD16. -- **Dependencies:** U3-U6. -- **Files:** `packages/execution-service/src/worker/adapters/registry.ts`, `packages/execution-service/src/worker/materializers/registry.ts`, `packages/execution-service/src/gateway/index.ts`, `packages/execution-service/src/gateway/github-app-webhook.ts`, `packages/execution-service/src/gateway/github-app-reconciler.ts`, `packages/execution-service/src/worker/index.ts`, `packages/execution-service/src/worker/supervisor.ts`, `packages/execution-service/src/worker/reaper.ts`, `packages/execution-service/src/execution/telemetry.ts`, `packages/execution-service/package.json`, `packages/execution-service/tsconfig.json`, `package.json`, `bun.lock`, `containers/gateway.Dockerfile`, `containers/worker.Dockerfile`, `.dockerignore`, `.github/workflows/ci.yml`, `.github/workflows/publish.yml`, `packages/execution-service/tests/unit/gateway/github-app-webhook.test.ts`, `packages/execution-service/tests/unit/gateway/github-app-reconciler.test.ts`, `packages/execution-service/tests/unit/worker/adapters/registry.test.ts`, `packages/execution-service/tests/unit/worker/materializers/registry.test.ts`, `packages/execution-service/tests/e2e/service-lifecycle.test.ts`. -- **Approach:** Register only Codex and Pi as backend adapters and register the - built-in Git/OCI materializers plus configured external materializers through - a separate closed registry. Add gateway and supervised worker entrypoints - inside the private Node 22 workspace. Register source credentials separately: - an authoritative gateway/control-plane lease controller, a trusted GitHub App - minter using focused `@octokit/auth-app`, its authenticated single-use - non-durable worker client, and an account-pinned GitHub CLI provider only on - trusted-local acquisition hosts. Wire authenticated GitHub App lifecycle - webhooks and bounded reconciliation to the durable entitlement-generation - store. Reject duplicate or ambiguous provider IDs, implicit enterprise host - detection, unpinned CLI accounts, ambient GitHub token variables, remote CLI - fallback, worker-side App private-key handles, fallback-on-error policy, and - provider/mapping configuration-digest disagreement. - Before readiness, validate named public TLS termination, every remote - worker's mTLS/equivalent transport and pinned identity/capabilities, the - complete central lease-controller/minter path for every remote App profile, - single-use grant consumption, fresh-token lifetime versus acquisition ceiling - and clock skew, current entitlement-generation authority, Unix-socket - locality, store, runtimes, matching gateway/worker materializer definition - digests, image digests, schemas, credential names, egress, limits, OCI - runner/sandbox isolation, monotonic command storage, acquisition and - provider/tool credential-boundary capabilities, supervisor boundary, orphan - roots, trust, quotas, and resource controls. Propagate `traceparent`, then - apply KTD10's small shared metadata allowlist and bounded filtering/redaction - before any structured log/span processor or OTLP exporter; neither - OpenInference nor backend-native attributes bypass it. Build a minimal - gateway/control-plane image with the lease controller and App minter but no - provider runtime, writable repository, or baked-in private key, and a - one-execution worker image whose init kills the complete boundary when the - worker server exits, including poisoned failed-quiescence exit. -- **Execution note:** Treat this as integration and packaging work; prove it with built-process and container smoke tests rather than source-shape assertions. -- **Patterns to follow:** `src/core/profile/adapters/registry.ts` for explicit adapter composition, root package scripts for workspace delegation, `src/core/mcp-http-stdio-proxy.ts` for server lifecycle, `.github/workflows/ci.yml` for quality gates, and `.github/workflows/publish.yml` for immutable releases. -- **Test scenarios:** - - Registry exposes exactly Codex and Pi, reports their capabilities/versions, accepts an injected fake registry in tests, and rejects OpenCode or unknown backend IDs before workspace creation. - - Materializer registry exposes built-in Git and OCI plus only configured - external IDs, resolves every external image to the configured digest, - rejects duplicates/tags/unknown IDs, and cannot be influenced by request - image, command, credential, or policy fields. - - Source-credential registry maps `github.com` and explicit enterprise - host/API pairs, selects only an operator-mapped App installation, and - permits GitHub CLI only for an account-pinned trusted-local no-mapping case. - The CLI invocation includes `--user` and no ambient GitHub token variables. - No request field can alter provider selection. Public output contains only - safe code/reason/retryability; operator provenance contains the non-secret - provider and installation/account identity. - - Remote App profiles fail readiness without the central minter and - authoritative lease controller, entitlement-generation webhook/ - reconciliation authority, safe fresh-token lifetime policy, single-use - grant support, matching provider/mapping configuration digest, or with an - App private-key handle in worker configuration. Runtime requests by active - attempt/fence derive every provider/repository/route field from current - durable state and reject replay, substitution, stale state, and duplicate - consumption. The same-host trusted case and a separately deployed minter - pass the same contract; neither uses snapshot delivery. - - Readiness rejects an acquisition ceiling that can exceed a fresh token's - safe lifetime. Near-expiry auth-app cache output is bypassed with - `refresh: true`, lease expiry is capped by token expiry, and failure never - falls through to the CLI. - - Gateway and supervised worker start from built outputs, become ready only after trusted transport/identity, credential and supervisor boundaries, dependencies, and orphan recovery pass, and stop gracefully on SIGTERM. - - Gateway readiness fails for malformed auth, missing/mismatched named TLS termination, plaintext production public ingress, invalid aggregate store, unavailable required worker, quota/free-space failure, or non-loopback unauthenticated bind. - - Worker-route readiness fails for plaintext remote URL, wrong/untrusted certificate, worker identity/capability mismatch, or replayed capability; mTLS/equivalent authenticated encryption and same-host Unix sockets pass. - - Worker readiness fails for concurrency above one, unsupported trust claim, unavailable OS credential/supervisor/resource enforcement, unproved or unrecoverable orphan roots, or unavailable/incompatible Codex or Pi runtime. - - Worker readiness fails when a profile allows a materializer absent from its - route, gateway and worker definition digests differ, an external image is - not digest-pinned, an input schema or output manifest version is - unsupported, named credentials or the OCI runner/sandbox are unavailable, - the runner control plane would be visible to the workspace, or configured - egress and resource enforcement cannot be provided. - - Killing or poisoning the worker server while an adapter child and invocation root exist makes the supervisor destroy the boundary; replacement readiness waits for root deletion/quarantine and never reuses it. - - Trace context crosses the authenticated private call and correlates result identities using only opaque owner correlation. Exporter probes for agent, model, tool, stale-event, and error spans contain allowlisted bounded metadata but no canary secret, prompt/output, tool argument/result, file body/source fragment, raw caller identity, or cross-owner fragment; exporter failure cannot change Task status. - - Gateway/control-plane images contain no Codex, Pi, Git workspace, coding - provider credentials, or baked-in GitHub App private key; the App key enters - only through its configured secret handle. Worker images and configuration - contain neither App issuer material nor user credential stores, pin both - coding runtimes, enforce provider/tool UID/process/mount separation or the - credential broker, confine one workspace/config root, disable repository Pi - extensions and unrestricted built-ins, enforce deployment limits, and - complete fake-provider security probes. - - Installing the root npm package on Node 18 does not load service dependencies; the private service workspace and containers enforce Node 22.19+. -- **Verification:** The registries dispatch both adapters and - source-credential providers through their respective contracts; built - services and images prove controller-authorized fresh App minting without - worker issuer material, single-use lease replay/staleness/config-digest - rejection, entitlement-generation webhook/reconciliation behavior, - account-pinned sanitized CLI eligibility, deterministic public failure - mapping with operator-only identities, and lifecycle/security behavior; - exporter-capture tests prove pre-processor metadata allowlisting, bounded - redaction, opaque owner correlation, and canary/cross-owner exclusion across - agent, model, tool, stale-event, and error spans; CI and publication bind - immutable image tags to the release commit. - -### U8. Cross-backend conformance, documentation, and release evidence - -- **Goal:** Prove standard A2A extension carriers, retained replay, trusted transport, monotonic cancellation, truthful result preservation, credential separation, metadata-only telemetry, worker recycling, trace-order conformance, and the shared backend contract end to end without leaking backend details into callers. -- **Requirements:** R1-R22; F1-F5; AE1-AE14. -- **Dependencies:** U1-U7. -- **Files:** `packages/execution-service/tests/e2e/execution-gateway.test.ts`, `packages/execution-service/tests/fixtures/execution/conformance-cases.ts`, `examples/gateway/gateway.yaml`, `examples/gateway/worker.yaml`, `docs/src/content/docs/guides/execution-gateway.mdx`, `docs/src/content/docs/reference/execution-gateway-configuration.mdx`, `README.md`, `CHANGELOG.md`. -- **Approach:** Run one conformance suite against the fake backend and each provider fixture, plus opt-in credentialed smoke cases, with gateway and supervised worker as separate processes. Each race fixture declares attempt/fence correlation, required durable transitions and observed effects, required happens-before edges, maximum occurrence counts, and effects forbidden after terminalization. A deliberately small test-side checker evaluates those constraints against durable records plus observed worker/process outcomes without calling the production selector. It remains coverage protection—not TLA+, a model checker, event sourcing, or a second lifecycle implementation. Document the exact extension URI and legal Agent Card/header/Message/Artifact carriers, both fixed Artifacts and four result states, retained-replay ordering, worker transport/identity, monotonic command tombstones, failed-quiescence recycling, the narrow OS credential boundary, reviewed-domain limitation, metadata-only telemetry and its separate operator access/retention, storage/HA limits, lack of execution resume, source hardening, quotas, retention, and operations. - Document all three source modes, the exact source discriminator, canonical - repository/namespace and materializer-input authorization, multi-repository - destinations, the standard workspace manifest and verification-method - labels, materializer registration, worker-derived definition digests, - profile allowlisting, digest/profile/idempotency boundaries, same-filesystem - publication, supervisor-owned runner cleanup, acquisition credential - teardown, authorization-scoped cache reuse/revocation, source-mode - capabilities, and the prohibition on caller-supplied acquisition code. - Document normalized host/API mapping, operator-owned - repository-to-installation mapping, GitHub App precedence and - repository/read/expiry token scope, account-pinned sanitized trusted-local - CLI eligibility, fail-closed selected-App behavior, focused - `@octokit/auth-app` ownership and `refresh: true`, central App private-key - custody, controller-derived single-use remote lease bindings, token/lease - lifetime rules, entitlement-generation webhooks/reconciliation and - cache-hit provenance, deterministic public source-auth mapping with - operator-only identities, remote readiness requirements, the one initial - token-minter path, and deferred versioned snapshot delivery. -- **Execution note:** Use a disposable local Git HTTP server, temporary gateway store, temporary worker root, and loopback ports. Never read the developer's real home, sessions, or credentials in deterministic tests. -- **Patterns to follow:** Existing `tests/e2e/*` built-process style, `tests/helpers/env.ts` home isolation, Starlight guide/reference organization under `docs/src/content/docs/`, and Buzz's required-critical-action coverage rule without importing its TLA+ model or production implementation. -- **Test scenarios:** - - Covers AE1-AE14 through built services with a fake backend and official A2A client. - - Agent Card required-extension advertisement, `A2A-Extensions`, `Message.extensions`, request `Message.metadata[uri]`, and the single fixed integrity Artifact carrier interoperate; missing/mismatched carriers and `Task.extensions` fail. - - Identical retained replay after deadline expiry, quota exhaustion, readiness loss, authorization change, or profile replacement returns the original Task; changed request/schema or inconsistent original bindings conflict. - - The same accepted schema, valid result, invalid result, missing result, and pre-output failure pass through Codex and Pi with identical decisions. A valid-result-then-check-failure and invalid-result-then-evidence-failure preserve the selected state and only the valid Artifact; `not_produced` remains pre-candidate only. - - Direct Git object/destination/authorization mismatch, OCI - digest/manifest/namespace mismatch, registered materializer - descriptor/input/resource/expected-output mismatch, direct known-secret - disclosure, and setup failure after worker acceptance produce the selected - `Submitted -> Working -> Failed` trace; provider invocation never begins, - and durable snapshots, streams, and conformance records agree. A policy - revocation, webhook-advanced entitlement generation, reconciliation result, - or unknown/stale App state before lookup cannot consume a previously - populated cache entry. A valid hit after provider-policy change records - `cache_hit`, original acquisition provider, and current selection separately - without minting. - - GitHub source cases prove operator-mapped App selection, account-pinned - sanitized trusted-local CLI selection only when no mapping applies, no CLI - invocation after any selected-App failure, explicit enterprise host/API - mapping, repository/read-only/expiry-only token narrowing, and an - authenticated controller-derived single-use lease carrying worker identity, - attempt, lease epoch, command revision, fence, repository, operation, and - expiry without worker issuer material. They reject replay, substitution, - stale state, duplicate grant consumption, and controller/minter - configuration-digest disagreement; bypass near-expiry auth-app cache output - with `refresh: true`; cap lease expiry by token expiry; fail readiness for an - unsafe acquisition ceiling; assert every safe source-auth - code/reason/retryability tuple and operator-only identity detail; and - publish a credential-free workspace. - - Pause dispatch after selection, complete unseen-attempt cancel, then release dispatch; the stale command creates no workspace/process. The small independent checker enforces each fixture's attempt/fence correlation, happens-before edges, maximum counts, and forbidden post-terminal effects. Deliberately bad traces that still contain every required action name fail for wrong order, wrong fence, duplicate-over-maximum effects, and an extra stale dispatch after terminalization. - - Concurrent callers cannot observe each other's Tasks, streams, cancellations, page tokens, quotas, or Artifacts; one worker serializes admitted work. - - Gateway restart, reconnect, ambiguous dispatch, duplicate/out-of-order - commands/events, worker crash, lease expiry, cancellation, provider failure, - evidence truncation, logical expiry, and cleanup failure preserve one - truthful terminal outcome without provider reattachment or replay. - - SIGKILL during external materialization and before or after provider spawn - forces supervisor-owned runner/process death, credential/staging mount - removal, and replacement orphan recovery before readiness. A child that - calls `setsid` and ignores graceful signals forces termination - unknown/failed, poisoned-worker exit, supervisor boundary destruction, and - replacement orphan recovery; no poisoned worker accepts a next reservation. - - Public plaintext, wrong TLS boundary, private plaintext, wrong certificate/worker identity, and capability replay fail readiness/dispatch; configured TLS, mTLS/equivalent overlay, and same-host Unix socket cases pass. - - Both adapters block model-tool probes of parent/sibling environments, procfs/process listings, known/discovered backend roots, and network secret exfiltration under the OS credential boundary. Environment filtering alone is never accepted as proof, and hostile-source/cross-tenant claims remain rejected. - - End-to-end exporter capture repeats the agent/model/tool/stale-event/error canary and cross-owner probes, proving only bounded allowlisted metadata and opaque owner correlation cross the telemetry boundary while Task/Artifact access and retention remain independent. - - Redirect/DNS-rebinding and unauthorized-resource cases cover every Git and - OCI registry/auth/manifest/blob connection. Secondary Git fetch, OCI tag, - foreign/external layer URL, cross-origin credential forwarding, - layer/manifest mismatch, unregistered or unpinned materializer, undeclared - materializer egress, malicious output manifest, resource exhaustion, - malicious file types/link swaps, repository Pi extensions, and unrestricted - built-ins remain blocked within the documented reviewed-source boundary. - - Same-filesystem publication succeeds by atomic rename; a cross-filesystem - staging root fails readiness and fault injection never observes a partial - final tree or copy fallback. Provenance distinguishes worker-verified and - trusted-service identities from materializer-attested claims. - - Examples validate with production schemas and use only secret variable - names. Docs state the three source modes and exact discriminator, standard - workspace manifest and provenance labels, materializer registry/profile - boundary, worker-derived definition digest, resource authorization and - entitlement-generation cache-revocation boundary, cache-hit/original/current - provider provenance, prohibition on caller-supplied acquisition code, - one selected remote token-minter/lease path with deferred snapshot delivery, - account-pinned sanitized CLI invocation, token versus lease scope, fresh - token and readiness lifetime rules, deterministic source-auth mapping, - same-filesystem publication, supervisor-owned runner cleanup, acquisition - credential teardown, two transport boundaries, one gateway replica, one - execution per worker, reviewed trust domain, narrow credential isolation - versus deferred hostile-code isolation, metadata-only telemetry with its - fixed pre-processor allowlist and separate operator access/retention, - runtime floors, and ephemeral provider sessions. - - Opt-in real-provider smoke tests record backend/runtime and credential-boundary prerequisites, skipping only when a named prerequisite is absent. -- **Verification:** A clean install builds root CLI and private service without raising the CLI engine floor; full suites and docs pass; the official A2A client exercises every advertised operation including the `Submitted -> Working -> Failed` source/setup path; exporter capture proves the telemetry canary/cross-owner contract; the independent checker rejects all-name-present traces with wrong order/fence/multiplicity or forbidden stale dispatch; and release evidence records each available real backend plus explicit skipped prerequisites. +- **Goal:** Run built-in and profile-backed Pi targets through strict RPC with the + same public lifecycle and honest capability reporting. +- **Requirements:** R7, R8, R13, R14, R15, R16; AE3, AE4, AE10, AE12, AE15, + AE17, AE18; KTD9, KTD11, KTD12. +- **Files:** Pi adapter, RPC parser, restricted policy extension, profile-context + and auth bridge, fixtures, conformance and optional credentialed smoke tests. +- **Approach:** Launch Pi with typed invocation configuration, its private + control-process auth view, strict JSONL RPC, explicit allowed tools/extensions, + per-MCP secret views, deterministic permissions, event validation, deadline/ + cancellation escalation, and settled completion. Pi-invoked tools receive no + provider or MCP credentials. Repository extensions and unrestricted built-ins + remain disabled. +- **Execution note:** Reuse the adapter contract exactly; record Pi-specific facts + as bounded native evidence rather than public schema branches. +- **Verification:** Shared adapter conformance, malformed/unknown RPC, deadline, + auth/MCP/tool-secret denial, and an opt-in credentialed smoke case. + +### U7. End-to-end delivery and documentation + +- **Goal:** Prove the built CLI and document the trusted-network operating model, + workspace configuration, credentials, sources, and risks. +- **Requirements:** R1-R18; F1-F5; AE1-AE20. +- **Files:** gateway guide/reference, configuration reference, README, CHANGELOG, + real example project/user workspaces, E2E fixtures, release evidence. +- **Approach:** After the final implementation review is resolved, build the CLI; + create project and user workspaces under `/tmp/`; expose Codex/Pi fixture + targets; serve on loopback and `0.0.0.0`; acquire from local Git and OCI + fixtures; run the official A2A client through negotiation, success, replay, + cancellation, deadline, shutdown, restart, and expiry. Document that network + reachability grants full authority and App/OCI secrets are process inputs, not + YAML. +- **Execution note:** The green smoke test must exercise the same built command + and `/tmp/` workspace shape as the recorded red E2E, not a test-only server. +- **Verification:** `bun run build`, focused and full tests, typecheck, lint, docs + build, schema drift check, and exact red/green E2E commands/results recorded in + the PR description. --- @@ -1532,106 +1011,65 @@ docs/src/content/docs/ | Gate | Applies to | Required evidence | |---|---|---| -| Contract generation | U1 | Exact extension URI/carriers, closed workspace source discriminator and manifest, algorithm-qualified materializer/profile/input/output digest preimages and vectors, verification-method vocabulary, authorization-scope/revocation/App-entitlement fields, cache-hit/original/current-provider provenance, worker request limited to active attempt/fence, controller-derived single-use non-durable lease fields and token-versus-lease scope, exact source-auth code/reason/retryability tuples, both fixed Artifact schemas, original replay bindings, command revisions/tombstones, result-state preservation, and positive/negative fixtures report no drift. | -| Focused unit tests | U1-U7 | Active-unit tests pass with replay ordering, fault injection, state races, unseen cancel, limits, result preservation, failed-quiescence exit, credential probes, account-pinned sanitized CLI execution, fresh-token lifetime boundaries, entitlement-generation authorization/cache revocation, provenance states, exact source-auth failures, and cleanup. | -| Gateway/worker integration | U3-U4, U7-U8 | Built processes agree on authenticated revisioned dispatch, worker identity, command tombstones, leases, direct Git/OCI/registered materialization, deterministic operator-mapped-App-before-account-pinned-CLI eligibility, central fresh App minting without worker issuer material, controller-derived single-use non-durable lease delivery, repository/read/expiry-only token scope, replay/substitution/stale/config-digest rejection, entitlement-generation cache revocation and hit provenance, exact failure mapping, fail-closed selected-App behavior, worker-derived registry digests, acquisition credential teardown, supervisor-owned materializer runners, same-filesystem atomic publication, workspace-manifest provenance labels, Task/Artifact persistence, poisoned exit, orphan recovery, and cleanup. | -| Backend conformance | U5-U8 | One shared suite passes against Codex and Pi, including the versioned schema subset, four result states, valid/invalid preservation across later failure, integrity Artifact carrier, and structured-result Artifact rule. | -| Credentialed provider smoke | U4-U6, U8 | An available centrally held GitHub App, account-pinned trusted-local `gh` login, operator-trusted registered materializer, and each available coding provider mutate disposable immutable workspaces while provider selection follows policy. App acquisition proves `refresh: true`, minimum remaining lifetime, lease-at-or-before-token expiry, repository/read-only/expiry token scope, and no worker issuer material; local CLI smoke proves `--user` and sanitized ambient token variables. Adversarial later-phase probes cannot directly access acquisition/provider credential environments, mounts, processes, roots, or runner control planes and literal canaries remain absent; missing credentials/runtime/boundary capability are recorded as skipped prerequisites. | -| A2A interoperability | U3, U8 | Official `@a2a-js/sdk` client passes required-extension negotiation and legal carriers, immediate/waiting send, stream, reconnect, get, list/filter/page, subscribe, retained replay, cancel races, expiry, and owner isolation without `Task.extensions`. | -| Security and abuse | U2-U4, U7-U8 | Fixtures prove trusted public/private transport and peer identity, auth-before-lookup, retained-claim-first replay, opaque owners, exact source-resource authorization, per-connection Git/OCI SSRF and credential-origin controls, operator-mapped GitHub provider eligibility without identity escalation, central issuer-key custody, worker request limited to active attempt/fence, authoritative current-state derivation, single-use lease replay/substitution/staleness/config-digest rejection, repository/read/expiry-only token scope, fresh-token lifetime safety, fail-closed selected-App errors, account-pinned sanitized CLI use, exact public failure mappings with operator-only identities, webhook/reconciliation-driven entitlement cache revocation, truthful cache-hit provenance, digest-pinned registered materializers, schema/manifest validation, acquisition and provider/tool credential separation, quotas, monotonic cancel/dispatch, failed-quiescence recycling, race-resistant capture, and trust-topology rejection. | -| Lifecycle trace conformance | U8 | The small test-side checker, independently of production selectors, validates attempt/fence correlation, required happens-before edges, maximum occurrence counts, and forbidden post-terminal effects against durable records plus observed worker/process outcomes; all-name-present bad traces fail for wrong order/fence/multiplicity and stale post-terminal dispatch. | -| Telemetry safety | U7-U8 | Exporter capture across agent, model, tool, stale-event, and error spans proves the pre-processor allowlist and bounded redaction exclude prompt/output/tool/source/file content, canary secrets, raw identities, and cross-owner fragments while retaining only bounded operational metadata and opaque owner correlation. | -| Service packaging | U7-U8 | Root Node 18 install, private Node 22 build with focused `@octokit/auth-app` and no full Octokit client, gateway/control-plane lease controller and minter plus supervised-worker smoke, entitlement webhook/reconciler, worker issuer-key exclusion, backend/materializer/source-credential registry readiness, transport and credential readiness, poisoned/crashed worker containment, orphan recovery, and both service container builds pass. | -| Repository quality | All | `bun run schema:check`, `bun run typecheck`, `bun run lint`, and `bun test` pass. | -| Documentation | U8 | `bun run docs:build` passes and examples validate against current schemas. | - -The authoritative behavioral proof is the built-process E2E path with the official A2A client and a separately started worker. Unit tests alone do not prove extension carriers, retained-replay ordering, trusted transport, durable aggregation, monotonic worker commands, credential/process isolation, cancellation, boundary recycling, cleanup integration, or telemetry export safety. The deliberately small independent trace checker supplements that path only by rejecting ordering, fence, multiplicity, and post-terminal-effect violations; it is not a production lifecycle model. - ---- +| Workspace schema | U1 | Project/user parsing, strict nested fields, built-in collision rules, generated-schema drift | +| Public contract | U1-U2 | Official A2A client, every-operation activation, metadata preservation, exact request/result/Artifact/error/canonicalization fixtures | +| Trusted-network model | U2, U7 | Loopback and `0.0.0.0`; shared Task visibility/cancellation; docs warning | +| Durable Task lifecycle | U2, U4 | Private safe state paths, lock, durable claim+Task, no early eviction, store faults, races, restart, atomic expiry | +| Repository acquisition | U3 | Declared-name revision resolution, hermetic Git, full commits, three-way App/`gh` eligibility and sub-budget | +| OCI acquisition | U3 | Declared repository, manifest/layer/workspace digests, safe extraction, no fallback | +| Credential and state isolation | U3-U7 | Separate provider/MCP/tool views; teardown; no cross-scope secrets, operator home, or state root | +| Supervisor lifecycle | U4 | Deadline, shutdown, cancellation races, non-escapable containment, unmanaged recovery, manager handoff, stale-root proof | +| Safe evidence | U4-U6 | Descriptor-relative no-follow reads; links/special files/Git indirection rejected; hermetic Git | +| Backend conformance | U4-U6 | Same suite for fake, Codex, and Pi; profile and built-in variants | +| Structured result | U1, U4-U6 | Exact subset and envelope, valid/invalid/not-produced states, Artifact cardinality, no false publication | +| Repository quality | All | Build, focused/full tests, typecheck, lint, schema check, docs build | +| Built CLI E2E | U7 | Recorded red then green built command under `/tmp/`, both sources, auth isolation, replay/cancel/deadline/shutdown/restart | ## Definition of Done ### Global -- Every R1-R22 requirement is implemented or explicitly shown in a passing conformance scenario. -- The exact required extension is advertised and negotiated through standard Agent Card/header/Message/Artifact surfaces; requests live only at `Message.metadata[uri]`, terminal integrity lives only in the fixed integrity Artifact, and no `Task.extensions` exists. -- Codex and Pi pass the same backend conformance suite, schema subset, and validator. Every terminal Task publishes the integrity Artifact; selected `valid`/`invalid` states survive later failures, and only `valid` publishes the separate fixed structured-result Artifact. -- Gateway and supervised worker run as separate Node 22 processes/images; the Node 18 root CLI does not import service dependencies, and the gateway has no provider runtime or writable repository. -- Authentication and bounded parsing precede owner-scoped retained lookup; identical replay uses stored original bindings before mutable admission, while current authorization/profile/readiness/deadline and quota apply only to atomic new claims. -- Production public ingress uses its named TLS boundary, remote worker routes authenticate and encrypt peers with worker identity/capability binding, and same-host Unix sockets are the only non-network alternative; unprotected remote endpoints fail readiness. -- Cancellation/deadlines use monotonic worker command tombstones and one native abort. Stale dispatch cannot create work, and failed quiescence poisons and exits the worker so supervisor destruction and replacement orphan recovery precede new admission. -- Workspace source validation and exact resource authorization, - per-connection direct Git/OCI controls, trusted-policy GitHub provider - resolution with operator-mapped App precedence, account-pinned sanitized - local-only CLI eligibility, no selected-App failure fallback, central App - private-key custody, controller-derived single-use authenticated lease - delivery, repository/read-only/expiry-only token scope, current-command - recheck and replay/substitution/stale/config-digest rejection, fresh-token and - readiness lifetime bounds, deterministic public source-auth mapping with - operator-only identities, authenticated webhook/reconciliation-driven App - entitlement generations, fail-closed unknown/stale cache authorization, and - separate cache-hit/original-acquisition/current-selection provenance are - enforced end to end. The initial remote path is central token minting and - non-durable lease delivery; versioned snapshot delivery remains deferred. - Worker-derived registered-materializer digests, the standard workspace - manifest and truthful provenance labels, same-filesystem atomic publication, - supervisor-owned runner cleanup, acquisition credential teardown, - OS-enforced provider/tool credential boundary, phase-scoped secrets, disabled - repository Pi extensions/unrestricted built-ins, one-execution - reviewed-domain policy, resource limits, Artifact race defenses, - completeness, provenance, and authenticated expiry are enforced without - accepting caller acquisition code or claiming hostile-source or cross-tenant - isolation. -- Metadata-only telemetry is filtered through the fixed allowlist and bounded redaction before processing/export; canary secrets, content, raw caller identities, and cross-owner fragments never reach exporters, and only opaque owner correlation crosses the separately governed operator boundary. -- Required source/setup and race traces satisfy attempt/fence, happens-before, maximum-count, and forbidden-post-terminal constraints in the independent test-side checker; all-name-present malformed traces fail without introducing a parallel lifecycle implementation. -- Focused tests, full repository gates, built-process smoke, container builds, docs build, and applicable credentialed backend smoke tests have recorded outcomes. -- Public documentation states extension carriers, retained replay, trusted transports, credential versus hostile-code boundaries, metadata-only telemetry and its separate operator access/retention, topology, storage/HA limitation, runtime pins, result preservation, poisoned/crashed-worker recovery, and deferred capabilities. -- Abandoned experiments, unused adapters, compatibility shims, generated scratch files, retained test workspaces, and stale documentation are removed. +- Every R1-R18 requirement is implemented or explicitly demonstrated by a + passing acceptance scenario. +- The gateway starts with no `gateway.yaml` or `worker.yaml`, defaults to + loopback, and accepts explicit `0.0.0.0`. +- Network reachability is the only caller trust boundary; Task visibility and + idempotency are deployment-wide and documented accurately. +- Project workspace declarations own repositories and named OCI snapshot + repositories; user workspace declarations own profile launcher exposure; + built-in target IDs cannot be shadowed. +- The A2A card, every-operation activation header, metadata preservation, strict + request and result-schema grammar, exact error mapping, integrity/produced + Artifacts, canonicalization, retention capacity, and cancellation semantics + pass official-client contract fixtures. +- Repository and OCI modes produce one validated workspace-manifest contract, + never fall back across source modes, and retain truthful provenance. +- GitHub App eligibility/unknown state, acquisition sub-budget, no-installation + `gh` fallback, selected-App failure, OCI auth containment, and pre-provider + source-credential teardown are proven. +- Typed preparation never runs workspace setup shell commands. Built-in and + profile targets authenticate through private provider-control views; every MCP + child is secret-scoped; model tools cannot reach provider/MCP/operator + credentials or gateway state. +- Deadline, cancellation/terminal races, shutdown, result states, safe private + state paths, retention capacity, store failure, descendant quiescence, + managed/unmanaged recovery, safe evidence, and cleanup pass fault tests. +- Evaluation behavior, public-Internet authentication, remote workers, custom + materializers, and multi-tenant policy remain absent. ### Per unit -- U1: Standard extension carriers, closed workspace source/manifest contracts, - algorithm-qualified materializer/profile/input/output digest preimages, - verification and authorization vocabulary, App entitlement generation and - cache provenance states, active-attempt/fence-only credential requests, - controller-derived single-use non-durable lease bindings, token-versus-lease - scope, exact source-auth code/reason/retryability tuples, - integrity/structured-result Artifact schemas, four result states, original - claim digests, command revisions/tombstones, fence rules, typed failures, and - fixtures are generated and stable. -- U2: Trusted ingress, auth, opaque owner isolation, retained-claim-first replay, original bindings, atomic new admission, CAS settlement, pagination, startup recovery, quotas, Artifact access, tombstones, and cleanup pass fault injection. -- U3: Every advertised A2A operation agrees across stream and lookup while extension negotiation, replay ordering, authenticated worker routes, fencing, monotonic cancellation, and races preserve one Task. -- U4: Worker command state, exact source authorization, direct - Git/OCI/registered materialization, operator-mapped GitHub - App-before-account-pinned-CLI eligibility with sanitized invocation and - fail-closed selected-App errors, central fresh App minting without worker - issuer material, authoritative single-use lease delivery with - replay/substitution/stale/config-digest rejection, - repository/read-only/expiry-only token scope and safe lifetime boundaries, - deterministic public failure mapping with operator-only identities, - webhook/reconciliation-driven entitlement cache revocation and truthful hit - provenance, workspace-manifest validation and provenance classification, - same-filesystem atomic publication, acquisition and provider credential - separation, supervisor-owned runner cleanup, poisoned-exit/orphan recovery, - and dispatch/materialization/setup/action/check/quiescence/evidence/cleanup - pass malicious, crashed, and faulted scenarios. -- U5: Codex direct-SDK streaming, schema/signal forwarding, validated output, result preservation, OS credential separation, native evidence, fresh threads, cancellation, and failure mapping pass adapter and applicable smoke verification. -- U6: Pi strict RPC/framing, terminating result, exact policy tools, disabled repository extensions/built-ins, OS-isolated credential store/provider runtime, result preservation, settlement, stats, abort, and process cleanup pass verification. -- U7: Closed backend, materializer, and source-credential registries, focused - `@octokit/auth-app` packaging without a full Octokit or root-CLI dependency, - authoritative lease controller, central App private-key custody and fresh - minter readiness, entitlement webhook/reconciler, account-pinned sanitized - local CLI, trusted transport/identity/readiness, acquisition/provider - credential and supervisor capability gating, poisoned-worker recycling, - metadata-only pre-export telemetry controls, Node-version separation, - tracing, shutdown, containers, and release artifacts work from built outputs. -- U8: Cross-backend E2E, all three workspace source modes, central GitHub App - and account-pinned trusted-local CLI credential-selection cases, remote - issuer-key exclusion, controller-derived single-use lease delivery, - fresh-token lifetime, token-versus-lease scope, exact failure mappings, - entitlement-driven cache invalidation and hit provenance, standard manifest - provenance, acquisition credential teardown, standard A2A carriers, retained - replay, selected materialization/setup transitions, independent race-trace - constraints, telemetry canary/cross-owner probes, transport and credential - abuse cases, command/quiescence races, examples, operator docs, changelog, - and release evidence are complete. +- U1: Runtime and generated schemas agree; invalid negotiation, request, + source/exposure/collision/configuration fixtures fail at expected paths. +- U2: Official A2A operations, global replay/visibility, project locks, store + faults, listeners, deadline, shutdown, restart, and retention pass. +- U3: Git and OCI fixtures pass; three-way provider eligibility, token lifetime, + and all no-fallback rules are observed; leak scans are clean. +- U4: Fake-adapter lifecycle proves terminal monotonicity, typed preparation, + isolation, containment, bounded/safe evidence, deadline/cancellation/shutdown, + poisoning, and cleanup. +- U5: Codex passes shared conformance and optional credentialed smoke evidence is + recorded when credentials exist. +- U6: Pi passes the same conformance and malformed RPC cannot produce success. +- U7: Final review is resolved; built CLI red/green E2E under `/tmp/`, complete + repository gates, schemas, docs, and reproducible PR instructions are complete. diff --git a/docs/research/agent-host-protocol-decision-inputs.md b/docs/research/agent-host-protocol-decision-inputs.md index 113ea6fc..e49d39f8 100644 --- a/docs/research/agent-host-protocol-decision-inputs.md +++ b/docs/research/agent-host-protocol-decision-inputs.md @@ -7,9 +7,9 @@ northbound contract. Treat the Agent Host Protocol (AHP) as an optional future protocol behind the gateway for a compatible backend or beside it for a collaborative session client. -AHP does not replace ADR 0002's Task identity, caller-scoped idempotency, -authorization, immutable source handling, cleanup, terminal evidence, or -bounded result retention. +AHP does not replace ADR 0002's deployment-wide Task identity and idempotency, +network trust boundary, immutable source handling, cleanup, terminal evidence, +or bounded result retention. The initial backend set is Codex and Pi; OpenCode is deferred. They are peer execution adapters behind one conformance contract; provider-specific process, @@ -27,14 +27,14 @@ source inspection, and full protocol comparison live in the AI Research Wiki: | Concern | AllAgents A2A gateway | AHP host/session layer | |---|---|---| -| Northbound consumer | AI Evals and future remote execution clients | IDE, browser, CLI, or collaborative operator client | +| Northbound consumer | AI Evals and future trusted-network execution clients | IDE, browser, CLI, or collaborative operator client | | Primary lifecycle | One addressable Task per accepted execution | Long-running session/chat with shared clients | | Public identity | Agent Card, Message, Task, Artifact, invocation key | Host, client, channel, session, chat, turn, tool call | -| State | Task status, messages, artifacts, retention | Snapshots, ordered actions, reducers, reconnect | -| Authorization | Authenticate/authorize service caller | Endpoint/resource auth and tool confirmation | +| State | Deployment-wide Task status, messages, artifacts, retention | Snapshots, ordered actions, reducers, reconnect | +| Authorization | Network reachability; no application caller identity | Endpoint/resource auth and tool confirmation | | Cancellation | Cancel Task, abort backend, terminate, clean up, report terminal outcome | Cancel interactive turn and call provider-native abort | | Evidence | Source, output, usage/cost, traces, file changes, artifacts, failures, cleanup, completeness, provenance | Live changesets and provider/session state | -| Isolation | Selected worker/backend boundary | Not supplied by the shared host process | +| Isolation | Single-process supervisor with invocation-owned child containment | Not supplied by the shared host process | The identities must be correlated rather than reused. At minimum retain the A2A Task ID, AllAgents invocation key, backend execution/session ID, @@ -45,16 +45,17 @@ provider-native thread/chat ID, and trace ID. 1. Define one narrow backend adapter contract for create/invoke, progress, permission decisions, cancellation, terminalization, evidence collection, shutdown, native evidence passthrough, and explicit capabilities. -2. Keep gateway responsibilities separate from worker/backend responsibilities. - The gateway owns caller authorization, Task/idempotency identity, backend - selection, normalized results, cancellation propagation, and retention. - Workers own source materialization, provider processes, mutable workspaces, - evidence capture, process termination, and cleanup. +2. Keep A2A and backend responsibilities separate inside one gateway service. + The A2A layer owns deployment-wide Task/idempotency identity, backend + selection, normalized results, cancellation propagation, and retention. The + invocation supervisor owns source acquisition, provider child processes, + mutable workspaces, evidence capture, process termination, and cleanup. 3. Propagate `CancelTask` and deadlines through the adapter to the provider-native abort primitive, then persist terminal status and cleanup outcome. Transport closure is not cancellation. -4. Separate caller authorization, execution permission policy, and - provider/resource credentials. +4. Separate network authorization, execution permission policy, and + provider/resource credentials. The initial gateway has no application caller + identity. 5. Combine normalized file operations with bounded provider-native diffs/checkpoints/trajectories. Declare attribution limits and incompleteness rather than treating the final working-tree diff as exact @@ -76,6 +77,8 @@ provider-native thread/chat ID, and trace ID. - Active-session reconnection beyond A2A Task lookup, subscription, and terminal result retrieval. - AHP local endpoint discovery, SSH host selection, and tunnel multiplexing. +- Remote worker ownership, routing, and session transport until the initial + single-process gateway needs a separate execution host. - Generic changeset review/operation state. - Long-lived session/chat catalogs and provider-native session adoption. diff --git a/docs/research/harbor-repository-materialization.md b/docs/research/harbor-repository-materialization.md index d4708d4c..38b6be3f 100644 --- a/docs/research/harbor-repository-materialization.md +++ b/docs/research/harbor-repository-materialization.md @@ -13,10 +13,10 @@ repository the agent edits is therefore benchmark- and task-owned: it may be bak an image, cloned by a Dockerfile, copied as task content, or otherwise prepared by the task author. -For AllAgents, repository and workspace provenance must remain explicit in the public -execution request and terminal evidence. Custom acquisition should be an -operator-registered, digest-pinned materializer behind the worker protocol, not an -arbitrary caller-supplied image or setup script. +For AllAgents, repository and workspace provenance must remain explicit in the +public execution request and terminal evidence. The initial gateway supports +only declared Git repositories and named digest-pinned OCI workspace snapshots; +custom materializers remain deferred. ## What Harbor fetches @@ -34,9 +34,11 @@ This is efficient for a large repository containing many independent Harbor task is not a mechanism for assembling several application repositories into one agent workspace. -Harbor also accepts an omitted commit or a mutable ref and resolves it to a commit. -That is convenient for an interactive local benchmark CLI, but it is weaker than the -AllAgents gateway requirement that an accepted request already name immutable source. +Harbor also accepts an omitted commit or a mutable ref and resolves it to a +commit. AllAgents permits a caller to override a declared repository with a +branch, tag, or commit for developer convenience, but resolves and records the +full commit before provider execution. Reproducibility-sensitive callers use a +full commit; OCI snapshots remain digest-pinned at admission. ### Task packages from the package registry @@ -77,10 +79,10 @@ sets that as `WORKDIR`; Harbor itself never clones that application repository. 1. **Separate descriptor acquisition from execution.** Resolve and validate immutable inputs before starting the coding-agent runtime. -2. **Use content-addressed caches.** Key reusable workspace snapshots by a digest of - normalized source identities, materializer version/digest, setup policy, current - authorization scope, and revocation epoch rather than a mutable name. Reauthorize - before lookup and make an old epoch ineligible after revocation. +2. **Use content-addressed snapshot caches.** Key reusable OCI workspace + snapshots by their immutable OCI and workspace-manifest digests. Direct Git + mode resolves revisions independently and records the resulting commits. + Reauthorize every remote acquisition. 3. **Avoid downloading irrelevant content.** For Git-backed descriptor catalogs, Harbor's tree-only discovery and sparse checkout are sound optimizations. For an application repository, use partial/shallow acquisition only when it preserves the @@ -93,43 +95,40 @@ sets that as `WORKDIR`; Harbor itself never clones that application repository. ### Adapt -Keep a first-class workspace manifest instead of hiding source inside an environment -image. Each materialized repository should retain at least: +Keep a first-class workspace manifest instead of hiding source inside an +environment image. Each materialized repository or snapshot should retain at +least: -- canonical source URL or snapshot identity; -- requested and resolved immutable commit or OCI digest; +- canonical source URL or configured snapshot identity; +- requested revision and resolved commit, or OCI manifest digest; - destination path and optional source subdirectory; -- materializer identity and version/digest; -- resulting tree/content identity; -- cache hit/miss and completeness facts. - -Use three explicit source modes: - -1. **Direct Git repositories** for the normal case, each with an exact commit and - collision-free destination. -2. **OCI workspace snapshots** for large, preassembled workspaces, referenced by digest - rather than tag and accompanied by a signed/validated workspace manifest. -3. **Operator-registered materializers** for JFrog, unusual monorepos, generated source, - or organization-specific setup. A request selects a configured materializer ID, - pins the expected workspace-manifest digest, and supplies validated, - resource-authorized structured inputs. The operator configuration pins the builder - image by digest, the worker derives the non-secret definition digest, credentials are - scoped only to materialization, and the builder must produce the standard workspace - manifest before the agent starts. The builder is operator-trusted deployment code; - deployments that cannot grant that trust need a broker or stronger acquisition - service. - -This retains Harbor's useful task-owned flexibility without allowing a caller to choose -an arbitrary executable image or shell script inside the trusted worker. +- acquisition implementation identity; +- resulting tree/content identity; and +- completeness and verification-versus-attestation facts. + +Use exactly two initial source modes: + +1. **Direct declared Git repositories** for the normal case. A request selects + configured repository names and may override only their revisions. The + gateway resolves and records full commits and enforces collision-free + destinations. +2. **Named OCI workspace snapshots** for large, preassembled workspaces. The + project workspace declares the repository; the request supplies immutable + OCI and workspace-manifest digests. + +Both modes produce the same standard workspace manifest. Neither mode falls +through to the other after admission. ### Do not copy -- Mutable Git refs, `HEAD`, image tags, or package `latest` as accepted execution - identities. -- Harbor's broad Git transport set (`http`, `ssh`, and `git` as well as HTTPS) at a - remote service boundary. The gateway should keep canonical credential-free HTTPS, - destination-policy revalidation, disabled redirects/helpers/filters/hooks/submodules, - and exact commit verification. +- Unresolved mutable Git refs as terminal execution identities. Branch and tag + overrides are valid only when the gateway resolves and records a full commit + before provider execution. +- Mutable OCI tags or package `latest` as accepted snapshot identities. +- Harbor's broad Git transport set (`http`, `ssh`, and `git` as well as HTTPS) at + a service boundary. The gateway keeps canonical credential-free HTTPS, + destination-policy revalidation, disabled redirects/helpers/filters/hooks/ + submodules, and full-commit verification. - A non-fatal Git LFS miss. If declared workspace content cannot be materialized, preparation must fail before provider execution. - Hashing a prebuilt image reference string as environment identity. Resolve and pin @@ -143,25 +142,31 @@ an arbitrary executable image or shell script inside the trusted worker. ## Recommended boundary -The worker should execute a dedicated materialization phase before any harness starts: - -1. Validate the normalized workspace request, exact source-resource authorization, - configured materializer, and current authorization scope before any cache lookup. -2. Resolve phase-scoped source credentials without exposing them to setup, the model, - or later evidence. -3. Populate a worker-owned staging directory on the final publication filesystem or - pull and unpack a digest-pinned workspace snapshot there. -4. Verify repository commits, paths, limits, content, the expected manifest digest, - and the standard workspace manifest; distinguish worker-verified identities from - materializer-attested claims. -5. Stop the acquisition process, revoke credentials, remove its mounts and runner - resource, and retain only the validated host-owned staging tree. +The gateway supervisor executes a dedicated acquisition phase before any +provider starts: + +1. Validate the normalized source request and its declared repository or + snapshot identities before any network access. +2. Resolve phase-scoped source credentials without exposing them to typed + provider preparation, the model, or later evidence collection. +3. Populate a gateway-owned staging directory on the final publication + filesystem, or pull and unpack a digest-pinned workspace snapshot there. +4. Verify repository commits, paths, limits, content, the expected manifest + digest, and the standard workspace manifest; distinguish gateway-verified + identities from snapshot-attested claims. +5. Stop acquisition processes, revoke credentials, remove helpers and mounts, + and retain only the validated credential-free staging tree. 6. Atomically rename that tree into the final workspace, record provenance, run - operator-owned setup, record the post-setup baseline, and only then launch the - harness-specific worker runtime. + adapter-owned typed preparation, record the baseline, and only then launch + the provider runtime. Project or user `setup` shell commands are not run. + +Operator-registered materializers, custom builders, and third source variants +are deferred until direct Git and OCI snapshots cannot satisfy a demonstrated +deployment need. Adding one requires a new decision for trust, configuration, +credential, provenance, and isolation boundaries. -The practical conclusion is narrow: Harbor is strong evidence for content-addressed -input bundles and environment-provider indirection. It is not evidence for making +The practical conclusion is narrow: Harbor is strong evidence for content- +addressed input bundles and staged publication. It is not evidence for making repository acquisition opaque or task-defined in the AllAgents public contract. ## Primary sources diff --git a/docs/research/source-credential-broker-precedents.md b/docs/research/source-credential-broker-precedents.md index b1be9054..f2c69f5b 100644 --- a/docs/research/source-credential-broker-precedents.md +++ b/docs/research/source-credential-broker-precedents.md @@ -2,29 +2,29 @@ ## Decision -The execution gateway does **not** need a mandatory standalone Git credential -broker for trusted local use. The settled local provider is an explicit, -account-pinned `gh auth token --hostname --user ` helper invoked -only when no App installation mapping applies. Its environment removes -`GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and -`GITHUB_ENTERPRISE_TOKEN`, and its token is exposed only to the one-shot -acquisition process. Git credential helpers and Git Credential Manager (GCM) -establish the process-boundary precedent, but arbitrary configured helpers are -not part of the selected implementation. A local helper is a broker in the -security sense; it is not a separately deployed network service. - -Remote or multi-tenant workers use one initial path: an authoritative -gateway/control-plane lease controller and trusted central token minter deliver -a fresh GitHub token over an authenticated, single-use, non-durable lease. The -GitHub bearer token is scoped only to the repository, read-only contents -permission, and GitHub expiry. Worker identity, attempt, lease epoch, command -revision, fence, operation, and delivery expiry are properties of the lease and -channel, not the token. Workers never inherit a person's credential helper, -credential store, SSH agent, or the App private key. A versioned central -snapshot-delivery protocol is deferred; it is not an alternative initial -readiness path. The minter may live inside the trusted control plane unless -private-key isolation, audit, scaling, or blast-radius requirements justify a -separate service process. +The execution gateway does **not** need a standalone Git credential broker for +the initial trusted-network deployment. It supports two in-process trusted +providers for `github.com`: a configured GitHub App and a configured, +account-pinned `gh auth token --hostname github.com --user ` fallback. + +The App is preferred whenever an App-authenticated repository-coverage check +proves an installation eligible. `gh` is considered only when the App is absent +or coverage is positively ineligible; unknown discovery, authentication, +permission, rate-limit, or service failures fail closed. Ambient `GH_TOKEN`, +`GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` are removed +from the CLI helper environment. + +Either token is exposed only to the one-shot acquisition process through an +invocation-scoped Git credential helper. The helper, token, and acquisition +process are gone before adapter preparation or provider execution. Git +credential helpers and Git Credential Manager establish the process-boundary +precedent, but arbitrary configured helpers are not part of the selected +implementation. A local helper is a broker in the security sense; it is not a +separately deployed network service. + +Central token minters, authenticated delivery leases, remote workers, and +multi-tenant credential policy are deferred until ADR 0002's deployment +boundary is reconsidered. ## Precedents @@ -65,12 +65,13 @@ local process/socket boundary, not a remotely reachable credential service. **Relevance.** Git helpers and GCM prove that a local credential provider can be an on-demand process rather than a network service. AllAgents does not, however, inherit or invoke an arbitrary configured helper chain. Its closed -provider registry permits only an explicit GitHub CLI provider pinned to a -configured non-secret account in a trusted-local profile, and only when no -configured GitHub App installation mapping applies. The helper invokes -`gh auth token --hostname --user ` without ambient GitHub token -variables. Its output reaches only the one-shot acquisition child; setup and -the coding harness inherit neither helper configuration nor the token. +provider registry permits only the selected GitHub App token or an explicit +GitHub CLI provider pinned to a configured non-secret account when App +eligibility is positively absent. The CLI invokes +`gh auth token --hostname github.com --user ` without ambient GitHub +token variables. Its output reaches only the one-shot acquisition child; +adapter preparation and the coding runtime inherit neither helper configuration +nor token. ### SSH agent forwarding @@ -136,11 +137,11 @@ long-term credential store, and its post-job deletion is defense in depth rather than the token's revocation mechanism. **Relevance.** This is the closest production precedent for AllAgents: keep the -App private key at a trusted central minter, issue one fresh least-privilege +App private key in the trusted gateway process, issue one fresh least-privilege token for a particular repository acquisition, expose it only during that phase, and remove its local material afterward. GitHub enforces repository, -read-only contents permission, and expiry; AllAgents separately enforces -attempt and operation bindings through its authenticated delivery lease. +read-only contents permission, and expiry; the gateway separately binds the +acquisition to the retained Task and effective configuration digest. ### BuildKit secret and SSH mounts @@ -179,82 +180,39 @@ credentials and does not eliminate the need for a central issuer in production. ## Recommendation for AllAgents -### Local mode - -1. Resolve `github.com` through the built-in GitHub backend and require explicit - host/API mappings for GitHub Enterprise Server hostnames. -2. Prefer a configured GitHub App installation that trusted operator policy - maps to the authorized repository. Do not use `@octokit/auth-app` to discover - installations. If no installation mapping applies, a trusted-local profile - may invoke the explicit - `gh auth token --hostname --user ` provider pinned to a - configured non-secret account. Include that account in the entitlement and - effective-profile digests, remove `GH_TOKEN`, `GITHUB_TOKEN`, - `GH_ENTERPRISE_TOKEN`, and `GITHUB_ENTERPRISE_TOKEN` from the helper - environment, and fail if the configured account cannot be resolved. Do not - inherit an arbitrary Git helper/GCM chain or forward an SSH agent. -3. Treat provider order as eligibility, not retry. Once the App provider is - selected, configuration, authentication, minting, authorization, rate-limit, - or service failure terminates acquisition without falling through to the - user identity. -4. Give the resolved token only to the dedicated acquisition subprocess through - a temporary helper channel, remove that channel, terminate the child, and - publish only a credential-free verified workspace before setup or the coding - harness starts. -5. Do **not** require or auto-start an AllAgents network credential service for - trusted local execution. The explicit account-pinned provider subprocess is - sufficient. - -### Production remote or multi-tenant workers - -1. Put GitHub App issuer material in a trusted central token-minter component. - Trusted operator configuration, not auth-app discovery, maps the repository - to an installation ID. For every cache-miss acquisition, use focused - [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) with - `refresh: true` to bypass its installation-token cache and mint a fresh token - narrowed to that repository and read-only contents permission. Require - remaining lifetime strictly greater than the acquisition deadline plus - clock-skew margin, expire the delivery lease no later than the token, and - fail readiness when the configured acquisition ceiling can exceed a fresh - token's safe lifetime. -2. Make the gateway/control-plane credential-lease controller authoritative. - The authenticated worker requests only by active attempt and fence. From - durable dispatch and policy state, the controller derives the - effective-profile digest, selected provider, host/API-mapping digest, - installation ID, repository, operation, worker route and identity, lease - epoch, command revision, and expiry. Immediately before issuance it rechecks - active command revision, tombstone, fence, and lease state. -3. Deliver one single-use, non-durable grant/response over the authenticated - acquisition channel. A separate minter must agree with the controller's - configuration digest and consume the grant atomically. Reject replay, - substituted fields or providers, stale command state, and configuration - disagreement. The bearer token itself remains scoped only by GitHub to the - repository, read-only contents permission, and expiry; worker, attempt, - fence, and operation bindings belong to the lease. -4. Advance a GitHub App entitlement generation from authenticated lifecycle - webhooks plus bounded reconciliation whenever an installation is uninstalled, - suspended, or changes repository selection. Unknown or stale installation - state fails cache authorization. Mint only on a cache miss. On a miss, record - the acquiring provider in operator provenance; on a hit, record `cache_hit`, - the cached original acquisition-provider metadata, and current policy - selection/entitlement binding separately. -5. Publish deterministic coarse failures: `source_auth_unavailable` / - `no_eligible_provider` (not retryable); `source_auth_denied` / - `installation_repository_denied` (not retryable); - `source_auth_failed` with `app_configuration_invalid`, - `app_authentication_failed`, or `app_mint_failed` (not retryable), - `provider_rate_limited` or `provider_unavailable` (retryable), or - `trusted_local_cli_failed` (not retryable). Keep provider, installation, and - account identifiers in operator-only provenance. -6. Never forward an operator's general SSH agent or reuse their desktop GCM - store in a remote worker. Those capabilities represent the person, not the - individual execution request. -7. Keep minting logically central even if it initially lives inside the trusted - gateway process. Split the minter into a standalone network service when - remote trust boundaries, private-key isolation, audit, scaling, or - blast-radius controls require it. A versioned central snapshot-delivery - protocol may be designed later, but is not part of the initial architecture. - -The resulting rule is: **local reuse may be subprocess-mediated; production -issuance must be centrally policy-mediated.** A process boundary is required in -both cases, but a standalone credential service is not. +### Initial trusted-network gateway + +1. Resolve only canonical `github.com` HTTPS origins in the initial delivery. +2. Determine App applicability through an App-authenticated GitHub API client, + or verify an explicitly configured installation ID against the repository. + Model the result as `eligible`, `ineligible`, or `unknown`. +3. For `eligible`, use focused + [`@octokit/auth-app`](https://github.com/octokit/auth-app.js) authentication + and mint a fresh token narrowed to the repository and read-only contents. + Require remaining lifetime greater than the gateway's at-most-900-second + acquisition sub-budget plus a 60-second clock-skew margin. +4. For a missing App or proven `ineligible`, a trusted local deployment may use + the configured `gh auth token --hostname github.com --user ` + provider. Include the account in the acquisition-policy digest and strip + ambient token variables. An `unknown` App result never falls through. +5. Treat provider order as eligibility, not retry. Once App is selected, + configuration, authentication, minting, authorization, repository coverage, + rate-limit, or service failure terminates acquisition. +6. Give the resolved token only to the dedicated acquisition subprocess through + a temporary helper channel. Remove the channel and terminate the process + before atomically publishing the credential-free verified workspace. +7. Do not require or auto-start a network credential service. Keep App issuer + material and GitHub/OCI auth stores inaccessible to the adapter process and + model-invoked tools. + +### Deferred remote or multi-tenant deployment + +A future deployment may require a central token minter, authenticated single-use +delivery leases, entitlement generations, revocation reconciliation, worker +identity, fencing, and a snapshot-delivery protocol. Those mechanisms are not +part of the selected single-process architecture. They require a separate +decision when remote workers or tenant isolation become product requirements. + +The resulting initial rule is: **credential reuse is acquisition-subprocess- +mediated and ends before provider execution.** Remote or multi-tenant issuance +policy remains deferred; a standalone credential service is not required now. From 25a45f54ad69de63bfed7e98ecccd3624635821c Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sat, 19 Sep 2026 13:15:52 +1000 Subject: [PATCH 09/12] docs(architecture): define Promptfoo gateway consumption --- ...-agent-execution-through-an-a2a-gateway.md | 96 +++++++- ...0837-feat-coding-execution-gateway-plan.md | 206 ++++++++++++++++-- 2 files changed, 271 insertions(+), 31 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 73afb298..b66c36c5 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -165,12 +165,14 @@ a `sha256:` workspace-manifest digest. The gateway constructs the full OCI reference server-side. Callers cannot supply a registry host, repository name, mutable tag, extraction destination, credential, or external-layer policy. -Both modes produce the same versioned workspace manifest. It records requested -and resolved repository identities, destinations, acquisition kind, relevant -OCI manifest and layer digests, the workspace-manifest digest, completeness, -and whether each fact was independently verified or snapshot-attested. A commit -listed inside an OCI snapshot is not described as independently verified unless -the gateway separately verifies it against its Git remote. +Both modes produce the same versioned, wire-visible workspace manifest. It +records declared logical repository names, requested revisions, resolved +commits, acquisition kind, relevant OCI manifest and layer digests, the +workspace-manifest digest, completeness, and whether each fact was independently +verified or snapshot-attested. It omits Git URLs, OCI repository origins, and +destination paths. A commit listed inside an OCI snapshot is not described as +independently verified unless the gateway separately verifies it against its +Git remote. Acquisition occurs in a gateway-owned staging directory. The gateway validates paths, collisions, file types, symlinks, layer and file counts, individual and @@ -179,6 +181,76 @@ the invocation workspace. Absolute paths, traversal, device files, sockets, escaping links, foreign or external OCI layers, and cross-origin credential forwarding are rejected. +### Consume the gateway from Promptfoo through an AI Evals provider + +Rejecting caller-supplied origins does not prevent AI Evals from selecting a +workspace in Promptfoo YAML. The two files have different ownership: + +- the AllAgents project workspace is the operator-controlled catalog that maps + repository and snapshot names to Git URLs, destinations, and OCI repositories; +- the Promptfoo configuration selects a target and source mode. Repository mode + materializes the complete configured repository set and may override + revisions by declared repository name. Snapshot mode selects one declared + snapshot name and supplies immutable digests. + +AI Evals owns a Promptfoo +[custom JavaScript/TypeScript provider](https://www.promptfoo.dev/docs/providers/custom-api/). +It implements `ApiProvider`: its constructor receives `ProviderOptions`, retains +`options.id`, validates `options.config`, and exposes `id()`. +`callApi(prompt, context, options)` reads bounded test variables from +`context.vars` and cancellation from `options?.abortSignal`. The provider +translates one `callApi` into one A2A Task: it creates an invocation key, puts +the prompt in the single `TextPart`, puts the target and closed source union in +the required extension metadata, waits or streams to terminal, and returns +output, normalized token usage, and logical provenance in Promptfoo's +`ProviderResponse`. + +For example, AI Evals can define two provider instances without sending either +origin over the wire: + +```yaml +providers: + - id: file://./providers/allagents-a2a.ts + label: codex-direct + config: + endpoint: http://allagents-gateway.tailnet:4732 + target: codex + source: + kind: repositories + revisions: + allagents: 0123456789abcdef0123456789abcdef01234567 + + - id: file://./providers/allagents-a2a.ts + label: codex-evaluation-snapshot + config: + endpoint: http://allagents-gateway.tailnet:4732 + target: codex + source: + kind: workspaceSnapshot + snapshot: evaluation + digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef + workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 +``` + +The first provider materializes the complete configured repository set and uses +the `allagents` key only to override that repository's revision. The second +provider's `evaluation` key resolves to the declared +`ghcr.io/entityprocess/allagents-workspaces` repository. + +Static provider config fixes the source kind and logical names. The only +per-test object is `context.vars.allagentsSource`: repository mode accepts +revision overrides only for statically listed names and only as full lowercase +40-hex commits; snapshot mode accepts only replacement OCI and workspace- +manifest `sha256:` digests. Missing leaves retain static values. A URL, +destination, mutable revision, credential, command, unknown member, or changed +source kind/name fails before submission. After Task acceptance, the provider's +bounded deadline or `options?.abortSignal` sends `CancelTask`. It maps gateway +input, output, cached-input, and total token counts to Promptfoo's `prompt`, +`completion`, `cached`, and `total` fields respectively; other usage and Task/ +Artifact evidence stays in metadata without origins. The provider belongs in AI +Evals. AllAgents exposes the A2A contract and consumer documentation without +taking a runtime dependency on Promptfoo. + ### Resolve GitHub credentials with App-first eligibility fallback The source request is credential-free and never selects a credential provider. @@ -315,7 +387,8 @@ Terminal evidence distinguishes: - agent output; - optional validated structured result; -- requested and resolved repository or OCI identities; +- logical repository names, requested revisions, resolved commits, or snapshot + names and digests, never source origins or destinations; - pre- and post-execution Git state where applicable; - produced artifacts; - usage and bounded provider-native evidence; @@ -367,6 +440,10 @@ retry. Consumers own those concerns. Kubernetes deployment in the initial product. - Project and user workspace files remain the sole declaration authority for source identities and exposed profile launchers. +- AI Evals can express the configured repository set with named revision + overrides, or select a prebuilt image through a snapshot handle, in Promptfoo + YAML. Its custom provider translates that closed source choice to A2A and + keeps raw origins under AllAgents operator control. - Network reachability grants access to every exposed target and retained Task. Operators must treat network policy as the authorization boundary. - GitHub App credentials support private repositories without forcing every @@ -412,8 +489,9 @@ never control commands or argv. ### Let callers provide repository URLs or OCI repositories Rejected because workspace configuration already defines trusted source -identities and destinations. Requests may select declared names and immutable -revisions or digests, not introduce new origins. +identities and destinations. Repository requests materialize the configured set +and may override revisions by declared name; snapshot requests select a declared +name and immutable digests. Neither variant introduces a new origin. ### Fall back from a selected GitHub App after runtime failure diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 4793ba0c..07ec97e9 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -15,12 +15,14 @@ execution: code - **Objective:** A developer can run one trusted-network A2A endpoint for one AllAgents workspace and invoke built-in or explicitly exposed profile targets - against either declared Git repositories or a digest-pinned OCI workspace - snapshot. + against either the complete configured Git repository set, with optional + named revision overrides, or a digest-pinned OCI workspace snapshot. AI Evals + can configure either source mode in Promptfoo YAML through a custom provider + without sending origins. - **Means:** Add `allagents gateway serve`, a private execution-service package, a bounded durable Task store, direct Codex and Pi adapters, GitHub App and - GitHub CLI acquisition providers, OCI snapshot acquisition, and one supervised - invocation lifecycle. + GitHub CLI acquisition providers, OCI snapshot acquisition, one supervised + invocation lifecycle, and a documented Promptfoo provider contract. - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) owns the public and trust boundaries. Project and user `workspace.yaml` files own source and profile declarations. A2A 1.0 owns core wire semantics. @@ -48,7 +50,10 @@ evaluation framework or multi-tenant platform. Callers use A2A Tasks and one required AllAgents extension. Network reachability is authorization. The service resolves configured targets and sources from existing workspace files, acquires a fresh invocation workspace, invokes Codex or Pi through a typed -adapter, and retains bounded terminal evidence. +adapter, and retains bounded terminal evidence. AI Evals consumes that boundary +through its own Promptfoo custom provider: evaluation YAML supplies named +revision overrides for the configured repository set, or one snapshot handle +and immutable digests, while AllAgents retains origin and credential authority. ### Problem Frame @@ -65,7 +70,8 @@ registry, or another profile configuration file for the initial use case. ### Actors - A1. **Trusted-network caller:** Any process able to reach the endpoint. All - callers have the same authority and Task visibility. + callers have the same authority and Task visibility. The first caller is an + AI Evals-owned Promptfoo custom provider that maps one `callApi` to one Task. - A2. **Execution gateway:** The A2A server and invocation supervisor. It owns deployment-wide Task identity, acquisition, routing, status, cancellation, evidence, retention, and cleanup. @@ -103,6 +109,10 @@ registry, or another profile configuration file for the initial use case. Governs R5, R13-R16. - **Keep evaluation outside AllAgents.** Consumers own datasets, repetitions, scoring, assertions, and evaluation Runs. Governs R17. +- **Bridge Promptfoo at the consumer boundary.** AI Evals owns a custom provider + that maps Promptfoo YAML and test variables to the closed A2A source modes and + maps terminal Tasks back to `ProviderResponse`. AllAgents owns no Promptfoo + runtime behavior. Governs R19. ### Requirements @@ -183,11 +193,14 @@ registry, or another profile configuration file for the initial use case. `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, workspaceManifestDigest: Digest, repositories }`. `repositories` contains 1-64 unique strict entries - `{ name: ConfigName, canonicalUrl: string, requestedRevision?: - RevisionText, resolvedCommit: string, verification: - "independentlyVerified" | "snapshotAttested" }`; `canonicalUrl` is a - canonical HTTPS URL of at most 2048 bytes, and `resolvedCommit` matches - `^[0-9a-f]{40}$`. + `{ name: ConfigName, requestedRevision?: RevisionText, + resolvedCommit: string, verification: + "independentlyVerified" | "snapshotAttested" }`; `resolvedCommit` matches + `^[0-9a-f]{40}$`. Gateway-generated source identity, workspace-manifest + fields, evidence metadata, and provider-added metadata never contain Git + URLs, OCI repository origins, or destination paths. This guarantee does not + inspect or sanitize opaque caller prompts, provider terminal output, or + produced-Artifact payloads. - optional `workspaceManifestDigest` is `Digest`. - `terminalOutput` is `{ text, truncated }`, where `text` is valid UTF-8 of at most 1 MiB and `truncated` is boolean. @@ -382,6 +395,28 @@ registry, or another profile configuration file for the initial use case. for listener, workspace, state/retention, GitHub, OCI, and Codex/Pi auth-file handles. Secret values never enter workspace files, requests, logs, Tasks, Artifacts, retained workspaces, or model-invoked tool environments. +- R19. Document AI Evals consumption through a Promptfoo custom + JavaScript/TypeScript provider implementing Promptfoo's `ApiProvider`. + `constructor(options: ProviderOptions)` retains `options.id`, validates + `options.config`, and `id()` returns the retained ID. Static config contains + the gateway endpoint, target ID, and exactly one closed source mode: + repository mode materializes the complete configured repository set and + carries only an optional revision map keyed by declared repository name; + snapshot mode carries one declared snapshot name with OCI and workspace- + manifest digests. `callApi(prompt, context, options)` may apply the exact + `context.vars.allagentsSource` leaf overrides defined below. Dynamic + repository revisions must be full lowercase 40-hex commit IDs; dynamic + snapshot values must be full lowercase `sha256:` digests. Source kind, + snapshot name, and repository origins never vary per test. Unknown members, + revision names absent from static config, URLs, destinations, tags, + credentials, commands, and permission policy fail before submission. + `options?.abortSignal` and the provider's bounded deadline both invoke A2A + `CancelTask` after acceptance. One `callApi` creates one A2A Task and maps + terminal output, usage, Task/Artifact IDs, structured result, and logical + provenance into `ProviderResponse`; admission or terminal failure maps to + `error`. AI Evals owns the provider implementation. AllAgents publishes the + protocol and YAML examples without importing Promptfoo provider code or adding + Promptfoo as a runtime dependency. ### Key Flows @@ -437,6 +472,24 @@ registry, or another profile configuration file for the initial use case. printing the platform recovery command. Managed mode may exit only after its validated external manager accepts containment ownership. +- F6. **Invoke from Promptfoo** + 1. Promptfoo constructs the AI Evals-owned TypeScript provider with + `ProviderOptions`; the provider retains the ID and validates + `options.config` containing the private-network endpoint, target, and one + closed source-mode object. + 2. `callApi(prompt, context, options)` applies only valid + `context.vars.allagentsSource` leaf overrides, creates one invocation key, + and sends one A2A Message with the prompt and required extension. + 3. The provider waits or streams until terminal. Its deadline or + `options?.abortSignal` sends `CancelTask` once after acceptance and waits + for the same terminal cleanup path. + 4. It returns terminal text or validated structured output in + `ProviderResponse.output`; maps `inputTokens -> prompt`, + `outputTokens -> completion`, `cachedInputTokens -> cached`, and + `totalTokens -> total`; and puts other usage plus Task, Artifact, logical + source, termination, and cleanup facts in `metadata`. Admission or terminal + execution failure returns `error`. + ### Acceptance Examples - AE1. A caller on a permitted Tailscale or firewalled network discovers the @@ -497,6 +550,17 @@ registry, or another profile configuration file for the initial use case. - AE20. Unrelated Message metadata survives request processing. Every profiled A2A operation requires activation, and a terminal Task may contain the single integrity Artifact plus referenced produced Artifacts. +- AE21. The AI Evals Promptfoo provider loads one repository-mode and one + snapshot-mode YAML instance. Repository mode materializes the complete + configured set and sends only optional revision overrides keyed by declared + name; snapshot mode sends one declared name and immutable digests. Neither + request source metadata nor response source-identity metadata contains a Git + URL, OCI repository, or destination. + Both calls return scorable `ProviderResponse.output`, the exact normalized + token mapping, and Task/Artifact/logical-provenance metadata. Per-test + repository overrides accept only full commits. An unknown variable member, + mutable revision, origin, destination, or undeclared name fails before + submission. ### Success Criteria @@ -505,6 +569,9 @@ registry, or another profile configuration file for the initial use case. - The official A2A client exercises required-extension negotiation, send, stream, get, list, subscribe, replay, cancel, terminal cancel errors, Artifact retrieval, and expiry. +- An AI Evals-style Promptfoo custom-provider fixture consumes representative + YAML for both source modes and maps a terminal Task to `ProviderResponse` + without adding Promptfoo to the AllAgents runtime. - Built-in Codex/Pi and exposed profile targets pass one conformance suite, including reserved-ID collisions. - Direct Git and OCI snapshot fixtures produce equivalent validated workspace @@ -722,6 +789,87 @@ The nested object is strict and initially contains only `expose: true`. Absence means not exposed. Exposure requires a launcher, an initial supported backend, and a healthy installed profile with matching declaration digest. +**Promptfoo custom-provider consumption** + +AI Evals implements Promptfoo's +[`ApiProvider`](https://www.promptfoo.dev/docs/providers/custom-api/) in +TypeScript. Its `constructor(options: ProviderOptions)` stores +`options.id ?? "allagents-a2a"` and validates `options.config`; `id()` returns +that stored value. Its +`callApi(prompt, context, options)` uses `context.vars` for test data and +`options?.abortSignal` for request cancellation. + +Static YAML defines the source mode and every logical name: + +```yaml +providers: + - id: file://./providers/allagents-a2a.ts + label: codex-direct + config: + endpoint: http://allagents-gateway.tailnet:4732 + target: codex + source: + kind: repositories + revisions: + allagents: 0123456789abcdef0123456789abcdef01234567 + + - id: file://./providers/allagents-a2a.ts + label: codex-evaluation-snapshot + config: + endpoint: http://allagents-gateway.tailnet:4732 + target: codex + source: + kind: workspaceSnapshot + snapshot: evaluation + digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef + workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 + +tests: + - description: direct repositories at an exact commit + providers: [codex-direct] + vars: + allagentsSource: + revisions: + allagents: fedcba9876543210fedcba9876543210fedcba98 + + - description: immutable prebuilt workspace + providers: [codex-evaluation-snapshot] + vars: + allagentsSource: + digest: sha256:fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210 + workspaceManifestDigest: sha256:6789abcdef0123456789abcdef0123456789abcdef0123456789abcdef012345 +``` + +`allagents` is a declared repository name used only as a revision-override key; +repository mode still materializes the complete configured set. `evaluation` is +the logical snapshot handle. The provider sends the source mode, optional named +revisions, and immutable digests, not +`https://github.com/EntityProcess/allagents.git` or +`ghcr.io/entityprocess/allagents-workspaces`. The gateway resolves origins and +credentials server-side and omits them from A2A source-identity responses. + +`context.vars.allagentsSource` is the only per-test override. In repository mode +it may contain exactly `revisions`, whose keys must already exist in static +`config.source.revisions` and whose values are full lowercase 40-hex commits. +In snapshot mode it may contain exactly `digest` and/or +`workspaceManifestDigest`, both full lowercase `sha256:` digests. Present leaves +replace static leaves; absent leaves retain static values. Source kind, +repository-name allowlist, and snapshot name remain static. Unknown members, +mutable revisions, origins, destinations, credentials, and commands fail before +A2A submission. + +Each `callApi` creates one invocation key and A2A Task. The provider sends +`CancelTask` when its bounded deadline or `options?.abortSignal` fires after +acceptance. It returns terminal text or the validated structured result as +`ProviderResponse.output`. It maps gateway usage exactly as +`inputTokens -> tokenUsage.prompt`, `outputTokens -> tokenUsage.completion`, +`cachedInputTokens -> tokenUsage.cached`, and +`totalTokens -> tokenUsage.total`; provider-specific counters stay in +`metadata`. Task ID, Artifact references, logical source identity, termination, +and cleanup evidence also remain in `metadata`, without origins or destination +paths. Admission and terminal failures use `ProviderResponse.error`. This +provider is AI Evals code; AllAgents has no Promptfoo runtime dependency. + ### Error and Status Mapping | Condition | Stable code and A2A outcome | Fresh-invocation retryable | @@ -988,22 +1136,29 @@ messages never enter either carrier. ### U7. End-to-end delivery and documentation - **Goal:** Prove the built CLI and document the trusted-network operating model, - workspace configuration, credentials, sources, and risks. -- **Requirements:** R1-R18; F1-F5; AE1-AE20. + workspace configuration, credentials, sources, Promptfoo consumption, and + risks. +- **Requirements:** R1-R19; F1-F6; AE1-AE21. - **Files:** gateway guide/reference, configuration reference, README, CHANGELOG, - real example project/user workspaces, E2E fixtures, release evidence. + real example project/user workspaces, AI Evals-style Promptfoo YAML and custom- + provider contract fixture, E2E fixtures, release evidence. - **Approach:** After the final implementation review is resolved, build the CLI; create project and user workspaces under `/tmp/`; expose Codex/Pi fixture targets; serve on loopback and `0.0.0.0`; acquire from local Git and OCI fixtures; run the official A2A client through negotiation, success, replay, - cancellation, deadline, shutdown, restart, and expiry. Document that network - reachability grants full authority and App/OCI secrets are process inputs, not - YAML. + cancellation, deadline, shutdown, restart, and expiry. Run a minimal custom- + provider fixture through one configured-repository-set invocation and one + named-snapshot invocation, proving Promptfoo configuration carries only + source mode, named revision overrides, snapshot handle, and digests while the + gateway resolves origins. Document that network reachability grants full + authority and App/OCI secrets are process inputs, not YAML. - **Execution note:** The green smoke test must exercise the same built command and `/tmp/` workspace shape as the recorded red E2E, not a test-only server. + The consumer fixture models AI Evals but remains test/documentation code; the + AllAgents runtime does not import Promptfoo. - **Verification:** `bun run build`, focused and full tests, typecheck, lint, docs - build, schema drift check, and exact red/green E2E commands/results recorded in - the PR description. + build, schema drift check, custom-provider contract fixture, and exact + red/green E2E commands/results recorded in the PR description. --- @@ -1024,12 +1179,13 @@ messages never enter either carrier. | Structured result | U1, U4-U6 | Exact subset and envelope, valid/invalid/not-produced states, Artifact cardinality, no false publication | | Repository quality | All | Build, focused/full tests, typecheck, lint, schema check, docs build | | Built CLI E2E | U7 | Recorded red then green built command under `/tmp/`, both sources, auth isolation, replay/cancel/deadline/shutdown/restart | +| Promptfoo consumption | U7 | AI Evals-style YAML for both source modes; request source metadata and gateway-generated response provenance omit origins; terminal Task maps to `ProviderResponse` | ## Definition of Done ### Global -- Every R1-R18 requirement is implemented or explicitly demonstrated by a +- Every R1-R19 requirement is implemented or explicitly demonstrated by a passing acceptance scenario. - The gateway starts with no `gateway.yaml` or `worker.yaml`, defaults to loopback, and accepts explicit `0.0.0.0`. @@ -1042,6 +1198,11 @@ messages never enter either carrier. request and result-schema grammar, exact error mapping, integrity/produced Artifacts, canonicalization, retention capacity, and cancellation semantics pass official-client contract fixtures. +- AI Evals-style Promptfoo YAML selects repository mode with optional named + revision overrides, or snapshot mode with one logical handle and immutable + digests. The custom-provider fixture maps one `callApi` to one Task, propagates + cancellation, normalizes usage, and returns output, Artifacts, and logical + provenance without sending origins or adding a Promptfoo runtime dependency. - Repository and OCI modes produce one validated workspace-manifest contract, never fall back across source modes, and retain truthful provenance. - GitHub App eligibility/unknown state, acquisition sub-budget, no-installation @@ -1071,5 +1232,6 @@ messages never enter either carrier. - U5: Codex passes shared conformance and optional credentialed smoke evidence is recorded when credentials exist. - U6: Pi passes the same conformance and malformed RPC cannot produce success. -- U7: Final review is resolved; built CLI red/green E2E under `/tmp/`, complete - repository gates, schemas, docs, and reproducible PR instructions are complete. +- U7: Final review is resolved; built CLI red/green E2E under `/tmp/`, Promptfoo + custom-provider contract fixture, complete repository gates, schemas, docs, + and reproducible PR instructions are complete. From 69a727ef1a75d89bf110a26e03eede748530e85a Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sat, 19 Sep 2026 15:13:01 +1000 Subject: [PATCH 10/12] docs(architecture): align execution gateway contracts --- ...-agent-execution-through-an-a2a-gateway.md | 404 ++-- ...0837-feat-coding-execution-gateway-plan.md | 1657 +++++++++++------ 2 files changed, 1309 insertions(+), 752 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index b66c36c5..22aacddc 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -40,8 +40,8 @@ entry point. It is separate from the interactive CLI command lifecycle but may run as a single local service process that supervises acquisition and provider child processes. -The gateway implements A2A 1.0 HTTP+JSON plus a required versioned AllAgents -coding-execution extension. It owns: +The gateway implements A2A protocol version `1.0` over the `HTTP+JSON` binding +plus a required versioned AllAgents coding-execution extension. It owns: - stable Task and idempotency identity; - execution-target selection; @@ -59,18 +59,31 @@ or durable evaluation Run ledger. The initial gateway has no application-level authentication or per-caller authorization. It may bind to loopback, a specific interface, or `0.0.0.0`. Loopback remains the default when no listen address is supplied, but an explicit -`0.0.0.0` binding is valid and requires no unsafe-mode flag. - -Every host able to reach the listener is equally trusted. Any reachable caller -may invoke every exposed target, list and retrieve retained Tasks and Artifacts, -and request cancellation. Task lookup and idempotency are deployment-wide, not +`0.0.0.0` binding is valid and requires no unsafe-mode flag. The Agent Card +advertises a separate absolute interface URL; non-loopback listeners require +that value explicitly because a wildcard bind address is not routable. +Production interfaces use HTTPS; direct HTTP is limited to loopback development. + +Every external host able to reach the listener is equally trusted. Any reachable +caller may invoke every available target, including built-in and gateway-enabled +profile targets; list or retrieve retained Tasks and their Artifacts; and +request cancellation. Task lookup and idempotency are deployment-wide, not scoped to a caller identity. Operators must use Tailscale ACLs, host firewalls, container networking, or equivalent network controls when the listener is not loopback-only. -TLS termination, OIDC, static bearer tokens, per-tenant ownership, and -multi-tenant information-hiding are deferred. They require a separate decision -when the service is exposed outside one trusted network boundary. +Invocation descendants are not network peers. Provider, MCP, and model-tool +processes run in role-specific network namespaces that cannot route to host +loopback, any gateway bind or advertised address, ingress proxies, or operator +management networks. Provider and MCP egress is default-deny except for +destinations compiled from adapter and MCP configuration; model tools receive no +network unless an explicit adapter policy grants the same constrained egress. + +Gateway-managed TLS termination, OIDC, static bearer tokens, per-tenant +ownership, and multi-tenant information-hiding are deferred. Production clients +reach the advertised HTTPS interface through operator-managed termination or an +encrypted private overlay. Application authentication requires a separate +decision when the service leaves one trusted network boundary. ### Use existing workspace files as the configuration authority @@ -82,8 +95,8 @@ canonical for repository identities, remote sources, destination paths, default revisions, workspace files, plugins, and named OCI snapshot sources. The user `~/.allagents/workspace.yaml` remains canonical for global profiles and -launcher-backed execution targets. A launcher-bearing profile client is exposed -only when it explicitly declares: +launcher-backed execution targets. A launcher-bearing profile client is gateway- +enabled only when it explicitly declares: ```yaml profiles: @@ -92,15 +105,15 @@ profiles: - name: codex launcher: codex-review gateway: - expose: true + enabled: true ``` The public target ID is the launcher basename. Launcher names are already portable and collision-checked across every user profile, while one profile may contain several clients and therefore several launchers. Internally the target resolves to exactly one `(profile, client)` pair. The gateway reserves built-in -target IDs, initially `codex` and `pi`; an exposed launcher whose portable -collision key matches a built-in ID is invalid. +target IDs, initially `codex` and `pi`; a gateway-enabled launcher whose +portable collision key matches a built-in ID is invalid. The built-in `codex` and `pi` targets remain available when their adapters are ready. Explicit launcher-backed targets add configured variants such as @@ -116,21 +129,25 @@ its typed adapter and invokes the provider's supported automation surface. Process-level options use exact flags and environment variables for: -- listener and workspace selection; +- listener, advertised-interface URL, and workspace selection; - a project-specific state-directory override; - terminal Task retention and bounded Artifact/event storage; - GitHub App identifiers and private-key file references; - the configured GitHub CLI account; -- an OCI credential file or fixed credential-helper executable; and +- a strict Docker-auth file or fixed Docker credential-helper executable; and - Codex and Pi auth-file handles. By default the state root is a deterministic child of `~/.allagents/gateway/` keyed by the canonical project-workspace identity. The -store persists and verifies that identity and holds an exclusive lock for the -process lifetime. The root is current-user owned, private, symlink- and hard- -link-resistant, and disjoint from project, profile, and invocation roots. - -Secret values never belong in either workspace file. +packaged Rust helper owns a private SQLite store in WAL/full-synchronization mode +through a descriptor-rooted VFS. Every database, WAL, SHM, journal, and temporary +file open uses `openat2` beneath/no-symlink resolution and rejects hard links. +The store persists claims, Tasks, one execution lease, internal outcome intent, +events, bounded Artifact bytes, containment identity, and expiry transactions. +It verifies workspace identity and holds an exclusive process-lifetime lock. +The root is current-user owned, private, link-resistant, and disjoint from +project, profile, and invocation roots. The listener exposes metadata-only +`/healthz` and `/readyz`; readiness is false whenever admission is unsafe. ### Support direct repositories and OCI workspace snapshots @@ -152,18 +169,43 @@ tags may be accepted for developer convenience, but the gateway resolves and records the full commit object ID before provider execution. Reproducibility- sensitive callers should supply full commit IDs. -For OCI snapshots, the project workspace declares the registry repository: +For OCI snapshots, the project workspace declares the registry repository and +any exact cross-origin layer-blob redirect hosts: ```yaml workspaceSnapshots: evaluation: repository: ghcr.io/entityprocess/allagents-workspaces + layerRedirectHosts: + - pkg-containers.githubusercontent.com ``` -The request supplies the name `evaluation`, a `sha256:` OCI manifest digest, and -a `sha256:` workspace-manifest digest. The gateway constructs the full OCI -reference server-side. Callers cannot supply a registry host, repository name, -mutable tag, extraction destination, credential, or external-layer policy. +The request supplies the name `evaluation`, a `sha256:` OCI image-manifest +digest, and a `sha256:` workspace-manifest digest. The gateway constructs the +full OCI reference server-side. Callers cannot supply a registry host, +repository name, mutable tag, extraction destination, credential, platform +selector, redirect host, or external-layer policy. + +V1 accepts only an OCI Image Manifest directly at the requested digest; image +indexes, descriptor URLs or embedded data, non-distributable layers, and +unknown media types are rejected. Its config is the RFC 8785 canonical +`application/vnd.allagents.workspace-manifest.v1+json` object and must match the +requested workspace-manifest digest. The gateway verifies the manifest body, +config, and every distributable tar/gzip/zstd layer descriptor before decoding, +then applies layers in manifest order with OCI whiteout and opaque-whiteout +semantics. + +Registry metadata remains same-origin. A cross-origin redirect is allowed only +for a layer-blob `GET` or `HEAD` to an exact operator-declared +`layerRedirectHosts` entry; an absent allowlist rejects it. Every bounded HTTPS +hop strips authorization, cookies, and client credentials, rejects URL +credentials, resolves and validates every address at connection time, and +rejects mixed answers, rebinding, downgrade, and unapproved destinations. +Loopback, link-local, private, reserved, or other non-global addresses are +permitted only when their exact host is the source's operator-declared +repository host or layer-redirect host. Token, manifest, and config redirects +remain same-origin. Descriptor size and digest verification remains mandatory +after redirects. Both modes produce the same versioned, wire-visible workspace manifest. It records declared logical repository names, requested revisions, resolved @@ -176,10 +218,10 @@ Git remote. Acquisition occurs in a gateway-owned staging directory. The gateway validates paths, collisions, file types, symlinks, layer and file counts, individual and -total sizes, digests, and the workspace manifest before atomically publishing -the invocation workspace. Absolute paths, traversal, device files, sockets, -escaping links, foreign or external OCI layers, and cross-origin credential -forwarding are rejected. +total compressed and expanded sizes, digests, and the workspace manifest before +atomically publishing the invocation workspace. Absolute paths, traversal, +device files, sockets, escaping links, foreign or external OCI layers, and +unapproved cross-origin access are rejected. ### Consume the gateway from Promptfoo through an AI Evals provider @@ -195,25 +237,40 @@ workspace in Promptfoo YAML. The two files have different ownership: AI Evals owns a Promptfoo [custom JavaScript/TypeScript provider](https://www.promptfoo.dev/docs/providers/custom-api/). -It implements `ApiProvider`: its constructor receives `ProviderOptions`, retains -`options.id`, validates `options.config`, and exposes `id()`. -`callApi(prompt, context, options)` reads bounded test variables from -`context.vars` and cancellation from `options?.abortSignal`. The provider -translates one `callApi` into one A2A Task: it creates an invocation key, puts -the prompt in the single `TextPart`, puts the target and closed source union in -the required extension metadata, waits or streams to terminal, and returns -output, normalized token usage, and logical provenance in Promptfoo's -`ProviderResponse`. +It implements `ApiProvider`: its constructor receives `ProviderOptions`, +requires and retains a nonempty `options.id`, validates `options.config`, and +exposes `id()`. +`callApi(prompt, context?, options?)` reads bounded test variables from +`context?.vars` when present and cancellation from `options?.abortSignal`. The +provider translates one `callApi` into one A2A Task: it creates and retains a +high-entropy invocation key, sends one Message whose sole Part has `text` set, +declares the extension in `Message.extensions`, puts the target and closed source +union in the matching metadata member, and calls `SendMessage` with +`returnImmediately: true`. It captures the Task ID and follows terminal state +through `SubscribeToTask`, with `GetTask` and bounded resubscription for races or +disconnects. It returns output, normalized token usage, stable failure metadata, +and logical provenance in Promptfoo's `ProviderResponse`. For example, AI Evals can define two provider instances without sending either origin over the wire: ```yaml +prompts: + - file://./prompts/coding-task.txt + +sharing: false +evaluateOptions: + maxConcurrency: 1 + cache: false +commandLineOptions: + write: false + share: false + providers: - id: file://./providers/allagents-a2a.ts label: codex-direct config: - endpoint: http://allagents-gateway.tailnet:4732 + endpoint: https://allagents-gateway.example.internal target: codex source: kind: repositories @@ -223,7 +280,7 @@ providers: - id: file://./providers/allagents-a2a.ts label: codex-evaluation-snapshot config: - endpoint: http://allagents-gateway.tailnet:4732 + endpoint: https://allagents-gateway.example.internal target: codex source: kind: workspaceSnapshot @@ -235,21 +292,32 @@ providers: The first provider materializes the complete configured repository set and uses the `allagents` key only to override that repository's revision. The second provider's `evaluation` key resolves to the declared -`ghcr.io/entityprocess/allagents-workspaces` repository. +`ghcr.io/entityprocess/allagents-workspaces` repository. The gateway enforces +one active invocation transactionally. Promptfoo keeps `maxConcurrency: 1` to +avoid predictably creating failed capacity Tasks; other trusted callers need no +external queue for correctness. The no-cache/no-write/no-share values are secure +defaults for confidential prompts and outputs; consumers may enable persistence +or sharing only after applying their own retention, access, destination, and +redaction policy. Static provider config fixes the source kind and logical names. The only -per-test object is `context.vars.allagentsSource`: repository mode accepts +per-test object is `context?.vars?.allagentsSource`: repository mode accepts revision overrides only for statically listed names and only as full lowercase 40-hex commits; snapshot mode accepts only replacement OCI and workspace- -manifest `sha256:` digests. Missing leaves retain static values. A URL, -destination, mutable revision, credential, command, unknown member, or changed -source kind/name fails before submission. After Task acceptance, the provider's -bounded deadline or `options?.abortSignal` sends `CancelTask`. It maps gateway -input, output, cached-input, and total token counts to Promptfoo's `prompt`, -`completion`, `cached`, and `total` fields respectively; other usage and Task/ -Artifact evidence stays in metadata without origins. The provider belongs in AI -Evals. AllAgents exposes the A2A contract and consumer documentation without -taking a runtime dependency on Promptfoo. +manifest `sha256:` digests. Missing context or leaves retain static values. A +URL, destination, mutable revision, credential, command, unknown member, or +changed source kind/name fails before submission. After Task acceptance, the +provider's bounded deadline or `options?.abortSignal` sends one `CancelTask` +using a fresh cleanup signal rather than the already aborted request signal. +It maps gateway input, output, cached-input, and total token counts to +Promptfoo's `prompt`, `completion`, `cached`, and `total` fields respectively. +Safe stable failure +code, retryability, accepted Task ID, other usage, and logical Task/Artifact +evidence stay in metadata without origins. Opaque prompts, terminal output, +structured results, native evidence, and produced-Artifact payloads remain +unredacted sensitive data. The provider belongs in AI Evals. AllAgents exposes +the A2A contract and consumer documentation without taking a runtime dependency +on Promptfoo. ### Resolve GitHub credentials with App-first eligibility fallback @@ -261,17 +329,20 @@ For `github.com`, the gateway supports two trusted providers: The App is preferred when it has an installation covering the configured repository. Installation applicability has three outcomes: `eligible`, -`ineligible`, and `unknown`. The gateway discovers applicability through an -App-authenticated GitHub API client, or verifies an explicitly configured -installation ID against the repository. `@octokit/auth-app` handles App JWT and -installation-token authentication; it is not treated as the repository- -discovery policy by itself. - -For an eligible installation, the gateway requests a fresh repository-scoped -installation token for each acquisition and grants only required read -permissions. Acquisition receives at most 900 seconds or the shorter remaining -Task deadline. The token must remain valid beyond that sub-budget plus a -60-second clock-skew margin. +`ineligible`, and `unknown`. An App-authenticated lookup that returns coverage +is eligible. A 404 is ineligible only after the configured GitHub CLI identity +independently proves that the repository exists; an uncorroborated 404 or any +authentication, permission, rate-limit, timeout, or service ambiguity is +unknown. An explicitly configured installation ID must positively verify +repository coverage. + +For an eligible installation, the gateway bypasses the SDK token cache and +requests a fresh repository-scoped, read-only token for each acquisition. It +validates repository selection, permissions, creation time, and expiry. +Acquisition receives at most 900 seconds or the shorter remaining Task deadline, +and the token must remain valid beyond that sub-budget plus a 60-second clock- +skew margin. The gateway revokes the token after acquisition; unconfirmed +revocation fails before provider execution. GitHub CLI is an eligibility fallback only when the App is not configured or applicability is positively `ineligible`. An `unknown` result caused by @@ -287,9 +358,9 @@ with `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and is part of the acquisition-policy digest. After an App installation is selected, App configuration, authentication, -token minting, permission, repository-coverage, rate-limit, or service failure -terminates acquisition. The gateway never retries the same Task through the -broader GitHub CLI identity. +token minting or validation, permission, repository coverage, revocation, +rate-limit, or service failure terminates acquisition. The gateway never retries +the same Task through the broader GitHub CLI identity. Git receives credentials only through an invocation-scoped helper under hermetic Git configuration. The gateway excludes system, global, and repository @@ -300,9 +371,10 @@ Artifacts, retained workspaces, profile setup, MCP processes, agent processes, or model-invoked tools. The helper and token are destroyed before provider execution. -OCI credentials come from a configured auth-file or standard credential helper, -are scoped to snapshot acquisition, and are removed before publication. Public -registries require no credential configuration. +OCI credentials come from either a strict Docker-auth subset that cannot name +executables or a fixed Docker credential helper using its standard `get` +protocol. They are scoped to snapshot acquisition and removed before +publication. Public registries require no credential configuration. ### Integrate providers through typed adapters @@ -312,9 +384,11 @@ capabilities, invocation, progress, deterministic permission handling, abort, terminal output, optional structured result, usage, native evidence, and disposal. -The Codex adapter depends directly on `@openai/codex-sdk`, creates one fresh -thread per Task, passes cancellation and optional output schema through the SDK, -and consumes structured events. +The Codex adapter depends directly on pinned `@openai/codex-sdk`, creates one +fresh thread per Task, passes cancellation, and consumes structured events. It +uses native `outputSchema` only for schemas supported by the pinned Structured +Outputs contract; other valid public schemas use explicit JSON guidance and the +same gateway-side validator used by every backend. The Pi adapter uses strict RPC mode with invocation-owned configuration and a restricted policy extension. Repository extensions and unrestricted built-ins @@ -332,56 +406,88 @@ Validated profile settings, plugins, MCP declarations, and deterministic workspace projections are applied through existing typed transforms. Provider control processes, MCP children, and model-invoked tools receive -distinct allowlisted environments and filesystem views. The provider control -process sees only its invocation-private auth channel; each MCP child sees only -its own resolved secrets; shell and other model-invoked tools see neither -provider nor MCP credentials. Every view excludes gateway state, operator home, -App keys, GitHub/OCI stores, acquisition helpers, unrelated adapter auth, and -the parent environment. A target is not ready unless its adapter can enforce -these separations. This credential/state isolation is required even though -general hostile-code sandboxing remains deferred. +distinct allowlisted filesystem, environment, descriptor, secret, and network +views. The provider control process sees only its invocation-private auth +channel; each MCP child sees only its own resolved secrets; shell and other +model-invoked tools see neither provider nor MCP credentials. Every view excludes +gateway state, operator home, App keys, GitHub/OCI stores, acquisition helpers, +unrelated adapter auth, the parent environment, gateway endpoints, host +loopback, and management networks. + +The pinned backend must expose a non-bypassable synchronous hook that delegates +every MCP and model-tool spawn to the Rust helper. The helper enters the role's +mount and network namespaces, replaces the environment, closes every +non-allowlisted descriptor, and only then executes untrusted code. Codex or Pi +is unavailable when its pinned surface can bypass that hook. This enforced +credential/state/network boundary is required even though general hostile-code +sandboxing remains deferred. ### Persist Task truth, not live provider execution The gateway durably stores Task identity, the canonical request, idempotency -claim, selected target and source, effective configuration digest, terminal -status, Artifact metadata, and retained evidence under the configured state -directory. A provider session is not a durable recovery checkpoint. - -An identical idempotency replay returns the existing Task. Reusing the key with -a different canonical request conflicts. Because the initial service has no -caller identity, the idempotency namespace and Task visibility are gateway-wide. - -Terminal Task records, Artifacts, events, and invocation claims expire -atomically after the configured TTL. The retained-count limit never evicts an +claim, selected target and source, effective configuration digest, one execution +lease, internal outcome intent, Artifact bytes, retained evidence, and +containment identity under the configured state directory. The official A2A +HTTP+JSON transport wraps an AllAgents-owned request handler; typed transactions +execute in the Rust helper's descriptor-rooted SQLite VFS. One transaction +arbitrates `createOrReplay`, UUIDv7 Task creation, execution-lease acquisition, +and containment binding; another atomically settles terminal status, result or +failure, evidence, Artifacts, cleanup, and lease release. A provider session is +not a durable recovery checkpoint. + +At most one Task holds the execution lease from acquisition through final +evidence collection. A second otherwise-valid request settles failed with +`execution_capacity_unavailable`. Its transient empty containment set is +destroyed after settlement without releasing the start gate or launching a +helper child. An identical idempotency replay returns the existing Task. Reusing +the key with a different canonical request conflicts. Clients generate at least +128 bits of randomness +once per logical invocation and reuse the same key plus request after an +ambiguous transport failure. Because the initial service has no caller identity, +the idempotency namespace and Task visibility are gateway-wide. + +Terminal Task records, Artifacts, events, and invocation claims expire in one +transaction after the configured TTL. The retained-count limit never evicts an unexpired Task; the gateway rejects new admission until expiry frees capacity. -State-store integrity or durability failure stops admission and prevents the -gateway from acknowledging creation or reporting terminal success. +State-store integrity, VFS, helper protocol, or durability failure stops +admission and prevents the gateway from acknowledging creation or reporting +terminal success. -On gateway restart, interrupted nonterminal Tasks settle failed; provider work -is not resumed or automatically replayed. A new invocation may start fresh. +On gateway restart, interrupted nonterminal Tasks settle failed only after +containment reconciliation; provider work is not resumed or automatically +replayed. A new invocation may start fresh. ### Make cancellation, evidence, and cleanup explicit -The gateway supervises every acquisition and provider process set. Cancellation -first invokes the provider's native abort or protocol cancellation, then applies -bounded forced termination to the complete descendant set. - -Terminal cleanup evidence is recorded only after the supervisor proves the -complete invocation process set quiescent through an enforceable, invocation- -owned containment primitive. If the platform cannot provide that guarantee, the -gateway fails readiness rather than relying on best-effort process enumeration. -If termination or proof fails, the Task records termination as unknown or -failed and the gateway rejects new work. An unmanaged foreground gateway stays -alive with poisoned readiness and continues reaping while printing the stable -containment identifier and platform recovery command. It may exit with a -nonempty set only after a validated external manager accepts cleanup ownership. - -On startup the gateway identifies every interrupted invocation's containment -set and proves it empty before binding or advertising readiness. It may -quarantine a stale filesystem root only after process quiescence is proven. A -reaping or proof failure terminalizes the Task with unknown/failed termination, -keeps readiness false, and enters the same managed or unmanaged recovery path. +The gateway supervises every acquisition and provider process set. The helper +allocates a stable empty containment set behind a start gate. The Task, +execution lease, and containment identifier commit durably before the helper may +release that gate or execute any child; a failed commit destroys the empty set. + +One durable compare-and-set arbitrates provider terminal outcome, caller +cancellation, deadline, and shutdown as an internal outcome intent while the +externally visible Task remains nonterminal. The winning intent owns the stable +result or failure code and drives one idempotent abort and quiescence path. +Cancellation first invokes the provider's native abort or protocol cancellation, +then applies bounded forced termination to the complete descendant set. + +Live provider events are bounded while execution runs. Filesystem, Git, and +produced-Artifact evidence is read only after the supervisor proves the complete +invocation process set quiescent through its invocation-owned containment. +Only then does one transaction atomically publish terminal status, the integrity +Artifact, bounded evidence, result or failure, produced Artifacts, termination, +cleanup, and lease release. If quiescence cannot be proven, that transaction +settles `execution_quiescence_unknown` without verified filesystem evidence. +The gateway rejects new work and stays alive with poisoned readiness while +continuing to reap; later recovery changes only internal recovery/readiness +state, never the settled Task. + +On startup the helper enumerates the entire project-owned containment namespace, +including unknown identifiers, and proves every set empty before binding, +releasing a retained lease, quarantining stale roots, or advertising readiness. +It prints the stable containment identifier and platform recovery command for +any nonempty set. An unsupported platform fails before binding rather than +relying on process enumeration. Terminal evidence distinguishes: @@ -401,24 +507,33 @@ source, or file contents are excluded from operational logs. ### Profile A2A instead of inventing an invocation API The gateway uses A2A Agent Cards, Messages, Tasks, Artifacts, operations, errors, -streaming, and cancellation. The Agent Card declares the AllAgents coding- -execution extension as required. Every operation that creates, returns, lists, -subscribes to, or mutates profiled Tasks or Artifacts activates -`https://allagents.dev/a2a/extensions/coding-execution/v1` through the -`A2A-Extensions` header. Unsupported calls receive the standard A2A extension- -support error, and responses echo the activated URI. - -The versioned extension carries the invocation key, execution target, closed +streaming, and cancellation. Its Agent Card advertises one interface with +`protocolBinding: "HTTP+JSON"`, `protocolVersion: "1.0"`, and +`capabilities.streaming: true`. The coding- +execution `AgentExtension` is required and has strict +`params: { targets: TargetId[] }`, populated from ready built-in and explicitly +gateway-enabled launcher-backed targets. It does not publish paths, commands, +arguments, environment selectors, credentials, exact source authorization +details, or transient worker state. + +Every A2A HTTP+JSON request carries `A2A-Version: 1.0`. Every operation that +creates, returns, lists, subscribes to, or mutates profiled Tasks or Artifacts +also activates `https://allagents.dev/a2a/extensions/coding-execution/v1` +through `A2A-Extensions`. Missing activation receives +`ExtensionSupportRequiredError`; an unsupported protocol version receives +`VersionNotSupportedError`. Unsuccessful HTTP responses use the A2A +`google.rpc.Status` JSON envelope with typed `google.rpc.ErrorInfo` details; +validation also uses `google.rpc.BadRequest`, never JSON-RPC error carriers. + +The published versioned extension specification defines Agent Card params, +activation, request/idempotency/replay, errors, and terminal Task/Artifact +schemas. Its request carries the invocation key, execution target, closed workspace source, bounded deadline, and optional bounded result schema in its own strict `Message.metadata` member without rejecting unrelated A2A metadata. -Every terminal Task has one fixed-name, versioned integrity Artifact plus zero -or more produced Artifacts. Breaking extension versions receive versioned cards -and endpoints rather than silent fallback. - -The Agent Card advertises built-in and explicitly exposed launcher-backed -targets through an allowlisted capability projection. It does not publish local -paths, commands, arguments, environment selectors, credentials, exact source -authorization details, or transient worker state. +The request Message lists the URI in `Message.extensions`. Every terminal Task +has one fixed-name, versioned integrity Artifact whose `Artifact.extensions` +lists the URI, plus zero or more produced Artifacts. Breaking extension versions +receive versioned cards and endpoints rather than silent fallback. ACP, app-server, SDK, and RPC protocols remain backend implementation details. W3C Trace Context may propagate correlation through HTTP and child-process @@ -439,22 +554,33 @@ retry. Consumers own those concerns. isolation, `gateway.yaml`, `worker.yaml`, remote worker protocol, or required Kubernetes deployment in the initial product. - Project and user workspace files remain the sole declaration authority for - source identities and exposed profile launchers. + source identities and gateway-enabled profile launchers. - AI Evals can express the configured repository set with named revision overrides, or select a prebuilt image through a snapshot handle, in Promptfoo YAML. Its custom provider translates that closed source choice to A2A and keeps raw origins under AllAgents operator control. -- Network reachability grants access to every exposed target and retained Task. - Operators must treat network policy as the authorization boundary. +- External network reachability grants access to every available target, + including built-in and gateway-enabled profile targets, plus every retained + Task. Operators treat network policy as the authorization boundary; invocation + descendants are isolated from that boundary and host-management networks. +- One durable execution lease enforces one active invocation independent of + consumer concurrency settings. - GitHub App credentials support private repositories without forcing every - developer to use one identity; GitHub CLI remains a local eligibility - fallback when no App installation applies. + developer to use one identity; GitHub CLI remains a local eligibility fallback + only when no App installation applies. - Direct repositories and digest-pinned OCI snapshots converge on one validated - workspace manifest and evidence contract. -- The gateway process remains a meaningful API and lifecycle boundary, but not - a hostile-code sandbox. Strong multi-tenant isolation remains future work. + workspace manifest and evidence contract. OCI metadata remains same-origin; + only layer blobs may redirect to exact operator-approved hosts. +- Gateway v1 execution is supported on Linux x64/arm64 with the packaged state/ + security helper, descriptor-rooted SQLite VFS, cgroup v2, mount/network + namespaces, nftables, pidfds, and safe-file operations. Matching helper + packages publish and verify before the root package; unsupported hosts or + missing capabilities fail before binding rather than degrading containment. +- The gateway process remains a meaningful API and lifecycle boundary, but not a + hostile-code sandbox. Strong multi-tenant isolation remains future work. - Codex and Pi share one conformance suite while retaining bounded native - evidence and honest capability differences. + evidence and honest capability differences. A backend is unavailable unless + its pinned surface can delegate every MCP/tool spawn through the helper. - A future deployment configuration becomes justified only when the product needs multiple worker routes, tenants, credential policies, custom materializers, centralized storage, or other operator-selected variants. diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index 07ec97e9..b2e14039 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -14,8 +14,8 @@ execution: code ## Goal Capsule - **Objective:** A developer can run one trusted-network A2A endpoint for one - AllAgents workspace and invoke built-in or explicitly exposed profile targets - against either the complete configured Git repository set, with optional + AllAgents workspace and invoke built-in or explicitly gateway-enabled profile + targets against either the complete configured Git repository set, with optional named revision overrides, or a digest-pinned OCI workspace snapshot. AI Evals can configure either source mode in Promptfoo YAML through a custom provider without sending origins. @@ -28,9 +28,9 @@ execution: code own source and profile declarations. A2A 1.0 owns core wire semantics. - **Execution order:** Capture a red built-CLI E2E for the missing gateway; freeze schemas and configuration projection; implement the Task store, A2A - server, acquisition, supervisor, Codex, and Pi; run a final implementation - review and fix important findings; then run the green built-CLI E2E, - repository gates, and documentation validation. + server, supervisor/helper, acquisition, Codex, and Pi; run a final + implementation review and fix important findings; then run the green built- + CLI E2E, repository gates, and documentation validation. - **Stop conditions:** Do not add application authentication, `gateway.yaml`, `worker.yaml`, remote worker routing, caller-supplied URLs or commands, mutable OCI tags, selected-provider failure fallback, evaluation behavior, or @@ -78,8 +78,8 @@ registry, or another profile configuration file for the initial use case. - A3. **Backend adapter:** The Codex or Pi implementation translating native automation events and cancellation into the common contract. - A4. **Operator/developer:** The person who selects the project workspace, - exposes profile launchers, supplies process flags and credential handles, and - controls network access. + gateway-enables profile launchers, supplies process flags and credential + handles, and controls network access. - A5. **GitHub/OCI source:** The remote content service used only during the acquisition phase. @@ -92,18 +92,18 @@ registry, or another profile configuration file for the initial use case. application authentication or caller ownership. Explicit `0.0.0.0` binding is valid. (session-settled: user-directed.) Governs R4-R5. - **Reuse workspace configuration.** Project `workspace.yaml` owns sources; - user `workspace.yaml` owns profiles, launchers, and exposure. There is no - `gateway.yaml`. (session-settled: user-directed.) Governs R6-R8, R18. + user `workspace.yaml` owns profiles, launchers, and gateway enablement. There + is no `gateway.yaml`. (session-settled: user-directed.) Governs R6-R8, R18. - **Support two acquisition modes.** Direct declared repositories and named, digest-pinned OCI workspace snapshots converge on one manifest and evidence contract. (session-settled: user-directed.) Governs R9-R11. - **Use App-first GitHub credential eligibility.** Prefer an applicable GitHub App; use a configured `gh` account only when no App installation applies; never fall back after selected-App failure. (session-settled: user-directed.) - Governs R10-R11. + Governs R12. - **Keep a typed backend seam.** Codex SDK and Pi RPC are the complete initial backend set. Launcher-backed profiles resolve through those adapters rather - than executing generated wrapper files. Governs R7-R8, R12-R15. + than executing generated wrapper files. Governs R7-R8, R13-R15. - **Persist Task truth, not provider sessions.** Restart settles interrupted work failed; it never resumes or automatically replays provider execution. Governs R5, R13-R16. @@ -120,16 +120,29 @@ registry, or another profile configuration file for the initial use case. - R1. Implement A2A 1.0 HTTP+JSON for Agent Card discovery, `SendMessage`, `GetTask`, `ListTasks`, `CancelTask`, streaming send, and active Task - subscription when advertised. The Agent Card declares + subscription. Every A2A request carries `A2A-Version: 1.0`; another version + receives `VersionNotSupportedError`. The Agent Card advertises exactly one + absolute interface URL with `protocolBinding: "HTTP+JSON"`, + `protocolVersion: "1.0"`, and `capabilities.streaming: true`. It declares `https://allagents.dev/a2a/extensions/coding-execution/v1` with - `required: true`. Every operation that creates, returns, lists, subscribes to, - or mutates profiled Tasks or Artifacts must include - `A2A-Extensions: https://allagents.dev/a2a/extensions/coding-execution/v1`; - responses echo the activated URI, and unsupported calls receive A2A + `required: true` and strict `params: { targets: TargetId[] }`, populated from + ready built-in and gateway-enabled profile targets. Production interface URLs + use HTTPS; direct HTTP is limited to loopback development. Every operation + that creates, returns, lists, subscribes to, or mutates profiled Tasks or + Artifacts includes that URI in `A2A-Extensions`; missing activation receives `ExtensionSupportRequiredError`. + + Honor both `SendMessageConfiguration.returnImmediately` modes. `ListTasks` + implements every standard filter, cursor pagination, `pageSize` 1-100 with a + default no greater than 50, descending status-timestamp order, and required + `tasks`, `nextPageToken`, `pageSize`, and `totalSize` fields. + `nextPageToken` is present and empty on the final page. With the default + `includeArtifacts: false`, each returned Task omits `artifacts`; `true` + includes the field. - R2. Generate a strict versioned request schema from Zod and place it only at - `Message.metadata[extensionUri]`. Strict objects reject every unlisted member. - V1 uses these wire scalars: + `Message.metadata[extensionUri]`; the Message also lists `extensionUri` in + `Message.extensions`. Strict objects reject every unlisted member. V1 uses + these wire scalars: - `InvocationKey` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. - `ConfigName` and `TargetId` match `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`. @@ -169,38 +182,53 @@ registry, or another profile configuration file for the initial use case. formats, defaults, coercion, non-finite numbers, duplicate canonical enum values, and unknown keywords are rejected. The canonical result schema is at most 64 KiB, 256 nodes, and 32 levels deep. Repository revision count cannot - exceed declared repositories. The Message contains exactly one `TextPart` - whose UTF-8 prompt is 1 byte to 1 MiB; other Part kinds are rejected. Only the - extension-owned metadata object is strict; unrelated A2A metadata and other - activated-extension keys are preserved or ignored according to A2A. - Canonicalization materializes defaults, normalizes extension strings to UTF-8 - NFC, sorts record keys, and hashes RFC 8785 extension JSON plus prompt bytes. - Do not add `Task.extensions` or backend-specific public fields. + exceed declared repositories. The Message contains exactly one `Part` with + `text` set to a UTF-8 prompt of 1 byte to 1 MiB; other Part content fields are + rejected. Only the extension-owned metadata object is strict; unrelated A2A + metadata and other activated-extension keys are preserved or ignored + according to A2A. Canonicalization materializes defaults, normalizes extension + strings to UTF-8 NFC, sorts record keys, and hashes RFC 8785 extension JSON + plus prompt bytes. Do not add `Task.extensions` or backend-specific public + fields. + + A client generates an opaque invocation key with at least 128 bits of + randomness once per logical execution, durably reuses that key and identical + canonical request after an ambiguous transport failure, and creates a new key + only for intentionally new execution. A2A `messageId` remains Message identity + and does not replace the extension idempotency key. - R3. One valid new request creates one addressable Task. Follow-up messages to an existing Task are unsupported. Every terminal Task has exactly one integrity Artifact plus zero or more produced Artifacts. The integrity Artifact has `artifactId` and `name` equal to - `allagents.execution-integrity` and one `DataPart` whose strict - `allagents.execution-integrity/v1` object has the following normative wire - shape. `SafeUInt` is an integer 0-9,007,199,254,740,991; `ShortText` is valid - UTF-8 of at most 4096 bytes; `ArtifactId` matches + `allagents.execution-integrity`, lists `extensionUri` in + `Artifact.extensions`, and has one `Part` with `data` set to the strict + `allagents.execution-integrity/v1` object and `mediaType: + "application/json"`. Its `taskId` equals the enclosing A2A `Task.id`. + `SafeUInt` is an integer 0-9,007,199,254,740,991; `ShortText` is valid UTF-8 + of at most 4096 bytes; `ArtifactId` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`; and `MediaType` is a valid RFC 6838 media type of at most 255 ASCII bytes. - `version` is the literal `"1"`; `taskId` is a lowercase canonical UUIDv7; and `target` is `TargetId`. - `sourceIdentity` is either - `{ kind: "repositories", repositories }` or + `{ kind: "repositories", complete, repositories }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, - workspaceManifestDigest: Digest, repositories }`. - `repositories` contains 1-64 unique strict entries + workspaceManifestDigest: Digest, layerDigests, complete, repositories }`. + `complete` is boolean. `layerDigests` contains 0-64 `Digest` values in + manifest order. `repositories` contains 0-64 unique strict entries `{ name: ConfigName, requestedRevision?: RevisionText, resolvedCommit: string, verification: "independentlyVerified" | "snapshotAttested" }`; `resolvedCommit` matches - `^[0-9a-f]{40}$`. Gateway-generated source identity, workspace-manifest - fields, evidence metadata, and provider-added metadata never contain Git - URLs, OCI repository origins, or destination paths. This guarantee does not - inspect or sanitize opaque caller prompts, provider terminal output, or - produced-Artifact payloads. + `^[0-9a-f]{40}$`. Before provider execution, `complete` must be true, + `repositories` must exactly match the configured catalog, and snapshot + identities must include every layer digest. Failed acquisition records only + verified members and sets `complete` false. + - Gateway-generated source identity, workspace-manifest fields, evidence + metadata, and provider-added metadata never contain Git URLs, OCI repository + origins, or destination paths. This guarantee applies only to + gateway-managed credentials and generated metadata; it does not inspect or + sanitize opaque prompts, terminal output, structured results, native + evidence, or produced-Artifact payloads. - optional `workspaceManifestDigest` is `Digest`. - `terminalOutput` is `{ text, truncated }`, where `text` is valid UTF-8 of at most 1 MiB and `truncated` is boolean. @@ -210,7 +238,7 @@ registry, or another profile configuration file for the initial use case. - `producedArtifacts` contains 0-128 strict entries `{ artifactId: ArtifactId, name?: ShortText, mediaType?: MediaType, size: SafeUInt, digest: Digest }`; each references one additional A2A - Artifact. + Artifact embedded in the Task. - `evidence` is `{ items, complete, truncated }`, where the booleans have their literal JSON meaning and `items` contains 0-256 strict entries `{ kind, artifactId?, digest?, summary? }`. `kind` is one of @@ -234,7 +262,7 @@ registry, or another profile configuration file for the initial use case. `{ path, keyword, message }` entries: `path` is an RFC 6901 JSON Pointer of at most 1024 bytes, `keyword` is one of the v1 `SchemaNode` member names, and `message` is `ShortText`. `reason` is one of - `notRequested | providerDidNotReturn | providerFailed | cancelled | + `notRequested | providerDidNotReturn | providerFailed | canceled | deadlineExceeded | invalidProviderPayload`. Failure, rejection, and cancellation retain every available field without @@ -247,43 +275,63 @@ registry, or another profile configuration file for the initial use case. - R4. Do not authenticate application callers. Allow loopback, specific-address, and explicit `0.0.0.0` listeners. Every reachable caller may create, list, - retrieve, subscribe to, cancel, and fetch Artifacts for every Task. Document - Tailscale ACLs, firewalls, or equivalent network controls as the authorization - boundary. + retrieve, subscribe to, and cancel every Task. Artifacts are retrieved only + inside Tasks through `GetTask` or `ListTasks(includeArtifacts: true)`; v1 adds + no separate Artifact endpoint. Document Tailscale ACLs, firewalls, or + equivalent network controls as the authorization boundary. - R5. Idempotency and Task visibility are deployment-wide. Atomically and durably bind an invocation key to the canonical request, selected target, source identity, optional result-schema digest, deadline, and effective - configuration digest before acknowledging Task creation. Identical replay - returns the existing Task; a changed request conflicts. The project-specific - state root persists the canonical workspace identity and holds an exclusive - process lock. The root and all state files must be current-user owned, use - `0700`/`0600`-equivalent permissions, be disjoint from project, profile, and - invocation roots, and be opened descriptor-relatively without following - symlinks or accepting hard-linked files. Startup verifies those invariants, - store integrity, and workspace identity; terminalizes interrupted Tasks - failed; and never resumes provider work. Store open, corruption, write, - rename, or fsync failure stops admission, aborts and contains active work, - prevents terminal success, and exits only after quiescence or a validated - external manager accepts cleanup ownership. + configuration digest before acknowledging Task creation. One transactional + `createOrReplay` operation arbitrates competing requests. Identical replay + returns the existing Task; a changed request conflicts. Status and terminal + settlement are monotonic. The project-specific state root persists the + canonical workspace identity and holds an exclusive process lock. The root + and all state files must be current-user owned, use `0700`/`0600`-equivalent + permissions, be disjoint from project, profile, and invocation roots, and be + opened descriptor-relatively without following symlinks or accepting hard- + linked files. Startup verifies those invariants, store integrity, and + workspace identity; terminalizes interrupted Tasks failed; and never resumes + provider work. A durable commit fsyncs every changed file and affected + containing directory before acknowledgment. Store open, corruption, write, + transaction, rename, or fsync failure stops admission, aborts and contains + active work, prevents terminal success, and keeps the process alive with + poisoned readiness until the containment set is proven empty. **Workspace and target configuration** - R6. One gateway process serves one project workspace selected by `--workspace` or cwd. Parse its `.allagents/workspace.yaml` through the authoritative project - schema. Repository names, URLs, destinations, default revisions, workspace - projection, plugins, and named OCI snapshot repositories come only from that - declaration. + schema, then compile a gateway-only repository catalog without tightening + ordinary workspace parsing. Derive each logical name from `name` or the + existing path-basename fallback; require 1-64 unique `ConfigName` values, + collision-free normalized destinations, and one supported canonical GitHub + origin resolved through existing `source`/`repo` semantics. Repository names, + origins, destinations, default revisions, workspace projection, plugins, and + named OCI snapshot repositories come only from that declaration. A catalog + failure makes the gateway not ready. - R7. Parse `~/.allagents/workspace.yaml` through the authoritative user schema. Built-in `codex` and `pi` targets are available when ready. A launcher-bearing - profile client adds a target only when `gateway.expose: true`. Its public ID is - the globally collision-checked launcher basename and resolves to exactly one - `(profile, client)` pair. Built-in IDs are reserved under the same portable - collision key; colliding exposure is a configuration error. Initially only + profile client adds a target only when `gateway.enabled: true`. Its public ID + is the globally collision-checked launcher basename and resolves to exactly + one `(profile, client)` pair. Built-in IDs are reserved under the same portable + collision key; colliding enablement is a configuration error. Initially only Codex and Pi profile clients are executable. - R8. A request selects a declared target, may set the bounded - `deadlineSeconds`, and may provide one bounded result schema. The overall - deadline covers acquisition, publication, typed preparation, provider - execution, and evidence collection. Acquisition receives + `deadlineSeconds`, and may provide one bounded result schema. The gateway owns + one durable execution lease covering acquisition through final evidence + collection. Admission claims that lease transactionally before launching any + helper child; at most one Task may hold it. A second otherwise-valid request + is accepted as a Task and settles failed with + `execution_capacity_unavailable`. It may transiently allocate one empty, + start-gated containment set, but never releases the gate, starts acquisition, + or executes a child and must destroy that set after the failure settlement. + Lease identity is stored with the Task, survives restart, and is released only + by the final settlement transaction or startup reconciliation after the + recorded containment set is proven empty. + + The overall deadline covers acquisition, publication, typed preparation, + provider execution, and evidence collection. Acquisition receives `min(900 seconds, remaining overall deadline)`; exceeding that sub-budget fails before provider execution. Overall expiry initiates abort and bounded forced termination. Cleanup then uses its own fixed bounded budget and the @@ -291,7 +339,7 @@ registry, or another profile configuration file for the initial use case. executable path, command, argv, environment, profile settings, plugins, MCP servers, repository URLs, destination paths, credential provider, setup behavior, or permission policy. Readiness rejects missing, partial, drifted, - unsupported, or declaration-missing exposed profiles. + unsupported, or declaration-missing gateway-enabled profiles. **Workspace acquisition** @@ -301,37 +349,68 @@ registry, or another profile configuration file for the initial use case. Unknown variants, cross-variant fields, undeclared names, mutable snapshot references, malformed digests, and destination overrides fail admission. Source-mode failure never falls through to the other mode. -- R10. Repository mode materializes every configured repository required by the - selected project workspace. Caller revisions may override only a declared - repository's default revision. Canonicalize HTTPS GitHub origins, resolve and - record full commits before provider execution, use hermetic Git configuration, - disable redirects and repository-controlled secondary fetch/exec features, - verify checkout identities, and reject path collisions or escapes. +- R10. Repository mode materializes every entry in the compiled gateway catalog. + Caller revisions may override only a declared repository's default revision. + Canonicalize HTTPS GitHub origins, resolve and record full commits before + provider execution, use hermetic Git configuration, disable redirects and + repository-controlled secondary fetch/exec features, verify checkout + identities, and reject path collisions or escapes. - R11. Snapshot mode maps `snapshot` to a declared OCI repository and constructs - `@` server-side. Validate registry origin, OCI manifest - and layer digests, expected workspace-manifest digest, paths, symlinks, file - types, file/layer counts, individual and total sizes, and manifest - completeness in staging before atomic publication. Reject external or foreign - layers and cross-origin credential forwarding. The common workspace manifest + `@` server-side. V1 accepts only + `application/vnd.oci.image.manifest.v1+json` with `schemaVersion: 2` directly + at the requested digest. Reject image indexes, nested indexes, descriptor + `urls` or embedded `data`, non-distributable layers, unknown media types, and + more than 64 layers. The config descriptor must use + `application/vnd.allagents.workspace-manifest.v1+json`; its digest must equal + `workspaceManifestDigest`, and its bytes are RFC 8785 canonical JSON. Accepted + layer media types are the OCI distributable tar, gzip, and zstd variants. + Verify the raw manifest body and every config/layer descriptor size and digest + while streaming, before decoding. Apply layers base-to-top with OCI whiteout + and opaque-whiteout semantics. + + The workspace manifest must contain every compiled project repository exactly + once at its operator-declared destination; reject missing, extra, renamed, + misplaced, or duplicate repositories and undeclared generated content. Apply + these fixed v1 ceilings across all processed layers, including overwritten or + whiteouted content: 4 MiB manifest, 4 MiB config, 2 GiB total compressed + layer bytes, 8 GiB total expanded bytes, 250,000 entries, 1 GiB per regular + file, 4096 UTF-8 bytes and 128 components per path, and 1 MiB per PAX or other + extended header. Abort before crossing a limit. Validate paths, collisions, + file types, modes, links, and manifest completeness in staging before atomic + publication. Reject absolute or traversing paths, devices, sockets, sparse + files, escaping links, credentials in redirect URLs, unapproved cross-origin + redirects, and external layers. Cross-origin redirects are limited to + layer-blob `GET`/`HEAD` requests and exact operator-declared + `layerRedirectHosts`; token, manifest, and config requests remain same-origin. + Private or otherwise non-global destinations are permitted only when the exact + host is the source's declared repository host or a declared layer-redirect + host, with per-hop rebinding checks. The common workspace manifest distinguishes independently verified Git facts from snapshot-attested facts. **Credential selection and containment** - R12. Repository requests never carry credentials or select providers. For - `github.com`, determine configured-App applicability as - `eligible | ineligible | unknown` through an App-authenticated GitHub API - client, or verify an explicit installation ID against the repository. For - `eligible`, mint a fresh repository-scoped, read-only installation token - through `@octokit/auth-app` and require remaining lifetime greater than the - R8 acquisition sub-budget plus a 60-second clock-skew margin. Use the - configured GitHub CLI account only when the App is absent or applicability is - positively `ineligible`. An `unknown` result or any selected-App - configuration, authentication, minting, permission, repository, rate-limit, - or service failure terminates acquisition without `gh` fallback. Run - `gh auth token --hostname github.com --user ` with ambient token + `github.com`, a configured App lookup returning installation coverage is + `eligible`. A 404 is `ineligible` only after the repository's existence is + independently proven through the configured GitHub CLI identity; an + uncorroborated 404, 401, 403, 429, timeout, or 5xx is `unknown`. An explicit + installation ID is eligible only after positive repository-coverage + verification. For `eligible`, call `@octokit/auth-app` with `refresh: true` + and the exact repository selection to mint a new read-only installation token + for every acquisition. Validate its repository selection, permissions, + creation time, and expiry, and require remaining lifetime greater than the R8 + acquisition sub-budget plus a 60-second clock-skew margin. + + Use the configured GitHub CLI account only when the App is absent or + applicability is positively `ineligible`. An `unknown` result or any + selected-App configuration, authentication, minting, permission, repository, + rate-limit, or service failure terminates acquisition without `gh` fallback. + Run `gh auth token --hostname github.com --user ` with ambient token variables removed. Deliver either token only through an invocation-scoped Git - credential helper and destroy it before typed preparation or provider - execution. OCI credentials likewise exist only during snapshot acquisition. + credential helper. Revoke an App token after acquisition and fail before + provider execution if revocation cannot be confirmed; destroy all local token + material before typed preparation. OCI credentials likewise exist only during + snapshot acquisition. **Execution, evidence, and cleanup** @@ -343,48 +422,92 @@ registry, or another profile configuration file for the initial use case. discover executables as targets from `PATH`, scrape a TUI, or append public input to argv. - R14. Codex uses pinned `@openai/codex-sdk`, one fresh thread per Task, - `AbortSignal`, streamed events, optional native `outputSchema`, and an - operator-selected Codex auth-file handle. Pi uses strict RPC, invocation-owned - configuration, an operator-selected Pi auth-file handle, and one restricted - policy extension; repository extensions and unrestricted built-ins do not - auto-load. The gateway copies only the selected adapter's required auth - material into an invocation-private, read-only control-process view and - removes it during cleanup. + `AbortSignal`, streamed events, and an operator-selected Codex auth-file + handle. It passes native `outputSchema` only when the public schema has an + object root, every object's `required` set equals its property set, nesting is + at most 10 levels, and every keyword is supported by the pinned model/API. + Other valid public schemas use explicit JSON prompt guidance plus the common + gateway-side validator without a native schema. Pi uses strict RPC, + invocation-owned configuration, an operator-selected Pi auth-file handle, and + one restricted policy extension; repository extensions and unrestricted + built-ins do not auto-load. The gateway copies only the selected adapter's + required auth material into an invocation-private, read-only control-process + view and removes it during cleanup. - R15. Acquire into a private staging root and atomically publish the invocation workspace. Run only adapter-owned typed preparation that projects validated project/profile settings, plugins, and MCP declarations through existing deterministic transforms; never execute project or user `setup` entries or - other configured shell commands. Enforce distinct process views: + other configured shell commands. + + Enforce distinct process views: - the provider control process receives only its invocation workspace, minimum non-secret profile configuration, and adapter auth channel; - each MCP child receives only its own resolved secret references; and - model-invoked shell/tools receive the workspace and no provider or MCP credentials. - All views exclude gateway state, operator home, App keys, GitHub/OCI stores, - source helpers, unrelated adapter credentials, and the parent environment. - Fail readiness for a target when its adapter cannot enforce those separations. - Acquisition credentials and mounts are absent first. Treat the mutated - workspace as untrusted during evidence collection: use descriptor-relative - no-follow reads; reject hard links, special/sparse files, path replacement, - out-of-root targets, and `.git` gitdir/core.worktree/alternates escapes; and - run Git inspection with hermetic configuration that disables hooks, filters, - drivers, fsmonitor, pagers, helpers, and external commands. + + The pinned backend must expose one non-bypassable synchronous spawn hook for + every MCP and model-tool process. The hook delegates execution to the security + helper, which enters the role-specific mount and network namespaces, replaces + the environment, closes every non-allowlisted descriptor, and only then + executes untrusted code. A backend that can spawn any tool without this hook + is not a v1 target and fails readiness; conformance fixtures alone cannot waive + that requirement. All views exclude gateway state, operator home, App keys, + GitHub/OCI stores, source helpers, unrelated adapter credentials, and the + parent environment. + + Invocation network namespaces cannot route to host loopback, any gateway bind + or advertised address, operator management networks, or ingress proxies. + Provider and MCP egress is default-deny except for role-specific destinations + compiled from adapter and MCP configuration; every resolved address is checked + at connection time, and gateway/host-management destinations remain denied + even when a hostname resolves to them. Model tools receive no network unless + the adapter's explicit policy grants similarly constrained egress. Fail target + readiness unless all filesystem, credential, descriptor, and network + separations are enforceable. + + Acquisition credentials and mounts are absent first. Capture bounded provider + events while the process is live. After the provider reports terminal, abort + and terminate its complete containment set and prove it empty before reading + Git state, hashing or copying files, or describing produced Artifacts as + verified. Treat the mutated workspace as untrusted: use descriptor-relative + no-follow reads; revalidate identity and size after open; reject hard links, + special/sparse files, path replacement, out-of-root targets, and `.git` + gitdir/core.worktree/alternates escapes; and run Git inspection with hermetic + configuration that disables hooks, filters, drivers, fsmonitor, pagers, + helpers, and external commands. If quiescence cannot be proven, retain only + truthful partial process evidence; do not publish filesystem evidence or + produced Artifacts as verified. - R16. Supervise the complete acquisition/provider descendant set inside an invocation-owned OS containment primitive whose membership children cannot - escape. Fail readiness when the platform cannot enforce and inspect that - boundary. Cancellation persists intent with an atomic state transition before - native abort, then applies bounded forced termination. A terminal commit that - wins first makes later cancellation not cancelable; cancellation intent that - wins settles cancelled after quiescence. Terminal cleanup evidence requires - proof that the containment set is empty. Failure records termination - unknown/failed and rejects admission. An unmanaged foreground gateway remains - alive with poisoned readiness and continues reaping; it prints the stable - containment identifier and platform recovery command. It may exit with a - nonempty set only after a validated external manager accepts ownership. Startup - proves every interrupted set empty before it may quarantine stale roots or - advertise readiness. Graceful shutdown stops admission atomically, persists - shutdown/cancellation intent, drains or aborts active work within a bounded - grace period, proves quiescence, settles once, and only then exits. + escape. Allocate its stable identifier and empty set first, then commit that + identity with the Task and execution lease before the helper may release its + start gate or execute any child. A failed commit destroys the still-empty set. + Startup enumerates the entire project-owned containment namespace, reconciles + both recorded and unknown identifiers, and refuses readiness while any + unknown or nonempty set remains. + + One durable compare-and-set arbitrates provider terminal outcome, caller + cancellation, overall deadline, and shutdown as an internal `outcomeIntent` + while the externally visible Task remains nonterminal. The winning intent + owns the stable result or failure code and drives one idempotent abort and + quiescence path. Only after quiescence, safe evidence collection, produced- + Artifact verification, and cleanup does one settlement transaction atomically + write terminal Task status, result/failure, bounded evidence, exactly one + integrity Artifact, produced Artifacts, termination outcome, lease release, + and cleanup outcome. + + Cancellation intent persists before native abort, followed by bounded forced + termination. If quiescence cannot be proven, settle once with + `execution_quiescence_unknown`, no verified filesystem evidence, and immutable + unknown/failed termination; reject admission, keep readiness false, and leave + the process alive to continue reaping. Later recovery changes only internal + recovery/readiness state, never the settled Task. Print the stable containment + identifier and platform recovery command. Startup proves every interrupted set + empty before it may quarantine stale roots or advertise readiness. Graceful + shutdown stops admission atomically, commits shutdown intent, drains or aborts + active work within a bounded grace period, follows the same settlement path, + and only then exits. **Scope and configuration** @@ -392,29 +515,44 @@ registry, or another profile configuration file for the initial use case. repetitions, experiment scheduling, or automatic Task retry. - R18. Do not add `gateway.yaml` or `worker.yaml`. Process configuration uses the exact CLI flags and environment variables in the Configuration Contract - for listener, workspace, state/retention, GitHub, OCI, and Codex/Pi auth-file - handles. Secret values never enter workspace files, requests, logs, Tasks, - Artifacts, retained workspaces, or model-invoked tool environments. + for listener, advertised interface URL, workspace, state/retention, GitHub, + OCI, and Codex/Pi auth-file handles. The listener also exposes unauthenticated + metadata-only `/healthz` and `/readyz` endpoints outside A2A: liveness returns + 200 while the process can serve; readiness returns 200 only while new + admission is safe and otherwise 503. They reveal no targets, sources, paths, + or failure details and do not require A2A headers. Gateway code never copies + acquisition or provider credential values into generated workspace files, + requests, logs, Task/Artifact metadata, retained workspaces, or model-tool + environments. This is not a redaction guarantee for opaque prompts, provider + output, structured results, native evidence, or produced-Artifact payloads. - R19. Document AI Evals consumption through a Promptfoo custom JavaScript/TypeScript provider implementing Promptfoo's `ApiProvider`. - `constructor(options: ProviderOptions)` retains `options.id`, validates - `options.config`, and `id()` returns the retained ID. Static config contains - the gateway endpoint, target ID, and exactly one closed source mode: - repository mode materializes the complete configured repository set and - carries only an optional revision map keyed by declared repository name; - snapshot mode carries one declared snapshot name with OCI and workspace- - manifest digests. `callApi(prompt, context, options)` may apply the exact - `context.vars.allagentsSource` leaf overrides defined below. Dynamic - repository revisions must be full lowercase 40-hex commit IDs; dynamic - snapshot values must be full lowercase `sha256:` digests. Source kind, - snapshot name, and repository origins never vary per test. Unknown members, - revision names absent from static config, URLs, destinations, tags, - credentials, commands, and permission policy fail before submission. - `options?.abortSignal` and the provider's bounded deadline both invoke A2A - `CancelTask` after acceptance. One `callApi` creates one A2A Task and maps - terminal output, usage, Task/Artifact IDs, structured result, and logical - provenance into `ProviderResponse`; admission or terminal failure maps to - `error`. AI Evals owns the provider implementation. AllAgents publishes the + `constructor(options: ProviderOptions)` requires and retains a nonempty + `options.id`, validates `options.config`, and `id()` returns that ID. Static + config contains the + gateway endpoint, target ID, and exactly one closed source mode: repository + mode materializes the complete configured repository set and carries only an + optional revision map keyed by declared repository name; snapshot mode carries + one declared snapshot name with OCI and workspace-manifest digests. + `callApi(prompt, context?, options?)` may apply the exact + `context?.vars?.allagentsSource` leaf overrides defined below; missing context + means no override. Dynamic repository revisions must be full lowercase + 40-hex commit IDs; dynamic snapshot values must be full lowercase `sha256:` + digests. Source kind, snapshot name, and repository origins never vary per + test. Unknown members, revision names absent from static config, URLs, + destinations, tags, credentials, commands, and permission policy fail before + submission. + + The provider sends `SendMessage` with `configuration.returnImmediately: true`, + captures the accepted Task ID, and calls `SubscribeToTask`; a terminal-before- + subscribe race or broken stream falls back to `GetTask` and resubscription + within the same deadline. A deadline or `options?.abortSignal` issues exactly + one `CancelTask` with a fresh bounded cleanup signal rather than the already + aborted request signal. One `callApi` creates one A2A Task and maps terminal + output, usage, Task/Artifact IDs, structured result, and logical provenance + into `ProviderResponse`. Admission or terminal failure maps a safe human + message to `error` and stable `code`, `retryable`, and accepted `taskId` to + `metadata`. AI Evals owns the provider implementation. AllAgents publishes the protocol and YAML examples without importing Promptfoo provider code or adding Promptfoo as a runtime dependency. @@ -422,169 +560,220 @@ registry, or another profile configuration file for the initial use case. - F1. **Start and advertise** 1. Resolve cwd or `--workspace`, user workspace, project-specific state root, - retention limits, listen address, source credentials, and provider auth + retention limits, listen address, advertised interface URL, source + credentials, and provider auth handles. + 2. Validate state-root ownership, permissions, links, disjointness, workspace + identity, compiled repository catalog, snapshots, target namespace, backend + availability, profile state, Linux containment/helper availability, + provider/MCP/tool mount, descriptor, and network views, and credential handles. - 2. Validate state-root ownership, permissions, links, disjointness, and - workspace identity; acquire the exclusive lock; validate repositories, - snapshots, target namespace, backend availability, profile state, - containment, separate provider/MCP/tool views, and credential handles. - 3. Reconcile interrupted Tasks and prove every stale containment set empty - before quarantining filesystem roots. - 4. Bind the requested address, including `0.0.0.0` when explicit, and publish - one Agent Card whose required extension and allowlisted targets match the + 3. Enumerate the entire project-owned containment namespace. Reconcile + recorded and unknown identifiers and prove every set empty before + quarantining filesystem roots or releasing a retained execution lease. + 4. Bind the requested address, including `0.0.0.0` when explicit; serve + metadata-only health/readiness probes; and publish one Agent Card whose + absolute interface URL, required extension, and target allowlist match the validated configuration. - F2. **Acquire repositories and execute** - 1. Negotiate the required extension and validate the strict request, one text - prompt, target, repository-name/revision map, result schema, deadline, and - deployment-wide idempotency claim. - 2. Durably commit the claim and Task before acknowledgment; create the - invocation containment and staging root. - 3. For each declared repository, classify App applicability, select App or - `gh` only by eligibility, resolve the revision, fetch hermetically, verify - the commit, and remove credentials. + 1. Negotiate A2A version and the required extension, then validate the strict + request, one text Part, target, repository-name/revision map, result schema, + deadline, and deployment-wide idempotency claim. + 2. Ask the helper to allocate a stable empty containment set behind a start + gate. In one transaction, create or replay the claim and Task, acquire the + execution lease, and bind the containment identifier before acknowledgment. + Capacity failure settles the Task with `execution_capacity_unavailable`, + then destroys the empty set without releasing the gate or launching a + child. Commit failure likewise destroys the empty set. + 3. Release the start gate. For each declared repository, classify App + applicability, select App or `gh` only by eligibility, resolve the revision, + fetch hermetically, verify the commit, revoke an App token, and remove every + acquisition credential. 4. Publish the complete workspace, run typed preparation, invoke the isolated - adapter, validate any structured result, collect evidence through safe - reads, terminate descendants, clean up, and settle the Task once. + adapter, and validate any structured result while capturing live events. + Terminate and prove the containment set empty before safe filesystem/Git + evidence reads and produced-Artifact verification. Atomically settle the + terminal Task, evidence, Artifacts, cleanup, and lease release. - F3. **Acquire an OCI snapshot and execute** - 1. Resolve the named snapshot repository and digest-pinned reference. - 2. Authenticate if required, pull and verify the OCI manifest and layers, - extract safely, and validate the workspace-manifest digest. + 1. Perform the same version/extension validation, gated empty-containment + allocation, and atomic claim+Task+lease+containment commit as F2. + 2. Resolve the named snapshot repository and digest-pinned reference. + Authenticate if required; pull and verify the direct image manifest, + workspace-manifest config blob, and distributable layers; apply changesets + in order; enforce all limits; and validate the exact project catalog. 3. Remove registry credentials, publish atomically, run typed preparation, - invoke the isolated adapter, collect safe evidence, clean up, and settle. + invoke the isolated adapter, and capture live events. Terminate and prove + quiescence before safe filesystem evidence and verified produced Artifacts, + then perform the same atomic settlement and lease release as F2. - F4. **Cancel** 1. Atomically persist cancellation intent if the Task remains cancelable. 2. Abort acquisition or provider work, escalate within the bounded termination - budget, prove containment quiescence, preserve partial evidence, clean up, - and settle cancelled. + budget, prove containment quiescence, preserve truthful partial evidence, + clean up, and settle canceled. 3. Repeated cancellation while intent is pending does not re-signal work. Cancellation after any terminal state returns A2A `TaskNotCancelableError`. - F5. **Shut down** 1. Stop new admission before signaling active work. - 2. Persist shutdown cancellation intent, abort and escalate, drain evidence, + 2. Persist shutdown intent, abort and escalate, drain live process evidence, prove quiescence, and settle the accepted Task once. 3. Exit only after durable settlement and empty containment. If proof fails, - unmanaged mode remains alive, not ready, and continues reaping while - printing the platform recovery command. Managed mode may exit only after - its validated external manager accepts containment ownership. + remain alive, not ready, and continue reaping while printing the stable + containment identifier and platform recovery command. - F6. **Invoke from Promptfoo** 1. Promptfoo constructs the AI Evals-owned TypeScript provider with `ProviderOptions`; the provider retains the ID and validates `options.config` containing the private-network endpoint, target, and one closed source-mode object. - 2. `callApi(prompt, context, options)` applies only valid - `context.vars.allagentsSource` leaf overrides, creates one invocation key, - and sends one A2A Message with the prompt and required extension. - 3. The provider waits or streams until terminal. Its deadline or - `options?.abortSignal` sends `CancelTask` once after acceptance and waits - for the same terminal cleanup path. - 4. It returns terminal text or validated structured output in - `ProviderResponse.output`; maps `inputTokens -> prompt`, + 2. `callApi(prompt, context?, options?)` applies only valid + `context?.vars?.allagentsSource` leaf overrides, creates and retains one + high-entropy invocation key, and sends one A2A Message with + `configuration.returnImmediately: true`. + 3. After receiving the Task ID, subscribe to terminal updates. Resolve a + terminal-before-subscribe or disconnected-stream race through `GetTask` + and bounded resubscription. Deadline or abort sends `CancelTask` once with + a fresh cleanup signal. + 4. Return terminal text or validated structured output in + `ProviderResponse.output`; map `inputTokens -> prompt`, `outputTokens -> completion`, `cachedInputTokens -> cached`, and - `totalTokens -> total`; and puts other usage plus Task, Artifact, logical - source, termination, and cleanup facts in `metadata`. Admission or terminal - execution failure returns `error`. + `totalTokens -> total`; and put other usage plus Task, Artifact, logical + source, termination, cleanup, and stable failure facts in `metadata`. + Admission or terminal failure returns a safe `error`. ### Acceptance Examples - AE1. A caller on a permitted Tailscale or firewalled network discovers the - gateway bound to `0.0.0.0`, selects `codex-review`, and receives one durable - Task without presenting an application credential. -- AE2. Any reachable caller can list, retrieve, cancel, and fetch Artifacts for - a Task created by another reachable caller; documentation states this shared - trust model without implying tenant privacy. -- AE3. A launcher-bearing Codex profile without `gateway.expose: true` is absent - from discovery and rejected when selected. An exposed but drifted profile - fails readiness/new admission. -- AE4. A multi-client profile exposes `codex-review` and `pi-review` as distinct - targets. Both resolve through adapters; neither generated wrapper is executed. + gateway through its configured HTTPS interface URL while it is bound to + `0.0.0.0`, selects `codex-review`, and receives one durable Task without an + application credential. +- AE2. Any reachable caller can list, retrieve, and cancel a Task created by + another reachable caller and inspect its embedded Artifacts through `GetTask` + or `ListTasks(includeArtifacts: true)`; documentation states this shared trust + model without implying tenant privacy. +- AE3. A launcher-bearing Codex profile without `gateway.enabled: true` is + absent from discovery and rejected when selected. An enabled but drifted + profile fails readiness/new admission. +- AE4. A multi-client profile gateway-enables `codex-review` and `pi-review` as + distinct targets. Both resolve through adapters; neither generated wrapper is + executed. - AE5. Repository mode accepts declared names and revision overrides, rejects an undeclared name or URL override, and records the resolved full commits. -- AE6. An applicable GitHub App mints a fresh repository-scoped token whose - lifetime exceeds the acquisition sub-budget plus skew. A repository with no - applicable installation uses the configured `gh` account. Unknown App - applicability, auth, or mint failure does not fall through to `gh`. -- AE7. Snapshot mode accepts a declared snapshot name and matching OCI/workspace - digests, rejects mutable tags, traversal, foreign layers, digest mismatch, or - undeclared registry repositories, and publishes only after full validation. -- AE8. Repository and snapshot modes produce the same workspace-manifest shape, - while OCI-contained commit identities remain marked snapshot-attested unless - independently verified. -- AE9. Identical invocation-key replay returns the original Task. Reusing the key - with a changed target, source, prompt, or result schema conflicts. +- AE6. An applicable GitHub App bypasses its token cache, mints a new + repository-scoped read-only token with adequate lifetime, validates the token, + and revokes it after acquisition. A corroborated existing repository with no + applicable installation uses the configured `gh` account. An uncorroborated + 404, unknown applicability, auth, mint, validation, or revocation failure does + not fall through to `gh` or start the provider. +- AE7. Snapshot mode accepts a direct image manifest with matching manifest, + config/workspace, and layer digests; applies gzip/zstd layers and whiteouts in + order; and enforces every fixed limit. Same-origin metadata redirects work; + only layer requests may cross origin to an exact declared host, with + credentials stripped and every resolved address checked. Mutable tags, + indexes, unknown/non-distributable media, descriptor URLs/data, traversal, + foreign layers, digest/size mismatch, malformed whiteouts, undeclared + repositories, redirect loops/rebinding, non-global destinations not declared + for that source, and unapproved origins fail. +- AE8. Repository and snapshot modes produce the same workspace-manifest shape + and exact compiled repository set/layout. OCI-contained commit identities are + snapshot-attested unless independently verified; source identity includes + completeness and ordered layer digests without origins. +- AE9. Identical invocation-key replay, including after a lost response, returns + the original Task. Reusing the key with a changed target, source, prompt, or + result schema conflicts; separate high-entropy keys create separate Tasks. - AE10. Cancellation during Git, OCI pull, Codex, or Pi terminates the complete - process set and records cleanup. Unproved quiescence poisons readiness; an - unmanaged foreground process stays alive and reaps, while managed exit - requires accepted external cleanup ownership. -- AE11. Restart turns interrupted Tasks into one terminal failure and never - resumes a provider session. Terminal Tasks and Artifacts remain retrievable - until expiry. + process set and records cleanup. Unproved quiescence poisons readiness; the + gateway stays alive, rejects admission, and continues reaping until empty. +- AE11. Kill fixtures before and after empty-containment creation, durable + Task/lease/containment binding, child clone, start-gate release, and response + acknowledgment leave no unrecorded live set. Restart enumerates the full + project-owned namespace, refuses unknown/nonempty sets, turns interrupted + Tasks into one terminal failure, never resumes a provider session, and keeps + terminal Tasks and embedded Artifacts retrievable until expiry. - AE12. A valid structured result survives later check or evidence failure as a valid result with an overall failed Task; invalid or absent results are never published as valid. -- AE13. An exposed launcher named `codex`, `pi`, or a portable case-equivalent - fails configuration compilation instead of shadowing a built-in target. +- AE13. A gateway-enabled launcher named `codex`, `pi`, or a portable case- + equivalent fails configuration compilation instead of shadowing a built-in + target. - AE14. Two gateways for different workspaces use distinct private state roots; a second process for the same root fails the exclusive lock. Wrong-owner, - permissive, linked, hard-linked, or overlapping roots fail startup. Store - fault injection cannot acknowledge an uncommitted Task or false success. -- AE15. Deadline expiry during Git, OCI, preparation, Codex, Pi, or evidence - initiates one abort/termination path and retains truthful partial evidence. + permissive, linked, hard-linked, or overlapping roots fail startup. The real + helper VFS rejects database, WAL, SHM, journal, temporary-file, symlink, + hard-link, and rename-swap attacks. Process-kill fixtures at transaction, file + sync, directory sync, and response boundaries recover either the complete old + or new generation and never lose an acknowledged Task or publish false + success. +- AE15. Barrier-controlled provider-terminal, caller-cancel, deadline, and + shutdown races durably select one internal intent and one abort/quiescence + path during Git, OCI, preparation, Codex, Pi, or evidence. Subscribers observe + no terminal Task until one transaction writes status, integrity Artifact, + bounded evidence, result/failure, termination, cleanup, and lease release. + Later reaping changes only internal readiness/recovery state. - AE16. Repeated cancel while cancellation is pending is idempotent; cancel - after cancelled, completed, failed, or rejected returns + after canceled, completed, failed, or rejected returns `TaskNotCancelableError`. - AE17. A workspace containing `setup` shell entries never executes them through - gateway acquisition or startup. Built-in Codex/Pi authenticate through their - selected private control-process auth views; model-invoked tools cannot read - provider or MCP secrets, operator stores, or gateway state. -- AE18. Evidence collection rejects a provider-created escaping link, hard link, - special file, sparse-file abuse, or `.git` indirection and runs Git inspection - without repository-controlled execution hooks. + gateway acquisition or startup. Real Codex/Pi child and grandchild tool paths + are helper-mediated: filesystem, environment, inherited descriptor, `/proc`, + and magic-link probes cannot read provider/MCP secrets, operator stores, or + gateway state. Agent Card, Task operations, host loopback, bind/advertised + addresses, ingress, and management-network probes fail from every invocation + role; only compiled role egress succeeds. +- AE18. Evidence is collected only after containment quiescence. An escaping + link, hard link, special file, sparse-file abuse, replaced inode, or `.git` + indirection is rejected and Git inspection runs without repository-controlled + execution. Unknown quiescence produces no verified filesystem Artifact. - AE19. The 1001st unexpired retained Task is rejected with - `retention_capacity_exhausted`; no retained Task is evicted before TTL. -- AE20. Unrelated Message metadata survives request processing. Every profiled - A2A operation requires activation, and a terminal Task may contain the single - integrity Artifact plus referenced produced Artifacts. -- AE21. The AI Evals Promptfoo provider loads one repository-mode and one - snapshot-mode YAML instance. Repository mode materializes the complete - configured set and sends only optional revision overrides keyed by declared - name; snapshot mode sends one declared name and immutable digests. Neither - request source metadata nor response source-identity metadata contains a Git - URL, OCI repository, or destination. - Both calls return scorable `ProviderResponse.output`, the exact normalized - token mapping, and Task/Artifact/logical-provenance metadata. Per-test - repository overrides accept only full commits. An unknown variable member, - mutable revision, origin, destination, or undeclared name fails before - submission. + `retention_capacity_exhausted`; no retained Task is evicted before TTL. While + one Task holds the execution lease, a barrier-controlled second request + settles `execution_capacity_unavailable` and launches no helper child; races + and restart never produce two lease holders. +- AE20. Official HTTP+JSON client fixtures send `A2A-Version: 1.0`, exercise + required-extension activation and both `SendMessage` modes, preserve unrelated + metadata, verify standard `google.rpc.Status` errors, and cover every + `ListTasks` filter, cursor, order, response field, and artifact-inclusion rule. + A terminal Task contains one extension-marked integrity Artifact plus + referenced produced Artifacts using unified Parts. +- AE21. The AI Evals Promptfoo fixture has a top-level prompt and disables + sharing, caching, result writes, and concurrency above one. It loads one + repository-mode and one snapshot-mode provider, sends only closed logical + source data, retains one invocation key across ambiguous retries, and cancels + an accepted Task on abort. Both calls return scorable output, normalized token + usage, and Task/Artifact/logical-provenance metadata. Safe failure metadata + includes code, retryability, and accepted Task ID. Calls with omitted context + work; unknown variables, mutable revisions, origins, destinations, or + undeclared names fail before submission. ### Success Criteria - `allagents gateway serve` starts from a real workspace with no deployment YAML. -- Explicit loopback, private-interface, and `0.0.0.0` listeners work. -- The official A2A client exercises required-extension negotiation, send, - stream, get, list, subscribe, replay, cancel, terminal cancel errors, Artifact - retrieval, and expiry. -- An AI Evals-style Promptfoo custom-provider fixture consumes representative - YAML for both source modes and maps a terminal Task to `ProviderResponse` - without adding Promptfoo to the AllAgents runtime. -- Built-in Codex/Pi and exposed profile targets pass one conformance suite, - including reserved-ID collisions. +- Explicit loopback, private-interface, and `0.0.0.0` listeners work with a + distinct valid advertised interface URL; health/readiness reflect admission. +- The official A2A client exercises version and extension negotiation, both send + modes, stream, get, complete list/pagination semantics, subscribe, replay, + cancel, terminal cancel errors, Task-embedded Artifacts, standard HTTP+JSON + errors, and expiry. +- An AI Evals-style Promptfoo custom-provider fixture consumes secure-default + YAML for both source modes, propagates post-acceptance cancellation, and maps a + terminal Task to `ProviderResponse` without adding Promptfoo to the AllAgents + runtime. +- Built-in Codex/Pi and gateway-enabled profile targets pass one conformance + suite, including reserved-ID collisions and Codex native-schema gating. - Direct Git and OCI snapshot fixtures produce equivalent validated workspace - manifests and truthful provenance. -- GitHub App eligibility, unknown failure, no-installation `gh` fallback, - selected-App failure, token lifetime, containment, and OCI credential cleanup - are proven end to end. + manifests and truthful complete provenance. +- GitHub App eligibility, 404 ambiguity, unknown failure, no-installation `gh` + fallback, fresh-token validation/revocation, OCI authentication and challenge + handling, and pre-provider credential teardown are proven end to end. - No request can supply a command, executable, URL, destination, credential, mutable OCI tag, backend override, or arbitrary environment value. -- State-store fault, deadline, cancellation-race, shutdown, descendant escape, - unsafe evidence, and stale-root scenarios fail closed. -- The built CLI passes a trusted-network smoke test against project and user - workspaces created under `/tmp/`. +- State-store crash, deadline/cancellation/terminal/shutdown race, descendant + escape, unsafe evidence, and stale-root scenarios fail closed. +- The bundled CLI and packaged Linux helper pass a trusted-network smoke test + against project and user workspaces created under `/tmp/`. ### Scope Boundaries @@ -592,7 +781,7 @@ registry, or another profile configuration file for the initial use case. - A2A 1.0 HTTP+JSON and the required AllAgents extension. - One process and one active invocation at a time initially. -- Built-in and exposed profile-backed Codex/Pi targets. +- Built-in and gateway-enabled profile-backed Codex/Pi targets. - Direct declared Git repositories and named OCI workspace snapshots. - GitHub App and configured GitHub CLI acquisition credentials. - Local durable Task/evidence storage, cancellation, cleanup, and provenance. @@ -610,6 +799,8 @@ registry, or another profile configuration file for the initial use case. delivery. - OpenCode, Claude, Copilot, OMP, arbitrary CLI, and TUI adapters. - Evaluation orchestration and automatic retries. +- Non-Linux gateway execution in v1; ordinary AllAgents CLI behavior remains + cross-platform. ### Sources @@ -617,12 +808,19 @@ registry, or another profile configuration file for the initial use case. - [AHP decision inputs](../research/agent-host-protocol-decision-inputs.md) - [Harbor repository materialization lessons](../research/harbor-repository-materialization.md) - [Source credential broker precedents](../research/source-credential-broker-precedents.md) -- [A2A 1.0 specification](https://a2a-protocol.org/latest/specification/) +- [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) +- [A2A extension guide](https://a2a-protocol.org/latest/topics/extensions/) +- [Promptfoo custom providers](https://www.promptfoo.dev/docs/providers/custom-api/) +- [Promptfoo configuration reference](https://github.com/promptfoo/promptfoo/blob/main/site/docs/configuration/reference.md) - [OpenAI Codex SDK](https://developers.openai.com/codex/sdk/) -- [OpenAI Codex app-server](https://developers.openai.com/codex/app-server/) +- [OpenAI structured outputs](https://developers.openai.com/api/docs/guides/structured-outputs/) - [GitHub App installation tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) - [Git credential helpers](https://git-scm.com/docs/gitcredentials) +- [Docker credential stores](https://docs.docker.com/reference/cli/docker/login/#credential-stores) +- [Node.js SQLite API](https://nodejs.org/docs/latest-v22.x/api/sqlite.html) - [OCI Image Specification](https://github.com/opencontainers/image-spec) +- [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec) +- [Linux cgroup v2](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html) --- @@ -630,59 +828,89 @@ registry, or another profile configuration file for the initial use case. ### Key Technical Decisions -- KTD1. **Use the official A2A JavaScript SDK behind a small AllAgents request - decorator.** The decorator validates the required extension and canonical - request, resolves the deployment-wide retained claim, performs new-admission - checks, and preserves standard Task/Artifact carriers. -- KTD2. **Generate public, Task-store, workspace-manifest, and adapter contracts - from canonical Zod schemas.** Keep the result-schema subset shared across - Codex and Pi and forbid backend-specific public fields. +- KTD1. **Use the official A2A JavaScript SDK transport around an + AllAgents-owned request handler.** Do not use `DefaultRequestHandler` or its + non-transactional `TaskStore` seam. Implement the SDK's request-handler + interface so AllAgents controls UUIDv7 creation, atomic `createOrReplay`, + monotonic settlement, listing, retention, expiry, and HTTP+JSON error details + while retaining standard Task/Artifact carriers. +- KTD2. **Generate and publish the extension and storage contracts from canonical + Zod schemas.** The versioned extension specification at its declared URI + defines Agent Card params, activation, Message metadata/extensions, request, + Task/Artifact, idempotency, error, replay, examples, and versioning. Generate + public JSON Schemas, Task-store, workspace-manifest, and adapter types from the + same source. Keep backend-specific fields private. - KTD3. **Use a single-process supervisor, not a remote worker protocol.** One service owns Task state, staging, publication, backend child processes, evidence, termination, and cleanup. Child processes remain contained behind an invocation lifecycle boundary. - KTD4. **Make application authentication intentionally absent.** All Tasks and Artifacts share one deployment namespace. The listener accepts explicit - `0.0.0.0`; network controls are external. (session-settled: user-directed.) -- KTD5. **Compile configuration from existing workspace files.** Add - `workspaceSnapshots` to the project schema and `gateway.expose` to strict - profile-client schemas. Resolve the public launcher ID to one profile/client. + `0.0.0.0`; network controls are external. Bind and advertised interface URL + are distinct. (session-settled: user-directed.) +- KTD5. **Compile gateway configuration from existing workspace files.** Add + `workspaceSnapshots` to the project schema and `gateway.enabled` to strict + profile-client schemas. A gateway-only compiler normalizes the project + repository catalog and resolves each public launcher ID to one profile/client. Add no deployment YAML. (session-settled: user-directed.) - KTD6. **Keep source input name-based and closed.** Repository requests carry only declared-name revisions; snapshot requests carry only a declared snapshot name and immutable digests. Compute one canonical source identity for idempotency and provenance. -- KTD7. **Use direct acquisition implementations.** Git runs with hermetic config - and an invocation credential helper. OCI pulls through a library or fixed - non-shell client interface that validates registry redirects and digests and - extracts without trusting archive paths. -- KTD8. **Select GitHub credentials by three-way eligibility.** Discover - repository coverage with an App-authenticated GitHub API client or verify an - explicit installation ID. Only positive `ineligible` permits the configured - `gh` account; `unknown` and selected-provider failure are terminal. - (session-settled: user-directed.) +- KTD7. **Freeze direct Git and OCI acquisition profiles.** Git runs with + hermetic config and an invocation credential helper. An AllAgents-owned + minimal OCI Distribution client in the Rust helper uses pinned `reqwest` + (rustls, redirects and ambient proxies disabled), `tar`, `flate2`, and `zstd` + crates for streaming pull, bounded authentication, decoding, and changeset + application behind the helper's typed protocol. The client implements only the + v1 direct-image manifest/config/layer profile, RFC 8785 workspace-manifest + config, fixed extraction limits, and explicit Distribution-Spec authentication + and redirect policy. A test-only deterministic reference packer produces the + conformance fixture that freezes the format. +- KTD8. **Select GitHub credentials by provable three-way eligibility.** App + lookup 200 is eligible; 404 is ineligible only with independent repository- + existence proof; all ambiguous outcomes are unknown. Fresh App tokens bypass + SDK cache, are validated and revoked, and only positive ineligibility permits + the configured `gh` account. (session-settled: user-directed.) - KTD9. **Keep one behavior-focused `codex | pi` adapter registry.** Direct - targets and exposed profile targets resolve to the same adapter types and - conformance tests; profile context modifies server-owned configuration, never - the public command line. Provider control, each MCP child, and model-invoked - tools receive separate secret scopes and filesystem/environment views. -- KTD10. **Store one immutable terminal Task generation.** A private, - project-specific locked local store supports one process, durable atomic - idempotency claim plus Task creation, monotonic status, bounded - events/Artifacts, no eviction before TTL, atomic expiry, and startup - terminalization. State paths are ownership/mode/link/disjointness checked. - Integrity or durability failure stops admission and prevents false success. -- KTD11. **Use enforceable invocation containment.** The platform implementation - owns a non-escapable descendant set, drains stdout/stderr, and covers - credential helpers, Git/OCI, MCP, and provider processes. Failure to inspect - or prove an empty set fails or poisons readiness. An unmanaged gateway remains - alive to reap and expose recovery instructions; managed exit requires an - external manager that already accepted containment ownership. -- KTD12. **Keep durable evidence separate and collect it defensively.** Evidence - retains bounded source, Git, provider, result, artifact, and cleanup facts. - Post-execution workspace reads are descriptor-relative and no-follow; Git - metadata indirections and repository-controlled execution are rejected. - Structured logs remain metadata-only and never retain secrets or unrestricted + targets and gateway-enabled profile targets resolve to the same adapter types + and conformance tests; profile context modifies server-owned configuration, + never the public command line. A target is ready only when its pinned backend + exposes a non-bypassable spawn hook through which the helper launches every + MCP and model-tool process with separate filesystem, environment, descriptor, + secret, and network views. +- KTD10. **Put durable Task truth behind the Rust helper's SQLite VFS.** The + helper owns the single process-lifetime SQLite connection and exposes typed + transactional store operations; TypeScript never opens the database by path. + A small audited VFS roots every database, WAL, SHM, journal, and temporary-file + open beneath a preopened private state-directory descriptor with `openat2` + beneath/no-symlink checks, rejects hard links, and fsyncs files and containing + directories. SQLite uses WAL, foreign keys, and `synchronous=FULL`. Claims, + Tasks, events, bounded Artifact bytes, execution lease, containment identity, + internal outcome intent, and expiry live in transactional tables. + `createOrReplay`, lease acquisition, and terminal settlement are transactions; + acknowledge only committed state. Crash recovery yields a complete old or new + generation, never a mixed or missing acknowledged Task. Integrity, VFS, helper + protocol, or durability failure stops admission and prevents false success. +- KTD11. **Package one enforceable Linux security and state helper.** V1 supports + Linux x64/arm64 with cgroup v2, `clone3(CLONE_INTO_CGROUP)`, pidfds, `openat2` + beneath/no-symlink resolution, mount and network namespaces, and nftables + through a small audited Rust helper distributed in platform-specific optional + packages. Its typed inherited-pipe protocol owns SQLite operations, creates + empty containment behind a durable start gate, atomically launches and tracks + the complete acquisition/provider descendant set, mediates every MCP/tool + spawn, builds role-specific filesystem/environment/descriptor/network views, + terminates and waits for membership, and performs safe file operations. + Missing kernel features, delegated cgroup/network access, helper package, + backend spawn mediation, or protocol compatibility fails before binding; + there is no weaker fallback. A poisoned process remains alive to reap until + the set is empty. +- KTD12. **Capture live events, then collect durable filesystem evidence only + after quiescence.** Evidence retains bounded source, Git, provider, result, + Artifact, and cleanup facts. Post-execution workspace reads use the helper's + descriptor-relative no-follow handles, revalidate identity/size, and reject + Git metadata indirections or repository-controlled execution. Structured logs + remain metadata-only and never retain secrets or unrestricted request/output/file bodies. ### High-Level Technical Design @@ -690,19 +918,22 @@ registry, or another profile configuration file for the initial use case. ```mermaid flowchart TB C[Trusted-network A2A caller] --> G[Gateway server] - G --> S[Local Task store] + G --> H[Linux security and state helper] + H --> S[SQLite Task store] G --> W[Workspace compiler] W --> PW[Project workspace.yaml] W --> UW[User workspace.yaml] G --> A[Acquisition supervisor] A --> Git[Declared Git repositories] A --> OCI[Named OCI snapshot] + A --> H A --> P[Atomically published invocation workspace] G --> R[Closed adapter registry] R --> Codex[Codex SDK] R --> Pi[Pi RPC] - Codex --> E[Evidence and cleanup] - Pi --> E + Codex --> H + Pi --> H + H --> E[Quiescence then evidence and cleanup] E --> S ``` @@ -715,44 +946,91 @@ No `gateway.yaml` or `worker.yaml` is introduced. | Concern | CLI | Environment | Default | |---|---|---|---| | Listener | `--listen` | `ALLAGENTS_GATEWAY_LISTEN` | `127.0.0.1:4732` | +| Advertised interface URL | `--advertise-url` | `ALLAGENTS_GATEWAY_ADVERTISE_URL` | `http://127.0.0.1:4732` only with the default loopback listener; otherwise required | | Project workspace | `--workspace` | `ALLAGENTS_GATEWAY_WORKSPACE` | cwd | | State directory | `--state-dir` | `ALLAGENTS_GATEWAY_STATE_DIR` | `~/.allagents/gateway/` | | Terminal Task TTL | `--task-ttl` | `ALLAGENTS_GATEWAY_TASK_TTL` | `24h` | | Retained Task limit | `--max-retained-tasks` | `ALLAGENTS_GATEWAY_MAX_RETAINED_TASKS` | `1000` | | Per-Task retained bytes | `--max-task-bytes` | `ALLAGENTS_GATEWAY_MAX_TASK_BYTES` | `64MiB` | -| GitHub App ID | `--github-app-id` | `ALLAGENTS_GITHUB_APP_ID` | unset | -| App private key file | `--github-app-private-key-file` | `ALLAGENTS_GITHUB_APP_PRIVATE_KEY_FILE` | unset | -| App installation ID | `--github-app-installation-id` | `ALLAGENTS_GITHUB_APP_INSTALLATION_ID` | discovered/unset | -| GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GITHUB_CLI_ACCOUNT` | unset | -| OCI auth file | `--oci-auth-file` | `ALLAGENTS_OCI_AUTH_FILE` | unset | -| OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_OCI_CREDENTIAL_HELPER` | unset | -| Codex auth file | `--codex-auth-file` | `ALLAGENTS_CODEX_AUTH_FILE` | supported Codex default if safe | -| Pi auth file | `--pi-auth-file` | `ALLAGENTS_PI_AUTH_FILE` | supported Pi default if safe | - -Precedence is CLI over environment over default. Credential options name file -handles, accounts, or IDs, never secret values. Auth files must be regular, -current-user/root-owned, non-hard-linked, and no broader than `0600`. Setting -both OCI options is a startup error. The OCI helper value is one absolute -executable path with no arguments; it must be current-user/root-owned and not -group/world-writable. The gateway implements Docker credential-helper `get` -directly, without a shell: argv is exactly `[helperPath, "get"]`; stdin is the -canonical registry origin `https://[:nondefault-port]` plus one -newline; and an exit-zero stdout must be one UTF-8 JSON object with exactly -nonempty string fields `Username` and `Secret`, each at most 64 KiB. Stdout over -128 KiB, a timeout, nonzero exit, signal, malformed UTF-8/JSON, an unknown -member, or an empty credential fails acquisition with -`source_auth_oci_failed`; stderr is bounded, treated as secret-bearing, and not -placed in logs or evidence. The helper is invoked once per registry origin and -its credential is scoped to that origin and destroyed after acquisition. -Provider defaults are eligible only when their resolved auth files pass the -same checks; otherwise the target is not ready. The gateway projects only the -selected provider auth into its control-process view. +| GitHub App ID | `--github-app-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_ID` | unset | +| App private key file | `--github-app-private-key-file` | `ALLAGENTS_GATEWAY_GITHUB_APP_PRIVATE_KEY_FILE` | unset | +| App installation ID | `--github-app-installation-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_INSTALLATION_ID` | discovered/unset | +| GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GATEWAY_GITHUB_CLI_ACCOUNT` | unset | +| OCI auth file | `--oci-auth-file` | `ALLAGENTS_GATEWAY_OCI_AUTH_FILE` | unset | +| OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_GATEWAY_OCI_CREDENTIAL_HELPER` | unset | +| Codex auth file | `--codex-auth-file` | `ALLAGENTS_GATEWAY_CODEX_AUTH_FILE` | supported Codex default if safe | +| Pi auth file | `--pi-auth-file` | `ALLAGENTS_GATEWAY_PI_AUTH_FILE` | supported Pi default if safe | + +Precedence is CLI over environment over default. The advertised value is the +absolute URL placed in `AgentCard.supportedInterfaces`; wildcard hosts are +invalid, non-loopback listeners require an explicit value, and production uses +HTTPS. Credential options name file handles, accounts, or IDs, never secret +values. + +The Linux helper resolves every key/auth/helper path from a verified root with +`openat2(RESOLVE_BENEATH | RESOLVE_NO_SYMLINKS | RESOLVE_NO_MAGICLINKS)`, rejects +group/world-writable parent directories and linked or non-regular leaves, opens +with close-on-exec/no-follow, and verifies owner, mode, link count, device, and +inode with `fstat` after open. Consumers read the verified descriptor rather than +reopening the path. The helper executes a credential-helper binary from that +verified inode; a path or inode swap fails. Auth leaves are current-user/root- +owned, have one link, and are no broader than `0600`; helper leaves are +current-user/root-owned and not group/world-writable. + +Setting both OCI options is a startup error. `--oci-auth-file` accepts at most +1 MiB of strict UTF-8 Docker-config JSON containing only `auths`. Each key is the +exact registry lookup key below and each strict entry contains exactly one of: +bounded base64 `auth` decoding to `username:secret`, or bounded nonempty +`identitytoken`. `credsStore`, `credHelpers`, proxy/plugin fields, unknown +members, commands, and duplicate keys are rejected; nothing named by the file +is executed. Credential selection is exact-key only. + +The fixed OCI helper receives argv `[helperPath, "get"]` without a shell. Stdin +is the raw Docker lookup key plus newline: lowercase `host[:nondefault-port]` +except Docker Hub, which uses `https://index.docker.io/v1/`. Exit-zero stdout is +one UTF-8 JSON object with required nonempty `Username` and `Secret` strings and +optional `ServerURL`, each at most 64 KiB. `ServerURL`, when present, must equal +the lookup key; `Username: ""` classifies `Secret` as an identity token. +Stdout over 128 KiB, timeout, nonzero exit, signal, malformed UTF-8/JSON, unknown +member, mismatch, or empty credential fails with `source_auth_oci_failed`. +Stderr is bounded, treated as secret-bearing, and never logged or retained. + +Registry access starts anonymously. Accept at most one well-formed HTTPS Bearer +challenge and one authenticated retry per request, with one token refresh after +an in-budget 401. Scope must exactly equal +`repository::pull`; service is bounded, +passed only as data, and must match the registry service +(`registry.docker.io` for Docker Hub). +Credentialed token exchange is allowed only at a same-origin HTTPS realm +or the exact Docker Hub realm `https://auth.docker.io/token`; other realms are +anonymous-only. Do not request offline access or accept refresh tokens. Validate +token type and bounded expiry. + +Redirect handling is manual and limited to three HTTPS hops. Same-origin +redirects are permitted. A cross-origin redirect is permitted only for a +layer-blob `GET`/`HEAD` when the destination's normalized `host[:port]` exactly +matches that snapshot source's `layerRedirectHosts`; token, manifest, and config +requests reject it. Every hop rejects URL credentials, strips authorization, +cookies, and client credentials, resolves DNS afresh, validates every A/AAAA +address, and connects to a validated address with the original hostname used +for Host/SNI. Loopback, link-local, multicast, unspecified, RFC1918, ULA, CGNAT, +and other non-global destinations are rejected unless that exact host is +operator-approved for the source. Redirect loops, downgrade, mixed approved and +unapproved answers, and rebinding fail. Final descriptor bytes still must match +size and digest. + +Credentials are invoked once per registry lookup key, scoped to that origin and +repository pull, zeroed after use, and destroyed before publication. + +Provider defaults are eligible only when their resolved auth files pass the same +descriptor checks; otherwise the target is not ready. The gateway projects only +the selected provider auth into its control-process view. The derived workspace ID is a stable digest of the canonical project-workspace -path and is verified against store metadata. Retention includes Task records, -Artifacts, events, and invocation-key claims; expiry is atomic. When the -unexpired Task-count limit is reached, new admission fails rather than evicting -retained Tasks. +path and is verified against SQLite metadata. Retention includes Task records, +Artifact bytes, events, and invocation-key claims; expiry is transactional. When +the unexpired Task-count limit is reached, new admission fails rather than +evicting retained Tasks. **Project workspace additions** @@ -766,12 +1044,15 @@ repositories: workspaceSnapshots: evaluation: repository: ghcr.io/entityprocess/allagents-workspaces + layerRedirectHosts: + - pkg-containers.githubusercontent.com ``` Snapshot names use the portable profile-name vocabulary. Repositories must have unique stable names for remote acquisition. Snapshot repository values contain -only scheme/host/repository identity and never tags, digests, credentials, or -extraction paths. +only scheme/host/repository identity and an optional exact +`layerRedirectHosts` allowlist; never tags, digests, credentials, or extraction +paths. An absent allowlist rejects cross-origin layer redirects. **User workspace additions** @@ -782,31 +1063,44 @@ profiles: - name: codex launcher: codex-review gateway: - expose: true + enabled: true ``` -The nested object is strict and initially contains only `expose: true`. Absence -means not exposed. Exposure requires a launcher, an initial supported backend, -and a healthy installed profile with matching declaration digest. +The nested object is strict and initially contains only `enabled: true`. +Absence or `false` keeps the client unavailable through the gateway. Enablement +requires a launcher, an initial supported backend, and a healthy installed +profile with matching declaration digest. **Promptfoo custom-provider consumption** AI Evals implements Promptfoo's [`ApiProvider`](https://www.promptfoo.dev/docs/providers/custom-api/) in -TypeScript. Its `constructor(options: ProviderOptions)` stores -`options.id ?? "allagents-a2a"` and validates `options.config`; `id()` returns -that stored value. Its -`callApi(prompt, context, options)` uses `context.vars` for test data and -`options?.abortSignal` for request cancellation. +TypeScript. Its `constructor(options: ProviderOptions)` requires and stores a +nonempty `options.id`, validates `options.config`, and `id()` returns that +stored value. +`callApi(prompt, context?, options?)` reads +`context?.vars?.allagentsSource` when present and +`options?.abortSignal` for cancellation. Static YAML defines the source mode and every logical name: ```yaml +prompts: + - file://./prompts/coding-task.txt + +sharing: false +evaluateOptions: + maxConcurrency: 1 + cache: false +commandLineOptions: + write: false + share: false + providers: - id: file://./providers/allagents-a2a.ts label: codex-direct config: - endpoint: http://allagents-gateway.tailnet:4732 + endpoint: https://allagents-gateway.example.internal target: codex source: kind: repositories @@ -816,7 +1110,7 @@ providers: - id: file://./providers/allagents-a2a.ts label: codex-evaluation-snapshot config: - endpoint: http://allagents-gateway.tailnet:4732 + endpoint: https://allagents-gateway.example.internal target: codex source: kind: workspaceSnapshot @@ -840,6 +1134,13 @@ tests: workspaceManifestDigest: sha256:6789abcdef0123456789abcdef0123456789abcdef0123456789abcdef012345 ``` +The gateway enforces one active invocation transactionally. Promptfoo keeps +`maxConcurrency: 1` to avoid predictably creating failed capacity Tasks; other +trusted callers need no external queue for correctness. Disabling cache, local +result writes, and sharing is the safe baseline for confidential prompts and +opaque provider output. Consumers may enable persistence or sharing only after +defining their own access, retention, destination, and redaction policy. + `allagents` is a declared repository name used only as a revision-override key; repository mode still materializes the complete configured set. `evaluation` is the logical snapshot handle. The provider sends the source mode, optional named @@ -848,8 +1149,8 @@ revisions, and immutable digests, not `ghcr.io/entityprocess/allagents-workspaces`. The gateway resolves origins and credentials server-side and omits them from A2A source-identity responses. -`context.vars.allagentsSource` is the only per-test override. In repository mode -it may contain exactly `revisions`, whose keys must already exist in static +`context?.vars?.allagentsSource` is the only per-test override. In repository +mode it may contain exactly `revisions`, whose keys must already exist in static `config.source.revisions` and whose values are full lowercase 40-hex commits. In snapshot mode it may contain exactly `digest` and/or `workspaceManifestDigest`, both full lowercase `sha256:` digests. Present leaves @@ -858,32 +1159,44 @@ repository-name allowlist, and snapshot name remain static. Unknown members, mutable revisions, origins, destinations, credentials, and commands fail before A2A submission. -Each `callApi` creates one invocation key and A2A Task. The provider sends -`CancelTask` when its bounded deadline or `options?.abortSignal` fires after -acceptance. It returns terminal text or the validated structured result as -`ProviderResponse.output`. It maps gateway usage exactly as +Each `callApi` creates one high-entropy invocation key and sends `SendMessage` +with `returnImmediately: true`, then follows the accepted Task through +`SubscribeToTask`, `GetTask`, and bounded resubscription. An abort or deadline +sends one `CancelTask` with a fresh cleanup signal. Ambiguous submission retry +reuses the same key and request. The provider returns terminal text or validated +structured result as `ProviderResponse.output`. It maps gateway usage exactly as `inputTokens -> tokenUsage.prompt`, `outputTokens -> tokenUsage.completion`, `cachedInputTokens -> tokenUsage.cached`, and -`totalTokens -> tokenUsage.total`; provider-specific counters stay in +`totalTokens -> tokenUsage.total`; provider-specific counters remain in `metadata`. Task ID, Artifact references, logical source identity, termination, -and cleanup evidence also remain in `metadata`, without origins or destination -paths. Admission and terminal failures use `ProviderResponse.error`. This -provider is AI Evals code; AllAgents has no Promptfoo runtime dependency. +cleanup, and stable failure `code`/`retryable`/accepted `taskId` also remain in +`metadata`, without origins or destination paths. Admission and terminal +failures use a safe `ProviderResponse.error`. This provider is AI Evals code; +AllAgents has no Promptfoo runtime dependency. ### Error and Status Mapping -| Condition | Stable code and A2A outcome | Fresh-invocation retryable | +Every unsuccessful HTTP response has `Content-Type: application/json` and the +A2A 1.0 `google.rpc.Status` JSON shape under `error`. Standard A2A errors include +`google.rpc.ErrorInfo` with domain `a2a-protocol.org` and the specified uppercase +reason. Custom admission errors include `google.rpc.ErrorInfo` with domain +`allagents.dev`, uppercase stable-code reason, and string metadata `code`, +`retryable`, and optional `taskId`; field validation also includes +`google.rpc.BadRequest`. No HTTP+JSON response uses JSON-RPC `.data`. + +| Condition | Stable code and A2A/HTTP+JSON outcome | Fresh-invocation retryable | |---|---|---| -| Missing required extension | A2A `ExtensionSupportRequiredError`; no Task | No | -| Malformed request, source, digest, schema, prompt, or unknown target/source | `invalid_execution_request` in A2A `InvalidParamsError.data`; no Task | No | -| Invocation-key conflict | `invocation_key_conflict` in A2A `InvalidParamsError.data`; no new Task | No | -| Identical retained invocation replay | Existing Task and Artifacts | N/A | -| Cancel after terminal state | A2A `TaskNotCancelableError` | No | -| Retained Task capacity exhausted | `retention_capacity_exhausted` in A2A `InternalError.data`; no Task | Yes, after expiry | +| Unsupported A2A version | HTTP 400 A2A `VersionNotSupportedError`; no Task | No | +| Missing required extension | HTTP 400 A2A `ExtensionSupportRequiredError`; no Task | No | +| Malformed request, source, digest, schema, prompt, or unknown target/source | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | +| Invocation-key conflict | HTTP 409 `ALREADY_EXISTS`; `invocation_key_conflict`; no new Task | No | +| Identical retained invocation replay | Existing Task with embedded Artifacts | N/A | +| Cancel after terminal state | HTTP 400 A2A `TaskNotCancelableError` | No | +| Retained Task capacity exhausted | HTTP 429 `RESOURCE_EXHAUSTED`; `retention_capacity_exhausted`; `Retry-After`; no Task | Yes, after expiry | | Runtime capacity unavailable after acceptance | `execution_capacity_unavailable`; failed Task | Yes | | App absent/ineligible and configured `gh` succeeds | Continue with recorded provider class | N/A | | App applicability unknown | `source_auth_applicability_unknown`; failed Task; no fallback | Yes for rate-limit/service causes only | -| Selected App config/auth/mint failure | `source_auth_failed`; failed Task; no fallback | No | +| Selected App config/auth/mint/validation/revocation failure | `source_auth_failed`; failed Task; no fallback | No | | Selected App permission/repository denial | `source_auth_denied`; failed Task; no fallback | No | | Selected App rate limit | `source_auth_rate_limited`; failed Task; no fallback | Yes | | Selected App service failure | `source_auth_unavailable`; failed Task; no fallback | Yes | @@ -891,108 +1204,144 @@ provider is AI Evals code; AllAgents has no Promptfoo runtime dependency. | Git revision/identity failure | `source_git_identity_invalid`; failed Task | No | | Git transport failure | `source_git_unavailable`; failed Task | Yes | | OCI helper timeout, process, protocol, or credential failure | `source_auth_oci_failed`; failed Task; no fallback | No | -| OCI auth/digest/manifest/extraction validation failure | `source_snapshot_invalid`; failed Task; no Git fallback | No | +| OCI auth/challenge/digest/manifest/extraction validation failure | `source_snapshot_invalid`; failed Task; no Git fallback | No | | OCI registry service failure | `source_snapshot_unavailable`; failed Task; no Git fallback | Yes | | Deadline expires | `execution_deadline_exceeded`; abort/terminate; failed Task | Yes | | Known provider permission denial | `execution_permission_denied`; rejected Task | No | | Unknown provider protocol or result shape | `provider_protocol_invalid`; failed Task | No | -| Cancellation with proven quiescence | `execution_cancelled`; cancelled Task | No | +| Cancellation with proven quiescence | `execution_canceled`; canceled Task | No | | Termination or cleanup cannot be proven | `execution_quiescence_unknown`; failed Task; readiness poisoned | No | | State store durability/integrity failure | `task_store_failed`; stop admission; abort/contain; no success | No | | Restart finds interrupted Task | `gateway_restarted`; failed Task; no provider resume | Yes as a new invocation | -| Retention expiry | A2A `TaskNotFoundError` | Yes as a new invocation | +| Retention expiry | HTTP 404 A2A `TaskNotFoundError` | Yes as a new invocation | Accepted-Task failures use the integrity Artifact's strict `failure` object with `code`, safe `message`, table-defined `retryable`, and one closed cause from `validation | capacity | sourceAuth | sourceGit | sourceSnapshot | deadline | permission | providerProtocol | cancellation | termination | stateStore | -restart`. Admission failures use the exact A2A error type in the table with the -same stable code and retryability in safe `data`. Retryability describes whether -a caller may create a fresh invocation; it never enables automatic Task retry -or fallback. Provider identifiers, credentials, paths, and raw upstream -messages never enter either carrier. +restart`. Retryability says whether a caller may create a fresh invocation; it +never enables automatic Task retry or provider/source fallback. Promptfoo copies +only the safe code, retryability, and accepted Task ID into metadata. Provider +identifiers, credentials, paths, and raw upstream messages enter neither +carrier. ### Phased Delivery 1. Build the current CLI and record the red E2E showing that `allagents gateway serve` is unavailable. Record the exact `/tmp/` workspace setup, command, and observed failure. -2. Freeze workspace additions, public extension, common manifests, result - schema, errors, and fixtures. -3. Build the deployment-wide Task store and unauthenticated A2A server against - a fake adapter. -4. Add repository and OCI acquisition with credential containment and manifest - validation. -5. Add the invocation supervisor, execution containment, safe evidence, and - shared backend contract. +2. Freeze workspace additions, the published extension, snapshot format, + common manifests, result schema, errors, packaging, and fixtures. Establish + the Rust helper protocol, safe SQLite VFS, platform packages, and ordered + release pipeline first. +3. Build the SQLite Task store through the helper, AllAgents A2A request handler, + HTTP+JSON server, minimal backend interface/registry, and fake adapter. +4. Extend the packaged helper with invocation supervision, execution + containment, spawn mediation, role-specific network/secret views, safe + evidence, and terminal arbitration around the fake adapter. +5. Add repository and OCI acquisition through the supervisor/helper with + credential containment and manifest validation. 6. Add Codex, then Pi, against the same conformance suite. 7. Run final implementation review and fix important correctness, security, contract, reliability, DRY, and coverage findings. -8. Run the green built-CLI `/tmp/` E2E, repository quality gates, user - documentation, and release evidence. +8. Run the green bundled-CLI `/tmp/` E2E, clean-registry install smoke, + repository quality gates, user documentation, and release evidence. ### System-Wide Impact -- **Package surface:** Add a private execution-service package and the public - `allagents gateway serve` command. Preserve existing profile and sync commands. -- **Schema surface:** Extend project workspace schemas with named snapshots and - user profile-client schemas with explicit exposure. Regenerate versioned JSON - Schemas and update configuration docs. -- **Dependency surface:** Add the official A2A SDK, pinned Codex SDK, - `@octokit/auth-app`, and a focused OCI client/extraction implementation to the - private execution package. -- **State surface:** Add a bounded gateway state root and per-invocation staging, - publication, evidence, and cleanup roots. Do not alter existing profile state. -- **Security surface:** The network is the authorization boundary. Source - credentials are phase-scoped; acquired code and agent tools never receive App, - `gh`, or OCI credentials. -- **Compatibility:** Existing workspace files remain valid because new fields are - optional. Older binaries reject the new strict nested profile field, so docs - must state the minimum supporting version. +- **Package surface:** Declare a root Bun workspace; add private + `packages/execution-service` and Rust `packages/execution-helper`; add the + service as a root `workspace:*` development dependency; distribute Linux + x64/arm64 helper binaries through versioned platform-specific optional + packages; and bundle the service into published `dist/index.js`. The release + scripts and Publish workflow version matching helper packages and root + dependency ranges, publish and verify both platform packages first, and + publish `allagents` only after their registry metadata and checksums resolve. + Add public `allagents gateway serve` without changing existing profile and + sync commands. Root build, typecheck, tests, and clean-registry install smoke + include the private workspace service and resolved helper binary. +- **Schema surface:** Extend project workspace schemas with named snapshots, + exact layer-redirect hosts, and user profile-client schemas with explicit + gateway enablement. Publish the versioned extension specification and + generated JSON Schemas; update configuration docs. +- **Dependency surface:** Put the official A2A SDK, pinned Codex SDK, and + `@octokit/auth-app` in the private service package. Pin SQLite, the custom VFS + bindings, `reqwest` with rustls, `tar`, `flate2`, and `zstd` in the Rust helper + lockfile together with the Rust toolchain/helper protocol; check helper release + checksums. +- **State surface:** Add one bounded SQLite gateway state root and per-invocation + staging, publication, evidence, and cleanup roots. Do not alter profile state. +- **Security surface:** The network is the caller authorization boundary. Source + credentials are phase-scoped; helper-mediated process and network views keep + acquired code and agent tools from App, `gh`, OCI, provider, MCP, operator, and + gateway credentials/state. Helper absence or capability loss fails closed. +- **Compatibility:** Existing workspace files remain valid because new fields + are optional. Gateway startup applies stricter repository-catalog rules. + Older binaries reject the new strict nested profile field, so docs state the + minimum supporting version. ### Risks and Mitigations - **Accidental network exposure:** Binding `0.0.0.0` is intentional and allowed; - startup output and docs state that every reachable host has full authority. + require a distinct advertised URL, use HTTPS in production, and state in + startup output/docs that every reachable host has full authority. - **Profile identity drift:** Derive targets only from current validated user declarations and matching installed state; never resurrect declaration-missing launchers from retained profile state. -- **Credential leakage:** Use fresh App tokens or one configured `gh` account, - invocation-only helpers, hermetic Git, and credential teardown before - publication. Separate provider-control, per-MCP, and model-tool views prevent - one secret scope from reading another. +- **Credential leakage or path swap:** Use fresh validated/revoked App tokens or + one configured `gh` account, descriptor-bound credential handles, hermetic + Git, strict Docker auth/helper protocols, and credential teardown before + publication. Non-bypassable helper spawn mediation replaces environments, + closes descriptors, and enters role-specific mount/network namespaces before + every MCP or model-tool exec; a backend lacking that hook is unavailable. - **Identity-changing fallback:** Classify App applicability as eligible, - ineligible, or unknown; only positive ineligibility permits `gh`. -- **OCI archive abuse:** Require immutable digests, configured repositories, - bounded extraction, path/type/link validation, and manifest verification. -- **Untrusted acquired code:** General hostile-code sandboxing is not claimed, - but model-invoked tools cannot reach provider/MCP/operator credentials or - gateway state. Project/user setup shell commands are never automatic. -- **Evidence-time attacks:** Treat the mutated workspace as untrusted, use - descriptor-relative no-follow reads, reject Git metadata indirection, and - disable repository-controlled Git execution features. -- **Provider/API churn:** Pin compatible SDK/CLI versions and retain versioned - native fixtures plus one adapter conformance suite. -- **Orphaned processes:** Require a platform containment primitive whose - descendants cannot escape; fail readiness when unavailable. On uncertain - quiescence, unmanaged mode stays alive to reap and managed mode exits only - after cleanup ownership transfer. -- **Store corruption or disclosure:** Validate ownership, modes, links, root - disjointness, lock, and workspace identity. Integrity/durability failure stops - admission and prevents terminal success. + ineligible, or unknown; require repository-existence proof for 404 + ineligibility; only positive ineligibility permits `gh`. +- **OCI registry/archive abuse:** Require immutable digests, a closed + manifest/config/layer profile, same-origin metadata, exact operator-approved + layer-redirect hosts with per-hop address validation, changeset semantics, + fixed extraction limits, safe paths/types/links, and exact project-catalog + manifest verification. +- **Untrusted acquired code:** General hostile-code sandboxing beyond the + declared Linux process/network namespace and secret boundary is not claimed. + Invocation routes deny gateway, host loopback, and management networks; + provider/MCP egress is allowlisted; and model tools cannot reach provider/MCP/ + operator credentials or gateway state. Project/user setup shell commands are + never automatic. +- **Evidence-time attacks:** Prove containment empty first, then use + descriptor-relative no-follow reads with identity/size revalidation, reject + Git metadata indirection, and disable repository-controlled Git execution. +- **Provider/API churn:** Pin compatible SDK/CLI/model versions and retain + versioned native fixtures plus one adapter conformance suite. Gate Codex native + schemas to the pinned Structured Outputs subset and backend availability to a + proven non-bypassable spawn hook. +- **Orphaned processes:** Persist a stable empty containment identity before + start-gate release; enumerate the full project-owned cgroup namespace on + startup. On uncertain quiescence, stay alive, reject admission, and continue + reaping until empty without mutating the settled Task. +- **Store corruption or disclosure:** Route SQLite and all sidecars through the + helper's descriptor-rooted no-follow VFS with full synchronization and + transactions; validate ownership, modes, links, root disjointness, lock, and + workspace identity. Integrity/durability failure stops admission and prevents + terminal success. ### Assumptions -- The initial deployment is one gateway process and one active invocation. -- Every network peer able to connect is trusted with all exposed targets and - retained Tasks. -- The selected project workspace is operator-controlled and uses supported - repository/snapshot declarations. +- The initial deployment is one gateway process and one transactionally enforced + active invocation. +- Every external network peer able to connect is trusted with all available + targets, including built-ins and gateway-enabled profiles, and all retained + Tasks. Invocation descendants are deliberately unable to reach that network + boundary. +- The selected project workspace is operator-controlled and compiles to 1-64 + uniquely named GitHub repositories with collision-free destinations. - GitHub.com is the only authenticated Git host in the initial delivery. -- OCI snapshots use registries reachable through HTTPS and immutable manifests. -- Codex and Pi automation surfaces remain compatible with the pinned versions. -- Supported platforms provide a non-escapable invocation containment strategy; - the gateway fails readiness where that invariant cannot be met. +- OCI snapshots use HTTPS registries and the frozen v1 direct-image format. +- Codex and Pi are available only when their pinned automation surfaces support + non-bypassable helper-mediated tool and MCP spawning. +- Gateway v1 execution supports Linux x64/arm64 hosts with cgroup v2, `clone3`, + pidfds, `openat2`, mount/network namespaces, nftables, and delegated + permissions. --- @@ -1000,100 +1349,145 @@ messages never enter either carrier. ### U1. Workspace, extension, and manifest contracts -- **Goal:** Freeze configuration additions and all versioned public/private data - contracts before runtime implementation. -- **Requirements:** R1, R2, R3, R6, R7, R8, R9, R18; AE3, AE4, AE5, AE7, - AE8, AE9, AE13, AE16, AE20; KTD1, KTD2, KTD5, KTD6. -- **Files:** `src/models/workspace-config.ts`, schema generation tests and - generated public schemas, `packages/execution-service/src/contracts/*`, - `packages/execution-service/tests/unit/contracts/*`, configuration docs. -- **Approach:** Add strict named `workspaceSnapshots` and nested profile-client - `gateway.expose`; preserve project/user scope and reserve built-in IDs. Define - activation on every profiled operation, the extension-owned request envelope, - other-metadata behavior, exact result-schema grammar, source union, deadline, - integrity and produced Artifacts, stable failures, workspace manifest, and - canonical digest preimages from Zod. -- **Execution note:** Start with fixtures that reject missing activation, cross- - variant/unknown extension fields, extra Message Parts, undeclared names, - mutable snapshot references, malformed digests, invalid deadlines, exposure - without launcher, built-in collisions, and unsupported clients while - preserving unrelated metadata. -- **Verification:** Focused workspace-schema and contract tests; generated schema - drift check; representative YAML and wire examples parse through runtime - schemas; canonicalization and Artifact-cardinality fixtures pass. +- **Goal:** Freeze configuration, packaging, safe state primitives, and every + versioned public/private contract before runtime implementation. +- **Requirements:** R1, R2, R3, R5, R6, R7, R8, R9, R11, R18; AE3, AE4, AE5, + AE7, AE8, AE9, AE13, AE14, AE16, AE20; KTD1, KTD2, KTD5, KTD6, KTD7, + KTD10, KTD11. +- **Files:** root `package.json`/build/typecheck configuration, + `packages/execution-service/package.json` and TypeScript config, Rust + `packages/execution-helper`, Linux x64/arm64 optional packages, typed helper + protocol, SQLite schema/migrations and descriptor-rooted VFS, + `scripts/release.ts`, `scripts/publish.ts`, `.github/workflows/publish.yml`, + `src/models/workspace-config.ts`, schema generation tests and generated public + schemas, execution-service contracts, + `docs/src/pages/a2a/extensions/coding-execution/v1.astro` at the exact + declared URI plus a generated schema asset beneath that route, a versioned + snapshot-format specification, deterministic reference packer/conformance + fixtures, and configuration docs. +- **Approach:** Declare the Bun workspace and root `workspace:*` development + edge so the private service is installed, checked, and bundled. Establish the + helper protocol and audited SQLite VFS before the server store client. Version + helper packages with matching root optional-dependency ranges; publish and + verify both platform packages before the root package. Verify a clean registry + install resolves the matching helper binary and checksum and that the packed + root manifest contains no workspace protocol. + + Add strict named `workspaceSnapshots` with exact layer-redirect hosts and + nested profile-client `gateway.enabled`; preserve ordinary project/user + parsing while compiling gateway repository and target catalogs. Publish Agent + Card params, version/header activation, Message metadata/extensions, unified + Parts, exact result-schema grammar, source union, deadline, idempotency/replay, + HTTP+JSON errors, integrity/produced Artifacts, workspace manifest, OCI media/ + change-set/limit profile, and canonical digest preimages from Zod. +- **Execution note:** Start with independent wire fixtures that use only the + published extension specification. Reject missing version/activation, cross- + variant/unknown fields, extra Message Parts, undeclared names, mutable + snapshot references, malformed digests, invalid deadlines, incomplete or + mismatched manifests, gateway enablement without launcher, built-in + collisions, and unsupported clients while preserving unrelated metadata. + Fault-inject database/WAL/SHM link and rename swaps through the real VFS. +- **Verification:** Focused workspace-schema, packaging, helper VFS, and + contract tests; generated schema/spec drift checks; representative YAML, HTTP + errors, and wire examples parse through runtime schemas; snapshot conformance, + canonicalization, Artifact-cardinality, clean-registry install, matching + helper version/checksum, and ordered publish dry-run fixtures pass. ### U2. Deployment-wide Task store and A2A server - **Goal:** Serve the A2A lifecycle without application authentication and keep - durable deployment-wide Task/idempotency truth. -- **Requirements:** R1, R2, R3, R4, R5, R8, R16, R17, R18; AE1, AE2, AE9, - AE11, AE14, AE15, AE16, AE19, AE20; KTD1, KTD2, KTD3, KTD4, KTD10. -- **Files:** execution-service Task repository, Agent Card, request handler, - server, pagination/retention, CLI gateway command, and focused tests. -- **Approach:** Implement flags/env precedence, private link-safe project state - and lock, loopback default, explicit `0.0.0.0`, startup integrity/ - reconciliation, per-operation extension negotiation, durable atomic - claim+Task creation, monotonic terminal settlement, bounded events/Artifacts, - no early eviction, atomic expiry, global listing/cancellation, deadline - handling, and fail-closed graceful shutdown. -- **Execution note:** Prove with the official A2A client that one caller can read - and cancel another caller's Task; this is expected behavior. Fault-inject - unsafe state paths plus open/write/rename/fsync boundaries before - acknowledgment, cancellation intent, Artifact, and terminal settlement. -- **Verification:** A2A discovery/send/stream/get/list/subscribe/cancel/replay/ - expiry integration tests on loopback and `0.0.0.0`; state-path, retained- - capacity, store-fault, competing-lock, deadline, shutdown, and restart tests. - -### U3. Git and OCI workspace acquisition + durable deployment-wide Task/idempotency truth behind a fake backend. +- **Requirements:** R1, R2, R3, R4, R5, R8, R13, R16, R17, R18; AE1, AE2, AE9, + AE11, AE14, AE15, AE16, AE19, AE20; KTD1, KTD2, KTD3, KTD4, KTD9, KTD10. +- **Files:** typed Task-store client, Agent Card, AllAgents request handler, + HTTP+JSON/SSE server, pagination/retention, minimal backend interface and + registry, fake adapter, health/readiness, CLI gateway command, focused tests. +- **Approach:** Implement flags/env precedence, bind/advertised-URL separation, + private project state and lock, helper-owned SQLite full-sync transactions, + startup integrity and full containment-namespace reconciliation, A2A version + and extension negotiation, exact `SendMessage` modes and `ListTasks` + semantics, standard/custom `google.rpc.Status` errors, durable + `createOrReplay`, one execution lease, internal outcome intent plus atomic + terminal settlement, bounded events/Artifact bytes, no early eviction, + transactional expiry, global listing/cancellation, deadline handling, and + fail-closed graceful shutdown against the fake adapter. +- **Execution note:** Prove with the official A2A client that one external caller + can read and cancel another caller's Task; this is expected behavior. Kill + subprocesses after transaction write/sync/commit/response boundaries and + fault-inject helper/VFS I/O, capacity races, cancellation intent, Artifact, + and terminal settlement. +- **Verification:** A2A discovery/send modes/stream/get/full list/subscribe/ + cancel/replay/expiry and HTTP-error integration tests on loopback plus explicit + `0.0.0.0`/advertised URL; health/readiness, state-path, retained and active + capacity, crash/store-fault, competing-lock, deadline, shutdown, and restart + tests. + +### U3. Invocation supervisor and backend contract + +- **Goal:** Run one fake-backed invocation through containment, typed + preparation, evidence, terminal arbitration, and cleanup with truthful + outcomes before real acquisition/adapters. +- **Requirements:** R3, R5, R8, R13, R14, R15, R16; AE9, AE10, AE11, AE12, + AE14, AE15, AE16, AE17, AE18; KTD3, KTD9, KTD10, KTD11, KTD12. +- **Files:** security/state helper extensions, provider/MCP/tool view and egress + compiler, invocation state machine, containment/start-gate controller, spawn + broker, typed preparation, evidence collector, result validator, + cleanup/reaper, and lifecycle tests. +- **Approach:** Extend the U1 helper to allocate an empty cgroup with a stable ID + and start gate, commit Task+lease+containment before release, and enumerate + recorded and unknown cgroups on startup. Launch every child into the cgroup; + mediate every backend MCP/tool spawn; enter role-specific mount and network + namespaces; replace environments; close descriptors; apply nftables egress + policy; use pidfds for termination/wait; and expose safe file operations. + Resolve targets through U2's typed fake adapter; never execute generated + launchers or setup commands. Commit one internal intent across provider, + cancel, deadline, and shutdown; capture live events; prove quiescence before + filesystem evidence; atomically settle status, evidence, Artifacts, cleanup, + and lease release; remain alive to reap when poisoned without mutating the + settled Task. +- **Execution note:** Fault-inject every boundary: capacity races and restart; + process death before/after empty-set creation, Task binding, child clone, and + start-gate release; pairwise and three-way outcome races; child fork/escape; + helper protocol/version/package mismatch; provider/MCP/tool attempts to reach + Agent Card, ListTasks, GetTask, SendMessage, CancelTask, host loopback, and + management networks; environment/path/inherited-FD/`/proc`/magic-link secret + reads by real child and grandchild processes; output truncation, malicious + evidence, valid-result-then-evidence-failure, and unknown cleanup. +- **Verification:** Deterministic lifecycle, single execution lease, helper + packaging/checksum, cgroup/pidfd/mount/network namespace containment, + non-bypassable spawn mediation, separate secret/descriptor/egress views, + typed preparation, safe-file/evidence, unknown-cgroup reconciliation, and + poison/reaping tests plus real child-process smoke on Linux x64/arm64 CI. + +### U4. Git and OCI workspace acquisition - **Goal:** Materialize declared repository sets and named OCI snapshots into the - same validated invocation workspace contract. + same validated invocation workspace through the U3 security helper. - **Requirements:** R6, R9, R10, R11, R12, R15, R16, R18; AE5, AE6, AE7, - AE8, AE10, AE15, AE17, AE18; KTD6, KTD7, KTD8, KTD11. + AE8, AE10, AE15, AE17, AE18; KTD6, KTD7, KTD8, KTD11, KTD12. - **Files:** acquisition coordinator, Git transport, GitHub provider selection, - OCI client/extractor, workspace-manifest validator, staging/publication helper, - fixtures and tests. -- **Approach:** Resolve name-based source requests from project workspace. - Implement hermetic Git and full-commit verification. Classify App - applicability as eligible/ineligible/unknown, mint a fresh token with adequate - lifetime, permit `gh` only for positive ineligibility, and use temporary - credential helpers. Pull digest-pinned OCI manifests from declared - repositories, validate every layer and extraction boundary, validate the - expected workspace-manifest digest, and publish atomically. Tear down every - acquisition credential before typed preparation. -- **Execution note:** Use local Git remotes and a local OCI test registry/fixture; - prove unknown/selected-App failures do not call `gh`, token lifetime is - enforced, and snapshot failures never invoke Git fallback. -- **Verification:** Focused three-way provider-selection tests, Git integration - tests including branch/tag resolution, OCI digest/path/limit tests, credential - leak scans, and equivalent manifest output across both acquisition modes. - -### U4. Invocation supervisor and backend contract - -- **Goal:** Run one invocation through acquisition, adapter execution, evidence, - cancellation, descendant termination, and cleanup with truthful terminal - outcomes. -- **Requirements:** R3, R5, R8, R13, R14, R15, R16; AE9, AE10, AE11, AE12, - AE14, AE15, AE16, AE17, AE18; KTD3, KTD9, KTD10, KTD11, KTD12. -- **Files:** backend interface/registry, typed preparation, provider/MCP/tool - secret-view compiler, invocation state machine, platform containment, evidence - collector, result validator, cleanup/reaper, fake adapter, and lifecycle tests. -- **Approach:** Resolve targets to typed adapter context; never use generated - launchers or workspace setup commands. Project validated configuration through - deterministic transforms. Give provider control, each MCP child, and model - tools separate minimal views; enforce deadlines; track the non-escapable - containment set; preserve result states; collect evidence through safe reads; - and require quiescence before cleanup success. Startup reconciles Tasks and - containment before roots. Unmanaged poisoned mode continues reaping; managed - exit proves cleanup ownership transfer. -- **Execution note:** Build the fake adapter first and fault-inject every - boundary: cancellation/terminal races, deadline, shutdown, child escape, - cross-scope provider/MCP/tool secret reads, unsafe state reads, output - truncation, malicious evidence, valid-result-then-evidence-failure, unknown - cleanup, and manager handoff. -- **Verification:** Deterministic lifecycle, containment, separate-secret-view, - preparation, and evidence tests plus one real child-process smoke fixture per - supported platform strategy. + strict Docker-auth/helper resolver, OCI Distribution client and changeset + applier, workspace-manifest validator, staging/publication helper, fixtures and + tests. +- **Approach:** Resolve name-based requests from the compiled project catalog. + Implement hermetic Git and full-commit verification. Apply the exact App + eligibility proof table, bypass token cache, validate/revoke each fresh token, + permit `gh` only for positive ineligibility, and use descriptor-bound temporary + helpers. Implement anonymous-first bounded Bearer authentication, redirect/ + credential-origin rules, the frozen direct-image media profile, streaming + descriptor verification, gzip/zstd changeset and whiteout semantics, all + extraction ceilings, exact project-manifest validation, and atomic publication. + Tear down every acquisition credential before typed preparation. +- **Execution note:** Use local Git remotes and a local OCI registry plus the U1 + producer fixture. Prove ambiguous/selected-App failures never call `gh`, two + sequential acquisitions mint distinct tokens, token validation/revocation and + lifetime are enforced, helper/auth-file swaps fail, and snapshot failure never + invokes Git fallback. +- **Verification:** Three-way provider-selection and real-response fixture tests; + Git branch/tag/full-commit integration; GHCR/Docker Hub helper fixtures; + malicious realm/scope/downgrade/redirect tests; OCI index/media/digest/size/ + limit/order/whiteout/path/catalog fixtures; credential leak scans; equivalent + complete manifest output across both acquisition modes. ### U5. Codex backend adapter @@ -1104,15 +1498,22 @@ messages never enter either carrier. AE15, AE17, AE18; KTD9, KTD11, KTD12. - **Files:** Codex adapter, profile-context and auth bridge, fixtures, conformance and optional credentialed smoke tests. -- **Approach:** Pin the SDK; create one fresh thread per Task; pass cwd, typed - profile configuration, abort signal, optional output schema, and the private - Codex control-process auth view inside containment. Keep Codex-invoked tools - outside that auth view; normalize events/usage; bound evidence; dispose fully. -- **Execution note:** Characterize the pinned SDK and its tool-sandbox/auth - separation with captured fixtures before implementing normalization. Do not - import Promptfoo provider code. -- **Verification:** Shared adapter conformance, deadline, auth-isolation, and - tool-secret-denial fixtures plus an opt-in credentialed smoke case. +- **Approach:** Pin SDK/model compatibility and first prove a non-bypassable + synchronous hook that delegates every MCP and model-tool spawn to the U3 + helper. If the pinned Codex surface can bypass that hook, Codex is unavailable + in v1 rather than relying on an asserted view. Create one fresh thread per + Task; pass cwd, typed profile configuration, abort signal, and the private + Codex control-process auth view inside containment. Pass native `outputSchema` + only for the pinned Structured Outputs subset; otherwise add JSON guidance and + use the common terminal validator. Normalize events/usage, bound evidence, and + dispose fully. +- **Execution note:** Characterize the pinned SDK/model's spawn, schema, tool- + sandbox, auth, abort, and event behavior with captured fixtures before + normalization. Do not import Promptfoo provider code. +- **Verification:** Shared adapter conformance, real SDK child/grandchild spawn + mediation, filesystem/environment/inherited-FD/`/proc` credential denial, + gateway/host-network denial, native-schema and validated-fallback paths, + deadline, and an opt-in credentialed smoke case. ### U6. Pi backend adapter @@ -1122,43 +1523,51 @@ messages never enter either carrier. AE17, AE18; KTD9, KTD11, KTD12. - **Files:** Pi adapter, RPC parser, restricted policy extension, profile-context and auth bridge, fixtures, conformance and optional credentialed smoke tests. -- **Approach:** Launch Pi with typed invocation configuration, its private - control-process auth view, strict JSONL RPC, explicit allowed tools/extensions, - per-MCP secret views, deterministic permissions, event validation, deadline/ - cancellation escalation, and settled completion. Pi-invoked tools receive no - provider or MCP credentials. Repository extensions and unrestricted built-ins - remain disabled. -- **Execution note:** Reuse the adapter contract exactly; record Pi-specific facts - as bounded native evidence rather than public schema branches. -- **Verification:** Shared adapter conformance, malformed/unknown RPC, deadline, - auth/MCP/tool-secret denial, and an opt-in credentialed smoke case. +- **Approach:** First prove strict RPC exposes a non-bypassable synchronous hook + that delegates every MCP and model-tool spawn to the U3 helper. If Pi can + bypass that hook, Pi is unavailable in v1. Launch Pi with typed invocation + configuration, its private control-process auth view, strict JSONL RPC, + explicit allowed tools/extensions, per-MCP secret declarations, + deterministic permissions, event validation, deadline/cancellation + escalation, and settled completion. Repository extensions and unrestricted + built-ins remain disabled. +- **Execution note:** Characterize and pin Pi's spawn/RPC contract; record + Pi-specific facts as bounded native evidence rather than public schema + branches. +- **Verification:** Shared adapter conformance, real RPC child/grandchild spawn + mediation, filesystem/environment/inherited-FD/`/proc` provider/MCP secret + denial, gateway/host-network denial, malformed/unknown RPC, deadline, and an + opt-in credentialed smoke case. ### U7. End-to-end delivery and documentation -- **Goal:** Prove the built CLI and document the trusted-network operating model, - workspace configuration, credentials, sources, Promptfoo consumption, and - risks. +- **Goal:** Prove the bundled/packed CLI and document the trusted-network + operating model, Linux requirements, workspace configuration, credentials, + sources, Promptfoo consumption, and risks. - **Requirements:** R1-R19; F1-F6; AE1-AE21. -- **Files:** gateway guide/reference, configuration reference, README, CHANGELOG, - real example project/user workspaces, AI Evals-style Promptfoo YAML and custom- - provider contract fixture, E2E fixtures, release evidence. -- **Approach:** After the final implementation review is resolved, build the CLI; - create project and user workspaces under `/tmp/`; expose Codex/Pi fixture - targets; serve on loopback and `0.0.0.0`; acquire from local Git and OCI - fixtures; run the official A2A client through negotiation, success, replay, - cancellation, deadline, shutdown, restart, and expiry. Run a minimal custom- - provider fixture through one configured-repository-set invocation and one - named-snapshot invocation, proving Promptfoo configuration carries only - source mode, named revision overrides, snapshot handle, and digests while the - gateway resolves origins. Document that network reachability grants full - authority and App/OCI secrets are process inputs, not YAML. -- **Execution note:** The green smoke test must exercise the same built command - and `/tmp/` workspace shape as the recorded red E2E, not a test-only server. - The consumer fixture models AI Evals but remains test/documentation code; the - AllAgents runtime does not import Promptfoo. -- **Verification:** `bun run build`, focused and full tests, typecheck, lint, docs - build, schema drift check, custom-provider contract fixture, and exact - red/green E2E commands/results recorded in the PR description. +- **Files:** published extension and snapshot-format pages, gateway guide/ + reference, configuration reference, README, CHANGELOG, real project/user + workspaces, AI Evals-style Promptfoo YAML and custom-provider contract fixture, + E2E fixtures, packed-install smoke, release evidence. +- **Approach:** After final implementation review, build and pack the CLI plus + both helper packages; install in a clean Linux environment; create project and + user workspaces under `/tmp/`; gateway-enable fixture targets; serve on + loopback and `0.0.0.0` with a valid advertised URL; exercise health/readiness; + acquire local Git and OCI fixtures; and run an independently generated + official A2A client through version/extension negotiation, errors, both send + modes, complete listing, success, replay, cancellation, deadline, shutdown, + restart, and expiry. Run the custom-provider fixture through repository and + snapshot invocations with secure Promptfoo defaults, proving requests contain + only logical source data while the gateway resolves origins. Document full + network-peer authority, sensitive opaque payloads, and process-only secrets. +- **Execution note:** Green smoke uses the same built command and `/tmp/` + workspace shape as red E2E, never a test-only server. The consumer fixture is + AI Evals-style test/documentation code; AllAgents runtime does not import + Promptfoo. +- **Verification:** `bun run build`, packed-install/helper checksum smoke, + focused and full tests, typecheck, lint, docs build, extension/schema drift, + custom-provider contract fixture, and exact red/green commands/results in the + PR description. --- @@ -1166,20 +1575,20 @@ messages never enter either carrier. | Gate | Applies to | Required evidence | |---|---|---| -| Workspace schema | U1 | Project/user parsing, strict nested fields, built-in collision rules, generated-schema drift | -| Public contract | U1-U2 | Official A2A client, every-operation activation, metadata preservation, exact request/result/Artifact/error/canonicalization fixtures | -| Trusted-network model | U2, U7 | Loopback and `0.0.0.0`; shared Task visibility/cancellation; docs warning | -| Durable Task lifecycle | U2, U4 | Private safe state paths, lock, durable claim+Task, no early eviction, store faults, races, restart, atomic expiry | -| Repository acquisition | U3 | Declared-name revision resolution, hermetic Git, full commits, three-way App/`gh` eligibility and sub-budget | -| OCI acquisition | U3 | Declared repository, manifest/layer/workspace digests, safe extraction, no fallback | -| Credential and state isolation | U3-U7 | Separate provider/MCP/tool views; teardown; no cross-scope secrets, operator home, or state root | -| Supervisor lifecycle | U4 | Deadline, shutdown, cancellation races, non-escapable containment, unmanaged recovery, manager handoff, stale-root proof | -| Safe evidence | U4-U6 | Descriptor-relative no-follow reads; links/special files/Git indirection rejected; hermetic Git | -| Backend conformance | U4-U6 | Same suite for fake, Codex, and Pi; profile and built-in variants | -| Structured result | U1, U4-U6 | Exact subset and envelope, valid/invalid/not-produced states, Artifact cardinality, no false publication | -| Repository quality | All | Build, focused/full tests, typecheck, lint, schema check, docs build | -| Built CLI E2E | U7 | Recorded red then green built command under `/tmp/`, both sources, auth isolation, replay/cancel/deadline/shutdown/restart | -| Promptfoo consumption | U7 | AI Evals-style YAML for both source modes; request source metadata and gateway-generated response provenance omit origins; terminal Task maps to `ProviderResponse` | +| Workspace/package schema | U1 | Root workspace install/build edge; ordered helper-package publication and clean-registry resolution; project/user parsing; compiled repository/target catalogs; generated schema/spec drift | +| Public contract | U1-U2 | Independent official HTTP+JSON client; card interface/params/streaming capability; A2A version and every-operation extension headers; unified Parts; both send modes; complete listing; metadata; `google.rpc.Status`; request/result/Artifact/canonicalization fixtures | +| Trusted-network model | U2-U3, U7 | Loopback and `0.0.0.0` with distinct advertised URL; HTTPS docs; shared external Task visibility/cancellation; invocation-to-gateway and host-network denial; metadata-only health/readiness | +| Durable Task lifecycle | U1-U3 | Descriptor-rooted SQLite VFS/full-sync transactions; private state/lock; create-or-replay; one execution lease; internal outcome intent and atomic terminal settlement; no early eviction; crash/store faults; restart; transactional expiry | +| Repository acquisition | U4 | Compiled-name resolution, hermetic Git, commits, 200/404/ambiguous App eligibility, cache bypass, token validation/revocation, `gh` fallback and sub-budget | +| OCI acquisition | U4 | Strict Docker auth/helper; exact layer-redirect allowlist and per-hop address checks; Bearer origin policy; direct-image/config/layer media; descriptor verification; changesets/whiteouts; fixed limits; exact project catalog; no fallback | +| Linux helper and isolation | U1, U3-U7 | x64/arm64 packages/checksums; kernel/cgroup readiness; gated durable containment; full namespace enumeration; pidfd termination; openat2 path/VFS handles; non-bypassable spawn mediation; separate mount/environment/descriptor/network views | +| Supervisor lifecycle | U3 | Capacity races; pre/post-gate crash points; provider/cancel/deadline/shutdown intent races; live-event capture; atomic evidence settlement; poison/readiness/reaping; unknown-set proof | +| Safe evidence | U3-U6 | Descriptor-relative reads with identity/size recheck; links/special/sparse/replaced files and Git indirection rejected; no verified FS evidence before quiescence | +| Backend conformance | U2-U3, U5-U6 | Same lifecycle suite for fake, Codex, and Pi; real child/grandchild spawn mediation; credential and gateway-network denial; profile and built-in variants | +| Structured result | U1, U3, U5-U6 | Public grammar, Codex native-subset gate and fallback, valid/invalid/not-produced states, Artifact cardinality, no false publication | +| Repository quality | All | Build, clean-registry install, focused/full tests, typecheck, lint, schema/spec checks, docs build | +| Bundled CLI E2E | U7 | Recorded red then green command under `/tmp/`, both sources, advertised URL/probes, auth and network isolation, capacity, replay/cancel/deadline/shutdown/restart | +| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both modes; optional context; nonblocking acceptance/subscription/cancel; source/provenance omit origins; output/usage/error metadata mapping | ## Definition of Done @@ -1188,50 +1597,72 @@ messages never enter either carrier. - Every R1-R19 requirement is implemented or explicitly demonstrated by a passing acceptance scenario. - The gateway starts with no `gateway.yaml` or `worker.yaml`, defaults to - loopback, and accepts explicit `0.0.0.0`. -- Network reachability is the only caller trust boundary; Task visibility and - idempotency are deployment-wide and documented accurately. -- Project workspace declarations own repositories and named OCI snapshot - repositories; user workspace declarations own profile launcher exposure; - built-in target IDs cannot be shadowed. -- The A2A card, every-operation activation header, metadata preservation, strict - request and result-schema grammar, exact error mapping, integrity/produced - Artifacts, canonicalization, retention capacity, and cancellation semantics - pass official-client contract fixtures. -- AI Evals-style Promptfoo YAML selects repository mode with optional named - revision overrides, or snapshot mode with one logical handle and immutable - digests. The custom-provider fixture maps one `callApi` to one Task, propagates - cancellation, normalizes usage, and returns output, Artifacts, and logical - provenance without sending origins or adding a Promptfoo runtime dependency. -- Repository and OCI modes produce one validated workspace-manifest contract, - never fall back across source modes, and retain truthful provenance. -- GitHub App eligibility/unknown state, acquisition sub-budget, no-installation - `gh` fallback, selected-App failure, OCI auth containment, and pre-provider - source-credential teardown are proven. -- Typed preparation never runs workspace setup shell commands. Built-in and - profile targets authenticate through private provider-control views; every MCP - child is secret-scoped; model tools cannot reach provider/MCP/operator - credentials or gateway state. -- Deadline, cancellation/terminal races, shutdown, result states, safe private - state paths, retention capacity, store failure, descendant quiescence, - managed/unmanaged recovery, safe evidence, and cleanup pass fault tests. + loopback HTTP, accepts explicit `0.0.0.0`, requires a separate advertised URL + off default loopback, documents production HTTPS, and exposes truthful + metadata-only health/readiness. +- Network reachability is the only external caller trust boundary; Task + visibility and idempotency are deployment-wide. Invocation descendants cannot + reach that boundary, host loopback, or management networks. +- Project workspace declarations compile to the exact repository/snapshot + catalog; user declarations own profile launcher gateway enablement; built-in + target IDs cannot be shadowed. +- The published extension, Agent Card interface/params, A2A version and + activation headers, unified Parts, both send modes, full ListTasks behavior, + metadata preservation, strict schemas, HTTP+JSON errors, embedded Artifacts, + canonicalization, retention, and cancellation pass independent official-client + fixtures. +- Secure-default AI Evals Promptfoo YAML selects repository mode with optional + named revision overrides or snapshot mode with one handle and immutable + digests. The provider maps one optional-context `callApi` to one nonblocking + Task, retains its high-entropy key across ambiguous retry, propagates + cancellation with a fresh cleanup signal, normalizes usage, and returns safe + error metadata and logical provenance without origins or runtime dependency. +- Git and OCI modes produce one complete workspace-manifest contract. OCI v1 + uses the direct-image/config/layer profile, exact project catalog, descriptor + verification, same-origin metadata, operator-approved layer redirect hosts + with per-hop address validation, changeset semantics, and fixed extraction + ceilings. Source modes never fall back and provenance never overclaims + verification. +- App eligibility and ambiguous 404 handling, acquisition sub-budget, positive- + ineligibility `gh` fallback, fresh token cache bypass/validation/revocation, + strict Docker auth/helper and registry challenge policy, and pre-provider + credential teardown are proven. +- Typed preparation never runs workspace setup commands. The packaged Linux + helper durably binds containment before releasing any child, enumerates + unknown cgroups, and mediates every MCP/tool exec into role-specific mount, + environment, descriptor, credential, and network views. Real Codex/Pi + child/grandchild tests prove model tools cannot reach provider/MCP/operator + credentials, gateway state, or the gateway/host-management network; a backend + without non-bypassable spawn mediation is unavailable. +- The descriptor-rooted SQLite VFS, execution lease, pre/post-start-gate crash + boundaries, internal outcome-intent races, atomic terminal evidence + settlement, result states, state-path safety, descendant quiescence, readiness + poisoning/reaping, immutable terminal Tasks, and cleanup pass fault tests. - Evaluation behavior, public-Internet authentication, remote workers, custom - materializers, and multi-tenant policy remain absent. + materializers, non-Linux gateway execution, and multi-tenant policy remain + absent. ### Per unit -- U1: Runtime and generated schemas agree; invalid negotiation, request, - source/exposure/collision/configuration fixtures fail at expected paths. -- U2: Official A2A operations, global replay/visibility, project locks, store - faults, listeners, deadline, shutdown, restart, and retention pass. -- U3: Git and OCI fixtures pass; three-way provider eligibility, token lifetime, - and all no-fallback rules are observed; leak scans are clean. -- U4: Fake-adapter lifecycle proves terminal monotonicity, typed preparation, - isolation, containment, bounded/safe evidence, deadline/cancellation/shutdown, - poisoning, and cleanup. -- U5: Codex passes shared conformance and optional credentialed smoke evidence is - recorded when credentials exist. +- U1: Root workspace packaging, ordered helper-platform publication, clean- + registry resolution, safe SQLite VFS, runtime/generated schemas, published + extension and snapshot format, producer fixture, and invalid negotiation/ + source/enablement/collision/configuration fixtures agree. +- U2: Official HTTP+JSON operations, version/extension/error/list semantics, + global replay/visibility, helper-owned SQLite locks/crashes, execution lease, + listeners/advertised URL, probes, deadline, shutdown, restart, and retention + pass against the fake backend. +- U3: The fake lifecycle proves gated durable containment, full namespace + reconciliation, atomic terminal settlement, typed preparation, spawn-mediated + secret/descriptor/network views, safe evidence ordering, poisoning, reaping, + and cleanup on Linux x64/arm64. +- U4: Git and OCI fixtures pass; App eligibility/cache bypass/token + validation/revocation, Docker credential/challenge and layer-redirect rules, + changesets, limits, exact catalog, and no-fallback rules are observed; leak + scans are clean. +- U5: Codex passes shared conformance and both schema paths; optional + credentialed smoke evidence is recorded when credentials exist. - U6: Pi passes the same conformance and malformed RPC cannot produce success. -- U7: Final review is resolved; built CLI red/green E2E under `/tmp/`, Promptfoo - custom-provider contract fixture, complete repository gates, schemas, docs, - and reproducible PR instructions are complete. +- U7: Final review is resolved; bundled and packed CLI red/green E2E under + `/tmp/`, Promptfoo fixture, complete repository gates, published schemas/specs, + docs, and reproducible PR instructions are complete. From 77a36c9a6bff7a349ae2a853a27aa469703c1938 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Sun, 20 Sep 2026 13:50:25 +1000 Subject: [PATCH 11/12] docs(architecture): define reusable execution workspaces --- ...-agent-execution-through-an-a2a-gateway.md | 603 ++-- ...0837-feat-coding-execution-gateway-plan.md | 2429 +++++++++++------ 2 files changed, 1959 insertions(+), 1073 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 22aacddc..1d78419a 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -2,7 +2,7 @@ - Status: Accepted; implementation pending - Date: 2026-09-17 -- Updated: 2026-09-19 +- Updated: 2026-09-20 ## Context @@ -22,10 +22,12 @@ loopback, a firewalled network, or a Tailscale network. Network reachability is the trust and authorization boundary. A coding-agent execution still includes more than a model request. The gateway -must acquire an immutable workspace, select a configured agent target, contain -credentials to their required phases, propagate cancellation, collect evidence, -terminate descendants, and clean up. Those responsibilities need one public -contract even when the initial deployment remains a single trusted process. +must acquire or reuse an immutable workspace base, select a configured agent +target, propagate cancellation, collect bounded evidence, and clean up. +The trusted CI job, VM, or container that runs the gateway is the execution and +secret boundary: provider code and model-invoked tools run with that runner's +authority. The gateway owns one public contract for the lifecycle without +claiming hostile-code containment inside that boundary. The contract must not turn AllAgents into an evaluation harness. Dataset expansion, repetitions, assertions, scoring, experiment scheduling, and durable @@ -35,10 +37,12 @@ evaluation Runs remain consumer concerns. ### Add a trusted-network execution gateway -AllAgents will provide an independently testable `allagents gateway serve` -entry point. It is separate from the interactive CLI command lifecycle but may -run as a single local service process that supervises acquisition and provider -child processes. +AllAgents will provide an independently testable, separately installed +`allagents-gateway serve` entry point. It runs as one Bun service process that +supervises phase-scoped acquisition and host provider processes. The ordinary +`allagents` CLI does not contain or depend on the gateway. A convenience +dispatcher may locate and execute a separately installed compatible +`allagents-gateway`, but it must not download or embed gateway artifacts. The gateway implements A2A protocol version `1.0` over the `HTTP+JSON` binding plus a required versioned AllAgents coding-execution extension. It owns: @@ -54,6 +58,56 @@ plus a required versioned AllAgents coding-execution extension. It owns: The gateway is not an evaluator, grader, experiment scheduler, retry authority, or durable evaluation Run ledger. +### Use one TypeScript/Bun workspace with narrow package boundaries + +The gateway and CLI are TypeScript products in one Bun workspace monorepo. The +private root package owns orchestration only. `apps/cli` publishes `allagents`; +`apps/gateway` publishes `allagents-gateway`; and `apps/acquirer` is built only +as a digest-pinned GHCR image, never as an npm package. Shared code is limited to +three justified packages: + +- `packages/workspace-config` owns the project and user workspace projections + consumed by the CLI and gateway; +- `packages/execution-contracts` owns the A2A coding-execution wire contract and + portable validation; and +- `packages/acquisition-contracts` owns the typed request and manifest exchanged + with the acquisition image. + +Generated, language-portable contract fixtures live under `contracts/`. The +repository does not introduce speculative `core`, `common`, native platform, or +provider-sharing packages. A package is added only for an already-demonstrated +ownership boundary. + +This architecture follows from the deployment boundary. V1 runs on one trusted +Linux CI runner, the gateway and official provider automation surfaces are +available in TypeScript, and Docker is needed only for untrusted repository and +OCI materialization. Adding another gateway implementation runtime and custom +in-job security layer would increase release and operational surface without +creating a boundary inside the already-trusted CI job. + +### Keep independent CLI and gateway release trains + +The `allagents` CLI and `allagents-gateway` have independent versions, tags, and +release triggers. A CLI release publishes only `apps/cli`; installing it fetches +neither the gateway package nor the acquisition image. + +A gateway release first builds the multi-architecture `apps/acquirer` image, +pushes it to GHCR, records the immutable image-index digest and each supported +architecture's manifest digest, and verifies acquisition against those exact +digests. It then packs the exact `apps/gateway` npm tarball and runs package and +registry conformance with that tarball and those image digests. Only after both +artifacts pass does the workflow publish `allagents-gateway`. It does not +publish `allagents`. + +Compatibility is a versioned contract, not equal npm versions. +`allagents-gateway compatibility --format json` reports the product, gateway +version, build identity, acquisition image digest, and supported A2A, +coding-extension, workspace, execution-contract, acquisition-contract, and +snapshot versions. The +optional CLI dispatcher may launch a separately installed gateway only when the +required contract-version intersections are non-empty. Compatibility does not +depend on target-specific npm wrappers or embedded native binaries. + ### Trust the network boundary instead of adding application authentication The initial gateway has no application-level authentication or per-caller @@ -72,12 +126,12 @@ scoped to a caller identity. Operators must use Tailscale ACLs, host firewalls, container networking, or equivalent network controls when the listener is not loopback-only. -Invocation descendants are not network peers. Provider, MCP, and model-tool -processes run in role-specific network namespaces that cannot route to host -loopback, any gateway bind or advertised address, ingress proxies, or operator -management networks. Provider and MCP egress is default-deny except for -destinations compiled from adapter and MCP configuration; model tools receive no -network unless an explicit adapter policy grants the same constrained egress. +Provider processes, MCP servers, and model-invoked tools are not separate +network principals. They execute on the same trusted CI runner as the gateway +and may exercise the authority available to that job. Operators must provision +the runner accordingly and must not rely on AllAgents to isolate host secrets, +the gateway listener, management networks, or arbitrary repository code from +model-invoked tools. Gateway-managed TLS termination, OIDC, static bearer tokens, per-tenant ownership, and multi-tenant information-hiding are deferred. Production clients @@ -131,23 +185,23 @@ Process-level options use exact flags and environment variables for: - listener, advertised-interface URL, and workspace selection; - a project-specific state-directory override; -- terminal Task retention and bounded Artifact/event storage; +- disjoint immutable-base cache and per-Task runtime/workspace roots; +- workspace materialization policy plus Task and cache retention limits; - GitHub App identifiers and private-key file references; -- the configured GitHub CLI account; -- a strict Docker-auth file or fixed Docker credential-helper executable; and -- Codex and Pi auth-file handles. +- the configured GitHub CLI account; and +- a strict Docker-auth file or fixed Docker credential-helper executable used + only for acquisition. By default the state root is a deterministic child of `~/.allagents/gateway/` keyed by the canonical project-workspace identity. The -packaged Rust helper owns a private SQLite store in WAL/full-synchronization mode -through a descriptor-rooted VFS. Every database, WAL, SHM, journal, and temporary -file open uses `openat2` beneath/no-symlink resolution and rejects hard links. -The store persists claims, Tasks, one execution lease, internal outcome intent, -events, bounded Artifact bytes, containment identity, and expiry transactions. -It verifies workspace identity and holds an exclusive process-lifetime lock. -The root is current-user owned, private, link-resistant, and disjoint from -project, profile, and invocation roots. The listener exposes metadata-only -`/healthz` and `/readyz`; readiness is false whenever admission is unsafe. +gateway owns an ordinary private Bun SQLite database with transactions, WAL +mode, and full synchronization. It persists claims, Tasks, one execution lease, +internal outcome intent, events, bounded Artifact bytes, and expiry state. The +gateway verifies workspace identity and holds an exclusive process-lifetime +lock. The root is current-user owned, private, and disjoint from project, +profile, and invocation roots. Standard Bun SQLite APIs are the entire storage +layer. The listener exposes metadata-only `/healthz` and `/readyz`; readiness is +false whenever admission is unsafe. ### Support direct repositories and OCI workspace snapshots @@ -169,8 +223,8 @@ tags may be accepted for developer convenience, but the gateway resolves and records the full commit object ID before provider execution. Reproducibility- sensitive callers should supply full commit IDs. -For OCI snapshots, the project workspace declares the registry repository and -any exact cross-origin layer-blob redirect hosts: +For OCI snapshots, the project workspace declares an operator-selected OCI +Distribution repository and any exact cross-origin layer-blob redirect hosts: ```yaml workspaceSnapshots: @@ -180,6 +234,26 @@ workspaceSnapshots: - pkg-containers.githubusercontent.com ``` +The repository field is registry-neutral. V1 must pull AllAgents-formatted +workspace snapshots from Docker Hub, GHCR, JFrog Artifactory/JFrog Container +Registry, and compatible private OCI Distribution registries. Registry choice +does not change the snapshot media types, digest requirements, extraction +rules, or caller-visible source contract. + +Registry conformance is tiered. Every pull request runs local Distribution +fixtures and a live public, digest-pinned GHCR pull through the exact gateway +package under test and the exact acquisition image index and architecture +manifest digests built for that pull request. A release workflow additionally +tests least-privilege authenticated GHCR and a digest-pinned disposable JFrog +Container Registry over HTTPS with a private CA and pull-only identity. Those +release checks install the exact gateway npm tarball and use the exact +multi-architecture acquisition image index and per-architecture manifests +intended for publication, for every architecture the registry and runner +support, without rebuilding either artifact. A report for another commit, +package, image digest, architecture manifest, build identity, or compatibility +output is rejected. Docker Hub behavior remains covered by protocol fixtures to +avoid public rate-limit dependence in pull-request CI. + The request supplies the name `evaluation`, a `sha256:` OCI image-manifest digest, and a `sha256:` workspace-manifest digest. The gateway constructs the full OCI reference server-side. Callers cannot supply a registry host, @@ -216,12 +290,74 @@ destination paths. A commit listed inside an OCI snapshot is not described as independently verified unless the gateway separately verifies it against its Git remote. -Acquisition occurs in a gateway-owned staging directory. The gateway validates -paths, collisions, file types, symlinks, layer and file counts, individual and -total compressed and expanded sizes, digests, and the workspace manifest before -atomically publishing the invocation workspace. Absolute paths, traversal, -device files, sockets, escaping links, foreign or external OCI layers, and -unapproved cross-origin access are rejected. +For a source without a reusable validated base, the host gateway creates a +staging directory and bind-mounts only that directory into the digest-pinned +acquisition image. The acquisition container receives only the selected +repository or registry credential plus the strict network, redirect, size, +file-count, and archive policy needed for that source. The host resolves GitHub +App eligibility and mints any installation token; the container never receives +the App private key, host home directory, provider authentication state, or +Docker socket. The image contains and downloads no Codex, Pi, or other coding +harness. + +The container materializes the repository or OCI source into staging, emits the +typed acquisition manifest, and exits. The gateway removes it before provider +execution, validates the manifest plus paths, collisions, file types, symlinks, +layer and file counts, individual and total compressed and expanded sizes, and +digests, then atomically promotes staging to a validated base. Absolute paths, +traversal, device files, sockets, escaping links, foreign or external OCI +layers, and unapproved cross-origin access are rejected. Every non-publication +path removes staging. Docker has no role after acquisition completes. + +The base-cache key binds the acquisition-contract version, compiled catalog and +layout digest, and immutable source identity: every effective repository commit, +or the OCI manifest and workspace-manifest digests. A repository request is +reusable only when every effective revision is a full commit ID. Mutable +branch or tag requests instead receive a non-reusable Task-owned base that is +removed during settlement or reconciliation. Cache hits mint no credential and +start no acquisition container. Active Tasks pin reusable bases; bounded cache +eviction removes only unpinned entries. + +The request optionally selects `workspaceAccess: "readOnly" | "readWrite"` and +defaults to `readWrite`. A read-only Task resolves its provider cwd directly +inside its validated base; exact immutable requests may share a reusable cached +base, while mutable branch or tag requests own a non-reusable base. Every Task +receives a private runtime directory for temporary, home, provider-state, and +evidence files. The gateway disables optional Git locks and asks the adapter for +its native read-only policy when available. It does not inspect the prompt or +add a per-Task mount, chmod pass, or full-tree verification. Read-only is a +cooperative contract and best-effort provider control, not a hostile-code +boundary; the consumer remains responsible for giving the Task work that does +not require project writes. A violating provider can contaminate a cached base +and later Tasks; the operator must evict that entry before reuse. + +A read-write Task receives a unique writable view under +`//workspace`. The host materializer prefers a +filesystem block clone, falls back to rootless OverlayFS on supported Linux +hosts, and supports an explicit ordinary-copy backend for portability. It never +uses hard links for writable files. The selected materializer is operator +configuration, not request input. After evidence collection, normal settlement +unmounts when needed and removes the Task-owned view plus any non-reusable base; +a non-settling provider retains them with the poisoned execution lease until +reconciliation. + +For either access mode, the provider cwd is resolved from an optional logical +`workingDirectory` selector: + +- `{ kind: "workspaceRoot" }` selects the effective workspace root and is the + default; or +- `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }` + selects a declared repository and an optional validated directory beneath it. + +The caller never supplies an absolute path, configured destination, materializer, +cache key, or physical workspace name. The gateway maps the repository name +through the compiled catalog, resolves the optional relative path, and requires +the result to be an existing directory whose resolved path remains beneath the +selected repository root. The logical selector and access mode are part of the +canonical request, idempotency identity, and integrity evidence. +Gateway-generated structured metadata and operational logs never contain the +physical path; opaque terminal output, native evidence, and produced Artifact +payloads are not sanitized and may contain it. ### Consume the gateway from Promptfoo through an AI Evals provider @@ -240,14 +376,17 @@ AI Evals owns a Promptfoo It implements `ApiProvider`: its constructor receives `ProviderOptions`, requires and retains a nonempty `options.id`, validates `options.config`, and exposes `id()`. -`callApi(prompt, context?, options?)` reads bounded test variables from -`context?.vars` when present and cancellation from `options?.abortSignal`. The -provider translates one `callApi` into one A2A Task: it creates and retains a -high-entropy invocation key, sends one Message whose sole Part has `text` set, -declares the extension in `Message.extensions`, puts the target and closed source -union in the matching metadata member, and calls `SendMessage` with -`returnImmediately: true`. It captures the Task ID and follows terminal state -through `SubscribeToTask`, with `GetTask` and bounded resubscription for races or +`callApi(prompt, context?, options?)` reads bounded source, working-directory, +and workspace-access test variables from `context?.vars` when present and +cancellation from `options?.abortSignal`. The provider translates one `callApi` +into one A2A Task: it creates and retains a high-entropy invocation key, resolves +the effective logical working-directory selector and `readOnly | readWrite` +access mode, sends one Message whose sole Part has `text` set, declares the +extension in `Message.extensions`, puts the target, closed source union, logical +working directory, and access mode in the matching metadata member, and calls +`SendMessage` with `returnImmediately: true`. +It captures the Task ID and follows terminal state through `SubscribeToTask`, +with `GetTask` and bounded resubscription for races or disconnects. It returns output, normalized token usage, stable failure metadata, and logical provenance in Promptfoo's `ProviderResponse`. @@ -272,6 +411,10 @@ providers: config: endpoint: https://allagents-gateway.example.internal target: codex + workingDirectory: + kind: repository + repository: allagents + workspaceAccess: readOnly source: kind: repositories revisions: @@ -282,11 +425,24 @@ providers: config: endpoint: https://allagents-gateway.example.internal target: codex + workingDirectory: + kind: repository + repository: allagents + workspaceAccess: readWrite source: kind: workspaceSnapshot snapshot: evaluation digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 + +tests: + - description: gateway package trial + providers: [codex-direct] + vars: + allagentsWorkingDirectory: + kind: repository + repository: allagents + path: apps/gateway ``` The first provider materializes the complete configured repository set and uses @@ -300,20 +456,26 @@ defaults for confidential prompts and outputs; consumers may enable persistence or sharing only after applying their own retention, access, destination, and redaction policy. -Static provider config fixes the source kind and logical names. The only -per-test object is `context?.vars?.allagentsSource`: repository mode accepts -revision overrides only for statically listed names and only as full lowercase -40-hex commits; snapshot mode accepts only replacement OCI and workspace- -manifest `sha256:` digests. Missing context or leaves retain static values. A -URL, destination, mutable revision, credential, command, unknown member, or -changed source kind/name fails before submission. After Task acceptance, the -provider's bounded deadline or `options?.abortSignal` sends one `CancelTask` -using a fresh cleanup signal rather than the already aborted request signal. +Static provider config fixes the source kind and logical names and may define a +default logical `workingDirectory` and `workspaceAccess`; absent values default +to `{ kind: "workspaceRoot" }` and `readWrite`. Per-test +`context?.vars?.allagentsWorkingDirectory` may replace the selector, while +`context?.vars?.allagentsWorkspaceAccess` may replace the access mode with the +exact string `readOnly` or `readWrite`. Separate read-only trials may share one +immutable physical base and cwd. Read-write trials receive distinct Task-owned +writable views even when their logical selectors are equal. +`context?.vars?.allagentsSource` remains limited to revision or digest leaves. +Missing variables retain static values. Absolute paths, `.` or `..` segments, +configured destinations, unknown repositories, URLs, mutable revisions, +credentials, commands, materializer choices, and unknown members fail before +provider execution. After Task acceptance, the provider's bounded deadline or +`options?.abortSignal` sends one `CancelTask` using a fresh cleanup signal +rather than the already aborted request signal. It maps gateway input, output, cached-input, and total token counts to Promptfoo's `prompt`, `completion`, `cached`, and `total` fields respectively. -Safe stable failure -code, retryability, accepted Task ID, other usage, and logical Task/Artifact -evidence stay in metadata without origins. Opaque prompts, terminal output, +Safe stable failure code, retryability, accepted Task ID, other usage, logical +working directory, and Task/Artifact evidence stay in metadata without origins, +configured destinations, or physical paths. Opaque prompts, terminal output, structured results, native evidence, and produced-Artifact payloads remain unredacted sensitive data. The provider belongs in AI Evals. AllAgents exposes the A2A contract and consumer documentation without taking a runtime dependency @@ -363,131 +525,154 @@ rate-limit, or service failure terminates acquisition. The gateway never retries the same Task through the broader GitHub CLI identity. Git receives credentials only through an invocation-scoped helper under -hermetic Git configuration. The gateway excludes system, global, and repository -credential helpers, Git Credential Manager, askpass, SSH agents, repository- -controlled secondary fetches, and executable Git configuration. Tokens never -appear in clone URLs, command arguments, Git configuration, logs, Tasks, -Artifacts, retained workspaces, profile setup, MCP processes, agent processes, -or model-invoked tools. The helper and token are destroyed before provider -execution. +hermetic Git configuration inside the acquisition container. The gateway +excludes system, global, and repository credential helpers, Git Credential +Manager, askpass, SSH agents, repository-controlled secondary fetches, and +executable Git configuration. Tokens never appear in clone URLs, command +arguments, Git configuration, logs, Tasks, Artifacts, or retained workspaces. +The helper and token are destroyed and the acquisition container is removed +before provider execution. OCI credentials come from either a strict Docker-auth subset that cannot name executables or a fixed Docker credential helper using its standard `get` -protocol. They are scoped to snapshot acquisition and removed before -publication. Public registries require no credential configuration. - -### Integrate providers through typed adapters +protocol. Acquisition supports anonymous pulls plus same-origin Basic and +Distribution Bearer challenge flows required by the declared registry, with the +documented Docker Hub token-service exception. Any other cross-origin Bearer +realm is rejected before a request is sent. System roots may be supplemented by +an operator map from exact registry `host[:port]` keys to verified PEM bundles; +each bundle is trusted only for connections to its key. Credentials and custom +trust material are scoped to snapshot acquisition and removed before +publication. Public registries require no credential or custom-CA +configuration. + +### Integrate host providers through narrow typed adapters The initial backend registry contains Codex and Pi, delivered in that order. -Each adapter implements one behavior-focused contract for availability, +Each adapter implements a narrow AllAgents-owned contract for availability, capabilities, invocation, progress, deterministic permission handling, abort, terminal output, optional structured result, usage, native evidence, and -disposal. - -The Codex adapter depends directly on pinned `@openai/codex-sdk`, creates one -fresh thread per Task, passes cancellation, and consumes structured events. It -uses native `outputSchema` only for schemas supported by the pinned Structured -Outputs contract; other valid public schemas use explicit JSON guidance and the -same gateway-side validator used by every backend. - -The Pi adapter uses strict RPC mode with invocation-owned configuration and a -restricted policy extension. Repository extensions and unrestricted built-ins -are not loaded merely because they exist in acquired source. - -CLI-backed compatibility adapters may be added later when a client has a stable -machine protocol. Missing controls are reported honestly as capability gaps. -The gateway never scrapes a TUI or exposes arbitrary installed executables. -OMP is Pi-derived and is added only for demonstrated OMP-specific value beyond -direct Pi. +disposal. The gateway does not adopt AI SDK Harnesses or make a third-party +cross-provider abstraction part of its execution contract. + +The Codex adapter uses a pinned `@openai/codex-sdk` release first. App-server is +permitted only when a required, demonstrated capability is absent from that +SDK; convenience or speculative parity is not enough. Native `outputSchema` is +used only for schemas supported by the pinned Structured Outputs contract; +other valid public schemas use explicit JSON guidance and the same gateway-side +validator used by every backend. The adapter does not scrape a TUI or use an +unstable bridge merely to preserve the target name. + +The Pi adapter uses a pinned, supported RPC or package surface with +invocation-owned configuration and a restricted policy extension. Repository +extensions and unrestricted built-ins are not loaded merely because they exist +in acquired source. OMP remains out of the initial registry and is added only +for demonstrated OMP-specific value beyond direct Pi. + +Provider runtimes are installed and pinned as part of the CI runner or gateway +installation; the gateway never downloads them per request. An operator may +select a globally installed binary override only when an exact version and +capability compatibility probe succeeds. Missing controls are reported honestly +as capability gaps, and the gateway never exposes arbitrary installed +executables. + +Codex and Pi execute bare metal, directly on the same trusted Linux CI runner as +the gateway. For a read-only Task, cwd resolves inside its validated base, +shared only when reusable; for a read-write Task, cwd resolves inside the Task's +unique writable view. When +explicit API credentials are absent, Codex reuses the runner's existing +`CODEX_HOME` and ChatGPT login, and Pi reuses its existing supported host +authentication. The gateway references those host paths in place; it does not +copy, mount, or import OAuth files. + +Each provider process receives an explicitly constructed environment containing +only the invocation configuration, selected provider settings, and required +host identity, executable, home, and authentication paths. This reduces +accidental ambient-variable leakage but is not an isolation or secret- +containment claim: MCP servers, provider descendants, and model-invoked tools +may exercise the same CI-job authority and reach secrets available to that +runner. The CI job, VM, or container must therefore be provisioned as the +security boundary. Provider preparation is adapter-owned and typed. The gateway never executes project or user `setup` shell entries as part of acquisition or invocation. Validated profile settings, plugins, MCP declarations, and deterministic workspace projections are applied through existing typed transforms. -Provider control processes, MCP children, and model-invoked tools receive -distinct allowlisted filesystem, environment, descriptor, secret, and network -views. The provider control process sees only its invocation-private auth -channel; each MCP child sees only its own resolved secrets; shell and other -model-invoked tools see neither provider nor MCP credentials. Every view excludes -gateway state, operator home, App keys, GitHub/OCI stores, acquisition helpers, -unrelated adapter auth, the parent environment, gateway endpoints, host -loopback, and management networks. - -The pinned backend must expose a non-bypassable synchronous hook that delegates -every MCP and model-tool spawn to the Rust helper. The helper enters the role's -mount and network namespaces, replaces the environment, closes every -non-allowlisted descriptor, and only then executes untrusted code. Codex or Pi -is unavailable when its pinned surface can bypass that hook. This enforced -credential/state/network boundary is required even though general hostile-code -sandboxing remains deferred. - ### Persist Task truth, not live provider execution The gateway durably stores Task identity, the canonical request, idempotency claim, selected target and source, effective configuration digest, one execution -lease, internal outcome intent, Artifact bytes, retained evidence, and -containment identity under the configured state directory. The official A2A -HTTP+JSON transport wraps an AllAgents-owned request handler; typed transactions -execute in the Rust helper's descriptor-rooted SQLite VFS. One transaction -arbitrates `createOrReplay`, UUIDv7 Task creation, execution-lease acquisition, -and containment binding; another atomically settles terminal status, result or +lease, internal outcome intent, Artifact bytes, retained evidence, cleanup +outcome, and expiry state under the configured state directory. AllAgents-owned +TypeScript handlers expose the A2A contract and use ordinary private Bun SQLite +ownership and transactions with full synchronization. One transaction +arbitrates `createOrReplay`, UUIDv7 Task creation, and execution-lease +acquisition. Normal terminal settlement atomically writes status, result or failure, evidence, Artifacts, cleanup, and lease release. A provider session is not a durable recovery checkpoint. At most one Task holds the execution lease from acquisition through final evidence collection. A second otherwise-valid request settles failed with -`execution_capacity_unavailable`. Its transient empty containment set is -destroyed after settlement without releasing the start gate or launching a -helper child. An identical idempotency replay returns the existing Task. Reusing -the key with a different canonical request conflicts. Clients generate at least -128 bits of randomness -once per logical invocation and reuse the same key plus request after an -ambiguous transport failure. Because the initial service has no caller identity, -the idempotency namespace and Task visibility are gateway-wide. +`execution_capacity_unavailable` without launching an acquisition container or +provider process. An identical idempotency replay returns the existing Task. +Reusing the key with a different canonical request conflicts. Clients generate +at least 128 bits of randomness once per logical invocation and reuse the same +key plus request after an ambiguous transport failure. Because the initial +service has no caller identity, the idempotency namespace and Task visibility +are gateway-wide. Terminal Task records, Artifacts, events, and invocation claims expire in one transaction after the configured TTL. The retained-count limit never evicts an unexpired Task; the gateway rejects new admission until expiry frees capacity. -State-store integrity, VFS, helper protocol, or durability failure stops -admission and prevents the gateway from acknowledging creation or reporting -terminal success. +State-store integrity or durability failure stops admission and prevents the +gateway from acknowledging creation or reporting terminal success. -On gateway restart, interrupted nonterminal Tasks settle failed only after -containment reconciliation; provider work is not resumed or automatically -replayed. A new invocation may start fresh. +On gateway restart, interrupted nonterminal Tasks settle failed; provider work +is not resumed or automatically replayed. Admission resumes only after any +recorded acquisition container is gone and the recorded provider process group +is confirmed absent. Otherwise the gateway remains unready with the lease held. ### Make cancellation, evidence, and cleanup explicit -The gateway supervises every acquisition and provider process set. The helper -allocates a stable empty containment set behind a start gate. The Task, -execution lease, and containment identifier commit durably before the helper may -release that gate or execute any child; a failed commit destroys the empty set. - One durable compare-and-set arbitrates provider terminal outcome, caller cancellation, deadline, and shutdown as an internal outcome intent while the externally visible Task remains nonterminal. The winning intent owns the stable -result or failure code and drives one idempotent abort and quiescence path. -Cancellation first invokes the provider's native abort or protocol cancellation, -then applies bounded forced termination to the complete descendant set. +result or failure code and drives one idempotent cancellation and settlement +path. + +On Linux, each direct provider process starts in its own process group. +Cancellation first invokes the provider's supported graceful abort, then sends +`SIGTERM` to the process group after a bounded grace period, and finally sends +`SIGKILL` after a second bounded period. This is best-effort lifecycle control, +not containment: descendants can deliberately detach or escape the group. CI +runner teardown is the final orphan boundary. Cancellation during acquisition +stops and removes the acquisition container and unpublished staging; provider +execution never occurs in that container. Live provider events are bounded while execution runs. Filesystem, Git, and -produced-Artifact evidence is read only after the supervisor proves the complete -invocation process set quiescent through its invocation-owned containment. -Only then does one transaction atomically publish terminal status, the integrity -Artifact, bounded evidence, result or failure, produced Artifacts, termination, -cleanup, and lease release. If quiescence cannot be proven, that transaction -settles `execution_quiescence_unknown` without verified filesystem evidence. -The gateway rejects new work and stays alive with poisoned readiness while -continuing to reap; later recovery changes only internal recovery/readiness -state, never the settled Task. - -On startup the helper enumerates the entire project-owned containment namespace, -including unknown identifiers, and proves every set empty before binding, -releasing a retained lease, quarantining stale roots, or advertising readiness. -It prints the stable containment identifier and platform recovery command for -any nonempty set. An unsupported platform fails before binding rather than -relying on process enumeration. +produced-Artifact evidence is collected only after the direct provider process +has settled and the configured process-group escalation has completed. The +gateway does not claim to prove full descendant quiescence. A settled read-only +Task removes its private runtime and any non-reusable base; it retains only a +reusable cached base. A settled read-write Task removes its writable view and +any non-reusable base after evidence collection. One transaction then atomically +publishes terminal status, the integrity Artifact, bounded evidence, result or +failure, produced Artifacts, observed termination and cleanup outcomes, and +lease release. Task-owned cleanup failure publishes `workspace_cleanup_failed` +and retains an internal cleanup record for reconciliation. If the direct process +does not settle after final escalation, the gateway instead publishes +`execution_termination_failed` without filesystem, Git, or produced-Artifact +evidence; retains any Task-owned runtime, writable view, non-reusable base, and +the lease; stops admission; and remains unready until runner teardown and +startup reconciliation confirm the recorded process group is absent and clean +the retained state. +Evidence describes only what the gateway actually observed; +escaped descendants and uncertain cleanup are never upgraded to verified +outcomes. + +V1 supports trusted Linux CI runners and one active invocation. Other operating +systems and concurrent execution require a separate lifecycle design rather +than silent degradation. Terminal evidence distinguishes: @@ -528,8 +713,9 @@ validation also uses `google.rpc.BadRequest`, never JSON-RPC error carriers. The published versioned extension specification defines Agent Card params, activation, request/idempotency/replay, errors, and terminal Task/Artifact schemas. Its request carries the invocation key, execution target, closed -workspace source, bounded deadline, and optional bounded result schema in its -own strict `Message.metadata` member without rejecting unrelated A2A metadata. +workspace source, logical working-directory selector, bounded deadline, and +optional bounded result schema in its own strict `Message.metadata` member +without rejecting unrelated A2A metadata. The request Message lists the URI in `Message.extensions`. Every terminal Task has one fixed-name, versioned integrity Artifact whose `Artifact.extensions` lists the URI, plus zero or more produced Artifacts. Breaking extension versions @@ -548,8 +734,9 @@ retry. Consumers own those concerns. ## Consequences -- Developers can start one endpoint with `allagents gateway serve` and use - loopback, `0.0.0.0`, a specific interface, Tailscale, or firewall policy. +- Developers who explicitly install the gateway package can start one endpoint + with `allagents-gateway serve` and use loopback, `0.0.0.0`, a specific + interface, Tailscale, or firewall policy. - There is no application authentication, per-caller authorization, tenant isolation, `gateway.yaml`, `worker.yaml`, remote worker protocol, or required Kubernetes deployment in the initial product. @@ -561,26 +748,38 @@ retry. Consumers own those concerns. keeps raw origins under AllAgents operator control. - External network reachability grants access to every available target, including built-in and gateway-enabled profile targets, plus every retained - Task. Operators treat network policy as the authorization boundary; invocation - descendants are isolated from that boundary and host-management networks. -- One durable execution lease enforces one active invocation independent of - consumer concurrency settings. + Task. Operators treat network policy as authorization and must restrict the + systems and secrets available to the trusted CI runner; AllAgents does not + isolate provider or model-tool descendants within that runner. +- One durable SQLite execution lease enforces one active invocation independent + of consumer concurrency settings. - GitHub App credentials support private repositories without forcing every developer to use one identity; GitHub CLI remains a local eligibility fallback only when no App installation applies. - Direct repositories and digest-pinned OCI snapshots converge on one validated - workspace manifest and evidence contract. OCI metadata remains same-origin; - only layer blobs may redirect to exact operator-approved hosts. -- Gateway v1 execution is supported on Linux x64/arm64 with the packaged state/ - security helper, descriptor-rooted SQLite VFS, cgroup v2, mount/network - namespaces, nftables, pidfds, and safe-file operations. Matching helper - packages publish and verify before the root package; unsupported hosts or - missing capabilities fail before binding rather than degrading containment. -- The gateway process remains a meaningful API and lifecycle boundary, but not a - hostile-code sandbox. Strong multi-tenant isolation remains future work. -- Codex and Pi share one conformance suite while retaining bounded native - evidence and honest capability differences. A backend is unavailable unless - its pinned surface can delegate every MCP/tool spawn through the helper. + immutable-base manifest and evidence contract. Immutable source identities may + reuse a cached base; OCI metadata remains same-origin and only layer blobs may + redirect to exact operator-approved hosts. +- Read-only Tasks may share that base and physical cwd while keeping private + runtime state. Read-write Tasks receive disposable independent writable views + through the selected copy-on-write or copy materializer. +- Docker is a short-lived base-acquisition boundary only when no reusable + validated base exists. The container receives staging plus source credentials, + emits a typed manifest, and is removed before Codex or Pi starts on the host + runner. +- The private Bun workspace root orchestrates `apps/cli`, `apps/gateway`, the + image-only `apps/acquirer`, and the three contract/configuration packages. + CLI-only installs fetch neither the gateway package nor acquisition image. +- CLI and gateway versions and releases remain independent. Gateway releases + verify the exact npm tarball and the exact digest-pinned multi-architecture + acquisition image before publishing. +- Codex and Pi use pinned supported automation surfaces and existing host + authentication through narrow adapters. Explicit environment construction + reduces accidental leakage but cannot hide runner secrets from model-invoked + tools. +- Linux process-group escalation provides bounded best-effort cancellation. + Runner teardown remains the final orphan boundary, and evidence never claims + full descendant quiescence. - A future deployment configuration becomes justified only when the product needs multiple worker routes, tenants, credential policies, custom materializers, centralized storage, or other operator-selected variants. @@ -619,6 +818,21 @@ identities and destinations. Repository requests materialize the configured set and may override revisions by declared name; snapshot requests select a declared name and immutable digests. Neither variant introduces a new origin. +### Let callers provide a host cwd + +Rejected because an absolute or configured destination path would let a caller +select unrelated host content and bypass gateway-owned acquisition. Promptfoo +gets the required runtime control through a logical workspace-root or declared- +repository selector; the gateway maps it into the access-appropriate reusable +or Task-owned base or writable view according to `workspaceAccess`. + +### Always allocate a unique full workspace + +Rejected because read-only Tasks have no mutable project state to isolate, and +copying a large immutable workspace for every trial wastes transfer, storage, +and I/O. They share one validated base. Writable Tasks isolate only their +changes through a disposable copy-on-write view or explicit portable copy. + ### Fall back from a selected GitHub App after runtime failure Rejected because it would silently change identity and authorization scope after @@ -646,14 +860,69 @@ versioned extension. Rejected because benchmark orchestration, verification, and persisted evaluation state remain consumer concerns. The gateway executes one coding-agent Task. +### Build v1 around Rust and kernel containment + +Rejected because the trusted CI job is already the execution boundary. A Rust +gateway plus custom cgroups, pidfds, `openat2` VFS behavior, namespaces, +`nftables`, or spawn mediation would add implementation and release risk without +isolating model-invoked tools from secrets available to that job. Reconsider +native or stronger containment only if hostile-code or in-job secret isolation +becomes a product requirement. + +### Adopt AI SDK Harnesses as the backend abstraction + +Rejected because AllAgents needs a small contract tailored to its A2A Task, +evidence, cancellation, and profile semantics. Depending on a broad +cross-provider abstraction would enlarge the compatibility surface without +removing the need to understand the official Codex and Pi automation APIs. + +### Run shared host provider daemons + +Rejected because a long-lived daemon introduces cross-invocation state, +ownership, cancellation, and authentication ambiguity. V1 starts one direct +provider process for the one active Task and treats provider sessions as +ephemeral. + +### Run providers in per-invocation containers + +Rejected because official Codex and Pi automation should reuse the trusted +runner's existing installation and authentication. Copying or mounting OAuth +state into a provider container complicates ownership without creating a +security boundary against model tools. Docker remains limited to acquisition. + +### Trust ambient unversioned provider binaries + +Rejected because PATH discovery can silently change behavior between runs. +Pinned SDK, RPC, or package surfaces are the default; a global binary override +must pass exact version and capability probes, and runtimes are never downloaded +per request. + +### Couple CLI and gateway versions or publish them together + +Rejected because the products have different dependencies and release cadence. +Compatibility is explicit at the contract boundary; gateway-only work must not +force a CLI release, and CLI-only installation must not fetch gateway or +acquisition artifacts. + +### Bundle the gateway into every CLI installation + +Rejected because plugin/skill-only users do not need the A2A server, SQLite, +provider adapters, or acquisition image. The gateway ships as the separately +installed `allagents-gateway` npm package, and its release independently binds +the digest-pinned GHCR acquisition image. + ## Reconsider when Revisit this decision when any of these become requirements: - callers outside one trusted network must share the endpoint; - per-caller Task privacy, authorization, or audit identity is required; +- provider or model-tool code must be isolated from runner secrets or treated as + hostile inside the execution environment; - multiple gateway replicas need transactional shared storage; -- execution must route among remote worker pools or hostile-code sandboxes; +- more than one active invocation or shared provider daemons are required; +- execution must route among remote worker pools or sandboxes; +- non-Linux runners need equivalent lifecycle and cancellation semantics; - custom materializers are needed beyond direct Git and OCI snapshots; - multiple GitHub hosts, Apps, CLI accounts, or ordered credential policies need declarative configuration; diff --git a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md index b2e14039..d6db4a43 100644 --- a/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md +++ b/docs/plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md @@ -1,7 +1,7 @@ --- title: "Coding-Agent Execution Gateway - Plan" date: 2026-09-18 -updated: 2026-09-19 +updated: 2026-09-20 type: feat artifact_contract: ce-unified-plan/v1 artifact_readiness: implementation-ready @@ -13,31 +13,49 @@ execution: code ## Goal Capsule -- **Objective:** A developer can run one trusted-network A2A endpoint for one - AllAgents workspace and invoke built-in or explicitly gateway-enabled profile - targets against either the complete configured Git repository set, with optional - named revision overrides, or a digest-pinned OCI workspace snapshot. AI Evals - can configure either source mode in Promptfoo YAML through a custom provider - without sending origins. -- **Means:** Add `allagents gateway serve`, a private execution-service package, - a bounded durable Task store, direct Codex and Pi adapters, GitHub App and - GitHub CLI acquisition providers, OCI snapshot acquisition, one supervised - invocation lifecycle, and a documented Promptfoo provider contract. +- **Objective:** A developer can install and run one trusted-network A2A + endpoint for one AllAgents workspace and invoke built-in or explicitly + gateway-enabled profile targets against either the complete configured Git + repository set, with optional named revision overrides, or a digest-pinned + OCI workspace snapshot. AI Evals can configure either source mode in + Promptfoo YAML through a custom provider without sending origins. +- **Means:** Convert the repository to a private Bun workspace monorepo with + independently released `allagents` and `allagents-gateway` applications, + versioned workspace/execution/acquisition contract packages, generated + portable fixtures under `contracts/`, a bounded SQLite Task store, direct + Codex and Pi host-process adapters, and one digest-pinned acquisition image + used only when no reusable validated base exists. Read-only Tasks share a + reusable base or own a non-reusable base for mutable revisions; read-write + Tasks receive disposable writable views through an automatic block-clone/ + OverlayFS materializer with an explicit portable copy backend. + Providers still run bare metal on the trusted Linux CI runner. - **Authority:** [ADR 0002](../decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md) - owns the public and trust boundaries. Project and user `workspace.yaml` files - own source and profile declarations. A2A 1.0 owns core wire semantics. -- **Execution order:** Capture a red built-CLI E2E for the missing gateway; - freeze schemas and configuration projection; implement the Task store, A2A - server, supervisor/helper, acquisition, Codex, and Pi; run a final - implementation review and fix important findings; then run the green built- - CLI E2E, repository gates, and documentation validation. -- **Stop conditions:** Do not add application authentication, `gateway.yaml`, - `worker.yaml`, remote worker routing, caller-supplied URLs or commands, - mutable OCI tags, selected-provider failure fallback, evaluation behavior, or - automatic execution retry. -- **Tail ownership:** The implementing workflow runs focused contract and - lifecycle tests, provider fixture tests, the repository quality gates, a - built-CLI trusted-network smoke test, and documentation validation. + owns the public, trust, runtime, and packaging boundaries. Project and user + `workspace.yaml` files own source and profile declarations. A2A 1.0 owns core + wire semantics. The CI job, VM, or deployment container is the only + operational execution and isolation boundary. AllAgents does not claim that + boundary contains hostile code or hides job secrets from model-invoked tools; + it owns process lifecycle and truthful evidence only. +- **Execution order:** Capture red CLI-only and standalone-gateway package + smokes; complete the Bun monorepo, A2A SDK, provider-surface, process-group, + Docker-acquirer, package, and release feasibility gate; freeze schemas, + generated fixtures, configuration projection, SQLite ownership, and release + binding; implement the Task store and A2A server, host supervisor, Docker-only + acquisition, Codex, and Pi; run final review; then run green packed-package, + exact-image, registry-conformance, repository, and documentation gates. +- **Stop conditions:** Stop dependent production work if the pinned A2A surface + cannot implement the required public protocol or if the acquisition-container + boundary and exact image/package release binding are infeasible. A missing + Codex or Pi capability makes that target unavailable rather than changing the + A2A, trust, source, or Task contracts. Do not add application authentication, + deployment YAML, remote workers, caller-supplied origins/commands, mutable OCI + tags, provider fallback, evaluation behavior, automatic retries, per-provider + Docker, or containment claims. +- **Tail ownership:** The implementing workflow runs focused contract, + lifecycle, provider-environment, process-group, acquisition-boundary, and + release-binding tests; repository quality gates; packed CLI/gateway and exact + acquisition-image smoke tests; registry conformance; and documentation + validation. --- @@ -49,11 +67,13 @@ AllAgents gains a single-workspace coding-execution service without becoming an evaluation framework or multi-tenant platform. Callers use A2A Tasks and one required AllAgents extension. Network reachability is authorization. The service resolves configured targets and sources from existing workspace files, -acquires a fresh invocation workspace, invokes Codex or Pi through a typed -adapter, and retains bounded terminal evidence. AI Evals consumes that boundary -through its own Promptfoo custom provider: evaluation YAML supplies named -revision overrides for the configured repository set, or one snapshot handle -and immutable digests, while AllAgents retains origin and credential authority. +acquires or reuses an immutable base, shares it for read-only Tasks, creates an +independent disposable view for read-write Tasks, invokes Codex or Pi through a +typed adapter, and retains bounded terminal evidence. AI Evals consumes that +boundary through its own Promptfoo custom provider: evaluation YAML supplies +named revision overrides for the configured repository set, or one snapshot +handle and immutable digests, while AllAgents retains origin and credential +authority. ### Problem Frame @@ -97,13 +117,21 @@ registry, or another profile configuration file for the initial use case. - **Support two acquisition modes.** Direct declared repositories and named, digest-pinned OCI workspace snapshots converge on one manifest and evidence contract. (session-settled: user-directed.) Governs R9-R11. +- **Share immutable bases; isolate writes.** `workspaceAccess` defaults to + `readWrite`. Read-only Tasks may reuse one validated physical base and cwd + with Task-private runtime state; read-write Tasks receive unique disposable + writable views. Reflink/block clone is preferred, rootless OverlayFS is the + Linux fallback, and an explicit copy backend preserves portability. + (session-settled: user-directed.) Governs R2-R3, R5, R8-R11, R15-R16, R18-R19. - **Use App-first GitHub credential eligibility.** Prefer an applicable GitHub App; use a configured `gh` account only when no App installation applies; never fall back after selected-App failure. (session-settled: user-directed.) Governs R12. -- **Keep a typed backend seam.** Codex SDK and Pi RPC are the complete initial - backend set. Launcher-backed profiles resolve through those adapters rather - than executing generated wrapper files. Governs R7-R8, R13-R15. +- **Keep a narrow typed backend seam.** Pinned supported Codex and Pi package or + RPC surfaces are the complete initial backend set. Launcher-backed profiles + resolve through AllAgents-owned adapters and execute on the gateway host + rather than through generated wrapper files or the acquisition container. + Governs R7-R8, R13-R15. - **Persist Task truth, not provider sessions.** Restart settles interrupted work failed; it never resumes or automatically replays provider execution. Governs R5, R13-R16. @@ -139,20 +167,27 @@ registry, or another profile configuration file for the initial use case. `nextPageToken` is present and empty on the final page. With the default `includeArtifacts: false`, each returned Task omits `artifacts`; `true` includes the field. -- R2. Generate a strict versioned request schema from Zod and place it only at - `Message.metadata[extensionUri]`; the Message also lists `extensionUri` in - `Message.extensions`. Strict objects reject every unlisted member. V1 uses - these wire scalars: +- R2. Generate a strict versioned request schema from the canonical domain type + and place it only at `Message.metadata[extensionUri]`; the Message also lists + `extensionUri` in `Message.extensions`. Strict objects reject every unlisted + member. V1 uses these wire scalars: - `InvocationKey` matches `^[A-Za-z0-9][A-Za-z0-9._:-]{0,127}$`. - `ConfigName` and `TargetId` match `^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$`. - `RevisionText` is NFC UTF-8, 1-255 bytes, with no U+0000-U+001F or U+007F. - `Digest` matches `^sha256:[0-9a-f]{64}$`. + - `RelativeDirectory` is NFC UTF-8 of 1-1024 bytes containing 1-32 + slash-separated segments. Each segment is 1-255 bytes, is neither `.` nor + `..`, and contains no slash, backslash, U+0000-U+001F, or U+007F. The request object is exactly: `version: "1"`; `invocationKey: InvocationKey`; `target: TargetId`; `source`, one of `{ kind: "repositories", revisions?: Record }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, workspaceManifestDigest: Digest }`; optional + `workingDirectory`, one of `{ kind: "workspaceRoot" }` or + `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }`, + defaulting to `{ kind: "workspaceRoot" }`; optional `workspaceAccess`, one of + `"readOnly" | "readWrite"`, defaulting to `"readWrite"`; optional `deadlineSeconds` (integer 1-3600, default 1800); and optional `resultSchema: { version: "1", schema: SchemaNode }`. @@ -210,6 +245,14 @@ registry, or another profile configuration file for the initial use case. media type of at most 255 ASCII bytes. - `version` is the literal `"1"`; `taskId` is a lowercase canonical UUIDv7; and `target` is `TargetId`. + - `workingDirectory` is the effective logical selector from the request: + `{ kind: "workspaceRoot" }` or + `{ kind: "repository", repository: ConfigName, + path?: RelativeDirectory }`. It never contains a physical path or configured + repository destination. + - `workspaceAccess` is the effective `"readOnly" | "readWrite"` value. + `readOnly` is a consumer-selected cooperative contract with best-effort + provider-policy enforcement, not hostile-code containment. - `sourceIdentity` is either `{ kind: "repositories", complete, repositories }` or `{ kind: "workspaceSnapshot", snapshot: ConfigName, digest: Digest, @@ -246,9 +289,17 @@ registry, or another profile configuration file for the initial use case. termination | cleanup`; `artifactId` and `digest` use the aliases above, `summary` is `ShortText`, and at least one of those three optional members is present. + `complete` means every configured bounded evidence category was attempted + after the direct provider process settled; it never means every descendant + was enumerated or quiescent. - `termination` is `{ status: "clean" | "failed" | "unknown", - reason?: ShortText }`; `cleanup` is - `{ workspace: "removed" | "retained" | "failed", reason?: ShortText }`. + reason?: ShortText }` and reports the direct provider/process-group + observation only. + - `cleanup` is + `{ workspace: "shared" | "removed" | "retained" | "failed", + reason?: ShortText }`. `shared` means a read-only Task removed its private + runtime state while retaining a reusable cached base; `removed` means every + Task-owned runtime, writable view, and non-reusable base was removed. - optional `failure` is `{ code, message, retryable, cause }`, where `code` is one stable code from the error table below, `cause` is one member of the closed cause union defined below that table, `message` is `ShortText`, and @@ -281,22 +332,33 @@ registry, or another profile configuration file for the initial use case. equivalent network controls as the authorization boundary. - R5. Idempotency and Task visibility are deployment-wide. Atomically and durably bind an invocation key to the canonical request, selected target, - source identity, optional result-schema digest, deadline, and effective - configuration digest before acknowledging Task creation. One transactional + source identity, effective logical working directory, effective workspace + access, optional result-schema digest, deadline, and effective configuration + digest before acknowledging Task creation. One transactional `createOrReplay` operation arbitrates competing requests. Identical replay returns the existing Task; a changed request conflicts. Status and terminal settlement are monotonic. The project-specific state root persists the - canonical workspace identity and holds an exclusive process lock. The root - and all state files must be current-user owned, use `0700`/`0600`-equivalent - permissions, be disjoint from project, profile, and invocation roots, and be - opened descriptor-relatively without following symlinks or accepting hard- - linked files. Startup verifies those invariants, store integrity, and - workspace identity; terminalizes interrupted Tasks failed; and never resumes - provider work. A durable commit fsyncs every changed file and affected - containing directory before acknowledgment. Store open, corruption, write, - transaction, rename, or fsync failure stops admission, aborts and contains - active work, prevents terminal success, and keeps the process alive with - poisoned readiness until the containment set is proven empty. + canonical workspace identity and holds an exclusive process lock. + + The Bun gateway privately owns one SQLite database through `bun:sqlite`. + Claims, Tasks, events, bounded Artifact bytes, the single execution lease, + internal outcome intent, acquisition-container identity, staging and + transient-base identity, direct provider process-group identity, and expiry + live in ordinary transactional tables. Enable foreign keys, use WAL where + supported, set `synchronous=FULL`, and acknowledge only committed + transactions. The current user owns the state root and database with + `0700`/`0600`-equivalent permissions; the root is disjoint from project, + staging, publication, profile, and provider-auth roots. No custom SQLite VFS + or native file primitive is introduced. Startup verifies the root, lock, + schema/integrity, and workspace identity; terminalizes interrupted Tasks + failed; removes recorded acquisition containers and orphan staging; and + attempts to terminate recorded provider process groups. It releases a stale + or termination-poisoned lease only after the container is gone, the recorded + process group is confirmed absent, and recorded Task-owned runtime, view, and + transient-base cleanup has completed. Otherwise readiness remains false and + admission stays stopped. Provider work is never resumed. + Store open, corruption, write, transaction, or synchronization failure stops + admission and prevents terminal success. **Workspace and target configuration** @@ -317,29 +379,63 @@ registry, or another profile configuration file for the initial use case. one `(profile, client)` pair. Built-in IDs are reserved under the same portable collision key; colliding enablement is a configuration error. Initially only Codex and Pi profile clients are executable. -- R8. A request selects a declared target, may set the bounded - `deadlineSeconds`, and may provide one bounded result schema. The gateway owns - one durable execution lease covering acquisition through final evidence - collection. Admission claims that lease transactionally before launching any - helper child; at most one Task may hold it. A second otherwise-valid request - is accepted as a Task and settles failed with - `execution_capacity_unavailable`. It may transiently allocate one empty, - start-gated containment set, but never releases the gate, starts acquisition, - or executes a child and must destroy that set after the failure settlement. - Lease identity is stored with the Task, survives restart, and is released only - by the final settlement transaction or startup reconciliation after the - recorded containment set is proven empty. - - The overall deadline covers acquisition, publication, typed preparation, - provider execution, and evidence collection. Acquisition receives +- R8. A request selects a declared target, may select one logical + `workingDirectory`, may select `workspaceAccess`, may set the bounded + `deadlineSeconds`, and may provide one bounded result schema. `workspaceAccess` + defaults to `readWrite`; it is never inferred from prompt text. + + A read-only Task resolves its cwd directly inside its validated base. An exact + immutable request may share a reusable cached base; a mutable branch or tag + request owns a non-reusable base for the Task lifetime. The Task receives a + private `//runtime` for temporary, home, + provider-state, and evidence files. Provider environments disable optional + Git locks, and adapters request native read-only policy when available. The + gateway does not inspect prompts or add a per-Task mount, chmod pass, or full- + tree verification. The consumer remains responsible for assigning work that + does not require project mutation. + + A read-write Task receives a unique writable view at + `//workspace`. The configured host materializer + prefers a filesystem block clone, falls back to rootless OverlayFS on + supported Linux hosts, and supports an explicit ordinary-copy backend for + portability. Writable files never use hard links. Normal settlement removes + the writable view and any non-reusable base after evidence collection; non- + settling execution retains them with the poisoned lease until verified + reconciliation. + + `{ kind: "workspaceRoot" }` selects the effective base or Task view. A + repository selector maps its declared name through the compiled catalog, + appends only the validated `RelativeDirectory`, resolves links without escape, + and must name an existing directory beneath that repository. Absolute paths, + configured destinations, undeclared repositories, non-directories, and + escaping resolutions fail before provider start. Gateway-generated requests, + structured Task/Artifact metadata, and operational logs never contain the + physical path. Opaque terminal output, native evidence, and produced-Artifact + payloads are not sanitized and may contain it. + + The gateway owns one durable execution lease covering base acquisition or + lookup through final evidence collection. Admission claims that lease + transactionally before starting an acquisition container or provider process; + at most one Task may hold it. A second otherwise-valid request is accepted as + a Task and settles failed with `execution_capacity_unavailable` without + creating a container or process. Lease identity is stored with the Task and + survives gateway restart. Normal final settlement releases it; a provider- + termination failure retains it until verified startup reconciliation confirms + the recorded process group is absent. + + The overall deadline covers cache lookup, Docker acquisition on miss, + publication, optional materialization, typed preparation, bare-metal provider + execution, and evidence collection. Acquisition receives `min(900 seconds, remaining overall deadline)`; exceeding that sub-budget - fails before provider execution. Overall expiry initiates abort and bounded - forced termination. Cleanup then uses its own fixed bounded budget and the - R16 fail-closed quiescence rule. A request cannot provide or override backend, - executable path, command, argv, environment, profile settings, plugins, MCP - servers, repository URLs, destination paths, credential provider, setup - behavior, or permission policy. Readiness rejects missing, partial, drifted, - unsupported, or declaration-missing gateway-enabled profiles. + removes the acquisition container and fails before provider execution. + Overall expiry initiates adapter abort and Linux process-group escalation. + Cleanup then uses its own fixed bounded budget. A request cannot provide or + override backend, executable path, command, argv, environment, provider home, + profile settings, plugins, MCP servers, repository URLs, destination paths, + credential provider, setup behavior, provider permission policy, materializer, + cache key, Docker options, image reference, or mounts. Readiness rejects + missing, partial, drifted, unsupported, or declaration-missing gateway-enabled + profiles. **Workspace acquisition** @@ -356,7 +452,10 @@ registry, or another profile configuration file for the initial use case. repository-controlled secondary fetch/exec features, verify checkout identities, and reject path collisions or escapes. - R11. Snapshot mode maps `snapshot` to a declared OCI repository and constructs - `@` server-side. V1 accepts only + `@` server-side. The same digest-pull contract must + interoperate with Docker Hub, GHCR, JFrog Artifactory/JFrog Container + Registry, and compatible private OCI Distribution registries; registry choice + does not alter the accepted snapshot format. V1 accepts only `application/vnd.oci.image.manifest.v1+json` with `schemaVersion: 2` directly at the requested digest. Reject image indexes, nested indexes, descriptor `urls` or embedded `data`, non-distributable layers, unknown media types, and @@ -368,24 +467,38 @@ registry, or another profile configuration file for the initial use case. while streaming, before decoding. Apply layers base-to-top with OCI whiteout and opaque-whiteout semantics. - The workspace manifest must contain every compiled project repository exactly - once at its operator-declared destination; reject missing, extra, renamed, - misplaced, or duplicate repositories and undeclared generated content. Apply - these fixed v1 ceilings across all processed layers, including overwritten or - whiteouted content: 4 MiB manifest, 4 MiB config, 2 GiB total compressed - layer bytes, 8 GiB total expanded bytes, 250,000 entries, 1 GiB per regular - file, 4096 UTF-8 bytes and 128 components per path, and 1 MiB per PAX or other - extended header. Abort before crossing a limit. Validate paths, collisions, - file types, modes, links, and manifest completeness in staging before atomic - publication. Reject absolute or traversing paths, devices, sockets, sparse - files, escaping links, credentials in redirect URLs, unapproved cross-origin - redirects, and external layers. Cross-origin redirects are limited to - layer-blob `GET`/`HEAD` requests and exact operator-declared - `layerRedirectHosts`; token, manifest, and config requests remain same-origin. - Private or otherwise non-global destinations are permitted only when the exact - host is the source's declared repository host or a declared layer-redirect - host, with per-hop rebinding checks. The common workspace manifest - distinguishes independently verified Git facts from snapshot-attested facts. + The wire-visible workspace manifest contains every compiled project + repository exactly once by logical name and omits destination paths. After + applying layers, the gateway uses the compiled operator catalog to verify that + each listed repository exists at its configured destination and that no + repository is missing, extra, renamed, misplaced, duplicated, or accompanied + by undeclared generated content. Apply these fixed v1 ceilings across all + processed layers, including overwritten or whiteouted content: 4 MiB manifest, + 4 MiB config, 8 GiB total compressed layer bytes, 32 GiB total expanded bytes, + 500,000 entries, 4 GiB per regular file, 4096 UTF-8 bytes and 128 components + per path, and 1 MiB per PAX or other extended header. Abort before crossing a + limit. Validate paths, collisions, file types, modes, links, and the compiled + filesystem layout in staging before atomic publication. Reject absolute or + traversing paths, devices, sockets, sparse files, escaping links, credentials + in redirect URLs, unapproved cross-origin redirects, and external layers. + Cross-origin redirects are limited to layer-blob `GET`/`HEAD` requests and + exact operator-declared `layerRedirectHosts`; token, manifest, and config + requests remain same-origin. Private or otherwise non-global destinations are + permitted only when the exact host is the source's declared repository host + or a declared layer-redirect host, with per-hop rebinding checks. The common + path-free workspace manifest distinguishes independently verified Git facts + from snapshot-attested facts; compiled destinations remain private validation + inputs. + + After host validation, atomically promote staging to a validated base. An OCI + identity is reusable under a gateway-owned cache key containing its manifest + and workspace-manifest digests. A repository identity is reusable only when + every effective revision is a full commit ID. Its cache key also binds the + acquisition-contract version, compiled catalog/layout digest, and every + commit. Branch and tag requests instead receive a non-reusable Task-owned base + and never populate or reuse a cache entry. A valid cache hit starts no + acquisition container and resolves no source credential. Active Tasks pin a + reusable base; bounded eviction removes only unpinned cache entries. **Credential selection and containment** @@ -395,119 +508,148 @@ registry, or another profile configuration file for the initial use case. independently proven through the configured GitHub CLI identity; an uncorroborated 404, 401, 403, 429, timeout, or 5xx is `unknown`. An explicit installation ID is eligible only after positive repository-coverage - verification. For `eligible`, call `@octokit/auth-app` with `refresh: true` - and the exact repository selection to mint a new read-only installation token - for every acquisition. Validate its repository selection, permissions, - creation time, and expiry, and require remaining lifetime greater than the R8 - acquisition sub-budget plus a 60-second clock-skew margin. + verification. For `eligible`, the host gateway creates the App JWT from the + configured private key, discovers and verifies installation coverage, and + mints a new repository-scoped read-only installation token for every cache- + miss acquisition. Credentials are never cached. Validate the token's + repository selection, + permissions, creation time, and expiry, and require remaining lifetime greater + than the R8 acquisition sub-budget plus a 60-second clock-skew margin. Only + the resulting installation token enters the acquisition container; the App + private key remains on the host. Use the configured GitHub CLI account only when the App is absent or applicability is positively `ineligible`. An `unknown` result or any selected-App configuration, authentication, minting, permission, repository, rate-limit, or service failure terminates acquisition without `gh` fallback. - Run `gh auth token --hostname github.com --user ` with ambient token - variables removed. Deliver either token only through an invocation-scoped Git - credential helper. Revoke an App token after acquisition and fail before - provider execution if revocation cannot be confirmed; destroy all local token - material before typed preparation. OCI credentials likewise exist only during - snapshot acquisition. + Resolve `gh auth token --hostname github.com --user ` on the host + with ambient token variables removed, then inject only the selected + invocation-scoped source credential into the acquisition container. Revoke an + App token after acquisition and fail before provider execution if revocation + cannot be confirmed. OCI acquisition accepts anonymous pulls or exact- + registry credentials from the strict Docker-auth/helper boundary and supports + same-origin Basic and Distribution Bearer challenges, the documented Docker + Hub token service, and an operator-supplied exact-host CA-bundle map. The + acquisition container receives source-only credentials and trust material; + they are destroyed with the container before provider preparation. **Execution, evidence, and cleanup** -- R13. Keep one closed `codex | pi` backend registry and one behavior-focused - interface covering availability, capabilities, invocation, progress, - deterministic permission handling, abort, terminal output, optional structured - result, usage, bounded native evidence, and disposal. Profile targets resolve +- R13. Keep one closed `codex | pi` backend registry behind a narrow + AllAgents-owned TypeScript interface covering availability, capabilities, + invocation, progress, deterministic permission handling, abort, direct + process settlement, terminal output, optional structured result, usage, + bounded native evidence, and disposal. The gateway owns contract + normalization rather than adopting AI SDK Harnesses. Profile targets resolve adapter-owned configuration directly; never execute generated launchers, - discover executables as targets from `PATH`, scrape a TUI, or append public - input to argv. -- R14. Codex uses pinned `@openai/codex-sdk`, one fresh thread per Task, - `AbortSignal`, streamed events, and an operator-selected Codex auth-file - handle. It passes native `outputSchema` only when the public schema has an - object root, every object's `required` set equals its property set, nesting is - at most 10 levels, and every keyword is supported by the pinned model/API. - Other valid public schemas use explicit JSON prompt guidance plus the common - gateway-side validator without a native schema. Pi uses strict RPC, - invocation-owned configuration, an operator-selected Pi auth-file handle, and - one restricted policy extension; repository extensions and unrestricted - built-ins do not auto-load. The gateway copies only the selected adapter's - required auth material into an invocation-private, read-only control-process - view and removes it during cleanup. -- R15. Acquire into a private staging root and atomically publish the invocation - workspace. Run only adapter-owned typed preparation that projects validated - project/profile settings, plugins, and MCP declarations through existing - deterministic transforms; never execute project or user `setup` entries or - other configured shell commands. - - Enforce distinct process views: - - the provider control process receives only its invocation workspace, - minimum non-secret profile configuration, and adapter auth channel; - - each MCP child receives only its own resolved secret references; and - - model-invoked shell/tools receive the workspace and no provider or MCP - credentials. - - The pinned backend must expose one non-bypassable synchronous spawn hook for - every MCP and model-tool process. The hook delegates execution to the security - helper, which enters the role-specific mount and network namespaces, replaces - the environment, closes every non-allowlisted descriptor, and only then - executes untrusted code. A backend that can spawn any tool without this hook - is not a v1 target and fails readiness; conformance fixtures alone cannot waive - that requirement. All views exclude gateway state, operator home, App keys, - GitHub/OCI stores, source helpers, unrelated adapter credentials, and the - parent environment. - - Invocation network namespaces cannot route to host loopback, any gateway bind - or advertised address, operator management networks, or ingress proxies. - Provider and MCP egress is default-deny except for role-specific destinations - compiled from adapter and MCP configuration; every resolved address is checked - at connection time, and gateway/host-management destinations remain denied - even when a hostname resolves to them. Model tools receive no network unless - the adapter's explicit policy grants similarly constrained egress. Fail target - readiness unless all filesystem, credential, descriptor, and network - separations are enforceable. - - Acquisition credentials and mounts are absent first. Capture bounded provider - events while the process is live. After the provider reports terminal, abort - and terminate its complete containment set and prove it empty before reading - Git state, hashing or copying files, or describing produced Artifacts as - verified. Treat the mutated workspace as untrusted: use descriptor-relative - no-follow reads; revalidate identity and size after open; reject hard links, - special/sparse files, path replacement, out-of-root targets, and `.git` - gitdir/core.worktree/alternates escapes; and run Git inspection with hermetic - configuration that disables hooks, filters, drivers, fsmonitor, pagers, - helpers, and external commands. If quiescence cannot be proven, retain only - truthful partial process evidence; do not publish filesystem evidence or - produced Artifacts as verified. -- R16. Supervise the complete acquisition/provider descendant set inside an - invocation-owned OS containment primitive whose membership children cannot - escape. Allocate its stable identifier and empty set first, then commit that - identity with the Task and execution lease before the helper may release its - start gate or execute any child. A failed commit destroys the still-empty set. - Startup enumerates the entire project-owned containment namespace, reconciles - both recorded and unknown identifiers, and refuses readiness while any - unknown or nonempty set remains. - - One durable compare-and-set arbitrates provider terminal outcome, caller + discover arbitrary executables as targets from `PATH`, scrape a TUI, append + public input to argv, or download a provider runtime per request. A configured + globally installed binary override is eligible only after an exact version + and protocol compatibility probe. +- R14. Codex uses a pinned `@openai/codex-sdk` directly from the Bun gateway. + Codex app-server is allowed only if U0 demonstrates a required capability + absent from that pinned SDK; the reason and tested protocol version must then + be recorded. Each Task receives one fresh SDK execution context, streamed + events, native cancellation, and an explicitly constructed child environment. + When API credentials are absent, preserve the existing host `CODEX_HOME` and + ChatGPT login in place; do not copy, mount, parse, or import OAuth files. + Pass native `outputSchema` only when the public schema has an object root, + every object's `required` set equals its property set, nesting is at most 10 + levels, and every keyword is supported by the pinned SDK/model. Other valid + public schemas use explicit JSON guidance plus the common gateway-side + validator. + + Pi uses a pinned supported package/RPC surface, invocation-owned + configuration, one restricted policy extension, and the existing host Pi + authentication location. Repository extensions and unrestricted built-ins do + not auto-load. Do not copy, mount, parse, or import Pi authentication files. + Both adapters preserve only required host identity, authentication paths, + executable lookup, locale, certificate, and proxy settings in an explicit + environment allowlist. This reduces accidental environment leakage; it is + not a secret-isolation guarantee because model-invoked tools run with the same + CI-job authority. +- R15. Start a fresh Docker container only when a request has no reusable + validated base, including a cache miss or a non-reusable branch/tag request. + Probe Docker and the exact digest-pinned `apps/acquirer` image at that point. + A repository-mode probe failure is `source_git_unavailable`; a snapshot-mode + probe failure is `source_snapshot_unavailable`. The gateway creates private + staging and starts the image with that directory as its only writable bind + mount. The container receives the canonical acquisition request, compiled + catalog, strict network/size/archive policy, source-only GitHub or OCI + credentials, and only required exact-host CA material. It receives no GitHub + App private key, host home, provider home, Docker socket, gateway database, + published base, unrelated credential, Codex, Pi, or other coding harness. It + never downloads a coding harness. Docker network access is limited to source + endpoints required by the selected Git or OCI mode. + + The acquirer writes content beneath staging and emits one typed manifest + through the bind mount, then exits. The host gateway waits for exit, removes + the container, destroys source credentials, validates the manifest and tree + against the compiled catalog and fixed limits, and atomically promotes staging + to a reusable cache entry or non-reusable Task-owned base. Every cancellation, + deadline, validation failure, or other non-publication path removes staging + idempotently; cleanup uncertainty fails `source_cleanup_failed` and stops + admission. Source-mode failure never falls through. + + A read-only Task uses the base directly plus Task-private runtime state. + Adapter-owned preparation for that mode must keep project files unchanged and + place invocation configuration outside the base. A read-write Task first + receives a unique block-cloned, overlaid, or copied view; typed preparation + may then project validated project/profile settings, plugins, and MCP + declarations into that view. Project or user `setup` entries and other + configured shell commands never run automatically. + + Codex and Pi execute as direct host processes on the same trusted Linux CI + runner as the gateway. The adapter receives the access-appropriate resolved + cwd selected by the logical `workingDirectory`, Task-private runtime paths, + and the effective access mode; it never receives a caller-supplied physical + path or materializer choice. Provider execution never reuses the acquisition + container and never creates a per-invocation provider container. The CI job, + VM, or deployment container is the isolation boundary. AllAgents does not + claim containment of hostile repository code, network access by model tools, + or provider/MCP/operator secrets from those tools. Capture bounded provider + events while the direct provider process is live. Collect filesystem/Git + evidence only after that direct process settles and process-group termination + attempts finish; phrase the evidence as observed after direct-process + settlement, never as proof that every descendant is quiescent. Run Git + inspection with hermetic configuration that disables hooks, filters, drivers, + fsmonitor, pagers, helpers, optional locks, and external commands. +- R16. On trusted Linux runners, start each direct provider in a new process + group and persist its leader PID plus Linux process-start marker with the Task + and execution lease before recording provider execution as started. One + durable compare-and-set arbitrates provider terminal outcome, caller cancellation, overall deadline, and shutdown as an internal `outcomeIntent` - while the externally visible Task remains nonterminal. The winning intent - owns the stable result or failure code and drives one idempotent abort and - quiescence path. Only after quiescence, safe evidence collection, produced- - Artifact verification, and cleanup does one settlement transaction atomically - write terminal Task status, result/failure, bounded evidence, exactly one - integrity Artifact, produced Artifacts, termination outcome, lease release, - and cleanup outcome. - - Cancellation intent persists before native abort, followed by bounded forced - termination. If quiescence cannot be proven, settle once with - `execution_quiescence_unknown`, no verified filesystem evidence, and immutable - unknown/failed termination; reject admission, keep readiness false, and leave - the process alive to continue reaping. Later recovery changes only internal - recovery/readiness state, never the settled Task. Print the stable containment - identifier and platform recovery command. Startup proves every interrupted set - empty before it may quarantine stale roots or advertise readiness. Graceful - shutdown stops admission atomically, commits shutdown intent, drains or aborts - active work within a bounded grace period, follows the same settlement path, - and only then exits. + while the external Task remains nonterminal. The winning intent owns the + stable result or failure code and drives one idempotent abort path: request + graceful adapter abort, wait the configured grace period, send `SIGTERM` to + the process group, then `SIGKILL` after the forced-termination period. + + After the direct provider process has settled and bounded evidence collection + finishes, cleanup removes a read-only Task's private runtime state, unmounts + and removes a read-write Task's writable view, and removes any non-reusable + base. One transaction then writes terminal Task status, result/failure, + bounded evidence, exactly one integrity Artifact, produced Artifacts, observed + termination, cleanup outcome, and lease release. Any Task-owned cleanup + failure settles `workspace_cleanup_failed` with + `cleanup.workspace: "failed"` and retains its internal cleanup record for + startup or operator repair; admission stops when an active mount or uncertain + writable view remains. If the direct process does not settle after `SIGKILL`, + one transaction instead writes a failed Task with + `execution_termination_failed`, live provider evidence, and observed + termination, but no filesystem, Git, or produced-Artifact evidence. It retains + the Task runtime, any writable view or non-reusable base, and the lease; makes + readiness false; and stops admission. Startup may release that poisoned lease + only after the recorded process group is confirmed absent following runner + teardown and retained Task-owned state is reconciled; otherwise it remains + unready. + + Repeated cancellation while intent is pending does not re-signal work. + Startup never resumes a session. Gateway shutdown stops admission, commits + shutdown intent, performs the same escalation and settlement rules, and + exits. CI runner teardown is the final orphan boundary. AllAgents does not use + cgroups, pidfds, namespaces, nftables, `openat2`, a native platform layer, or + non-bypassable spawn mediation, and does not claim complete descendant + enumeration or hostile-code containment. **Scope and configuration** @@ -515,33 +657,40 @@ registry, or another profile configuration file for the initial use case. repetitions, experiment scheduling, or automatic Task retry. - R18. Do not add `gateway.yaml` or `worker.yaml`. Process configuration uses the exact CLI flags and environment variables in the Configuration Contract - for listener, advertised interface URL, workspace, state/retention, GitHub, - OCI, and Codex/Pi auth-file handles. The listener also exposes unauthenticated - metadata-only `/healthz` and `/readyz` endpoints outside A2A: liveness returns - 200 while the process can serve; readiness returns 200 only while new - admission is safe and otherwise 503. They reveal no targets, sources, paths, - or failure details and do not require A2A headers. Gateway code never copies - acquisition or provider credential values into generated workspace files, - requests, logs, Task/Artifact metadata, retained workspaces, or model-tool - environments. This is not a redaction guarantee for opaque prompts, provider - output, structured results, native evidence, or produced-Artifact payloads. + for listener, advertised interface URL, workspace, state/retention, + immutable-base cache, workspace materializer, acquisition image and Docker + access, GitHub/OCI source credentials, provider executable overrides, provider + home/auth paths, and process-group timeouts. The listener also exposes + unauthenticated metadata-only `/healthz` and `/readyz` endpoints outside A2A: + liveness returns 200 while the process can serve; readiness returns 200 only + while new admission is safe and otherwise 503. They reveal no targets, + sources, paths, or failure details and do not require A2A headers. Gateway code + never copies acquisition credential values into generated workspace files, + requests, logs, Task/Artifact metadata, retained Task views, cache entries, or + provider environments. This is not a redaction or isolation guarantee for + opaque prompts, provider/tool output, structured results, inherited host + authentication, native evidence, or produced-Artifact payloads. - R19. Document AI Evals consumption through a Promptfoo custom JavaScript/TypeScript provider implementing Promptfoo's `ApiProvider`. `constructor(options: ProviderOptions)` requires and retains a nonempty `options.id`, validates `options.config`, and `id()` returns that ID. Static - config contains the - gateway endpoint, target ID, and exactly one closed source mode: repository - mode materializes the complete configured repository set and carries only an - optional revision map keyed by declared repository name; snapshot mode carries - one declared snapshot name with OCI and workspace-manifest digests. + config contains the gateway endpoint, target ID, optional default logical + `workingDirectory`, optional `workspaceAccess` defaulting to `readWrite`, and + exactly one closed source mode: repository mode materializes the complete + configured repository set and carries only an optional revision map keyed by + declared repository name; snapshot mode carries one declared snapshot name + with OCI and workspace-manifest digests. `callApi(prompt, context?, options?)` may apply the exact - `context?.vars?.allagentsSource` leaf overrides defined below; missing context - means no override. Dynamic repository revisions must be full lowercase - 40-hex commit IDs; dynamic snapshot values must be full lowercase `sha256:` - digests. Source kind, snapshot name, and repository origins never vary per - test. Unknown members, revision names absent from static config, URLs, - destinations, tags, credentials, commands, and permission policy fail before - submission. + `context?.vars?.allagentsSource` leaf overrides, may replace the default + selector through `context?.vars?.allagentsWorkingDirectory`, and may replace + access through `context?.vars?.allagentsWorkspaceAccess`; missing context + retains static values. Dynamic source values remain limited as defined below. + The working-directory variable is exactly `{ kind: "workspaceRoot" }` or + `{ kind: "repository", repository: ConfigName, + path?: RelativeDirectory }`; access is exactly `readOnly` or `readWrite`. + Unknown members, invalid relative paths, URLs, physical or configured + destination paths, credentials, commands, Docker options, materializer + choices, and provider permission policy fail before provider execution. The provider sends `SendMessage` with `configuration.returnImmediately: true`, captures the accepted Task ID, and calls `SubscribeToTask`; a terminal-before- @@ -560,78 +709,112 @@ registry, or another profile configuration file for the initial use case. - F1. **Start and advertise** 1. Resolve cwd or `--workspace`, user workspace, project-specific state root, - retention limits, listen address, advertised interface URL, source - credentials, and provider auth handles. - 2. Validate state-root ownership, permissions, links, disjointness, workspace - identity, compiled repository catalog, snapshots, target namespace, backend - availability, profile state, Linux containment/helper availability, - provider/MCP/tool mount, descriptor, and network views, and credential - handles. - 3. Enumerate the entire project-owned containment namespace. Reconcile - recorded and unknown identifiers and prove every set empty before - quarantining filesystem roots or releasing a retained execution lease. + disjoint immutable-base cache and invocation roots, cache/task retention, + workspace materializer, listener, advertised URL, digest-pinned acquisition + image, Docker endpoint, source credentials, provider homes, and configured + provider executable overrides. + 2. Validate the SQLite state root, cache/invocation roots, workspace identity, + materializer policy, and static acquisition-image reference; compile + repository, snapshot, and target catalogs; verify Codex SDK and Pi RPC/ + package compatibility; and check any global binary override exactly. Do not + contact Docker or the acquisition registry at startup. + 3. Reconcile interrupted Tasks by removing any recorded acquisition container + and orphan staging, terminating any recorded Linux provider process group, + and marking the Task failed without resuming it. Release the durable lease + only after container removal, confirmed process-group absence, and cleanup + of recorded Task-owned runtime, view, and non-reusable base; otherwise keep + readiness false and the lease poisoned. 4. Bind the requested address, including `0.0.0.0` when explicit; serve metadata-only health/readiness probes; and publish one Agent Card whose absolute interface URL, required extension, and target allowlist match the validated configuration. -- F2. **Acquire repositories and execute** +- F2. **Acquire or reuse repositories and execute** 1. Negotiate A2A version and the required extension, then validate the strict - request, one text Part, target, repository-name/revision map, result schema, - deadline, and deployment-wide idempotency claim. - 2. Ask the helper to allocate a stable empty containment set behind a start - gate. In one transaction, create or replay the claim and Task, acquire the - execution lease, and bind the containment identifier before acknowledgment. - Capacity failure settles the Task with `execution_capacity_unavailable`, - then destroys the empty set without releasing the gate or launching a - child. Commit failure likewise destroys the empty set. - 3. Release the start gate. For each declared repository, classify App - applicability, select App or `gh` only by eligibility, resolve the revision, - fetch hermetically, verify the commit, revoke an App token, and remove every - acquisition credential. - 4. Publish the complete workspace, run typed preparation, invoke the isolated - adapter, and validate any structured result while capturing live events. - Terminate and prove the containment set empty before safe filesystem/Git - evidence reads and produced-Artifact verification. Atomically settle the - terminal Task, evidence, Artifacts, cleanup, and lease release. - -- F3. **Acquire an OCI snapshot and execute** - 1. Perform the same version/extension validation, gated empty-containment - allocation, and atomic claim+Task+lease+containment commit as F2. - 2. Resolve the named snapshot repository and digest-pinned reference. - Authenticate if required; pull and verify the direct image manifest, - workspace-manifest config blob, and distributable layers; apply changesets - in order; enforce all limits; and validate the exact project catalog. - 3. Remove registry credentials, publish atomically, run typed preparation, - invoke the isolated adapter, and capture live events. Terminate and prove - quiescence before safe filesystem evidence and verified produced Artifacts, - then perform the same atomic settlement and lease release as F2. + request, one text Part, target, repository-name/revision map, logical + working-directory selector, workspace access, result schema, deadline, and + deployment-wide idempotency claim. + 2. In one SQLite transaction, create or replay the claim and Task and acquire + the execution lease before starting work. Capacity failure settles the Task + with `execution_capacity_unavailable` and launches neither Docker nor a + provider. + 3. When every effective revision is a full commit, derive the immutable-base + key and pin a matching validated cache entry. On a miss or for mutable + branch/tag revisions, classify App applicability and mint a fresh + installation token or resolve the configured `gh` token only according to + eligibility. Probe Docker and the exact digest-pinned acquisition image, + then start it with only private staging, compiled request/policy, and the + selected token. The container fetches hermetically, verifies full commits, + writes the typed manifest, exits, and is removed. + 4. On acquisition, revoke any App token, destroy source credentials, validate + the manifest/staging on the host, and atomically promote it to either a + reusable cache entry or a non-reusable Task-owned base. A cache hit performs + none of those acquisition, Docker, or credential operations. Every + non-publication path removes staging. + 5. For `readOnly`, resolve the logical cwd directly in the base and create only + Task-private runtime state. For `readWrite`, create the unique writable view + through the selected materializer, then resolve cwd in that view. Run + access-appropriate typed preparation and start the adapter there as a direct + host process group with explicit environment and existing host auth. + Validate structured results while capturing bounded live events. + 6. After the direct provider process settles and cancellation escalation + finishes, collect bounded truthful evidence; remove private runtime, any + writable view, and any non-reusable base; unpin a reusable base; and + atomically settle the Task, Artifacts, observed termination, cleanup, and + lease. If the process does not settle after `SIGKILL`, publish + `execution_termination_failed` without filesystem/Git evidence, retain + Task-owned state and the lease for runner teardown, make readiness false, + and stop admission until startup reconciliation confirms the process group + absent and cleans retained state. + +- F3. **Acquire or reuse an OCI snapshot and execute** + 1. Perform the same version/extension validation and atomic + claim+Task+lease transaction as F2. + 2. Pin a cache entry matching the named snapshot, manifest digest, workspace- + manifest digest, catalog/layout digest, and acquisition-contract version. + On a miss, probe Docker and the exact digest-pinned acquisition image, then + start it with the digest-pinned reference, staging mount, exact-host + registry credentials/CA material, and frozen network/archive policy. Pull + and verify the direct image manifest, workspace-manifest config, and + distributable layers; apply changesets in order; enforce all limits; and + emit the typed manifest. + 3. On a miss, remove the container and registry material, validate and publish + the immutable base on the host, or remove staging on every non-publication + path. Then select the read-only shared base or read-write Task view and + settle through the same bare-metal and cleanup path as F2. Provider + execution never occurs in the acquisition container. - F4. **Cancel** 1. Atomically persist cancellation intent if the Task remains cancelable. - 2. Abort acquisition or provider work, escalate within the bounded termination - budget, prove containment quiescence, preserve truthful partial evidence, - clean up, and settle canceled. + 2. For acquisition, stop and remove the Docker container, source material, and + unpublished staging. For provider work, request graceful adapter abort, + then escalate to process-group `SIGTERM` and `SIGKILL` within bounded + periods. Preserve only observed, bounded evidence; clean up and settle + canceled or failed according to the durable winning intent. 3. Repeated cancellation while intent is pending does not re-signal work. Cancellation after any terminal state returns A2A `TaskNotCancelableError`. - F5. **Shut down** 1. Stop new admission before signaling active work. - 2. Persist shutdown intent, abort and escalate, drain live process evidence, - prove quiescence, and settle the accepted Task once. - 3. Exit only after durable settlement and empty containment. If proof fails, - remain alive, not ready, and continue reaping while printing the stable - containment identifier and platform recovery command. + 2. Persist shutdown intent, remove active acquisition Docker work or escalate + the direct provider process group, collect evidence only after the direct + provider settles, and settle the accepted Task once. + 3. Exit after the bounded settlement and cleanup path. Document that CI runner + teardown is the final orphan boundary and that gateway shutdown does not + prove every model-tool descendant is gone. - F6. **Invoke from Promptfoo** 1. Promptfoo constructs the AI Evals-owned TypeScript provider with `ProviderOptions`; the provider retains the ID and validates - `options.config` containing the private-network endpoint, target, and one - closed source-mode object. + `options.config` containing the private-network endpoint, target, optional + default logical working directory, optional default workspace access, and + one closed source-mode object. 2. `callApi(prompt, context?, options?)` applies only valid - `context?.vars?.allagentsSource` leaf overrides, creates and retains one - high-entropy invocation key, and sends one A2A Message with + `context?.vars?.allagentsSource` leaves and optional strict + `context?.vars?.allagentsWorkingDirectory` and + `context?.vars?.allagentsWorkspaceAccess` replacements, creates and retains + one high-entropy invocation key, and sends one A2A Message with `configuration.returnImmediately: true`. 3. After receiving the Task ID, subscribe to terminal updates. Resolve a terminal-before-subscribe or disconnected-stream race through `GetTask` @@ -662,12 +845,14 @@ registry, or another profile configuration file for the initial use case. executed. - AE5. Repository mode accepts declared names and revision overrides, rejects an undeclared name or URL override, and records the resolved full commits. -- AE6. An applicable GitHub App bypasses its token cache, mints a new - repository-scoped read-only token with adequate lifetime, validates the token, - and revokes it after acquisition. A corroborated existing repository with no - applicable installation uses the configured `gh` account. An uncorroborated - 404, unknown applicability, auth, mint, validation, or revocation failure does - not fall through to `gh` or start the provider. +- AE6. On a base-acquisition miss, including a mutable branch/tag request, an + applicable GitHub App bypasses its token cache, mints a repository-scoped + read-only token with adequate lifetime, validates and revokes it after + acquisition. A cache hit resolves no source credential. + A corroborated existing repository with no applicable installation uses the + configured `gh` account. An uncorroborated 404, unknown applicability, auth, + mint, validation, or revocation failure does not fall through to `gh` or start + the provider. - AE7. Snapshot mode accepts a direct image manifest with matching manifest, config/workspace, and layer digests; applies gzip/zstd layers and whiteouts in order; and enforces every fixed limit. Same-origin metadata redirects work; @@ -677,22 +862,38 @@ registry, or another profile configuration file for the initial use case. foreign layers, digest/size mismatch, malformed whiteouts, undeclared repositories, redirect loops/rebinding, non-global destinations not declared for that source, and unapproved origins fail. -- AE8. Repository and snapshot modes produce the same workspace-manifest shape - and exact compiled repository set/layout. OCI-contained commit identities are - snapshot-attested unless independently verified; source identity includes - completeness and ordered layer digests without origins. +- AE8. Repository and snapshot modes produce the same path-free wire-visible + workspace-manifest shape and logical repository set. The gateway separately + validates the acquired base against the exact compiled private destinations. + OCI-contained commit identities are snapshot-attested unless independently + verified; source identity includes completeness and ordered layer digests + without origins. One hundred Tasks using the same immutable identity perform + one full acquisition while the entry remains cached and pinned correctly. + A branch or tag request acquires a non-reusable Task-owned base, never enters + the reusable cache, and removes that base during settlement or reconciliation. - AE9. Identical invocation-key replay, including after a lost response, returns - the original Task. Reusing the key with a changed target, source, prompt, or - result schema conflicts; separate high-entropy keys create separate Tasks. -- AE10. Cancellation during Git, OCI pull, Codex, or Pi terminates the complete - process set and records cleanup. Unproved quiescence poisons readiness; the - gateway stays alive, rejects admission, and continues reaping until empty. -- AE11. Kill fixtures before and after empty-containment creation, durable - Task/lease/containment binding, child clone, start-gate release, and response - acknowledgment leave no unrecorded live set. Restart enumerates the full - project-owned namespace, refuses unknown/nonempty sets, turns interrupted - Tasks into one terminal failure, never resumes a provider session, and keeps - terminal Tasks and embedded Artifacts retrievable until expiry. + the original Task. Reusing the key with changed target, source, prompt, logical + working directory, workspace access, or result schema conflicts. Separate + read-only Tasks may share one physical base and cwd while keeping private + runtime state. Separate read-write Tasks receive independent writable views + even when their logical working-directory selectors are equal. +- AE10. Cancellation during a Git or OCI base acquisition stops and removes the + acquisition container and unpublished staging. Cancellation during Codex or + Pi requests graceful abort, then sends process-group `SIGTERM` and `SIGKILL` + on schedule. + The Task records observed termination and cleanup without claiming complete + descendant quiescence. If the direct process does not settle, the gateway + retains the Task's runtime and any writable view plus the lease, omits + filesystem/Git evidence, stops admission, and remains unready until post- + teardown startup reconciliation confirms the group absent. +- AE11. Kill fixtures before and after durable Task/lease creation, acquisition- + container start, provider process-group recording, and response + acknowledgment leave one recoverable SQLite truth. Restart removes the + recorded acquisition container, orphan staging, and safe Task-owned state; + best-effort terminates the recorded process group; and turns the interrupted + Task into one terminal failure without resuming a provider session. It retains + the lease and stays unready unless provider absence and required cleanup are + confirmed. Terminal Tasks and Artifacts remain until expiry. - AE12. A valid structured result survives later check or evidence failure as a valid result with an overall failed Task; invalid or absent results are never published as valid. @@ -701,56 +902,63 @@ registry, or another profile configuration file for the initial use case. target. - AE14. Two gateways for different workspaces use distinct private state roots; a second process for the same root fails the exclusive lock. Wrong-owner, - permissive, linked, hard-linked, or overlapping roots fail startup. The real - helper VFS rejects database, WAL, SHM, journal, temporary-file, symlink, - hard-link, and rename-swap attacks. Process-kill fixtures at transaction, file - sync, directory sync, and response boundaries recover either the complete old - or new generation and never lose an acknowledged Task or publish false - success. + permissive, linked, or overlapping roots fail startup. Ordinary Bun SQLite + transactions with foreign keys and `synchronous=FULL` recover a committed + Task/claim/lease generation after process-kill fixtures and never acknowledge + an uncommitted Task or publish false success; no custom VFS is required. - AE15. Barrier-controlled provider-terminal, caller-cancel, deadline, and - shutdown races durably select one internal intent and one abort/quiescence - path during Git, OCI, preparation, Codex, Pi, or evidence. Subscribers observe - no terminal Task until one transaction writes status, integrity Artifact, - bounded evidence, result/failure, termination, cleanup, and lease release. - Later reaping changes only internal readiness/recovery state. + shutdown races durably select one internal intent and one abort path during + Docker acquisition, preparation, Codex, Pi, or evidence. Normal settlement + writes status, integrity Artifact, bounded evidence, result/failure, observed + termination, cleanup, and lease release in one transaction. The + `execution_termination_failed` exception writes the terminal failure without + filesystem/Git evidence and deliberately retains the poisoned lease. - AE16. Repeated cancel while cancellation is pending is idempotent; cancel after canceled, completed, failed, or rejected returns `TaskNotCancelableError`. -- AE17. A workspace containing `setup` shell entries never executes them through - gateway acquisition or startup. Real Codex/Pi child and grandchild tool paths - are helper-mediated: filesystem, environment, inherited descriptor, `/proc`, - and magic-link probes cannot read provider/MCP secrets, operator stores, or - gateway state. Agent Card, Task operations, host loopback, bind/advertised - addresses, ingress, and management-network probes fail from every invocation - role; only compiled role egress succeeds. -- AE18. Evidence is collected only after containment quiescence. An escaping - link, hard link, special file, sparse-file abuse, replaced inode, or `.git` - indirection is rejected and Git inspection runs without repository-controlled - execution. Unknown quiescence produces no verified filesystem Artifact. +- AE17. A workspace containing `setup` shell entries never executes them during + acquisition or startup. The acquisition image receives only staging, + source-only credentials, exact source network policy, and archive limits; it + receives no host home, Docker socket, gateway state, provider auth, Codex, Pi, + or coding harness. Codex and Pi run afterward as direct host processes with + explicit environments that preserve required host identity/auth paths and + omit unrelated ambient values. +- AE18. Evidence collection starts only after the direct provider process has + settled and process-group escalation has completed. Git inspection disables + repository-controlled execution, and the integrity Artifact distinguishes + observed direct-process termination and cleanup from full descendant + quiescence. Documentation explicitly states that AllAgents provides no + hostile-code or model-tool secret-isolation guarantee. - AE19. The 1001st unexpired retained Task is rejected with `retention_capacity_exhausted`; no retained Task is evicted before TTL. While one Task holds the execution lease, a barrier-controlled second request - settles `execution_capacity_unavailable` and launches no helper child; races - and restart never produce two lease holders. + settles `execution_capacity_unavailable` and launches no acquisition or + provider child; races and restart never produce two lease holders. - AE20. Official HTTP+JSON client fixtures send `A2A-Version: 1.0`, exercise required-extension activation and both `SendMessage` modes, preserve unrelated metadata, verify standard `google.rpc.Status` errors, and cover every `ListTasks` filter, cursor, order, response field, and artifact-inclusion rule. - A terminal Task contains one extension-marked integrity Artifact plus - referenced produced Artifacts using unified Parts. + A terminal Task contains one extension-marked integrity Artifact with its + effective logical working directory, workspace access, and referenced + produced Artifacts using unified Parts. - AE21. The AI Evals Promptfoo fixture has a top-level prompt and disables sharing, caching, result writes, and concurrency above one. It loads one repository-mode and one snapshot-mode provider, sends only closed logical - source data, retains one invocation key across ambiguous retries, and cancels - an accepted Task on abort. Both calls return scorable output, normalized token + source, working-directory, and workspace-access data, replaces cwd and access + per trial through `allagentsWorkingDirectory` and + `allagentsWorkspaceAccess`, retains one invocation key across ambiguous + retries, and cancels an accepted Task on abort. Two read-only trials for the + same immutable source share the validated base; two read-write trials receive + independent disposable views. Both return scorable output, normalized token usage, and Task/Artifact/logical-provenance metadata. Safe failure metadata includes code, retryability, and accepted Task ID. Calls with omitted context - work; unknown variables, mutable revisions, origins, destinations, or - undeclared names fail before submission. + work; unknown variables, invalid or escaping relative directories, physical + paths, mutable revisions, origins, destinations, materializer choices, or + undeclared names fail before provider execution. ### Success Criteria -- `allagents gateway serve` starts from a real workspace with no deployment YAML. +- `allagents-gateway serve` starts from a real workspace with no deployment YAML. - Explicit loopback, private-interface, and `0.0.0.0` listeners work with a distinct valid advertised interface URL; health/readiness reflect admission. - The official A2A client exercises version and extension negotiation, both send @@ -758,33 +966,49 @@ registry, or another profile configuration file for the initial use case. cancel, terminal cancel errors, Task-embedded Artifacts, standard HTTP+JSON errors, and expiry. - An AI Evals-style Promptfoo custom-provider fixture consumes secure-default - YAML for both source modes, propagates post-acceptance cancellation, and maps a + YAML for both source modes, selects a logical cwd and access mode per trial, + proves shared-base reuse for read-only trials and independent disposable views + for read-write trials, propagates post-acceptance cancellation, and maps a terminal Task to `ProviderResponse` without adding Promptfoo to the AllAgents runtime. -- Built-in Codex/Pi and gateway-enabled profile targets pass one conformance - suite, including reserved-ID collisions and Codex native-schema gating. -- Direct Git and OCI snapshot fixtures produce equivalent validated workspace - manifests and truthful complete provenance. -- GitHub App eligibility, 404 ambiguity, unknown failure, no-installation `gh` - fallback, fresh-token validation/revocation, OCI authentication and challenge - handling, and pre-provider credential teardown are proven end to end. -- No request can supply a command, executable, URL, destination, credential, - mutable OCI tag, backend override, or arbitrary environment value. -- State-store crash, deadline/cancellation/terminal/shutdown race, descendant - escape, unsafe evidence, and stale-root scenarios fail closed. -- The bundled CLI and packaged Linux helper pass a trusted-network smoke test - against project and user workspaces created under `/tmp/`. +- Built-in Codex/Pi and gateway-enabled profile targets pass one backend + conformance suite, including reserved-ID collisions, existing-host-auth + behavior, explicit environment construction, cancellation escalation, and + Codex native-schema gating. +- Direct Git and OCI snapshot acquisition in the exact digest-pinned image + produces equivalent typed manifests and truthful provenance; repeated + immutable requests reuse one validated base and the image is removed before + provider execution. GitHub App eligibility, 404 ambiguity, unknown failure, + no-installation `gh` fallback, base-acquisition token validation/revocation, OCI + authentication/challenge handling, staging validation, and pre-provider + credential teardown are proven end to end. +- No request can supply a command, executable, URL, physical cwd, configured + destination, credential, mutable OCI tag, backend or materializer override, + arbitrary environment value, Docker option, image reference, or mount. +- SQLite crash/race/restart, acquisition-container cleanup, host provider + process-group cancellation, and truthful post-settlement evidence scenarios + pass without claiming complete descendant containment. +- The independently packaged Bun gateway passes a trusted-network smoke against + project and user workspaces under `/tmp/`; a CLI-only install fetches neither + the gateway package nor the acquisition image. ### Scope Boundaries **In scope** - A2A 1.0 HTTP+JSON and the required AllAgents extension. -- One process and one active invocation at a time initially. -- Built-in and gateway-enabled profile-backed Codex/Pi targets. -- Direct declared Git repositories and named OCI workspace snapshots. +- One gateway process and one active invocation at a time initially. +- Built-in and gateway-enabled profile-backed Codex/Pi host execution. +- Docker-only acquisition of direct declared Git repositories and named OCI + workspace snapshots when no reusable validated base exists. +- Reusable immutable bases and Task-private runtime state for read-only + execution; non-reusable Task-owned bases for mutable revisions; unique + disposable writable views for read-write execution. +- Logical workspace-root or declared-repository-relative provider cwd plus + explicit `readOnly | readWrite` access selected at runtime. - GitHub App and configured GitHub CLI acquisition credentials. -- Local durable Task/evidence storage, cancellation, cleanup, and provenance. +- Local durable Task/evidence storage, bounded base caching, process-group + cancellation, materialization cleanup, and provenance. - Listen addresses including `0.0.0.0`. **Out of scope** @@ -793,13 +1017,17 @@ registry, or another profile configuration file for the initial use case. Internet hardening. - `gateway.yaml`, `worker.yaml`, remote workers, mTLS worker links, Kubernetes routing, autoscaling, and multiple gateway replicas. -- Caller-provided repository or registry origins, mutable OCI tags, custom - materializers, Dockerfiles, Compose files, or acquisition commands. +- Caller-provided physical workspaces/cwds, repository or registry origins, + mutable OCI tags, custom materializers, Dockerfiles, Compose files, or + acquisition commands. - GitHub Enterprise Server and multiple ordered Apps/accounts in the initial delivery. - OpenCode, Claude, Copilot, OMP, arbitrary CLI, and TUI adapters. - Evaluation orchestration and automatic retries. -- Non-Linux gateway execution in v1; ordinary AllAgents CLI behavior remains +- Per-provider containers; cgroups, pidfds, namespaces, nftables, `openat2`, a + native platform layer, non-bypassable spawn mediation, hostile-code + containment, and secret isolation from model-invoked tools. +- Non-Linux gateway execution in v1; ordinary `allagents` CLI behavior remains cross-platform. ### Sources @@ -810,6 +1038,9 @@ registry, or another profile configuration file for the initial use case. - [Source credential broker precedents](../research/source-credential-broker-precedents.md) - [A2A 1.0 specification](https://a2a-protocol.org/v1.0.0/specification/) - [A2A extension guide](https://a2a-protocol.org/latest/topics/extensions/) +- [Official A2A JavaScript SDK](https://github.com/a2aproject/a2a-js) +- [Bun workspaces](https://bun.sh/docs/install/workspaces) +- [Bun SQLite](https://bun.sh/docs/api/sqlite) - [Promptfoo custom providers](https://www.promptfoo.dev/docs/providers/custom-api/) - [Promptfoo configuration reference](https://github.com/promptfoo/promptfoo/blob/main/site/docs/configuration/reference.md) - [OpenAI Codex SDK](https://developers.openai.com/codex/sdk/) @@ -817,10 +1048,15 @@ registry, or another profile configuration file for the initial use case. - [GitHub App installation tokens](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/generating-an-installation-access-token-for-a-github-app) - [Git credential helpers](https://git-scm.com/docs/gitcredentials) - [Docker credential stores](https://docs.docker.com/reference/cli/docker/login/#credential-stores) -- [Node.js SQLite API](https://nodejs.org/docs/latest-v22.x/api/sqlite.html) - [OCI Image Specification](https://github.com/opencontainers/image-spec) - [OCI Distribution Specification](https://github.com/opencontainers/distribution-spec) -- [Linux cgroup v2](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html) +- [GitHub Container registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) +- [JFrog Artifactory Docker repositories](https://jfrog.com/help/r/jfrog-artifactory-documentation/docker-repositories) +- [JFrog Container Registry image](https://hub.docker.com/r/jfrog/artifactory-jcr) +- [Docker OverlayFS storage driver](https://docs.docker.com/engine/storage/drivers/overlayfs-driver/) +- [Docker VFS copy fallback](https://docs.docker.com/engine/storage/drivers/vfs-driver/) +- [Windows ReFS block cloning](https://learn.microsoft.com/en-us/windows-server/storage/refs/block-cloning) +- [GitHub-hosted runners](https://docs.github.com/en/actions/reference/runners/github-hosted-runners) --- @@ -828,22 +1064,30 @@ registry, or another profile configuration file for the initial use case. ### Key Technical Decisions -- KTD1. **Use the official A2A JavaScript SDK transport around an - AllAgents-owned request handler.** Do not use `DefaultRequestHandler` or its - non-transactional `TaskStore` seam. Implement the SDK's request-handler - interface so AllAgents controls UUIDv7 creation, atomic `createOrReplay`, - monotonic settlement, listing, retention, expiry, and HTTP+JSON error details - while retaining standard Task/Artifact carriers. -- KTD2. **Generate and publish the extension and storage contracts from canonical - Zod schemas.** The versioned extension specification at its declared URI - defines Agent Card params, activation, Message metadata/extensions, request, - Task/Artifact, idempotency, error, replay, examples, and versioning. Generate - public JSON Schemas, Task-store, workspace-manifest, and adapter types from the - same source. Keep backend-specific fields private. -- KTD3. **Use a single-process supervisor, not a remote worker protocol.** One - service owns Task state, staging, publication, backend child processes, - evidence, termination, and cleanup. Child processes remain contained behind - an invocation lifecycle boundary. +- KTD1. **Gate the official A2A JavaScript SDK in the shipped Bun server + direction before adopting it.** Pin the exact SDK version and prove Agent Card + discovery, both send modes, streaming, Task get/list/cancel, resubscription, + extension negotiation, metadata preservation, and HTTP error envelopes by + driving the production gateway server with an independent official client + fixture. Implement the SDK's public request-handler seam while AllAgents owns + UUIDv7 creation, atomic `createOrReplay`, monotonic settlement, listing, + retention, expiry, and HTTP+JSON error details. Do not use an SDK default store + as the transaction boundary or replace A2A with a bespoke protocol. +- KTD2. **Keep contracts portable and generated from narrow TypeScript + packages.** `packages/workspace-config` owns project/user parsing and compiled + catalogs; `packages/execution-contracts` owns A2A extension, request, + Task/Artifact, idempotency, result, error, adapter, and evidence schemas; + `packages/acquisition-contracts` owns acquisition requests, typed manifests, + OCI snapshot rules, and fixed limits. Check generated JSON Schemas and golden + accepted/rejected examples into `contracts/` for the host gateway, acquirer + image, public docs, and consumer fixtures. Do not create `core`, `common`, or + a speculative shared package. +- KTD3. **Use one gateway supervisor, not a remote worker protocol.** The Bun + gateway owns Task state, immutable-base caching, Task runtime/view + materialization, provider child processes, evidence, termination, and cleanup. + It creates an ephemeral Docker container only when no reusable validated base + exists, removes it before provider execution, and launches Codex/Pi directly + on the trusted host in Linux process groups. - KTD4. **Make application authentication intentionally absent.** All Tasks and Artifacts share one deployment namespace. The listener accepts explicit `0.0.0.0`; network controls are external. Bind and advertised interface URL @@ -853,88 +1097,161 @@ registry, or another profile configuration file for the initial use case. profile-client schemas. A gateway-only compiler normalizes the project repository catalog and resolves each public launcher ID to one profile/client. Add no deployment YAML. (session-settled: user-directed.) -- KTD6. **Keep source input name-based and closed.** Repository requests carry - only declared-name revisions; snapshot requests carry only a declared snapshot - name and immutable digests. Compute one canonical source identity for - idempotency and provenance. -- KTD7. **Freeze direct Git and OCI acquisition profiles.** Git runs with - hermetic config and an invocation credential helper. An AllAgents-owned - minimal OCI Distribution client in the Rust helper uses pinned `reqwest` - (rustls, redirects and ambient proxies disabled), `tar`, `flate2`, and `zstd` - crates for streaming pull, bounded authentication, decoding, and changeset - application behind the helper's typed protocol. The client implements only the - v1 direct-image manifest/config/layer profile, RFC 8785 workspace-manifest - config, fixed extraction limits, and explicit Distribution-Spec authentication - and redirect policy. A test-only deterministic reference packer produces the - conformance fixture that freezes the format. -- KTD8. **Select GitHub credentials by provable three-way eligibility.** App - lookup 200 is eligible; 404 is ineligible only with independent repository- - existence proof; all ambiguous outcomes are unknown. Fresh App tokens bypass - SDK cache, are validated and revoked, and only positive ineligibility permits - the configured `gh` account. (session-settled: user-directed.) +- KTD6. **Keep source, cwd, and access input logical and closed.** Repository + requests carry only declared-name revisions; snapshot requests carry only a + declared snapshot name and immutable digests. Working-directory requests + select only the effective workspace root or a declared repository plus a + bounded relative directory. `workspaceAccess` is exactly `readOnly` or + `readWrite`. Read-only Tasks may share the immutable base; read-write Tasks + receive Task-ID-derived views. The gateway never accepts or returns a caller + path or materializer choice. Include canonical source identity, logical cwd, + and access in idempotency and provenance. +- KTD7. **Freeze Docker-only base acquisition.** Build `apps/acquirer` once as a + multi-architecture GHCR image and select it by manifest digest. Each request + without a reusable validated base starts a fresh container with one staging + bind mount, a typed acquisition request, source-only credentials, strict + source network policy, fixed size/archive limits, no host home, and no Docker + socket. + The image owns hermetic Git plus the minimal OCI Distribution client + and implements only the v1 direct-image manifest/config/layer profile, + RFC 8785 workspace-manifest config, explicit authentication/redirect rules, + streaming digest checks, and changeset application. It emits a typed manifest, + exits, and is removed before host validation/publication. It contains and + downloads no Codex, Pi, or other coding harness. A deterministic producer + fixture freezes the format. +- KTD8. **Select GitHub credentials by provable three-way eligibility.** On a + base-acquisition miss, App lookup 200 is eligible; 404 is ineligible only with + independent repository-existence proof; all ambiguous outcomes are unknown. + Fresh App tokens bypass credential cache, are validated and revoked, and only + positive ineligibility permits the configured `gh` account. The selected token + enters only the acquisition container; base-cache hits resolve no credential. + (session-settled: user-directed.) - KTD9. **Keep one behavior-focused `codex | pi` adapter registry.** Direct - targets and gateway-enabled profile targets resolve to the same adapter types - and conformance tests; profile context modifies server-owned configuration, - never the public command line. A target is ready only when its pinned backend - exposes a non-bypassable spawn hook through which the helper launches every - MCP and model-tool process with separate filesystem, environment, descriptor, - secret, and network views. -- KTD10. **Put durable Task truth behind the Rust helper's SQLite VFS.** The - helper owns the single process-lifetime SQLite connection and exposes typed - transactional store operations; TypeScript never opens the database by path. - A small audited VFS roots every database, WAL, SHM, journal, and temporary-file - open beneath a preopened private state-directory descriptor with `openat2` - beneath/no-symlink checks, rejects hard links, and fsyncs files and containing - directories. SQLite uses WAL, foreign keys, and `synchronous=FULL`. Claims, - Tasks, events, bounded Artifact bytes, execution lease, containment identity, - internal outcome intent, and expiry live in transactional tables. - `createOrReplay`, lease acquisition, and terminal settlement are transactions; - acknowledge only committed state. Crash recovery yields a complete old or new - generation, never a mixed or missing acknowledged Task. Integrity, VFS, helper - protocol, or durability failure stops admission and prevents false success. -- KTD11. **Package one enforceable Linux security and state helper.** V1 supports - Linux x64/arm64 with cgroup v2, `clone3(CLONE_INTO_CGROUP)`, pidfds, `openat2` - beneath/no-symlink resolution, mount and network namespaces, and nftables - through a small audited Rust helper distributed in platform-specific optional - packages. Its typed inherited-pipe protocol owns SQLite operations, creates - empty containment behind a durable start gate, atomically launches and tracks - the complete acquisition/provider descendant set, mediates every MCP/tool - spawn, builds role-specific filesystem/environment/descriptor/network views, - terminates and waits for membership, and performs safe file operations. - Missing kernel features, delegated cgroup/network access, helper package, - backend spawn mediation, or protocol compatibility fails before binding; - there is no weaker fallback. A poisoned process remains alive to reap until - the set is empty. -- KTD12. **Capture live events, then collect durable filesystem evidence only - after quiescence.** Evidence retains bounded source, Git, provider, result, - Artifact, and cleanup facts. Post-execution workspace reads use the helper's - descriptor-relative no-follow handles, revalidate identity/size, and reject - Git metadata indirections or repository-controlled execution. Structured logs - remain metadata-only and never retain secrets or unrestricted - request/output/file bodies. + targets and gateway-enabled profile targets resolve to the same narrow + AllAgents-owned TypeScript adapter contract and conformance suite; profile + context modifies server-owned configuration, never public argv. Codex uses + pinned `@openai/codex-sdk` first; app-server is allowed only for a proven + required SDK gap. Pi uses a pinned supported package/RPC surface. Neither + adapter downloads runtimes per request or adopts AI SDK Harnesses. A global + binary override requires an exact compatibility probe. +- KTD10. **Keep durable Task truth inside ordinary Bun SQLite ownership.** The + gateway holds the process-lifetime `bun:sqlite` connection, private state + root, and exclusive lock. SQLite uses foreign keys, transactional + `createOrReplay`/lease/settlement/expiry operations, WAL where supported, and + `synchronous=FULL`; acknowledge only committed state. Claims, Tasks, events, + bounded Artifact bytes, execution lease, acquisition-container/staging/ + transient-base identity, provider process-group identity, internal outcome + intent, and expiry live in tables. Startup integrity or durability failure + stops admission and prevents false success. Do not build a custom VFS or + native file layer. +- KTD11. **Treat the trusted Linux CI job as the provider isolation boundary.** + The gateway uses Docker only when no reusable validated base exists. Codex and + Pi run bare metal with the same CI-job authority as the gateway and existing + host + auth. Read-only is a consumer-selected cooperative contract with private + runtime state, optional-lock suppression, and native provider policy where + available; it is not hostile-code containment. + Construct provider environments explicitly to preserve required identity/auth + paths while omitting unrelated ambient values, but do not claim this protects + secrets from model-invoked tools. Linux cancellation is adapter abort, then + process-group `SIGTERM`, then `SIGKILL`; runner teardown is the final orphan + boundary. Do not add cgroups, pidfds, `openat2`, namespaces, nftables, native + containment packages, per-provider Docker, or spawn mediation. +- KTD12. **Capture live events, then collect bounded evidence after the direct + provider settles.** Evidence retains bounded source, Git, provider, result, + Artifact, observed termination, and cleanup facts. Git inspection disables + repository-controlled execution. Evidence and docs must not turn process- + group termination into a claim that all descendants are quiescent or that + model-tool output is redacted. +- KTD13. **Use one private Bun workspace without coupling releases.** The root + package is private orchestration. `apps/cli` publishes `allagents`; + `apps/gateway` publishes `allagents-gateway`; `apps/acquirer` is never + published to npm and ships only as a digest-pinned multi-architecture GHCR + image. Shared packages are limited to `packages/workspace-config`, + `packages/execution-contracts`, and `packages/acquisition-contracts`; + generated portable fixtures live under `contracts/`. + + CLI and gateway have independent versions, tags, changelogs, triggers, npm + tarballs, and release jobs. A CLI-only install resolves neither the gateway nor + the acquisition image. A gateway release first builds the acquisition image + once for the exact commit, resolves and records its multi-architecture + manifest plus supported platform digests, runs package and registry checks + against those exact immutable artifacts, and only then publishes the exact + `allagents-gateway` npm tarball. A gateway-only release never publishes + `allagents`; no Rust, Cargo, native binary, or platform npm package exists. +- KTD14. **Use tiered OCI registry conformance bound to exact release + artifacts.** Every pull request runs a local Distribution fixture and a live + public digest-pinned GHCR snapshot pull through the exact acquirer image. A + reusable release workflow adds authenticated least-privilege GHCR and pinned + private-CA JFrog Artifactory/JCR coverage. + + The callable workflow receives the exact gateway npm tarball, acquisition + multi-architecture manifest digest, per-platform image digests where the + registry supports them, build commit, and expected compatibility output; it + never rebuilds either artifact. Reports record the tested commit, npm tarball + digest, acquisition manifest/platform digests, architecture, image/registry + identity, auth mode, snapshot descriptor digests, and compatibility output, + including partial evidence on red paths. They cover valid anonymous and + authenticated pulls plus wrong credentials, insufficient permissions, digest + mismatch, missing/wrong CA, invalid media, and repository-path failures. The + gateway release must verify GHCR and JFrog against those exact artifacts + before npm publication; the JFrog target need not run on every pull request. + +### Package compatibility contract + +`allagents-gateway compatibility --format json` emits one strict, versioned +object containing `product: "allagents-gateway"`, `gatewayVersion`, +`buildCommit`, `runtime: "bun"`, the pinned acquisition image repository and +multi-architecture manifest digest, supported acquisition platforms/digests, +and supported A2A, coding-extension, workspace, execution-contract, acquisition- +contract, and snapshot versions. The packed npm tarball, clean-install smoke, +registry workflow, and release workflow consume this same object. + +An optional `allagents gateway ...` dispatcher locates but never installs the +separate gateway. It accepts independent CLI and gateway versions only when the +product identity and required contract-version ranges intersect; otherwise it +prints a clear install/upgrade error and does not start the service. The gateway +rejects an acquisition image whose manifest digest, platform digest, build +identity, or acquisition-contract version differs from its release metadata. +Golden fixtures cover exact matches, supported CLI/gateway version skew, +unsupported contract versions, wrong image manifests/platforms, divergent npm +tarball or image build identities, and newest/oldest supported pairs. There are +no platform npm packages or native-binary compatibility checks. ### High-Level Technical Design ```mermaid flowchart TB - C[Trusted-network A2A caller] --> G[Gateway server] - G --> H[Linux security and state helper] - H --> S[SQLite Task store] - G --> W[Workspace compiler] + C[Trusted-network A2A caller] --> G[Bun gateway host process] + G --> S[Bun SQLite Task store] + G --> W[workspace-config compiler] W --> PW[Project workspace.yaml] W --> UW[User workspace.yaml] - G --> A[Acquisition supervisor] + G --> BL[Reusable immutable-base lookup] + BL -->|hit| RB[Validated reusable base and pin] + BL -->|miss or mutable revision| D[Docker acquisition coordinator] + D --> A[Digest-pinned acquirer container] A --> Git[Declared Git repositories] A --> OCI[Named OCI snapshot] - A --> H - A --> P[Atomically published invocation workspace] - G --> R[Closed adapter registry] - R --> Codex[Codex SDK] - R --> Pi[Pi RPC] - Codex --> H - Pi --> H - H --> E[Quiescence then evidence and cleanup] - E --> S + A --> ST[Staging plus typed manifest] + ST --> V[Host validation and atomic base promotion] + V -->|exact identity| RB + V -->|mutable revision| TB[Task-owned transient base] + RB --> RO[Read-only base plus private runtime] + TB --> RO + RB --> M[Block clone or rootless OverlayFS or copy] + TB --> M + M --> RW[Task-owned writable view] + RO --> WD[Logical cwd resolver] + RW --> WD + WD --> R[Closed host adapter registry] + R --> Codex[Pinned Codex SDK] + R --> Pi[Pinned Pi RPC/package] + Codex --> PG[Linux provider process group] + Pi --> PG + PG --> E[Direct-process settlement then bounded evidence] + E --> C[Remove Task runtime, view, and transient base] + C --> S ``` ### Configuration Contract @@ -949,88 +1266,117 @@ No `gateway.yaml` or `worker.yaml` is introduced. | Advertised interface URL | `--advertise-url` | `ALLAGENTS_GATEWAY_ADVERTISE_URL` | `http://127.0.0.1:4732` only with the default loopback listener; otherwise required | | Project workspace | `--workspace` | `ALLAGENTS_GATEWAY_WORKSPACE` | cwd | | State directory | `--state-dir` | `ALLAGENTS_GATEWAY_STATE_DIR` | `~/.allagents/gateway/` | +| Invocation workspace root | `--invocation-root` | `ALLAGENTS_GATEWAY_INVOCATION_ROOT` | `~/.allagents/gateway-workspaces/` | +| Immutable-base cache root | `--base-cache-dir` | `ALLAGENTS_GATEWAY_BASE_CACHE_DIR` | `~/.allagents/gateway-cache/` | +| Immutable-base cache budget | `--base-cache-max-bytes` | `ALLAGENTS_GATEWAY_BASE_CACHE_MAX_BYTES` | `64GiB` | +| Workspace materializer | `--workspace-materializer` | `ALLAGENTS_GATEWAY_WORKSPACE_MATERIALIZER` | `auto` (`auto | cow | copy`) | +| Automatic copy ceiling | `--max-auto-copy-bytes` | `ALLAGENTS_GATEWAY_MAX_AUTO_COPY_BYTES` | `1GiB` | | Terminal Task TTL | `--task-ttl` | `ALLAGENTS_GATEWAY_TASK_TTL` | `24h` | | Retained Task limit | `--max-retained-tasks` | `ALLAGENTS_GATEWAY_MAX_RETAINED_TASKS` | `1000` | | Per-Task retained bytes | `--max-task-bytes` | `ALLAGENTS_GATEWAY_MAX_TASK_BYTES` | `64MiB` | +| Acquisition image | `--acquisition-image` | `ALLAGENTS_GATEWAY_ACQUISITION_IMAGE` | release-embedded `ghcr.io/.../allagents-acquirer@sha256:` | +| Docker endpoint | `--docker-host` | `ALLAGENTS_GATEWAY_DOCKER_HOST` | existing local Docker context/socket | +| Docker acquisition network | `--acquisition-network` | `ALLAGENTS_GATEWAY_ACQUISITION_NETWORK` | release-documented acquisition-only network | +| Acquisition timeout | `--acquisition-timeout` | `ALLAGENTS_GATEWAY_ACQUISITION_TIMEOUT` | `900s`, capped by remaining Task deadline | | GitHub App ID | `--github-app-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_ID` | unset | | App private key file | `--github-app-private-key-file` | `ALLAGENTS_GATEWAY_GITHUB_APP_PRIVATE_KEY_FILE` | unset | | App installation ID | `--github-app-installation-id` | `ALLAGENTS_GATEWAY_GITHUB_APP_INSTALLATION_ID` | discovered/unset | | GitHub CLI account | `--github-cli-account` | `ALLAGENTS_GATEWAY_GITHUB_CLI_ACCOUNT` | unset | | OCI auth file | `--oci-auth-file` | `ALLAGENTS_GATEWAY_OCI_AUTH_FILE` | unset | | OCI credential helper | `--oci-credential-helper` | `ALLAGENTS_GATEWAY_OCI_CREDENTIAL_HELPER` | unset | -| Codex auth file | `--codex-auth-file` | `ALLAGENTS_GATEWAY_CODEX_AUTH_FILE` | supported Codex default if safe | -| Pi auth file | `--pi-auth-file` | `ALLAGENTS_GATEWAY_PI_AUTH_FILE` | supported Pi default if safe | - -Precedence is CLI over environment over default. The advertised value is the -absolute URL placed in `AgentCard.supportedInterfaces`; wildcard hosts are -invalid, non-loopback listeners require an explicit value, and production uses -HTTPS. Credential options name file handles, accounts, or IDs, never secret -values. - -The Linux helper resolves every key/auth/helper path from a verified root with -`openat2(RESOLVE_BENEATH | RESOLVE_NO_SYMLINKS | RESOLVE_NO_MAGICLINKS)`, rejects -group/world-writable parent directories and linked or non-regular leaves, opens -with close-on-exec/no-follow, and verifies owner, mode, link count, device, and -inode with `fstat` after open. Consumers read the verified descriptor rather than -reopening the path. The helper executes a credential-helper binary from that -verified inode; a path or inode swap fails. Auth leaves are current-user/root- -owned, have one link, and are no broader than `0600`; helper leaves are -current-user/root-owned and not group/world-writable. - -Setting both OCI options is a startup error. `--oci-auth-file` accepts at most -1 MiB of strict UTF-8 Docker-config JSON containing only `auths`. Each key is the -exact registry lookup key below and each strict entry contains exactly one of: -bounded base64 `auth` decoding to `username:secret`, or bounded nonempty -`identitytoken`. `credsStore`, `credHelpers`, proxy/plugin fields, unknown -members, commands, and duplicate keys are rejected; nothing named by the file -is executed. Credential selection is exact-key only. - -The fixed OCI helper receives argv `[helperPath, "get"]` without a shell. Stdin -is the raw Docker lookup key plus newline: lowercase `host[:nondefault-port]` -except Docker Hub, which uses `https://index.docker.io/v1/`. Exit-zero stdout is -one UTF-8 JSON object with required nonempty `Username` and `Secret` strings and -optional `ServerURL`, each at most 64 KiB. `ServerURL`, when present, must equal -the lookup key; `Username: ""` classifies `Secret` as an identity token. -Stdout over 128 KiB, timeout, nonzero exit, signal, malformed UTF-8/JSON, unknown -member, mismatch, or empty credential fails with `source_auth_oci_failed`. -Stderr is bounded, treated as secret-bearing, and never logged or retained. - -Registry access starts anonymously. Accept at most one well-formed HTTPS Bearer -challenge and one authenticated retry per request, with one token refresh after -an in-budget 401. Scope must exactly equal -`repository::pull`; service is bounded, -passed only as data, and must match the registry service -(`registry.docker.io` for Docker Hub). -Credentialed token exchange is allowed only at a same-origin HTTPS realm -or the exact Docker Hub realm `https://auth.docker.io/token`; other realms are -anonymous-only. Do not request offline access or accept refresh tokens. Validate -token type and bounded expiry. - -Redirect handling is manual and limited to three HTTPS hops. Same-origin -redirects are permitted. A cross-origin redirect is permitted only for a -layer-blob `GET`/`HEAD` when the destination's normalized `host[:port]` exactly -matches that snapshot source's `layerRedirectHosts`; token, manifest, and config -requests reject it. Every hop rejects URL credentials, strips authorization, -cookies, and client credentials, resolves DNS afresh, validates every A/AAAA -address, and connects to a validated address with the original hostname used -for Host/SNI. Loopback, link-local, multicast, unspecified, RFC1918, ULA, CGNAT, -and other non-global destinations are rejected unless that exact host is -operator-approved for the source. Redirect loops, downgrade, mixed approved and -unapproved answers, and rebinding fail. Final descriptor bytes still must match -size and digest. - -Credentials are invoked once per registry lookup key, scoped to that origin and -repository pull, zeroed after use, and destroyed before publication. - -Provider defaults are eligible only when their resolved auth files pass the same -descriptor checks; otherwise the target is not ready. The gateway projects only -the selected provider auth into its control-process view. +| OCI CA bundle map | `--oci-ca-bundle-map` | `ALLAGENTS_GATEWAY_OCI_CA_BUNDLE_MAP` | system roots only | +| Codex home | `--codex-home` | `ALLAGENTS_GATEWAY_CODEX_HOME`, then `CODEX_HOME` | existing supported host Codex home | +| Codex binary override | `--codex-bin` | `ALLAGENTS_GATEWAY_CODEX_BIN` | pinned SDK-managed surface; unset | +| Pi home | `--pi-home` | `ALLAGENTS_GATEWAY_PI_HOME` | existing supported host Pi home | +| Pi binary override | `--pi-bin` | `ALLAGENTS_GATEWAY_PI_BIN` | pinned package/RPC surface; unset | +| Graceful abort period | `--abort-grace` | `ALLAGENTS_GATEWAY_ABORT_GRACE` | `10s` | +| SIGTERM period | `--term-grace` | `ALLAGENTS_GATEWAY_TERM_GRACE` | `10s` | +| Final cleanup period | `--cleanup-timeout` | `ALLAGENTS_GATEWAY_CLEANUP_TIMEOUT` | `30s` | + +Precedence is CLI over gateway-specific environment over provider-standard +environment over default. For Codex this is `--codex-home`, +`ALLAGENTS_GATEWAY_CODEX_HOME`, then the ordinary `CODEX_HOME` identity +location. The advertised value is the absolute URL placed in +`AgentCard.supportedInterfaces`; wildcard hosts are invalid, non-loopback +listeners require an explicit value, and production uses HTTPS. The acquisition +image must be a full `repository@sha256:` reference; tags are rejected. +The gateway verifies that the local platform resolves to the release-recorded +platform digest before starting acquisition. + +Docker is a base-acquisition dependency only when no reusable validated base +exists. The configured endpoint must support creating, waiting for, stopping, +and removing a container plus bind-mounting gateway-created staging. A validated +cache hit does not contact Docker or resolve a source credential. The gateway +never passes the +Docker socket into the container. The acquisition network is preconfigured by +the operator to reach only declared Git/OCI source hosts and required auth/ +redirect hosts; the gateway supplies the stricter per-request host policy to the +acquirer. No Docker flag, mount, network, image, or environment override is +accepted from A2A. + +Credential and CA paths are resolved on the trusted host, must be current-user +owned regular files with private permissions, and are read only for acquisition. +Setting both OCI credential options is a startup error. `--oci-auth-file` +accepts at most 1 MiB of strict UTF-8 Docker-config JSON containing only +`auths`; each exact registry key contains one bounded `auth` or +`identitytoken`. `credsStore`, `credHelpers`, proxy/plugin fields, commands, +duplicate keys, and unknown members are rejected. + +The fixed OCI helper receives argv `[helperPath, "get"]` without a shell and the +raw exact Docker lookup key on stdin. Exit-zero stdout is one bounded strict JSON +object with nonempty `Username` and `Secret` plus optional matching `ServerURL`. +Timeout, nonzero exit, signal, malformed output, mismatch, or empty credentials +fails with `source_auth_oci_failed`; stderr is secret-bearing and never logged. + +`--oci-ca-bundle-map` names a bounded strict JSON file mapping exact normalized +`host[:port]` keys to private PEM CA files. Only the bundle for the exact +registry, token service, or declared layer-redirect host augments system roots; +there is no insecure-TLS switch. Registry access begins anonymously and accepts +only bounded same-origin Basic or Distribution Bearer behavior plus the +documented Docker Hub token service. Cross-origin redirects remain limited to +layer `GET`/`HEAD` requests for exact declared hosts, with credentials stripped +and every hop checked. The gateway passes only the selected source credential +and exact CA material into the acquisition container and destroys both before +provider execution. + +Provider homes are never copied, mounted into Docker, parsed by AllAgents, or +imported into another store. The direct Codex/Pi host process receives the +selected home path and required host identity/auth environment in place. +Binary overrides are absolute host paths and must pass the pinned adapter's +exact version/protocol probe at readiness; they are not request-selectable. +The explicit provider environment starts from an allowlist rather than the +gateway's complete environment, but this is leakage reduction, not isolation. + +The immutable-base cache and invocation roots are current-user owned, private, +and disjoint from state, project, profile, provider-auth, and each other. +Acquisition writes a unique directory under `/.staging`; host +validation completes before an atomic same-filesystem rename to either the final +cache-key directory or `/transient/` for a non-reusable +base. Active Task references pin reusable entries. Least-recently-used eviction +enforces the byte budget and removes only unpinned reusable bases. Every non- +publication path removes its staging directory, and startup reconciles orphan +staging and recorded transient bases before readiness. + +For read-only access, every Task owns +`//runtime`; its cwd resolves in a reusable cached base +or its non-reusable transient base. For read-write access, the Task also owns +`//workspace`. `auto` probes same-filesystem block +clone first, then rootless OverlayFS on Linux, then ordinary copy only when the +base does not exceed `--max-auto-copy-bytes`. `cow` requires block clone or +rootless OverlayFS and fails readiness when neither is available. `copy` is the +explicit portable, higher-I/O backend and may exceed the automatic copy ceiling. +The explicit `copy` backend has no Linux-only filesystem requirement, but it +does not by itself make the v1 gateway available on Windows; process lifecycle +and cancellation remain Linux-only in this plan. Startup logs the selected +capabilities without paths. No mode uses writable hard links. Startup rejects +overlapping roots and stale mounts it cannot safely reconcile. The derived workspace ID is a stable digest of the canonical project-workspace path and is verified against SQLite metadata. Retention includes Task records, -Artifact bytes, events, and invocation-key claims; expiry is transactional. When -the unexpired Task-count limit is reached, new admission fails rather than -evicting retained Tasks. +Artifact bytes, events, and invocation-key claims; expiry is transactional. Task +expiry does not evict a pinned base, and base eviction does not remove retained +Task metadata. When the unexpired Task-count limit is reached, new admission +fails rather than evicting retained Tasks. **Project workspace additions** @@ -1046,13 +1392,25 @@ workspaceSnapshots: repository: ghcr.io/entityprocess/allagents-workspaces layerRedirectHosts: - pkg-containers.githubusercontent.com + enterprise: + repository: company.jfrog.io/docker-local/allagents-workspaces ``` Snapshot names use the portable profile-name vocabulary. Repositories must have -unique stable names for remote acquisition. Snapshot repository values contain -only scheme/host/repository identity and an optional exact -`layerRedirectHosts` allowlist; never tags, digests, credentials, or extraction -paths. An absent allowlist rejects cross-origin layer redirects. +unique stable names for remote acquisition. Non-Docker-Hub repository values +contain only an exact registry `host[:port]/repository-path` identity and an +optional exact `layerRedirectHosts` allowlist; never tags, digests, credentials, +or extraction paths. + +Docker Hub uses only the canonical declaration +`docker.io//` with an explicit namespace. The gateway +maps that declaration to API origin `https://registry-1.docker.io`, Docker +credential lookup key `https://index.docker.io/v1/`, Bearer service +`registry.docker.io`, and token realm `https://auth.docker.io/token`; +`index.docker.io` and `registry-1.docker.io` declarations are rejected as +aliases. GHCR, JFrog Artifactory/JCR, and compatible private OCI registries keep +their declared exact host. An absent allowlist rejects cross-origin layer +redirects. **User workspace additions** @@ -1079,10 +1437,13 @@ TypeScript. Its `constructor(options: ProviderOptions)` requires and stores a nonempty `options.id`, validates `options.config`, and `id()` returns that stored value. `callApi(prompt, context?, options?)` reads -`context?.vars?.allagentsSource` when present and +`context?.vars?.allagentsSource`, +`context?.vars?.allagentsWorkingDirectory`, and +`context?.vars?.allagentsWorkspaceAccess` when present, plus `options?.abortSignal` for cancellation. -Static YAML defines the source mode and every logical name: +Static YAML defines the source mode, logical names, and optional default logical +working directory and workspace access: ```yaml prompts: @@ -1102,6 +1463,10 @@ providers: config: endpoint: https://allagents-gateway.example.internal target: codex + workingDirectory: + kind: repository + repository: allagents + workspaceAccess: readOnly source: kind: repositories revisions: @@ -1112,6 +1477,10 @@ providers: config: endpoint: https://allagents-gateway.example.internal target: codex + workingDirectory: + kind: repository + repository: allagents + workspaceAccess: readWrite source: kind: workspaceSnapshot snapshot: evaluation @@ -1125,6 +1494,11 @@ tests: allagentsSource: revisions: allagents: fedcba9876543210fedcba9876543210fedcba98 + allagentsWorkingDirectory: + kind: repository + repository: allagents + path: apps/gateway + allagentsWorkspaceAccess: readOnly - description: immutable prebuilt workspace providers: [codex-evaluation-snapshot] @@ -1132,6 +1506,11 @@ tests: allagentsSource: digest: sha256:fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210 workspaceManifestDigest: sha256:6789abcdef0123456789abcdef0123456789abcdef0123456789abcdef012345 + allagentsWorkingDirectory: + kind: repository + repository: allagents + path: apps/gateway + allagentsWorkspaceAccess: readWrite ``` The gateway enforces one active invocation transactionally. Promptfoo keeps @@ -1149,29 +1528,43 @@ revisions, and immutable digests, not `ghcr.io/entityprocess/allagents-workspaces`. The gateway resolves origins and credentials server-side and omits them from A2A source-identity responses. -`context?.vars?.allagentsSource` is the only per-test override. In repository +`context?.vars?.allagentsSource` remains limited to source leaves. In repository mode it may contain exactly `revisions`, whose keys must already exist in static -`config.source.revisions` and whose values are full lowercase 40-hex commits. -In snapshot mode it may contain exactly `digest` and/or +`config.source.revisions` and whose values are full lowercase 40-hex commits. In +snapshot mode it may contain exactly `digest` and/or `workspaceManifestDigest`, both full lowercase `sha256:` digests. Present leaves replace static leaves; absent leaves retain static values. Source kind, -repository-name allowlist, and snapshot name remain static. Unknown members, -mutable revisions, origins, destinations, credentials, and commands fail before -A2A submission. +repository-name allowlist, and snapshot name remain static. + +`context?.vars?.allagentsWorkingDirectory` replaces the complete static +selector for that trial. It is exactly `workspaceRoot` or a declared repository +name plus an optional `RelativeDirectory`; the gateway performs catalog and +post-acquisition directory validation. `allagentsWorkspaceAccess` replaces the +static access value with exactly `readOnly` or `readWrite`. Missing access +defaults to `readWrite`. Neither variable accepts an absolute path, configured +destination, materializer, cache key, `.` or `..` segment, backslash, symlink +escape, or non-directory. Unknown members, mutable revisions, origins, +destinations, credentials, and commands fail before provider execution. Each `callApi` creates one high-entropy invocation key and sends `SendMessage` with `returnImmediately: true`, then follows the accepted Task through -`SubscribeToTask`, `GetTask`, and bounded resubscription. An abort or deadline +`SubscribeToTask`, `GetTask`, and bounded resubscription. A read-only Task may +share its immutable physical base and cwd with other Tasks while keeping private +runtime state; a read-write Task receives a unique disposable writable view. The +caller chooses neither physical path nor materializer. An abort or deadline sends one `CancelTask` with a fresh cleanup signal. Ambiguous submission retry -reuses the same key and request. The provider returns terminal text or validated -structured result as `ProviderResponse.output`. It maps gateway usage exactly as +reuses the same key, canonical request, Task, base/view, cwd, and access mode. +The provider returns terminal text or validated structured result as +`ProviderResponse.output`. It maps gateway usage exactly as `inputTokens -> tokenUsage.prompt`, `outputTokens -> tokenUsage.completion`, `cachedInputTokens -> tokenUsage.cached`, and `totalTokens -> tokenUsage.total`; provider-specific counters remain in -`metadata`. Task ID, Artifact references, logical source identity, termination, -cleanup, and stable failure `code`/`retryable`/accepted `taskId` also remain in -`metadata`, without origins or destination paths. Admission and terminal -failures use a safe `ProviderResponse.error`. This provider is AI Evals code; +`metadata`. Task ID, Artifact references, logical source identity, logical +working directory, workspace access, termination, cleanup, and stable failure +`code`/`retryable`/accepted `taskId` also remain in metadata, without origins, +configured destinations, or physical paths. +Admission and terminal failures use a safe `ProviderResponse.error`. This +provider is AI Evals code; AllAgents has no Promptfoo runtime dependency. ### Error and Status Mapping @@ -1188,12 +1581,15 @@ reason. Custom admission errors include `google.rpc.ErrorInfo` with domain |---|---|---| | Unsupported A2A version | HTTP 400 A2A `VersionNotSupportedError`; no Task | No | | Missing required extension | HTTP 400 A2A `ExtensionSupportRequiredError`; no Task | No | -| Malformed request, source, digest, schema, prompt, or unknown target/source | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | +| Malformed request, source, working-directory selector, workspace access, digest, schema, prompt, or unknown target/source/repository | HTTP 400 `INVALID_ARGUMENT`; `invalid_execution_request`; no Task | No | | Invocation-key conflict | HTTP 409 `ALREADY_EXISTS`; `invocation_key_conflict`; no new Task | No | | Identical retained invocation replay | Existing Task with embedded Artifacts | N/A | | Cancel after terminal state | HTTP 400 A2A `TaskNotCancelableError` | No | | Retained Task capacity exhausted | HTTP 429 `RESOURCE_EXHAUSTED`; `retention_capacity_exhausted`; `Retry-After`; no Task | Yes, after expiry | | Runtime capacity unavailable after acceptance | `execution_capacity_unavailable`; failed Task | Yes | +| Valid logical cwd resolves to a missing, non-directory, or escaping path after acquisition | `execution_working_directory_invalid`; failed Task; no provider start; no physical path returned | No | +| Required copy-on-write materializer unavailable, or `auto` would copy above its ceiling | `workspace_materialization_unavailable`; failed Task; no provider start | No | +| Task-private runtime, non-reusable base, or writable-view creation/removal fails | `workspace_cleanup_failed`; failed Task; `cleanup.workspace: "failed"`; retain cleanup record; stop admission if an active mount or uncertain writable view remains | Yes only as a fresh invocation after operator repair | | App absent/ineligible and configured `gh` succeeds | Continue with recorded provider class | N/A | | App applicability unknown | `source_auth_applicability_unknown`; failed Task; no fallback | Yes for rate-limit/service causes only | | Selected App config/auth/mint/validation/revocation failure | `source_auth_failed`; failed Task; no fallback | No | @@ -1209,15 +1605,17 @@ reason. Custom admission errors include `google.rpc.ErrorInfo` with domain | Deadline expires | `execution_deadline_exceeded`; abort/terminate; failed Task | Yes | | Known provider permission denial | `execution_permission_denied`; rejected Task | No | | Unknown provider protocol or result shape | `provider_protocol_invalid`; failed Task | No | -| Cancellation with proven quiescence | `execution_canceled`; canceled Task | No | -| Termination or cleanup cannot be proven | `execution_quiescence_unknown`; failed Task; readiness poisoned | No | -| State store durability/integrity failure | `task_store_failed`; stop admission; abort/contain; no success | No | +| Cancellation after acquisition removal or direct provider settlement | `execution_canceled`; canceled Task | No | +| Acquisition container or unpublished staging cannot be removed | `source_cleanup_failed`; failed Task; stop admission | No | +| Direct provider does not settle after abort/`SIGTERM`/`SIGKILL` | `execution_termination_failed`; failed Task; no filesystem/Git evidence; retain Task-owned runtime, view, or non-reusable base and lease; stop admission until post-teardown reconciliation | No | +| State store durability/integrity failure | `task_store_failed`; stop admission; request active-work abort; no success | No | | Restart finds interrupted Task | `gateway_restarted`; failed Task; no provider resume | Yes as a new invocation | | Retention expiry | HTTP 404 A2A `TaskNotFoundError` | Yes as a new invocation | Accepted-Task failures use the integrity Artifact's strict `failure` object with `code`, safe `message`, table-defined `retryable`, and one closed cause from -`validation | capacity | sourceAuth | sourceGit | sourceSnapshot | deadline | +`validation | capacity | sourceAuth | sourceGit | sourceSnapshot | sourceCleanup | +workingDirectory | workspaceMaterialization | workspaceCleanup | deadline | permission | providerProtocol | cancellation | termination | stateStore | restart`. Retryability says whether a caller may create a fresh invocation; it never enables automatic Task retry or provider/source fallback. Promptfoo copies @@ -1227,347 +1625,507 @@ carrier. ### Phased Delivery -1. Build the current CLI and record the red E2E showing that - `allagents gateway serve` is unavailable. Record the exact `/tmp/` workspace - setup, command, and observed failure. -2. Freeze workspace additions, the published extension, snapshot format, - common manifests, result schema, errors, packaging, and fixtures. Establish - the Rust helper protocol, safe SQLite VFS, platform packages, and ordered - release pipeline first. -3. Build the SQLite Task store through the helper, AllAgents A2A request handler, - HTTP+JSON server, minimal backend interface/registry, and fake adapter. -4. Extend the packaged helper with invocation supervision, execution - containment, spawn mediation, role-specific network/secret views, safe - evidence, and terminal arbitration around the fake adapter. -5. Add repository and OCI acquisition through the supervisor/helper with - credential containment and manifest validation. -6. Add Codex, then Pi, against the same conformance suite. -7. Run final implementation review and fix important correctness, security, +1. In a clean `/tmp/` npm prefix, install the current `allagents` package and + record the red E2E showing that `allagents-gateway serve` is unavailable and + that no acquisition image is fetched. +2. Execute U0 as a bounded feasibility gate: establish the private Bun + workspace layout; prove the shipped A2A server with the official JavaScript + client; pin and probe Codex SDK and Pi RPC/package surfaces; characterize + explicit provider environments and Linux process groups; build and run the + digest-pinned acquisition image for both supported architectures; and prove + independent CLI/gateway packaging plus exact release binding. +3. Freeze workspace additions, published extension, snapshot format, execution + and acquisition contracts, generated portable fixtures, error vocabulary, + SQLite schema/transactions, compatibility output, and release manifest. +4. Build the Bun SQLite Task store, AllAgents A2A request handler, HTTP+JSON/SSE + server, minimal backend interface/registry, and fake adapter. +5. Add direct host-process supervision, explicit environment construction, + read-only runtime separation, read-write materialization, process-group + cancellation, typed preparation, bounded evidence, terminal arbitration, + restart handling, and cleanup around the fake adapter. +6. Add Docker-only Git and OCI acquisition for requests without reusable bases, + with source-only credentials, strict mount/network/archive limits, typed + manifest emission, host validation, reusable-cache or non-reusable-base + publication, pinning, cleanup, reuse, and eviction. +7. Add Codex through the pinned SDK, then Pi through the pinned supported + package/RPC surface, against the same conformance suite. +8. Run final implementation review and fix important correctness, security, contract, reliability, DRY, and coverage findings. -8. Run the green bundled-CLI `/tmp/` E2E, clean-registry install smoke, - repository quality gates, user documentation, and release evidence. +9. Run green packed CLI/gateway `/tmp/` smokes, exact multi-architecture + acquirer-image tests, public GHCR conformance, exact-release authenticated + GHCR/JFrog conformance, repository quality gates, user documentation, and + release evidence. ### System-Wide Impact -- **Package surface:** Declare a root Bun workspace; add private - `packages/execution-service` and Rust `packages/execution-helper`; add the - service as a root `workspace:*` development dependency; distribute Linux - x64/arm64 helper binaries through versioned platform-specific optional - packages; and bundle the service into published `dist/index.js`. The release - scripts and Publish workflow version matching helper packages and root - dependency ranges, publish and verify both platform packages first, and - publish `allagents` only after their registry metadata and checksums resolve. - Add public `allagents gateway serve` without changing existing profile and - sync commands. Root build, typecheck, tests, and clean-registry install smoke - include the private workspace service and resolved helper binary. -- **Schema surface:** Extend project workspace schemas with named snapshots, - exact layer-redirect hosts, and user profile-client schemas with explicit - gateway enablement. Publish the versioned extension specification and - generated JSON Schemas; update configuration docs. -- **Dependency surface:** Put the official A2A SDK, pinned Codex SDK, and - `@octokit/auth-app` in the private service package. Pin SQLite, the custom VFS - bindings, `reqwest` with rustls, `tar`, `flate2`, and `zstd` in the Rust helper - lockfile together with the Rust toolchain/helper protocol; check helper release - checksums. -- **State surface:** Add one bounded SQLite gateway state root and per-invocation - staging, publication, evidence, and cleanup roots. Do not alter profile state. -- **Security surface:** The network is the caller authorization boundary. Source - credentials are phase-scoped; helper-mediated process and network views keep - acquired code and agent tools from App, `gh`, OCI, provider, MCP, operator, and - gateway credentials/state. Helper absence or capability loss fails closed. +- **Package surface:** Convert the root to private Bun workspace orchestration. + `apps/cli` publishes `allagents`; `apps/gateway` publishes + `allagents-gateway`; `apps/acquirer` publishes no npm package and builds only + the digest-pinned GHCR image. The ordinary CLI has no gateway dependency. + GitHub Actions has independent CLI and gateway release triggers. Gateway + release builds and verifies the acquisition image first, then publishes the + exact npm tarball; a gateway-only run never publishes the CLI. +- **Runtime surface:** `apps/gateway` owns gateway behavior end to end. Docker + exists only at the base-acquisition boundary when no reusable validated base + exists. Codex and Pi execute directly on the trusted Linux runner with + existing host + authentication. Packages share contracts and configuration, not generic + implementation helpers; do not add `core`, `common`, native IPC, or dual + implementations. +- **Schema surface:** `packages/workspace-config` extends project schemas with + named snapshots/exact redirect hosts and user profile-client schemas with + gateway enablement. `packages/execution-contracts` and + `packages/acquisition-contracts` generate versioned JSON Schemas and fixtures + under `contracts/`; update extension, snapshot-format, and configuration docs. +- **Dependency surface:** Pin Bun, the official A2A JavaScript SDK, + `@openai/codex-sdk`, the supported Pi package/RPC dependency, and the minimal + Git/OCI/archive dependencies used by `apps/acquirer` in the Bun lockfile. + Minimize dependencies per workspace and scan both the npm tarball and image. +- **State surface:** Add one bounded private Bun SQLite state root, one bounded + immutable-base cache, and per-Task runtime plus optional writable-view roots. + Do not alter provider profile or authentication state. +- **Security surface:** Network reachability authorizes callers. The acquisition + container has staging, source-only credentials, and strict source policy but + no host home or Docker socket. Provider execution has trusted CI-job + authority; explicit environments reduce accidental leakage but do not isolate + secrets or hostile code from model tools. - **Compatibility:** Existing workspace files remain valid because new fields - are optional. Gateway startup applies stricter repository-catalog rules. - Older binaries reject the new strict nested profile field, so docs state the - minimum supporting version. + are optional; request access defaults to `readWrite`. Gateway startup applies + stricter catalog, cache-root, materializer, and provider-readiness rules. + CLI/gateway version skew is governed by contract ranges; gateway/image + compatibility is exact by manifest digest and acquisition-contract version. ### Risks and Mitigations +- **A2A or provider-surface immaturity:** Pin exact JavaScript package versions + and run U0 wire/provider probes before production units. If the Codex SDK + lacks a required capability, document proof before selecting pinned app-server; + if neither works, the target is unavailable rather than silently scraped. +- **Contract drift:** Generate public and private schemas plus accepted/rejected + fixtures from the three narrow packages and run drift checks in the gateway, + acquirer, docs, and consumer fixtures. +- **Install-size regression:** Keep CLI and gateway workspace dependency graphs + separate, report packed/installed sizes, enforce budgets, and fail CLI-only + smoke if it resolves the gateway or acquisition image. - **Accidental network exposure:** Binding `0.0.0.0` is intentional and allowed; require a distinct advertised URL, use HTTPS in production, and state in - startup output/docs that every reachable host has full authority. + startup output/docs that every reachable peer has full authority. - **Profile identity drift:** Derive targets only from current validated user declarations and matching installed state; never resurrect declaration-missing launchers from retained profile state. -- **Credential leakage or path swap:** Use fresh validated/revoked App tokens or - one configured `gh` account, descriptor-bound credential handles, hermetic - Git, strict Docker auth/helper protocols, and credential teardown before - publication. Non-bypassable helper spawn mediation replaces environments, - closes descriptors, and enters role-specific mount/network namespaces before - every MCP or model-tool exec; a backend lacking that hook is unavailable. +- **Working-directory escape or mutable cross-trial reuse:** Accept only the + closed logical selector and `RelativeDirectory` grammar, resolve through the + compiled catalog, and require an existing directory beneath the selected + repository. Read-only Tasks share only the gateway-managed immutable base and + keep private runtime state; read-write views derive from Task IDs. Never expose + or accept a resolved host path. +- **Read-only contract violated by the prompt or provider:** Do not inspect + prompts or claim a sandbox. Disable optional Git locks, request native provider + read-only policy when available, isolate runtime writes, and document that + consumers must choose `readWrite` when project mutation is required. Do not + add a per-Task mount, chmod traversal, or full-tree verification in v1. A + violating provider can contaminate the base and later Tasks; the operator must + evict that entry before reuse. +- **Large workspace duplication or unsupported copy-on-write:** Acquire each + immutable identity once, pin shared bases, prefer block clone, fall back to + rootless OverlayFS, and retain explicit `copy` for portability. `auto` refuses + a full copy above its byte ceiling; startup reports capabilities, and CI users + provision enough disk or choose a larger/self-hosted runner. +- **Base-cache corruption or unbounded growth:** Bind keys to immutable source, + catalog/layout, and acquisition-contract identity; publish atomically; keep + roots private; pin active entries; evict only unpinned least-recently-used + entries under a byte budget; and stop admission on detected metadata or + filesystem inconsistency. This is trusted-runner state, not a hostile-process + integrity boundary. +- **Acquisition credential leakage:** Mount only staging, inject only the + selected source credential and exact-host CA material, never mount host home + or Docker socket, remove the container before provider execution, and scan + the manifest/staging/logs for gateway-managed credential values. - **Identity-changing fallback:** Classify App applicability as eligible, ineligible, or unknown; require repository-existence proof for 404 ineligibility; only positive ineligibility permits `gh`. - **OCI registry/archive abuse:** Require immutable digests, a closed - manifest/config/layer profile, same-origin metadata, exact operator-approved - layer-redirect hosts with per-hop address validation, changeset semantics, - fixed extraction limits, safe paths/types/links, and exact project-catalog - manifest verification. -- **Untrusted acquired code:** General hostile-code sandboxing beyond the - declared Linux process/network namespace and secret boundary is not claimed. - Invocation routes deny gateway, host loopback, and management networks; - provider/MCP egress is allowlisted; and model tools cannot reach provider/MCP/ - operator credentials or gateway state. Project/user setup shell commands are - never automatic. -- **Evidence-time attacks:** Prove containment empty first, then use - descriptor-relative no-follow reads with identity/size revalidation, reject - Git metadata indirection, and disable repository-controlled Git execution. -- **Provider/API churn:** Pin compatible SDK/CLI/model versions and retain - versioned native fixtures plus one adapter conformance suite. Gate Codex native - schemas to the pinned Structured Outputs subset and backend availability to a - proven non-bypassable spawn hook. -- **Orphaned processes:** Persist a stable empty containment identity before - start-gate release; enumerate the full project-owned cgroup namespace on - startup. On uncertain quiescence, stay alive, reject admission, and continue - reaping until empty without mutating the settled Task. -- **Store corruption or disclosure:** Route SQLite and all sidecars through the - helper's descriptor-rooted no-follow VFS with full synchronization and - transactions; validate ownership, modes, links, root disjointness, lock, and - workspace identity. Integrity/durability failure stops admission and prevents - terminal success. + manifest/config/layer profile, exact host/redirect policy, changeset + semantics, fixed extraction limits, safe paths/types/links, and exact catalog + validation inside the image and again at the host publication boundary. +- **Untrusted provider execution:** The acquired workspace and model tools run + with the same authority as the trusted CI job. Mitigate by using ephemeral + runners or an operator-managed VM/container boundary, least-privilege CI + credentials, explicit provider environments, no automatic setup commands, + and clear documentation. Do not describe AllAgents as a sandbox. +- **Evidence overclaim:** Collect only after the direct provider process settles, + keep evidence bounded, record process-group signals and observed cleanup, and + explicitly avoid claiming full descendant quiescence or output redaction. +- **Provider/API churn:** Pin SDK/package/protocol/model compatibility, require + exact probes for binary overrides, retain native fixtures, and share one + adapter conformance suite. Never download a provider runtime per request. +- **Orphaned processes:** Use a new Linux process group per direct provider, + persist the leader identity, escalate abort to `SIGTERM` and `SIGKILL`, and + rely on CI runner teardown as the final orphan boundary. +- **Store corruption or disclosure:** Use a current-user private state root, + exclusive gateway lock, ordinary Bun SQLite transactions, foreign keys, + `synchronous=FULL`, integrity checks, and bounded data. Integrity/durability + failure stops admission and prevents false success. +- **Artifact mismatch:** Bind every release report to the exact gateway npm + tarball digest and acquisition manifest/platform digests. Reject rebuilt, + mutable-tagged, wrong-commit, or contract-incompatible substitutes. ### Assumptions - The initial deployment is one gateway process and one transactionally enforced - active invocation. + active invocation on a trusted Linux CI runner. - Every external network peer able to connect is trusted with all available - targets, including built-ins and gateway-enabled profiles, and all retained - Tasks. Invocation descendants are deliberately unable to reach that network - boundary. + targets and retained Tasks. +- The CI job, VM, or deployment container is the isolation boundary. AllAgents + does not isolate hostile repository code, provider credentials, MCP secrets, + or host network access from model-invoked tools. - The selected project workspace is operator-controlled and compiles to 1-64 uniquely named GitHub repositories with collision-free destinations. - GitHub.com is the only authenticated Git host in the initial delivery. -- OCI snapshots use HTTPS registries and the frozen v1 direct-image format. -- Codex and Pi are available only when their pinned automation surfaces support - non-bypassable helper-mediated tool and MCP spawning. -- Gateway v1 execution supports Linux x64/arm64 hosts with cgroup v2, `clone3`, - pidfds, `openat2`, mount/network namespaces, nftables, and delegated - permissions. +- OCI snapshots use HTTPS Docker Hub, GHCR, JFrog Artifactory/JCR, or compatible + private OCI registries and the frozen v1 direct-image format. +- Docker is available solely for base-acquisition containers when no reusable + validated base exists, and the operator-provided acquisition network enforces + the deployment's source egress boundary. Immutable repository cache reuse + requires full commit IDs. +- Codex and Pi are installed or provided by pinned workspace dependencies before + gateway start and can reuse their existing host authentication locations. +- The implementation units after U0 assume the Bun/A2A/provider/process/acquirer + feasibility gates passed. A failed provider probe disables that target; a + failed architecture or release-binding gate stops the affected release rather + than introducing Rust, native platform packages, or a split runtime. --- ## Implementation Units -### U1. Workspace, extension, and manifest contracts - -- **Goal:** Freeze configuration, packaging, safe state primitives, and every - versioned public/private contract before runtime implementation. -- **Requirements:** R1, R2, R3, R5, R6, R7, R8, R9, R11, R18; AE3, AE4, AE5, - AE7, AE8, AE9, AE13, AE14, AE16, AE20; KTD1, KTD2, KTD5, KTD6, KTD7, - KTD10, KTD11. -- **Files:** root `package.json`/build/typecheck configuration, - `packages/execution-service/package.json` and TypeScript config, Rust - `packages/execution-helper`, Linux x64/arm64 optional packages, typed helper - protocol, SQLite schema/migrations and descriptor-rooted VFS, - `scripts/release.ts`, `scripts/publish.ts`, `.github/workflows/publish.yml`, - `src/models/workspace-config.ts`, schema generation tests and generated public - schemas, execution-service contracts, - `docs/src/pages/a2a/extensions/coding-execution/v1.astro` at the exact - declared URI plus a generated schema asset beneath that route, a versioned - snapshot-format specification, deterministic reference packer/conformance - fixtures, and configuration docs. -- **Approach:** Declare the Bun workspace and root `workspace:*` development - edge so the private service is installed, checked, and bundled. Establish the - helper protocol and audited SQLite VFS before the server store client. Version - helper packages with matching root optional-dependency ranges; publish and - verify both platform packages before the root package. Verify a clean registry - install resolves the matching helper binary and checksum and that the packed - root manifest contains no workspace protocol. - - Add strict named `workspaceSnapshots` with exact layer-redirect hosts and - nested profile-client `gateway.enabled`; preserve ordinary project/user - parsing while compiling gateway repository and target catalogs. Publish Agent - Card params, version/header activation, Message metadata/extensions, unified - Parts, exact result-schema grammar, source union, deadline, idempotency/replay, - HTTP+JSON errors, integrity/produced Artifacts, workspace manifest, OCI media/ - change-set/limit profile, and canonical digest preimages from Zod. -- **Execution note:** Start with independent wire fixtures that use only the - published extension specification. Reject missing version/activation, cross- - variant/unknown fields, extra Message Parts, undeclared names, mutable - snapshot references, malformed digests, invalid deadlines, incomplete or - mismatched manifests, gateway enablement without launcher, built-in - collisions, and unsupported clients while preserving unrelated metadata. - Fault-inject database/WAL/SHM link and rename swaps through the real VFS. -- **Verification:** Focused workspace-schema, packaging, helper VFS, and - contract tests; generated schema/spec drift checks; representative YAML, HTTP - errors, and wire examples parse through runtime schemas; snapshot conformance, - canonicalization, Artifact-cardinality, clean-registry install, matching - helper version/checksum, and ordered publish dry-run fixtures pass. +### U0. Bun monorepo, provider, process, and acquirer feasibility + +- **Goal:** Prove the settled Bun architecture can preserve A2A behavior, + independent distribution, supported provider control, Linux cancellation, + read-only shared-base execution, read-write materialization, and exact + acquisition-image release binding before production implementation. +- **Requirements:** R1-R2, R8, R13-R16, R18; AE8-AE11, AE17-AE18, AE20; + KTD1-KTD3, KTD6-KTD7, KTD9, KTD11-KTD14. +- **Files:** private root `package.json`/`bun.lock`, `apps/cli`, + `apps/gateway`, `apps/acquirer`, the three named `packages/` workspaces, + representative generated fixtures under `contracts/`, acquisition Dockerfile/ + image metadata, provider and workspace-materializer feasibility probes, + process-group probe, independent CLI/gateway pack scripts, and gateway release + workflow skeleton. +- **Approach:** Move the existing CLI into `apps/cli` without changing its + public package or behavior. Establish `apps/gateway` as the separately packed + Bun executable package and `apps/acquirer` as image-only code. Pin the + official A2A JavaScript SDK and drive a minimal production-direction server + through every required operation. Probe `@openai/codex-sdk` for invocation, + events, native abort, usage, structured-output support, and existing + `CODEX_HOME` behavior; consider app-server only when a named required + capability is proven absent. Probe the supported Pi package/RPC surface for + invocation, events, abort, usage, and existing host auth. Prove exact + compatibility rejection for global binary overrides. + + Run a real Linux child in a new process group and demonstrate graceful abort, + `SIGTERM`, and `SIGKILL` escalation plus the limit that unrelated/escaped + descendants are not proven gone. Prove one immutable base can serve repeated + read-only Tasks with private runtime state; probe block cloning and rootless + OverlayFS; verify independent writable changes and removal; and prove explicit + copy behavior plus the automatic copy ceiling. Build the acquirer image for + every supported architecture, run it with only a staging mount and synthetic + source secret, verify typed manifest output and container removal, and prove + the image has no provider runtime or Docker socket. Pack CLI and gateway + separately, prove a CLI-only install fetches neither gateway nor image, and + define the immutable gateway-tarball/acquisition-manifest release record. +- **Execution note:** U0 is a feasibility gate, not partial production + scaffolding. Do not paper over missing SDK/RPC behavior with TUI scraping, + AI SDK Harnesses, per-request downloads, per-provider Docker, or native + containment machinery. A missing provider capability disables that provider; + failed A2A, process, acquisition, or release-binding feasibility returns the + affected design for revision before dependent units. +- **Verification:** Official JavaScript client fixtures pass against the Bun + server; provider probes record exact pinned versions and auth-path behavior; + shared read-only base, private runtime, reflink, rootless-overlay, explicit + copy, cleanup, environment, and process-group probes pass on Linux; multi- + architecture image manifests/digests are recorded and the image boundary + rejects extra mounts/credentials/network; independent packed CLI/gateway + installs and compatibility fixtures pass; CLI-only installation fetches + neither gateway nor acquisition image. + +### U1. Workspace packages, contracts, SQLite, and release foundation + +- **Goal:** Freeze the monorepo ownership, workspace configuration, public and + acquisition contracts, ordinary SQLite transactions, and exact release + artifact binding before runtime implementation. +- **Requirements:** R1-R3, R5-R9, R11-R12, R18; AE3-AE9, AE13-AE14, AE16, + AE20; KTD1-KTD2, KTD5-KTD8, KTD10, KTD13-KTD14. +- **Files:** `packages/workspace-config`, `packages/execution-contracts`, + `packages/acquisition-contracts`, generated `contracts/` schemas and golden + examples, gateway SQLite schema/migrations, release scripts/workflows, + deterministic snapshot producer/conformance fixture, published extension and + snapshot-format assets, and configuration docs. +- **Approach:** Move authoritative project/user parsing and gateway catalog + compilation into `workspace-config`; add strict named `workspaceSnapshots`, + exact redirect hosts, and nested `gateway.enabled` without changing ordinary + CLI behavior. Define execution contracts for Agent Card params, version/header + activation, Message metadata/extensions, unified Parts, source union, logical + working-directory union and relative-path grammar, workspace access/default, + result-schema grammar, deadline, idempotency/replay, materialization errors, + HTTP errors, integrity/produced Artifacts, adapter events/results, and + evidence. Define acquisition contracts for the closed request, path-free typed + manifest, immutable-base cache key, private compiled-layout checks, OCI + media/change-set profile, fixed limits, and canonical digests. Generate + portable accepted/rejected fixtures beneath `contracts/`. + + Add private `bun:sqlite` ownership with foreign keys, WAL where supported, + `synchronous=FULL`, migrations, one execution lease, `createOrReplay`, + immutable-base metadata and active pins, internal outcome intent, atomic + settlement, transactional expiry, and recorded acquisition-container/provider- + process identities. Establish independent CLI/gateway versions and release + triggers. The gateway release record binds the exact npm tarball digest to the + acquirer multi-architecture manifest and supported platform digests; the image + is verified before npm publication. +- **Execution note:** Do not add `core`, `common`, a custom VFS, native file + primitives, native/platform npm packages, or runtime compatibility shims. + Start with external wire/manifest fixtures and stable rejection codes. Fault + SQLite transactions and process exit around commit/acknowledgment boundaries, + not filesystem attacks the ordinary SQLite contract does not claim to defeat. +- **Verification:** Workspace parsing/catalog fixtures, generated-schema drift, + wire/manifest accepted/rejected examples, canonicalization, Artifact + cardinality, SQLite commit/replay/lease/settlement/expiry/crash fixtures, + independent package versions, CLI-only and gateway clean-registry installs, + compatibility skew/image-mismatch matrix, exact tarball/image release record, + and idempotent absent/identical/divergent publication fixtures pass. ### U2. Deployment-wide Task store and A2A server - **Goal:** Serve the A2A lifecycle without application authentication and keep durable deployment-wide Task/idempotency truth behind a fake backend. -- **Requirements:** R1, R2, R3, R4, R5, R8, R13, R16, R17, R18; AE1, AE2, AE9, - AE11, AE14, AE15, AE16, AE19, AE20; KTD1, KTD2, KTD3, KTD4, KTD9, KTD10. -- **Files:** typed Task-store client, Agent Card, AllAgents request handler, - HTTP+JSON/SSE server, pagination/retention, minimal backend interface and - registry, fake adapter, health/readiness, CLI gateway command, focused tests. -- **Approach:** Implement flags/env precedence, bind/advertised-URL separation, - private project state and lock, helper-owned SQLite full-sync transactions, - startup integrity and full containment-namespace reconciliation, A2A version - and extension negotiation, exact `SendMessage` modes and `ListTasks` - semantics, standard/custom `google.rpc.Status` errors, durable - `createOrReplay`, one execution lease, internal outcome intent plus atomic - terminal settlement, bounded events/Artifact bytes, no early eviction, - transactional expiry, global listing/cancellation, deadline handling, and - fail-closed graceful shutdown against the fake adapter. -- **Execution note:** Prove with the official A2A client that one external caller - can read and cancel another caller's Task; this is expected behavior. Kill - subprocesses after transaction write/sync/commit/response boundaries and - fault-inject helper/VFS I/O, capacity races, cancellation intent, Artifact, - and terminal settlement. -- **Verification:** A2A discovery/send modes/stream/get/full list/subscribe/ - cancel/replay/expiry and HTTP-error integration tests on loopback plus explicit - `0.0.0.0`/advertised URL; health/readiness, state-path, retained and active - capacity, crash/store-fault, competing-lock, deadline, shutdown, and restart - tests. - -### U3. Invocation supervisor and backend contract - -- **Goal:** Run one fake-backed invocation through containment, typed - preparation, evidence, terminal arbitration, and cleanup with truthful - outcomes before real acquisition/adapters. -- **Requirements:** R3, R5, R8, R13, R14, R15, R16; AE9, AE10, AE11, AE12, - AE14, AE15, AE16, AE17, AE18; KTD3, KTD9, KTD10, KTD11, KTD12. -- **Files:** security/state helper extensions, provider/MCP/tool view and egress - compiler, invocation state machine, containment/start-gate controller, spawn - broker, typed preparation, evidence collector, result validator, - cleanup/reaper, and lifecycle tests. -- **Approach:** Extend the U1 helper to allocate an empty cgroup with a stable ID - and start gate, commit Task+lease+containment before release, and enumerate - recorded and unknown cgroups on startup. Launch every child into the cgroup; - mediate every backend MCP/tool spawn; enter role-specific mount and network - namespaces; replace environments; close descriptors; apply nftables egress - policy; use pidfds for termination/wait; and expose safe file operations. - Resolve targets through U2's typed fake adapter; never execute generated - launchers or setup commands. Commit one internal intent across provider, - cancel, deadline, and shutdown; capture live events; prove quiescence before - filesystem evidence; atomically settle status, evidence, Artifacts, cleanup, - and lease release; remain alive to reap when poisoned without mutating the - settled Task. -- **Execution note:** Fault-inject every boundary: capacity races and restart; - process death before/after empty-set creation, Task binding, child clone, and - start-gate release; pairwise and three-way outcome races; child fork/escape; - helper protocol/version/package mismatch; provider/MCP/tool attempts to reach - Agent Card, ListTasks, GetTask, SendMessage, CancelTask, host loopback, and - management networks; environment/path/inherited-FD/`/proc`/magic-link secret - reads by real child and grandchild processes; output truncation, malicious - evidence, valid-result-then-evidence-failure, and unknown cleanup. -- **Verification:** Deterministic lifecycle, single execution lease, helper - packaging/checksum, cgroup/pidfd/mount/network namespace containment, - non-bypassable spawn mediation, separate secret/descriptor/egress views, - typed preparation, safe-file/evidence, unknown-cgroup reconciliation, and - poison/reaping tests plus real child-process smoke on Linux x64/arm64 CI. - -### U4. Git and OCI workspace acquisition - -- **Goal:** Materialize declared repository sets and named OCI snapshots into the - same validated invocation workspace through the U3 security helper. -- **Requirements:** R6, R9, R10, R11, R12, R15, R16, R18; AE5, AE6, AE7, - AE8, AE10, AE15, AE17, AE18; KTD6, KTD7, KTD8, KTD11, KTD12. -- **Files:** acquisition coordinator, Git transport, GitHub provider selection, - strict Docker-auth/helper resolver, OCI Distribution client and changeset - applier, workspace-manifest validator, staging/publication helper, fixtures and - tests. -- **Approach:** Resolve name-based requests from the compiled project catalog. - Implement hermetic Git and full-commit verification. Apply the exact App - eligibility proof table, bypass token cache, validate/revoke each fresh token, - permit `gh` only for positive ineligibility, and use descriptor-bound temporary - helpers. Implement anonymous-first bounded Bearer authentication, redirect/ - credential-origin rules, the frozen direct-image media profile, streaming - descriptor verification, gzip/zstd changeset and whiteout semantics, all - extraction ceilings, exact project-manifest validation, and atomic publication. - Tear down every acquisition credential before typed preparation. -- **Execution note:** Use local Git remotes and a local OCI registry plus the U1 - producer fixture. Prove ambiguous/selected-App failures never call `gh`, two - sequential acquisitions mint distinct tokens, token validation/revocation and - lifetime are enforced, helper/auth-file swaps fail, and snapshot failure never - invokes Git fallback. -- **Verification:** Three-way provider-selection and real-response fixture tests; - Git branch/tag/full-commit integration; GHCR/Docker Hub helper fixtures; - malicious realm/scope/downgrade/redirect tests; OCI index/media/digest/size/ - limit/order/whiteout/path/catalog fixtures; credential leak scans; equivalent - complete manifest output across both acquisition modes. - -### U5. Codex backend adapter - -- **Goal:** Run built-in and profile-backed Codex targets through the supported - SDK while preserving structured progress, result, usage, cancellation, and - native evidence. -- **Requirements:** R7, R8, R13, R14, R15, R16; AE1, AE3, AE4, AE10, AE12, - AE15, AE17, AE18; KTD9, KTD11, KTD12. -- **Files:** Codex adapter, profile-context and auth bridge, fixtures, - conformance and optional credentialed smoke tests. -- **Approach:** Pin SDK/model compatibility and first prove a non-bypassable - synchronous hook that delegates every MCP and model-tool spawn to the U3 - helper. If the pinned Codex surface can bypass that hook, Codex is unavailable - in v1 rather than relying on an asserted view. Create one fresh thread per - Task; pass cwd, typed profile configuration, abort signal, and the private - Codex control-process auth view inside containment. Pass native `outputSchema` - only for the pinned Structured Outputs subset; otherwise add JSON guidance and - use the common terminal validator. Normalize events/usage, bound evidence, and - dispose fully. -- **Execution note:** Characterize the pinned SDK/model's spawn, schema, tool- - sandbox, auth, abort, and event behavior with captured fixtures before - normalization. Do not import Promptfoo provider code. -- **Verification:** Shared adapter conformance, real SDK child/grandchild spawn - mediation, filesystem/environment/inherited-FD/`/proc` credential denial, - gateway/host-network denial, native-schema and validated-fallback paths, - deadline, and an opt-in credentialed smoke case. - -### U6. Pi backend adapter - -- **Goal:** Run built-in and profile-backed Pi targets through strict RPC with the - same public lifecycle and honest capability reporting. -- **Requirements:** R7, R8, R13, R14, R15, R16; AE3, AE4, AE10, AE12, AE15, - AE17, AE18; KTD9, KTD11, KTD12. -- **Files:** Pi adapter, RPC parser, restricted policy extension, profile-context - and auth bridge, fixtures, conformance and optional credentialed smoke tests. -- **Approach:** First prove strict RPC exposes a non-bypassable synchronous hook - that delegates every MCP and model-tool spawn to the U3 helper. If Pi can - bypass that hook, Pi is unavailable in v1. Launch Pi with typed invocation - configuration, its private control-process auth view, strict JSONL RPC, - explicit allowed tools/extensions, per-MCP secret declarations, - deterministic permissions, event validation, deadline/cancellation - escalation, and settled completion. Repository extensions and unrestricted - built-ins remain disabled. -- **Execution note:** Characterize and pin Pi's spawn/RPC contract; record - Pi-specific facts as bounded native evidence rather than public schema - branches. -- **Verification:** Shared adapter conformance, real RPC child/grandchild spawn - mediation, filesystem/environment/inherited-FD/`/proc` provider/MCP secret - denial, gateway/host-network denial, malformed/unknown RPC, deadline, and an - opt-in credentialed smoke case. +- **Requirements:** R1-R5, R8, R13, R16-R18; AE1-AE2, AE9, AE11-AE16, + AE19-AE20; KTD1-KTD4, KTD9-KTD10. +- **Files:** `apps/gateway` Task-store module, Agent Card, A2A request handler, + HTTP+JSON/SSE server, pagination/retention, backend registry/fake adapter, + health/readiness, `allagents-gateway` command, and focused integration tests. +- **Approach:** Implement flags/environment precedence, bind/advertised-URL + separation, private state/lock, SQLite transactions, startup integrity and + interrupted-Task reconciliation, A2A version/extension negotiation, exact + `SendMessage` modes and `ListTasks` semantics, standard/custom + `google.rpc.Status` errors, durable `createOrReplay`, one execution lease, + internal outcome intent plus atomic terminal settlement, bounded + events/Artifact bytes, no early eviction, transactional expiry, deployment- + wide listing/cancellation, deadline handling, and graceful shutdown against a + fake adapter. +- **Execution note:** Use an independent official JavaScript A2A client to prove + one external caller can read and cancel another caller's Task; that is expected + trusted-network behavior. Kill gateway subprocesses around SQLite transaction, + commit, acknowledgment, cancellation-intent, Artifact, and settlement + boundaries. Do not add caller ownership or an application credential. +- **Verification:** Discovery, both send modes, stream/get/full list/subscribe/ + cancel/replay/expiry, HTTP errors, loopback and explicit + `0.0.0.0`/advertised URL, probes, retained/active capacity, competing lock, + SQLite crash/fault, deadline, shutdown, restart, and fake-backend tests pass. + +### U3. Host process supervisor and backend contract + +- **Goal:** Run fake-backed direct host invocations through shared read-only and + independent read-write workspace selection, logical cwd resolution, typed + preparation, explicit environment construction, process-group cancellation, + evidence, terminal arbitration, and cleanup with truthful limits before real + adapters. +- **Requirements:** R3, R5, R8, R13-R16, R18; AE8-AE12, AE14-AE18; + KTD3, KTD6, KTD9-KTD12. +- **Files:** `apps/gateway` backend types/registry, immutable-base manager, + workspace materializer, provider environment builder, Linux process-group + supervisor, invocation state machine, typed preparation, evidence collector, + result validator, cleanup/restart reconciliation, fake process fixtures, and + lifecycle tests. +- **Approach:** Define the minimal adapter contract for availability, + capabilities, access-aware invoke/events, graceful abort, direct-process + settlement, result/usage/evidence, and disposal. Resolve fake targets without + executing generated launchers or setup commands. For read-only, resolve cwd in + immutable base and allocate private runtime state. For read-write, materialize + a Task-ID-derived view via block clone, rootless OverlayFS, + or explicit copy. Resolve workspace-root and repository-relative selectors, + reject missing/non-directory/escaping paths, and pass only the effective cwd, + runtime paths, and access mode to the adapter. Start each direct provider in a + new process group, persist its leader PID and process-start marker before + marking execution started, and build its environment from a reviewed allowlist + that preserves required host identity/auth paths. Commit one internal intent + across provider terminal, cancel, deadline, and shutdown. Escalate adapter + abort to process-group `SIGTERM` and `SIGKILL`; capture bounded live events; + collect filesystem/Git evidence only after the direct process settles; remove + Task runtime or writable view; and atomically settle status, evidence, + Artifacts, observed termination, cleanup, and lease release. The non-settling + path emits only termination failure and live evidence, retains Task-owned + state plus lease, and blocks admission until verified reconciliation. +- **Execution note:** Fixtures must distinguish what AllAgents observes from what + it cannot guarantee. Exercise child and grandchild processes, including one + that escapes or outlives the direct process, and assert the gateway never + labels process-group cleanup as complete descendant quiescence. The CI runner + teardown is the final orphan boundary. No cgroups, pidfds, namespaces, + nftables, `openat2`, spawn broker, provider container, or isolation claim. +- **Verification:** Deterministic lifecycle; shared-base reuse without shared + runtime state; adapter-native read-only policy where available; independent + reflink, rootless-overlay, and copy views; automatic copy ceiling; cwd + resolution and escape rejection; single lease; explicit environment + inclusion/exclusion; + required host-auth preservation; binary-override compatibility rejection; + graceful/TERM/KILL timing; cancellation/deadline/shutdown races; poisoned- + lease behavior; post-teardown reconciliation; evidence ordering; result + states; cleanup outcomes; and truthful orphan-limit fixtures pass on trusted + Linux CI. + +### U4. Docker-only Git and OCI immutable-base acquisition + +- **Goal:** Materialize declared repository sets and named OCI snapshots when no + reusable validated base exists, validate and promote bases on the host, and + prove reuse and non-reusable cleanup without placing providers in Docker. +- **Requirements:** R6, R8-R12, R15-R16, R18; AE5-AE11, AE15, AE17-AE19; + KTD3, KTD6-KTD8, KTD10-KTD14. +- **Files:** `apps/acquirer` Git/OCI implementations and entrypoint, + `packages/acquisition-contracts`, `apps/gateway` Docker coordinator, + immutable-base cache/pin/eviction manager and host staging/manifest validator, + deterministic producer fixture, local/GHCR/JFrog fixtures, reusable registry- + conformance workflow, and focused tests. +- **Approach:** Derive cacheability and keys from the compiled catalog, + acquisition-contract version, layout digest, and immutable source identity. + A valid hit pins the base and starts no container or credential flow. On a + miss, the host creates private staging, resolves only the selected source + credential, and starts the exact digest-pinned image with staging as its sole + writable bind, no host home, no Docker socket, and per-request source policy. + In repository mode the host coordinator implements the App eligibility table, + mints and injects only the fresh repository-scoped token, validates/revokes it, + selects `gh` only for positive ineligibility, and never falls back after + selected-provider failure. Branch/tag requests bypass reusable bases. In + snapshot mode implement anonymous-first bounded Basic/Bearer authentication, + canonical Docker Hub normalization, exact-host CA/realm/redirect rules, + direct-image media profile, streaming digest verification, gzip/zstd + changesets/whiteouts, fixed limits, and path-free manifest/private layout + checks. + + The acquirer emits only typed manifest and staging content, then exits. The + gateway removes it, destroys source material, validates manifest, limits, and + exact catalog again on the host, and atomically publishes the base. Private- + root, pinning, budgeted unpinned-LRU eviction, and + restart fixtures cover cache lifecycle. Image probes prove no Codex/Pi/harness, + provider auth, host home, gateway state, or Docker control reaches acquisition. + Snapshot failure never invokes Git fallback. +- **Execution note:** Every PR runs local Git, local Distribution, and live + public digest-pinned GHCR against the exact built image. Release conformance + reuses the exact gateway npm tarball plus multi-architecture acquisition + manifest/platform digests without rebuilding. Authenticated GHCR uses least- + privilege pull credentials; pinned JFrog JCR uses HTTPS/private CA/private + repository/pull-only identity. Run platform-specific cases only where the + registry/runner supports that architecture and record coverage explicitly. +- **Verification:** App three-way selection, base-acquisition token lifetime/ + validation/revocation, cache-hit no-credential/no-container behavior, `gh` + fallback, Git revisions, immutable key invalidation, pin/eviction/restart, + non-reusable branch/tag base cleanup, Docker mount/env/network/credential/limit + enforcement, container and orphan-staging removal, + host revalidation/atomic publication, OCI auth/realm/redirect/CA/media/digest/ + size/whiteout/path/catalog cases, clean leak scans, equivalent typed manifests, + and exact local/public GHCR/authenticated GHCR/private-CA JFrog reports pass. + +### U5. Codex SDK adapter + +- **Goal:** Run built-in and profile-backed Codex targets on the trusted host + through the pinned SDK while preserving progress, result, usage, cancellation, + existing authentication, and truthful evidence. +- **Requirements:** R7-R8, R13-R16, R18; AE1, AE3-AE4, AE10-AE12, + AE15, AE17-AE18; KTD9, KTD11-KTD12. +- **Files:** `apps/gateway` Codex adapter, typed profile projection, environment + policy, SDK fixtures, shared conformance tests, and optional credentialed + smoke tests. +- **Approach:** Use pinned `@openai/codex-sdk` first. Create one fresh execution + context per Task; pass the resolved cwd, access mode, Task-private runtime + paths, and typed profile settings. For `readOnly`, request the native read-only + policy when supported and keep preparation outside the base. Preserve the + existing host `CODEX_HOME`/ChatGPT login when API credentials are absent; + stream/normalize events and usage; connect native abort to U3; bound evidence; + and dispose. + Use app-server only if U0 recorded a specific required SDK gap and pin/probe + its protocol. + Pass native `outputSchema` only for the supported Structured Outputs subset; + otherwise add JSON guidance and use the common terminal validator. +- **Execution note:** Characterize pinned SDK/model auth, abort, event, tool, and + schema behavior before normalization. Provider and model tools retain trusted + CI-job authority; tests inspect the explicit environment but make no hostile- + code, network, or secret-isolation claim. Do not import Promptfoo or AI SDK + Harnesses and do not download Codex per request. +- **Verification:** Shared adapter conformance; built-in/profile targets; + existing `CODEX_HOME` and API-credential paths; environment allowlist; exact + override probe; event/usage/result normalization; graceful/TERM/KILL + cancellation; native-schema and validated-fallback paths; deadline; malformed + provider payload; and opt-in credentialed smoke pass outside Docker. + +### U6. Pi RPC adapter + +- **Goal:** Run built-in and profile-backed Pi targets on the trusted host through + the pinned supported package/RPC surface with the same public lifecycle and + honest capability reporting. +- **Requirements:** R7-R8, R13-R16, R18; AE3-AE4, AE10-AE12, AE15, + AE17-AE18; KTD9, KTD11-KTD12. +- **Files:** `apps/gateway` Pi adapter/RPC parser, restricted policy extension, + typed profile projection, environment policy, fixtures, shared conformance + tests, and optional credentialed smoke tests. +- **Approach:** Launch Pi directly in the resolved cwd with access mode, + Task-private runtime paths, typed invocation configuration, existing host Pi + authentication location, strict RPC, explicit supported tools/extensions, + deterministic permissions, validated events, bounded evidence, and U3 + cancellation escalation. Request a native read-only policy when supported. + Never copy, mount, parse, or import Pi auth. + Repository extensions and unrestricted built-ins remain disabled. A global + Pi binary override must pass the exact pinned version/protocol probe. +- **Execution note:** Characterize and pin Pi's RPC/auth/abort/event contract. + Pi-specific facts remain bounded native evidence rather than public schema + branches. Model tools retain trusted CI-job authority; do not claim the + explicit environment isolates provider/MCP/operator secrets. +- **Verification:** Shared adapter conformance; built-in/profile targets; + existing host auth; environment allowlist; exact override probe; strict + malformed/unknown RPC rejection; event/usage/result normalization; graceful/ + TERM/KILL cancellation; deadline; and opt-in credentialed smoke pass outside + Docker. Malformed RPC can never produce success. ### U7. End-to-end delivery and documentation -- **Goal:** Prove the bundled/packed CLI and document the trusted-network - operating model, Linux requirements, workspace configuration, credentials, - sources, Promptfoo consumption, and risks. -- **Requirements:** R1-R19; F1-F6; AE1-AE21. -- **Files:** published extension and snapshot-format pages, gateway guide/ - reference, configuration reference, README, CHANGELOG, real project/user - workspaces, AI Evals-style Promptfoo YAML and custom-provider contract fixture, - E2E fixtures, packed-install smoke, release evidence. -- **Approach:** After final implementation review, build and pack the CLI plus - both helper packages; install in a clean Linux environment; create project and - user workspaces under `/tmp/`; gateway-enable fixture targets; serve on - loopback and `0.0.0.0` with a valid advertised URL; exercise health/readiness; - acquire local Git and OCI fixtures; and run an independently generated - official A2A client through version/extension negotiation, errors, both send - modes, complete listing, success, replay, cancellation, deadline, shutdown, - restart, and expiry. Run the custom-provider fixture through repository and - snapshot invocations with secure Promptfoo defaults, proving requests contain - only logical source data while the gateway resolves origins. Document full - network-peer authority, sensitive opaque payloads, and process-only secrets. -- **Execution note:** Green smoke uses the same built command and `/tmp/` - workspace shape as red E2E, never a test-only server. The consumer fixture is - AI Evals-style test/documentation code; AllAgents runtime does not import - Promptfoo. -- **Verification:** `bun run build`, packed-install/helper checksum smoke, - focused and full tests, typecheck, lint, docs build, extension/schema drift, - custom-provider contract fixture, and exact red/green commands/results in the - PR description. +- **Goal:** Prove independently released Bun CLI/gateway packages and the exact + acquisition image, then document the trusted-network and trusted-runner model, + workspace/source configuration, host auth, registry coverage, Promptfoo + consumption, installation, release ordering, and limits. +- **Requirements:** R1-R19; F1-F6; AE1-AE21; KTD1-KTD14. +- **Files:** published extension/snapshot-format pages, gateway guide/reference, + configuration reference, README, CHANGELOGs, real project/user workspaces, + AI Evals-style Promptfoo YAML/provider contract fixture, E2E fixtures, + CLI-only and gateway packed-install smokes, acquisition-image release record, + GHCR/JFrog reports, size/SBOM evidence, and independent release evidence. +- **Approach:** After final review, pack `apps/cli` and `apps/gateway` + independently without publishing. Prove CLI-only installation resolves + neither gateway nor image; install the gateway tarball in a clean trusted + Linux environment with Docker and pre-existing Codex/Pi host auth. Create + project/user workspaces under `/tmp/`; serve on loopback and `0.0.0.0`; test + probes and the complete A2A lifecycle; acquire local Git/OCI plus live registry + fixtures through the exact image; and run Codex/Pi on the host. Exercise the + Promptfoo consumer fixture in both source modes with per-trial logical cwd and + workspace access. Prove read-only Tasks reuse one immutable base without + shared runtime state, read-write Tasks receive independent disposable views, + and requests carry only logical source, cwd, and access data. + + Run registry workflows with the exact gateway tarball, acquisition manifest, + supported platform digests, and build commit. Gateway publication is blocked + until the image has passed required GHCR/JFrog conformance. Document that + network peers have full Task authority, providers/model tools have CI-job + authority, explicit environments are not isolation, evidence follows only + direct-process settlement, Docker is acquisition-only, and ephemeral runner + teardown is the final orphan boundary. +- **Execution note:** Green smoke uses the release-candidate npm tarball and + exact acquisition image artifacts, never a checkout rebuild. The consumer + fixture is AI Evals-owned test/documentation code; AllAgents runtime does not + import Promptfoo. +- **Verification:** CLI-only/gateway clean installs and sizes, independent + release dry runs, compatibility/image mismatch fixtures, local Distribution + and public digest-pinned GHCR on every PR, authenticated GHCR and private-CA + JFrog release conformance against exact artifacts, complete A2A/Task/provider/ + acquisition E2E, Promptfoo contract fixture, Bun typecheck/lint/test/build, + dependency/image scans, generated contract/docs drift, docs build, and exact + red/green commands/results in the PR description. --- @@ -1575,94 +2133,153 @@ carrier. | Gate | Applies to | Required evidence | |---|---|---| -| Workspace/package schema | U1 | Root workspace install/build edge; ordered helper-package publication and clean-registry resolution; project/user parsing; compiled repository/target catalogs; generated schema/spec drift | -| Public contract | U1-U2 | Independent official HTTP+JSON client; card interface/params/streaming capability; A2A version and every-operation extension headers; unified Parts; both send modes; complete listing; metadata; `google.rpc.Status`; request/result/Artifact/canonicalization fixtures | -| Trusted-network model | U2-U3, U7 | Loopback and `0.0.0.0` with distinct advertised URL; HTTPS docs; shared external Task visibility/cancellation; invocation-to-gateway and host-network denial; metadata-only health/readiness | -| Durable Task lifecycle | U1-U3 | Descriptor-rooted SQLite VFS/full-sync transactions; private state/lock; create-or-replay; one execution lease; internal outcome intent and atomic terminal settlement; no early eviction; crash/store faults; restart; transactional expiry | -| Repository acquisition | U4 | Compiled-name resolution, hermetic Git, commits, 200/404/ambiguous App eligibility, cache bypass, token validation/revocation, `gh` fallback and sub-budget | -| OCI acquisition | U4 | Strict Docker auth/helper; exact layer-redirect allowlist and per-hop address checks; Bearer origin policy; direct-image/config/layer media; descriptor verification; changesets/whiteouts; fixed limits; exact project catalog; no fallback | -| Linux helper and isolation | U1, U3-U7 | x64/arm64 packages/checksums; kernel/cgroup readiness; gated durable containment; full namespace enumeration; pidfd termination; openat2 path/VFS handles; non-bypassable spawn mediation; separate mount/environment/descriptor/network views | -| Supervisor lifecycle | U3 | Capacity races; pre/post-gate crash points; provider/cancel/deadline/shutdown intent races; live-event capture; atomic evidence settlement; poison/readiness/reaping; unknown-set proof | -| Safe evidence | U3-U6 | Descriptor-relative reads with identity/size recheck; links/special/sparse/replaced files and Git indirection rejected; no verified FS evidence before quiescence | -| Backend conformance | U2-U3, U5-U6 | Same lifecycle suite for fake, Codex, and Pi; real child/grandchild spawn mediation; credential and gateway-network denial; profile and built-in variants | -| Structured result | U1, U3, U5-U6 | Public grammar, Codex native-subset gate and fallback, valid/invalid/not-produced states, Artifact cardinality, no false publication | -| Repository quality | All | Build, clean-registry install, focused/full tests, typecheck, lint, schema/spec checks, docs build | -| Bundled CLI E2E | U7 | Recorded red then green command under `/tmp/`, both sources, advertised URL/probes, auth and network isolation, capacity, replay/cancel/deadline/shutdown/restart | -| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both modes; optional context; nonblocking acceptance/subscription/cancel; source/provenance omit origins; output/usage/error metadata mapping | +| Bun architecture feasibility | U0 | Exact A2A JavaScript SDK pin and official-client server-direction operations; pinned Codex SDK and Pi RPC/package probes; existing host-auth behavior; shared read-only base/private runtime; reflink, rootless-overlay, and copy probes; Linux abort/TERM/KILL process-group probe with truthful descendant limit; exact multi-architecture acquirer image; independent packed CLI/gateway installs; immutable tarball/image binding | +| Bun repository quality | U0-U7 | One lockfile; private root orchestration; workspace-scoped typecheck/lint/test/build; dependency and image scans; generated-contract drift; minimized runtime dependencies; packed and installed size budgets | +| Package and release separation | U0-U1, U7 | Independent `allagents` and `allagents-gateway` versions/tags/triggers/tarballs; image-first gateway release; exact npm tarball plus acquisition manifest/platform digests; idempotent publication; CLI-only install fetches neither gateway nor image; gateway-only release never publishes the CLI | +| Workspace and contract packages | U0-U1 | Only `workspace-config`, `execution-contracts`, and `acquisition-contracts` shared packages; generated portable `contracts/` fixtures; normalized catalogs/defaults/order/collision keys/stable errors; project/user parsing; schema/spec drift; no `core`/`common` | +| Public contract | U0-U2 | Official JavaScript client against the Bun gateway; card interface/params/streaming; A2A version and extension headers; unified Parts; logical cwd and workspace-access schema/default/canonicalization/integrity evidence; both send modes; complete listing; metadata; `google.rpc.Status`; request/result/Artifact fixtures | +| Trusted-network and runner model | U2-U7 | Loopback and `0.0.0.0` with distinct advertised URL; HTTPS docs; shared external Task visibility/cancellation; trusted Linux CI job/VM/deployment container as provider isolation boundary; read-only described as cooperative best-effort; explicit no-hostile-code/no-secret-isolation wording; metadata-only probes | +| Durable Task lifecycle | U1-U3 | Private gateway-owned `bun:sqlite`; foreign keys and `synchronous=FULL`; transactions for create/replay, base pins, lease, intent, settlement, and expiry; no early eviction; process-kill/store faults; lock/restart/interrupted-Task reconciliation | +| Acquisition-container boundary | U0, U4, U7 | One fresh container when no reusable validated base exists and none on cache hit; exact digest-pinned image; staging-only writable bind; selected repository/registry credentials and CA material only; no App private key, host home, Docker socket, gateway state, provider auth, Codex, Pi, or harness downloads; strict source network/size/archive policy; typed manifest; exit/removal before host validation and provider execution; orphan-staging cleanup | +| Immutable-base cache | U0-U4, U7 | Key binds acquisition contract, compiled catalog/layout, and immutable source; exact commit/digest reuse; branch/tag bypass into non-reusable Task-owned bases; atomic promotion; active pins; unpinned LRU byte-budget eviction; one acquisition across 100 identical read-only trials; no cached credentials; transient-base cleanup | +| Repository acquisition | U0, U4 | Compiled-name resolution; hermetic Git/full commits; App 200/404/ambiguous eligibility; fresh base-acquisition token validation/revocation; cache-hit no credential; `gh` only after positive ineligibility; acquisition sub-budget; no provider start on failure | +| OCI acquisition | U4 | Canonical Docker Hub plus GHCR/JFrog/private-registry matrix; exact-key auth/helper/CA; bounded Basic/Bearer; redirect/rebinding policy; direct-image/config/layer media; descriptor verification; path-free manifest/private layout; changesets/whiteouts; fixed limits; no fallback | +| Registry and exact-artifact conformance | U4, U7 | Local Distribution and public digest-pinned GHCR on every PR; authenticated GHCR and private-CA JFrog release targets; exact gateway npm tarball plus acquisition multi-architecture manifest/platform digests without rebuild; positive/negative auth/permission/CA/media/path cases; explicit architecture coverage | +| Host supervisor lifecycle | U3 | One active lease; reusable-base read-only/private-runtime and non-reusable-base paths; independent writable views and cleanup; provider PID/start identity persisted before started state; explicit environment allowlist and host auth paths; cancel/deadline/shutdown races; graceful abort then TERM/KILL; non-settling failure retains Task-owned state/lease and blocks readiness; verified post-teardown reconciliation | +| Truthful bounded evidence | U3-U6 | Live bounded events; collection only after the direct provider settles and escalation finishes; hermetic Git inspection; observed termination/cleanup recorded; no claim of full descendant quiescence, hostile-code containment, secret isolation, or opaque-output redaction | +| Backend conformance | U0, U2-U3, U5-U6 | Narrow access-aware AllAgents adapter contract; same lifecycle suite for fake, Codex SDK, and Pi RPC/package; built-in/profile variants; reusable or non-reusable read-only bases and independent read-write views; resolved logical cwd/runtime/access passed to providers; existing host auth; exact binary override probes; direct host execution outside acquisition Docker; no AI SDK Harnesses or per-request runtime download | +| Structured result | U1, U3, U5-U6 | Public grammar; Codex native-subset gate and validated fallback; valid/invalid/not-produced states; Artifact cardinality; malformed provider/RPC payload cannot publish success | +| Repository quality | All | Bun install/typecheck/lint/test/build; focused and full suites; clean-registry packed installs; dependency/image audit; generated schema/spec checks; docs build | +| Packaged gateway E2E | U7 | Recorded red/green `/tmp/` commands; explicit gateway install; exact acquisition image; Git/local OCI/GHCR/JFrog sources; base reuse/materialization/cleanup; advertised URL/probes; capacity/replay/cancel/deadline/shutdown/restart; host Codex/Pi auth; truthful trust documentation | +| Promptfoo consumption | U7 | Secure-default AI Evals YAML for both source modes; optional context; per-trial `allagentsWorkingDirectory` and `allagentsWorkspaceAccess`; shared-base read-only trials; independent disposable read-write views; nonblocking acceptance/subscription/cancel; logical source/cwd/access provenance without origins or physical paths; output/usage/error metadata mapping; no AllAgents Promptfoo runtime dependency | ## Definition of Done ### Global - Every R1-R19 requirement is implemented or explicitly demonstrated by a - passing acceptance scenario. -- The gateway starts with no `gateway.yaml` or `worker.yaml`, defaults to - loopback HTTP, accepts explicit `0.0.0.0`, requires a separate advertised URL - off default loopback, documents production HTTPS, and exposes truthful + passing acceptance scenario; F1-F6 and AE1-AE21 agree with the implementation + and error table. +- U0's Bun/A2A/provider/process/acquirer/package gate passes before dependent + units. The private root, three apps, three named packages, and generated + `contracts/` fixtures are the complete shared layout; no speculative shared + package, native sidecar, or split runtime remains. +- `allagents` and `allagents-gateway` remain independently versioned and + released. A CLI-only install fetches neither gateway nor acquisition image. + Gateway release builds/verifies the exact acquisition image and registry + reports before publishing the bound npm tarball. +- The gateway starts without `gateway.yaml` or `worker.yaml`, defaults to + loopback HTTP, accepts explicit `0.0.0.0`, requires a distinct advertised URL + away from default loopback, documents production HTTPS, and exposes truthful metadata-only health/readiness. -- Network reachability is the only external caller trust boundary; Task - visibility and idempotency are deployment-wide. Invocation descendants cannot - reach that boundary, host loopback, or management networks. +- Network reachability is the external caller authorization boundary; Task + visibility and idempotency are deployment-wide. Provider execution uses the + trusted Linux CI job/VM/deployment-container boundary and existing host auth. + Documentation explicitly says AllAgents does not contain hostile repository + code or isolate provider/MCP/operator secrets from model-invoked tools. - Project workspace declarations compile to the exact repository/snapshot catalog; user declarations own profile launcher gateway enablement; built-in - target IDs cannot be shadowed. + IDs cannot be shadowed. Requests may select only the effective workspace root + or a declared repository plus a bounded relative directory and may select only + `readOnly | readWrite` access. They cannot supply physical/configured + destination paths, origins, credentials, commands, provider environments, + materializers, cache keys, Docker images/options/mounts, or provider permission + policy. - The published extension, Agent Card interface/params, A2A version and - activation headers, unified Parts, both send modes, full ListTasks behavior, - metadata preservation, strict schemas, HTTP+JSON errors, embedded Artifacts, - canonicalization, retention, and cancellation pass independent official-client - fixtures. + activation headers, logical cwd and workspace-access unions/defaults, unified + Parts, both send modes, full `ListTasks`, metadata preservation, strict + schemas, HTTP+JSON errors, embedded Artifacts, canonicalization, retention, + and cancellation pass official-client fixtures. - Secure-default AI Evals Promptfoo YAML selects repository mode with optional - named revision overrides or snapshot mode with one handle and immutable - digests. The provider maps one optional-context `callApi` to one nonblocking - Task, retains its high-entropy key across ambiguous retry, propagates - cancellation with a fresh cleanup signal, normalizes usage, and returns safe - error metadata and logical provenance without origins or runtime dependency. -- Git and OCI modes produce one complete workspace-manifest contract. OCI v1 - uses the direct-image/config/layer profile, exact project catalog, descriptor - verification, same-origin metadata, operator-approved layer redirect hosts - with per-hop address validation, changeset semantics, and fixed extraction - ceilings. Source modes never fall back and provenance never overclaims + named revisions or snapshot mode with immutable digests and can replace the + logical cwd and access per trial. Each `callApi` maps to one nonblocking Task; + read-only Tasks may share the immutable base and physical cwd, while read- + write Tasks receive independent disposable views. Ambiguous retries retain the + same key, base/view, cwd, and access. The provider propagates cancellation, + normalizes usage, and returns safe failure/logical provenance without origins, + configured destinations, physical paths, or an AllAgents Promptfoo dependency. +- Every request without a reusable validated base starts the exact digest-pinned + image with staging as its only writable bind plus source-only credentials and + strict policy. A valid cache hit starts no container and resolves no + credential. The image has + no host home, Docker socket, gateway state, provider auth, Codex, Pi, or + harness download path. It emits a typed manifest, exits, and is removed before + host validation, immutable-base publication, typed preparation, or provider + execution. Git and OCI modes produce one path-free manifest contract while the + host validates exact private destinations. OCI v1 remains registry-neutral + across Docker Hub, GHCR, JFrog, and compatible private registries with + immutable digests, descriptor verification, exact redirect/auth/CA rules, + changesets, fixed limits, no cross-mode fallback, and truthful source verification. -- App eligibility and ambiguous 404 handling, acquisition sub-budget, positive- - ineligibility `gh` fallback, fresh token cache bypass/validation/revocation, - strict Docker auth/helper and registry challenge policy, and pre-provider - credential teardown are proven. -- Typed preparation never runs workspace setup commands. The packaged Linux - helper durably binds containment before releasing any child, enumerates - unknown cgroups, and mediates every MCP/tool exec into role-specific mount, - environment, descriptor, credential, and network views. Real Codex/Pi - child/grandchild tests prove model tools cannot reach provider/MCP/operator - credentials, gateway state, or the gateway/host-management network; a backend - without non-bypassable spawn mediation is unavailable. -- The descriptor-rooted SQLite VFS, execution lease, pre/post-start-gate crash - boundaries, internal outcome-intent races, atomic terminal evidence - settlement, result states, state-path safety, descendant quiescence, readiness - poisoning/reaping, immutable terminal Tasks, and cleanup pass fault tests. -- Evaluation behavior, public-Internet authentication, remote workers, custom - materializers, non-Linux gateway execution, and multi-tenant policy remain - absent. +- App eligibility/ambiguous 404 handling, base-acquisition fresh token + validation/revocation, cache-hit credential avoidance, positive-ineligibility + selection, OCI auth/challenges, exact-host CA, and acquisition credential + teardown pass. Local Distribution and public digest-pinned GHCR run on every + PR; authenticated GHCR and private-CA JFrog release reports match the exact + gateway tarball, acquisition manifest, supported platform digests, commit, and + compatibility output. +- Codex uses pinned `@openai/codex-sdk` first and existing `CODEX_HOME`/ChatGPT + login when API credentials are absent; app-server is used only for a recorded + SDK capability gap. Pi uses its pinned supported package/RPC surface and + existing host auth. No OAuth/auth files are copied, mounted, parsed, or + imported, and no provider runtime is downloaded per request. +- Direct providers start in Linux process groups with explicit environments that + preserve required identity/auth paths. Read-only Tasks use shared immutable + bases with private runtime state and best-effort provider policy; read-write + Tasks use independent disposable block-cloned, overlaid, or copied views. + Cancellation escalates adapter abort to `SIGTERM` to `SIGKILL`. Evidence is + bounded and begins only after the direct provider settles. A non-settling + provider publishes no filesystem/Git evidence, retains its Task-owned runtime + and any writable view plus the lease, and blocks readiness until verified + post-teardown reconciliation. Evidence reports observed termination/cleanup, + not full descendant quiescence; CI runner teardown is the final orphan + boundary. +- Ordinary private Bun SQLite ownership, foreign keys, full synchronization, + create/replay, base pins, one lease, internal outcome races, atomic settlement, + immutable terminal Tasks, expiry, crash/restart reconciliation, cache + eviction, and cleanup pass fault tests without a custom VFS or native file + layer. +- Evaluation behavior, public-Internet authentication, remote workers, caller- + selected custom materializers, per-provider Docker, native containment + primitives, non-Linux gateway execution, and multi-tenant policy remain absent. ### Per unit -- U1: Root workspace packaging, ordered helper-platform publication, clean- - registry resolution, safe SQLite VFS, runtime/generated schemas, published - extension and snapshot format, producer fixture, and invalid negotiation/ - source/enablement/collision/configuration fixtures agree. -- U2: Official HTTP+JSON operations, version/extension/error/list semantics, - global replay/visibility, helper-owned SQLite locks/crashes, execution lease, - listeners/advertised URL, probes, deadline, shutdown, restart, and retention - pass against the fake backend. -- U3: The fake lifecycle proves gated durable containment, full namespace - reconciliation, atomic terminal settlement, typed preparation, spawn-mediated - secret/descriptor/network views, safe evidence ordering, poisoning, reaping, - and cleanup on Linux x64/arm64. -- U4: Git and OCI fixtures pass; App eligibility/cache bypass/token - validation/revocation, Docker credential/challenge and layer-redirect rules, - changesets, limits, exact catalog, and no-fallback rules are observed; leak - scans are clean. -- U5: Codex passes shared conformance and both schema paths; optional - credentialed smoke evidence is recorded when credentials exist. -- U6: Pi passes the same conformance and malformed RPC cannot produce success. -- U7: Final review is resolved; bundled and packed CLI red/green E2E under - `/tmp/`, Promptfoo fixture, complete repository gates, published schemas/specs, - docs, and reproducible PR instructions are complete. +- U0: Bun workspace layout, official-client A2A server direction, pinned Codex + SDK/Pi surface probes, host-auth behavior, shared read-only/private-runtime and + read-write materializer probes, explicit environment/process-group + feasibility, multi-architecture acquirer image, independent packed installs, + and exact tarball/image release binding all pass. +- U1: Three narrow packages and generated fixtures, workspace additions, + execution/acquisition contracts including logical cwd, workspace access, + relative-path grammar, base-cache identity, and materialization errors, Bun + SQLite transactions/pins, published extension/snapshot format, independent + versions, compatibility matrix, and image-first release fixtures agree. +- U2: Official-client operations, version/extension/error/list semantics, + logical cwd/access defaults/canonicalization/integrity evidence, deployment- + wide replay/visibility, SQLite lock/crash/lease behavior, listeners/advertised + URL/probes, deadline/shutdown/restart/retention, and fake backend pass. +- U3: The fake lifecycle proves shared immutable-base read-only execution with + private runtime state, independent read-write views across every configured + materializer, cwd resolution/escape rejection, explicit provider environments, + required host-auth preservation, process-group abort/TERM/KILL, outcome races, + atomic settlement, evidence ordering, restart cleanup, and truthful orphan + limitations on Linux. +- U4: Git/OCI fixtures, App/`gh` selection, cache hit/miss/key/pin/eviction, + exact acquisition image boundary, staging-only mount, source credentials/ + network/limits, typed manifest, host revalidation/publication, local/public/ + authenticated GHCR, private-CA JFrog, exact manifest/platform digests, no + fallback, and leak scans pass. +- U5: Codex passes shared access-aware conformance and both schema paths through + the pinned SDK or documented required app-server fallback, receives resolved + cwd/runtime/access, reuses existing host auth, runs outside Docker, and records + optional credentialed smoke evidence. +- U6: Pi passes the same host-process conformance through pinned RPC/package + support with resolved cwd/runtime/access, reuses existing auth, and malformed + RPC cannot produce success. +- U7: Final review is resolved; CLI-only/gateway packed smokes and `/tmp/` E2E, + independent release/size/SBOM evidence, public GHCR on every PR, + authenticated GHCR/private-CA JFrog exact-artifact reports, Promptfoo + per-trial logical-cwd/access shared-base/read-write-view fixture, repository + gates, published schemas/specs, truthful threat-model docs, and reproducible PR + instructions are complete. From 1ee6c4d33a619e34e404ca50d8686c59d71fb9a4 Mon Sep 17 00:00:00 2001 From: Christopher Tso Date: Mon, 21 Sep 2026 16:37:16 +1000 Subject: [PATCH 12/12] docs(gateway): clarify execution gateway decision Focus the ADR on user and operator impact while keeping implementation plumbing in the linked plan. --- ...-agent-execution-through-an-a2a-gateway.md | 1130 +++-------------- 1 file changed, 209 insertions(+), 921 deletions(-) diff --git a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md index 1d78419a..4d087379 100644 --- a/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md +++ b/docs/decisions/0002-serve-coding-agent-execution-through-an-a2a-gateway.md @@ -2,930 +2,218 @@ - Status: Accepted; implementation pending - Date: 2026-09-17 -- Updated: 2026-09-20 - -## Context - -AllAgents already owns cross-client agent configuration, project workspace -knowledge, global profiles, plugins, hooks, MCP configuration, and generated -launchers. External systems also need to invoke those agents without importing -AllAgents internals or driving an interactive terminal. - -The first planned consumer is AI Evals. It needs one remote coding-agent call to -return terminal output, usage, traces, file changes, produced artifacts, -failures, cleanup outcomes, and execution provenance. Future trusted tools on a -private developer network may need the same execution boundary. - -The initial product is not a public multi-tenant control plane. Developers are -expected to run one gateway for one AllAgents project workspace and expose it on -loopback, a firewalled network, or a Tailscale network. Network reachability is -the trust and authorization boundary. - -A coding-agent execution still includes more than a model request. The gateway -must acquire or reuse an immutable workspace base, select a configured agent -target, propagate cancellation, collect bounded evidence, and clean up. -The trusted CI job, VM, or container that runs the gateway is the execution and -secret boundary: provider code and model-invoked tools run with that runner's -authority. The gateway owns one public contract for the lifecycle without -claiming hostile-code containment inside that boundary. - -The contract must not turn AllAgents into an evaluation harness. Dataset -expansion, repetitions, assertions, scoring, experiment scheduling, and durable -evaluation Runs remain consumer concerns. +- Updated: 2026-09-21 ## Decision -### Add a trusted-network execution gateway - -AllAgents will provide an independently testable, separately installed -`allagents-gateway serve` entry point. It runs as one Bun service process that -supervises phase-scoped acquisition and host provider processes. The ordinary -`allagents` CLI does not contain or depend on the gateway. A convenience -dispatcher may locate and execute a separately installed compatible -`allagents-gateway`, but it must not download or embed gateway artifacts. - -The gateway implements A2A protocol version `1.0` over the `HTTP+JSON` binding -plus a required versioned AllAgents coding-execution extension. It owns: - -- stable Task and idempotency identity; -- execution-target selection; -- workspace acquisition; -- deadline and cancellation propagation; -- normalized terminal output and evidence; -- bounded Task and Artifact retention; and -- enforcement of the coding-execution contract across every backend. - -The gateway is not an evaluator, grader, experiment scheduler, retry authority, -or durable evaluation Run ledger. - -### Use one TypeScript/Bun workspace with narrow package boundaries - -The gateway and CLI are TypeScript products in one Bun workspace monorepo. The -private root package owns orchestration only. `apps/cli` publishes `allagents`; -`apps/gateway` publishes `allagents-gateway`; and `apps/acquirer` is built only -as a digest-pinned GHCR image, never as an npm package. Shared code is limited to -three justified packages: - -- `packages/workspace-config` owns the project and user workspace projections - consumed by the CLI and gateway; -- `packages/execution-contracts` owns the A2A coding-execution wire contract and - portable validation; and -- `packages/acquisition-contracts` owns the typed request and manifest exchanged - with the acquisition image. - -Generated, language-portable contract fixtures live under `contracts/`. The -repository does not introduce speculative `core`, `common`, native platform, or -provider-sharing packages. A package is added only for an already-demonstrated -ownership boundary. - -This architecture follows from the deployment boundary. V1 runs on one trusted -Linux CI runner, the gateway and official provider automation surfaces are -available in TypeScript, and Docker is needed only for untrusted repository and -OCI materialization. Adding another gateway implementation runtime and custom -in-job security layer would increase release and operational surface without -creating a boundary inside the already-trusted CI job. - -### Keep independent CLI and gateway release trains - -The `allagents` CLI and `allagents-gateway` have independent versions, tags, and -release triggers. A CLI release publishes only `apps/cli`; installing it fetches -neither the gateway package nor the acquisition image. - -A gateway release first builds the multi-architecture `apps/acquirer` image, -pushes it to GHCR, records the immutable image-index digest and each supported -architecture's manifest digest, and verifies acquisition against those exact -digests. It then packs the exact `apps/gateway` npm tarball and runs package and -registry conformance with that tarball and those image digests. Only after both -artifacts pass does the workflow publish `allagents-gateway`. It does not -publish `allagents`. - -Compatibility is a versioned contract, not equal npm versions. -`allagents-gateway compatibility --format json` reports the product, gateway -version, build identity, acquisition image digest, and supported A2A, -coding-extension, workspace, execution-contract, acquisition-contract, and -snapshot versions. The -optional CLI dispatcher may launch a separately installed gateway only when the -required contract-version intersections are non-empty. Compatibility does not -depend on target-specific npm wrappers or embedded native binaries. - -### Trust the network boundary instead of adding application authentication - -The initial gateway has no application-level authentication or per-caller -authorization. It may bind to loopback, a specific interface, or `0.0.0.0`. -Loopback remains the default when no listen address is supplied, but an explicit -`0.0.0.0` binding is valid and requires no unsafe-mode flag. The Agent Card -advertises a separate absolute interface URL; non-loopback listeners require -that value explicitly because a wildcard bind address is not routable. -Production interfaces use HTTPS; direct HTTP is limited to loopback development. - -Every external host able to reach the listener is equally trusted. Any reachable -caller may invoke every available target, including built-in and gateway-enabled -profile targets; list or retrieve retained Tasks and their Artifacts; and -request cancellation. Task lookup and idempotency are deployment-wide, not -scoped to a caller identity. Operators must use Tailscale ACLs, host firewalls, -container networking, or equivalent network controls when the listener is not -loopback-only. - -Provider processes, MCP servers, and model-invoked tools are not separate -network principals. They execute on the same trusted CI runner as the gateway -and may exercise the authority available to that job. Operators must provision -the runner accordingly and must not rely on AllAgents to isolate host secrets, -the gateway listener, management networks, or arbitrary repository code from -model-invoked tools. - -Gateway-managed TLS termination, OIDC, static bearer tokens, per-tenant -ownership, and multi-tenant information-hiding are deferred. Production clients -reach the advertised HTTPS interface through operator-managed termination or an -encrypted private overlay. Application authentication requires a separate -decision when the service leaves one trusted network boundary. - -### Use existing workspace files as the configuration authority - -The initial gateway has no `gateway.yaml` or `worker.yaml`. - -One gateway process serves one project workspace selected by `--workspace` or -the current directory. The project `.allagents/workspace.yaml` remains -canonical for repository identities, remote sources, destination paths, -default revisions, workspace files, plugins, and named OCI snapshot sources. - -The user `~/.allagents/workspace.yaml` remains canonical for global profiles and -launcher-backed execution targets. A launcher-bearing profile client is gateway- -enabled only when it explicitly declares: - -```yaml -profiles: - review: - clients: - - name: codex - launcher: codex-review - gateway: - enabled: true -``` - -The public target ID is the launcher basename. Launcher names are already -portable and collision-checked across every user profile, while one profile may -contain several clients and therefore several launchers. Internally the target -resolves to exactly one `(profile, client)` pair. The gateway reserves built-in -target IDs, initially `codex` and `pi`; a gateway-enabled launcher whose -portable collision key matches a built-in ID is invalid. - -The built-in `codex` and `pi` targets remain available when their adapters are -ready. Explicit launcher-backed targets add configured variants such as -`codex-review` and `pi-tools`. Initially only Codex and Pi profile clients are -gateway-executable; other launcher-bearing clients become eligible only after a -reviewed adapter implements the common execution contract. - -The generated launcher file is a local UX artifact, not the remote execution -boundary. The gateway never discovers launchers from `PATH`, accepts a command, -executable path, arbitrary arguments, or environment overrides from a request, -or appends request data to a generated launcher. It resolves the profile through -its typed adapter and invokes the provider's supported automation surface. - -Process-level options use exact flags and environment variables for: - -- listener, advertised-interface URL, and workspace selection; -- a project-specific state-directory override; -- disjoint immutable-base cache and per-Task runtime/workspace roots; -- workspace materialization policy plus Task and cache retention limits; -- GitHub App identifiers and private-key file references; -- the configured GitHub CLI account; and -- a strict Docker-auth file or fixed Docker credential-helper executable used - only for acquisition. - -By default the state root is a deterministic child of -`~/.allagents/gateway/` keyed by the canonical project-workspace identity. The -gateway owns an ordinary private Bun SQLite database with transactions, WAL -mode, and full synchronization. It persists claims, Tasks, one execution lease, -internal outcome intent, events, bounded Artifact bytes, and expiry state. The -gateway verifies workspace identity and holds an exclusive process-lifetime -lock. The root is current-user owned, private, and disjoint from project, -profile, and invocation roots. Standard Bun SQLite APIs are the entire storage -layer. The listener exposes metadata-only `/healthz` and `/readyz`; readiness is -false whenever admission is unsafe. - -### Support direct repositories and OCI workspace snapshots - -Each request selects exactly one closed workspace source variant: - -1. `repositories`, which materializes the repositories declared by name in the - project workspace and accepts only optional revision overrides; or -2. `workspaceSnapshot`, which selects a named OCI snapshot repository declared - in the project workspace and supplies an immutable OCI manifest digest plus - the expected AllAgents workspace-manifest digest. - -Fields from another variant are invalid. The gateway does not fall back from an -OCI snapshot to Git repositories, or from Git repositories to a snapshot, after -a Task selects its source mode. - -For direct repositories, callers cannot override repository URLs or destination -paths. A revision override is keyed by a declared repository name. Branches and -tags may be accepted for developer convenience, but the gateway resolves and -records the full commit object ID before provider execution. Reproducibility- -sensitive callers should supply full commit IDs. - -For OCI snapshots, the project workspace declares an operator-selected OCI -Distribution repository and any exact cross-origin layer-blob redirect hosts: - -```yaml -workspaceSnapshots: - evaluation: - repository: ghcr.io/entityprocess/allagents-workspaces - layerRedirectHosts: - - pkg-containers.githubusercontent.com -``` - -The repository field is registry-neutral. V1 must pull AllAgents-formatted -workspace snapshots from Docker Hub, GHCR, JFrog Artifactory/JFrog Container -Registry, and compatible private OCI Distribution registries. Registry choice -does not change the snapshot media types, digest requirements, extraction -rules, or caller-visible source contract. - -Registry conformance is tiered. Every pull request runs local Distribution -fixtures and a live public, digest-pinned GHCR pull through the exact gateway -package under test and the exact acquisition image index and architecture -manifest digests built for that pull request. A release workflow additionally -tests least-privilege authenticated GHCR and a digest-pinned disposable JFrog -Container Registry over HTTPS with a private CA and pull-only identity. Those -release checks install the exact gateway npm tarball and use the exact -multi-architecture acquisition image index and per-architecture manifests -intended for publication, for every architecture the registry and runner -support, without rebuilding either artifact. A report for another commit, -package, image digest, architecture manifest, build identity, or compatibility -output is rejected. Docker Hub behavior remains covered by protocol fixtures to -avoid public rate-limit dependence in pull-request CI. - -The request supplies the name `evaluation`, a `sha256:` OCI image-manifest -digest, and a `sha256:` workspace-manifest digest. The gateway constructs the -full OCI reference server-side. Callers cannot supply a registry host, -repository name, mutable tag, extraction destination, credential, platform -selector, redirect host, or external-layer policy. - -V1 accepts only an OCI Image Manifest directly at the requested digest; image -indexes, descriptor URLs or embedded data, non-distributable layers, and -unknown media types are rejected. Its config is the RFC 8785 canonical -`application/vnd.allagents.workspace-manifest.v1+json` object and must match the -requested workspace-manifest digest. The gateway verifies the manifest body, -config, and every distributable tar/gzip/zstd layer descriptor before decoding, -then applies layers in manifest order with OCI whiteout and opaque-whiteout -semantics. - -Registry metadata remains same-origin. A cross-origin redirect is allowed only -for a layer-blob `GET` or `HEAD` to an exact operator-declared -`layerRedirectHosts` entry; an absent allowlist rejects it. Every bounded HTTPS -hop strips authorization, cookies, and client credentials, rejects URL -credentials, resolves and validates every address at connection time, and -rejects mixed answers, rebinding, downgrade, and unapproved destinations. -Loopback, link-local, private, reserved, or other non-global addresses are -permitted only when their exact host is the source's operator-declared -repository host or layer-redirect host. Token, manifest, and config redirects -remain same-origin. Descriptor size and digest verification remains mandatory -after redirects. - -Both modes produce the same versioned, wire-visible workspace manifest. It -records declared logical repository names, requested revisions, resolved -commits, acquisition kind, relevant OCI manifest and layer digests, the -workspace-manifest digest, completeness, and whether each fact was independently -verified or snapshot-attested. It omits Git URLs, OCI repository origins, and -destination paths. A commit listed inside an OCI snapshot is not described as -independently verified unless the gateway separately verifies it against its -Git remote. - -For a source without a reusable validated base, the host gateway creates a -staging directory and bind-mounts only that directory into the digest-pinned -acquisition image. The acquisition container receives only the selected -repository or registry credential plus the strict network, redirect, size, -file-count, and archive policy needed for that source. The host resolves GitHub -App eligibility and mints any installation token; the container never receives -the App private key, host home directory, provider authentication state, or -Docker socket. The image contains and downloads no Codex, Pi, or other coding -harness. - -The container materializes the repository or OCI source into staging, emits the -typed acquisition manifest, and exits. The gateway removes it before provider -execution, validates the manifest plus paths, collisions, file types, symlinks, -layer and file counts, individual and total compressed and expanded sizes, and -digests, then atomically promotes staging to a validated base. Absolute paths, -traversal, device files, sockets, escaping links, foreign or external OCI -layers, and unapproved cross-origin access are rejected. Every non-publication -path removes staging. Docker has no role after acquisition completes. - -The base-cache key binds the acquisition-contract version, compiled catalog and -layout digest, and immutable source identity: every effective repository commit, -or the OCI manifest and workspace-manifest digests. A repository request is -reusable only when every effective revision is a full commit ID. Mutable -branch or tag requests instead receive a non-reusable Task-owned base that is -removed during settlement or reconciliation. Cache hits mint no credential and -start no acquisition container. Active Tasks pin reusable bases; bounded cache -eviction removes only unpinned entries. - -The request optionally selects `workspaceAccess: "readOnly" | "readWrite"` and -defaults to `readWrite`. A read-only Task resolves its provider cwd directly -inside its validated base; exact immutable requests may share a reusable cached -base, while mutable branch or tag requests own a non-reusable base. Every Task -receives a private runtime directory for temporary, home, provider-state, and -evidence files. The gateway disables optional Git locks and asks the adapter for -its native read-only policy when available. It does not inspect the prompt or -add a per-Task mount, chmod pass, or full-tree verification. Read-only is a -cooperative contract and best-effort provider control, not a hostile-code -boundary; the consumer remains responsible for giving the Task work that does -not require project writes. A violating provider can contaminate a cached base -and later Tasks; the operator must evict that entry before reuse. - -A read-write Task receives a unique writable view under -`//workspace`. The host materializer prefers a -filesystem block clone, falls back to rootless OverlayFS on supported Linux -hosts, and supports an explicit ordinary-copy backend for portability. It never -uses hard links for writable files. The selected materializer is operator -configuration, not request input. After evidence collection, normal settlement -unmounts when needed and removes the Task-owned view plus any non-reusable base; -a non-settling provider retains them with the poisoned execution lease until -reconciliation. - -For either access mode, the provider cwd is resolved from an optional logical -`workingDirectory` selector: - -- `{ kind: "workspaceRoot" }` selects the effective workspace root and is the - default; or -- `{ kind: "repository", repository: ConfigName, path?: RelativeDirectory }` - selects a declared repository and an optional validated directory beneath it. - -The caller never supplies an absolute path, configured destination, materializer, -cache key, or physical workspace name. The gateway maps the repository name -through the compiled catalog, resolves the optional relative path, and requires -the result to be an existing directory whose resolved path remains beneath the -selected repository root. The logical selector and access mode are part of the -canonical request, idempotency identity, and integrity evidence. -Gateway-generated structured metadata and operational logs never contain the -physical path; opaque terminal output, native evidence, and produced Artifact -payloads are not sanitized and may contain it. - -### Consume the gateway from Promptfoo through an AI Evals provider - -Rejecting caller-supplied origins does not prevent AI Evals from selecting a -workspace in Promptfoo YAML. The two files have different ownership: - -- the AllAgents project workspace is the operator-controlled catalog that maps - repository and snapshot names to Git URLs, destinations, and OCI repositories; -- the Promptfoo configuration selects a target and source mode. Repository mode - materializes the complete configured repository set and may override - revisions by declared repository name. Snapshot mode selects one declared - snapshot name and supplies immutable digests. - -AI Evals owns a Promptfoo -[custom JavaScript/TypeScript provider](https://www.promptfoo.dev/docs/providers/custom-api/). -It implements `ApiProvider`: its constructor receives `ProviderOptions`, -requires and retains a nonempty `options.id`, validates `options.config`, and -exposes `id()`. -`callApi(prompt, context?, options?)` reads bounded source, working-directory, -and workspace-access test variables from `context?.vars` when present and -cancellation from `options?.abortSignal`. The provider translates one `callApi` -into one A2A Task: it creates and retains a high-entropy invocation key, resolves -the effective logical working-directory selector and `readOnly | readWrite` -access mode, sends one Message whose sole Part has `text` set, declares the -extension in `Message.extensions`, puts the target, closed source union, logical -working directory, and access mode in the matching metadata member, and calls -`SendMessage` with `returnImmediately: true`. -It captures the Task ID and follows terminal state through `SubscribeToTask`, -with `GetTask` and bounded resubscription for races or -disconnects. It returns output, normalized token usage, stable failure metadata, -and logical provenance in Promptfoo's `ProviderResponse`. - -For example, AI Evals can define two provider instances without sending either -origin over the wire: - -```yaml -prompts: - - file://./prompts/coding-task.txt - -sharing: false -evaluateOptions: - maxConcurrency: 1 - cache: false -commandLineOptions: - write: false - share: false - -providers: - - id: file://./providers/allagents-a2a.ts - label: codex-direct - config: - endpoint: https://allagents-gateway.example.internal - target: codex - workingDirectory: - kind: repository - repository: allagents - workspaceAccess: readOnly - source: - kind: repositories - revisions: - allagents: 0123456789abcdef0123456789abcdef01234567 - - - id: file://./providers/allagents-a2a.ts - label: codex-evaluation-snapshot - config: - endpoint: https://allagents-gateway.example.internal - target: codex - workingDirectory: - kind: repository - repository: allagents - workspaceAccess: readWrite - source: - kind: workspaceSnapshot - snapshot: evaluation - digest: sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef - workspaceManifestDigest: sha256:abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789 - -tests: - - description: gateway package trial - providers: [codex-direct] - vars: - allagentsWorkingDirectory: - kind: repository - repository: allagents - path: apps/gateway -``` - -The first provider materializes the complete configured repository set and uses -the `allagents` key only to override that repository's revision. The second -provider's `evaluation` key resolves to the declared -`ghcr.io/entityprocess/allagents-workspaces` repository. The gateway enforces -one active invocation transactionally. Promptfoo keeps `maxConcurrency: 1` to -avoid predictably creating failed capacity Tasks; other trusted callers need no -external queue for correctness. The no-cache/no-write/no-share values are secure -defaults for confidential prompts and outputs; consumers may enable persistence -or sharing only after applying their own retention, access, destination, and -redaction policy. - -Static provider config fixes the source kind and logical names and may define a -default logical `workingDirectory` and `workspaceAccess`; absent values default -to `{ kind: "workspaceRoot" }` and `readWrite`. Per-test -`context?.vars?.allagentsWorkingDirectory` may replace the selector, while -`context?.vars?.allagentsWorkspaceAccess` may replace the access mode with the -exact string `readOnly` or `readWrite`. Separate read-only trials may share one -immutable physical base and cwd. Read-write trials receive distinct Task-owned -writable views even when their logical selectors are equal. -`context?.vars?.allagentsSource` remains limited to revision or digest leaves. -Missing variables retain static values. Absolute paths, `.` or `..` segments, -configured destinations, unknown repositories, URLs, mutable revisions, -credentials, commands, materializer choices, and unknown members fail before -provider execution. After Task acceptance, the provider's bounded deadline or -`options?.abortSignal` sends one `CancelTask` using a fresh cleanup signal -rather than the already aborted request signal. -It maps gateway input, output, cached-input, and total token counts to -Promptfoo's `prompt`, `completion`, `cached`, and `total` fields respectively. -Safe stable failure code, retryability, accepted Task ID, other usage, logical -working directory, and Task/Artifact evidence stay in metadata without origins, -configured destinations, or physical paths. Opaque prompts, terminal output, -structured results, native evidence, and produced-Artifact payloads remain -unredacted sensitive data. The provider belongs in AI Evals. AllAgents exposes -the A2A contract and consumer documentation without taking a runtime dependency -on Promptfoo. - -### Resolve GitHub credentials with App-first eligibility fallback - -The source request is credential-free and never selects a credential provider. -For `github.com`, the gateway supports two trusted providers: - -1. a configured GitHub App; and -2. a configured GitHub CLI account. - -The App is preferred when it has an installation covering the configured -repository. Installation applicability has three outcomes: `eligible`, -`ineligible`, and `unknown`. An App-authenticated lookup that returns coverage -is eligible. A 404 is ineligible only after the configured GitHub CLI identity -independently proves that the repository exists; an uncorroborated 404 or any -authentication, permission, rate-limit, timeout, or service ambiguity is -unknown. An explicitly configured installation ID must positively verify -repository coverage. - -For an eligible installation, the gateway bypasses the SDK token cache and -requests a fresh repository-scoped, read-only token for each acquisition. It -validates repository selection, permissions, creation time, and expiry. -Acquisition receives at most 900 seconds or the shorter remaining Task deadline, -and the token must remain valid beyond that sub-budget plus a 60-second clock- -skew margin. The gateway revokes the token after acquisition; unconfirmed -revocation fails before provider execution. - -GitHub CLI is an eligibility fallback only when the App is not configured or -applicability is positively `ineligible`. An `unknown` result caused by -configuration, authentication, rate-limit, permission, or service failure -terminates acquisition. The CLI provider invokes: - -```text -gh auth token --hostname github.com --user -``` - -with `GH_TOKEN`, `GITHUB_TOKEN`, `GH_ENTERPRISE_TOKEN`, and -`GITHUB_ENTERPRISE_TOKEN` removed from its environment. The configured account -is part of the acquisition-policy digest. - -After an App installation is selected, App configuration, authentication, -token minting or validation, permission, repository coverage, revocation, -rate-limit, or service failure terminates acquisition. The gateway never retries -the same Task through the broader GitHub CLI identity. - -Git receives credentials only through an invocation-scoped helper under -hermetic Git configuration inside the acquisition container. The gateway -excludes system, global, and repository credential helpers, Git Credential -Manager, askpass, SSH agents, repository-controlled secondary fetches, and -executable Git configuration. Tokens never appear in clone URLs, command -arguments, Git configuration, logs, Tasks, Artifacts, or retained workspaces. -The helper and token are destroyed and the acquisition container is removed -before provider execution. - -OCI credentials come from either a strict Docker-auth subset that cannot name -executables or a fixed Docker credential helper using its standard `get` -protocol. Acquisition supports anonymous pulls plus same-origin Basic and -Distribution Bearer challenge flows required by the declared registry, with the -documented Docker Hub token-service exception. Any other cross-origin Bearer -realm is rejected before a request is sent. System roots may be supplemented by -an operator map from exact registry `host[:port]` keys to verified PEM bundles; -each bundle is trusted only for connections to its key. Credentials and custom -trust material are scoped to snapshot acquisition and removed before -publication. Public registries require no credential or custom-CA -configuration. - -### Integrate host providers through narrow typed adapters - -The initial backend registry contains Codex and Pi, delivered in that order. -Each adapter implements a narrow AllAgents-owned contract for availability, -capabilities, invocation, progress, deterministic permission handling, abort, -terminal output, optional structured result, usage, native evidence, and -disposal. The gateway does not adopt AI SDK Harnesses or make a third-party -cross-provider abstraction part of its execution contract. - -The Codex adapter uses a pinned `@openai/codex-sdk` release first. App-server is -permitted only when a required, demonstrated capability is absent from that -SDK; convenience or speculative parity is not enough. Native `outputSchema` is -used only for schemas supported by the pinned Structured Outputs contract; -other valid public schemas use explicit JSON guidance and the same gateway-side -validator used by every backend. The adapter does not scrape a TUI or use an -unstable bridge merely to preserve the target name. - -The Pi adapter uses a pinned, supported RPC or package surface with -invocation-owned configuration and a restricted policy extension. Repository -extensions and unrestricted built-ins are not loaded merely because they exist -in acquired source. OMP remains out of the initial registry and is added only -for demonstrated OMP-specific value beyond direct Pi. - -Provider runtimes are installed and pinned as part of the CI runner or gateway -installation; the gateway never downloads them per request. An operator may -select a globally installed binary override only when an exact version and -capability compatibility probe succeeds. Missing controls are reported honestly -as capability gaps, and the gateway never exposes arbitrary installed -executables. - -Codex and Pi execute bare metal, directly on the same trusted Linux CI runner as -the gateway. For a read-only Task, cwd resolves inside its validated base, -shared only when reusable; for a read-write Task, cwd resolves inside the Task's -unique writable view. When -explicit API credentials are absent, Codex reuses the runner's existing -`CODEX_HOME` and ChatGPT login, and Pi reuses its existing supported host -authentication. The gateway references those host paths in place; it does not -copy, mount, or import OAuth files. - -Each provider process receives an explicitly constructed environment containing -only the invocation configuration, selected provider settings, and required -host identity, executable, home, and authentication paths. This reduces -accidental ambient-variable leakage but is not an isolation or secret- -containment claim: MCP servers, provider descendants, and model-invoked tools -may exercise the same CI-job authority and reach secrets available to that -runner. The CI job, VM, or container must therefore be provisioned as the -security boundary. - -Provider preparation is adapter-owned and typed. The gateway never executes -project or user `setup` shell entries as part of acquisition or invocation. -Validated profile settings, plugins, MCP declarations, and deterministic -workspace projections are applied through existing typed transforms. - -### Persist Task truth, not live provider execution - -The gateway durably stores Task identity, the canonical request, idempotency -claim, selected target and source, effective configuration digest, one execution -lease, internal outcome intent, Artifact bytes, retained evidence, cleanup -outcome, and expiry state under the configured state directory. AllAgents-owned -TypeScript handlers expose the A2A contract and use ordinary private Bun SQLite -ownership and transactions with full synchronization. One transaction -arbitrates `createOrReplay`, UUIDv7 Task creation, and execution-lease -acquisition. Normal terminal settlement atomically writes status, result or -failure, evidence, Artifacts, cleanup, and lease release. A provider session is -not a durable recovery checkpoint. - -At most one Task holds the execution lease from acquisition through final -evidence collection. A second otherwise-valid request settles failed with -`execution_capacity_unavailable` without launching an acquisition container or -provider process. An identical idempotency replay returns the existing Task. -Reusing the key with a different canonical request conflicts. Clients generate -at least 128 bits of randomness once per logical invocation and reuse the same -key plus request after an ambiguous transport failure. Because the initial -service has no caller identity, the idempotency namespace and Task visibility -are gateway-wide. - -Terminal Task records, Artifacts, events, and invocation claims expire in one -transaction after the configured TTL. The retained-count limit never evicts an -unexpired Task; the gateway rejects new admission until expiry frees capacity. -State-store integrity or durability failure stops admission and prevents the -gateway from acknowledging creation or reporting terminal success. - -On gateway restart, interrupted nonterminal Tasks settle failed; provider work -is not resumed or automatically replayed. Admission resumes only after any -recorded acquisition container is gone and the recorded provider process group -is confirmed absent. Otherwise the gateway remains unready with the lease held. - -### Make cancellation, evidence, and cleanup explicit - -One durable compare-and-set arbitrates provider terminal outcome, caller -cancellation, deadline, and shutdown as an internal outcome intent while the -externally visible Task remains nonterminal. The winning intent owns the stable -result or failure code and drives one idempotent cancellation and settlement -path. - -On Linux, each direct provider process starts in its own process group. -Cancellation first invokes the provider's supported graceful abort, then sends -`SIGTERM` to the process group after a bounded grace period, and finally sends -`SIGKILL` after a second bounded period. This is best-effort lifecycle control, -not containment: descendants can deliberately detach or escape the group. CI -runner teardown is the final orphan boundary. Cancellation during acquisition -stops and removes the acquisition container and unpublished staging; provider -execution never occurs in that container. - -Live provider events are bounded while execution runs. Filesystem, Git, and -produced-Artifact evidence is collected only after the direct provider process -has settled and the configured process-group escalation has completed. The -gateway does not claim to prove full descendant quiescence. A settled read-only -Task removes its private runtime and any non-reusable base; it retains only a -reusable cached base. A settled read-write Task removes its writable view and -any non-reusable base after evidence collection. One transaction then atomically -publishes terminal status, the integrity Artifact, bounded evidence, result or -failure, produced Artifacts, observed termination and cleanup outcomes, and -lease release. Task-owned cleanup failure publishes `workspace_cleanup_failed` -and retains an internal cleanup record for reconciliation. If the direct process -does not settle after final escalation, the gateway instead publishes -`execution_termination_failed` without filesystem, Git, or produced-Artifact -evidence; retains any Task-owned runtime, writable view, non-reusable base, and -the lease; stops admission; and remains unready until runner teardown and -startup reconciliation confirm the recorded process group is absent and clean -the retained state. -Evidence describes only what the gateway actually observed; -escaped descendants and uncertain cleanup are never upgraded to verified -outcomes. - -V1 supports trusted Linux CI runners and one active invocation. Other operating -systems and concurrent execution require a separate lifecycle design rather -than silent degradation. - -Terminal evidence distinguishes: - -- agent output; -- optional validated structured result; -- logical repository names, requested revisions, resolved commits, or snapshot - names and digests, never source origins or destinations; -- pre- and post-execution Git state where applicable; -- produced artifacts; -- usage and bounded provider-native evidence; -- cancellation and termination outcomes; and -- workspace cleanup outcome. - -Credentials, raw secret-bearing paths, and unrestricted prompt, output, tool, -source, or file contents are excluded from operational logs. - -### Profile A2A instead of inventing an invocation API - -The gateway uses A2A Agent Cards, Messages, Tasks, Artifacts, operations, errors, -streaming, and cancellation. Its Agent Card advertises one interface with -`protocolBinding: "HTTP+JSON"`, `protocolVersion: "1.0"`, and -`capabilities.streaming: true`. The coding- -execution `AgentExtension` is required and has strict -`params: { targets: TargetId[] }`, populated from ready built-in and explicitly -gateway-enabled launcher-backed targets. It does not publish paths, commands, -arguments, environment selectors, credentials, exact source authorization -details, or transient worker state. - -Every A2A HTTP+JSON request carries `A2A-Version: 1.0`. Every operation that -creates, returns, lists, subscribes to, or mutates profiled Tasks or Artifacts -also activates `https://allagents.dev/a2a/extensions/coding-execution/v1` -through `A2A-Extensions`. Missing activation receives -`ExtensionSupportRequiredError`; an unsupported protocol version receives -`VersionNotSupportedError`. Unsuccessful HTTP responses use the A2A -`google.rpc.Status` JSON envelope with typed `google.rpc.ErrorInfo` details; -validation also uses `google.rpc.BadRequest`, never JSON-RPC error carriers. - -The published versioned extension specification defines Agent Card params, -activation, request/idempotency/replay, errors, and terminal Task/Artifact -schemas. Its request carries the invocation key, execution target, closed -workspace source, logical working-directory selector, bounded deadline, and -optional bounded result schema in its own strict `Message.metadata` member -without rejecting unrelated A2A metadata. -The request Message lists the URI in `Message.extensions`. Every terminal Task -has one fixed-name, versioned integrity Artifact whose `Artifact.extensions` -lists the URI, plus zero or more produced Artifacts. Breaking extension versions -receive versioned cards and endpoints rather than silent fallback. - -ACP, app-server, SDK, and RPC protocols remain backend implementation details. -W3C Trace Context may propagate correlation through HTTP and child-process -boundaries. OpenTelemetry and provider-native evidence remain optional, -separate layers; neither replaces durable Task evidence. - -### Keep evaluation commands out of scope - -This decision does not add `allagents eval`, benchmark authoring, assertions, -scoring, datasets, repetitions, experiment scheduling, or automatic execution -retry. Consumers own those concerns. - -## Consequences - -- Developers who explicitly install the gateway package can start one endpoint - with `allagents-gateway serve` and use loopback, `0.0.0.0`, a specific - interface, Tailscale, or firewall policy. -- There is no application authentication, per-caller authorization, tenant - isolation, `gateway.yaml`, `worker.yaml`, remote worker protocol, or required - Kubernetes deployment in the initial product. -- Project and user workspace files remain the sole declaration authority for - source identities and gateway-enabled profile launchers. -- AI Evals can express the configured repository set with named revision - overrides, or select a prebuilt image through a snapshot handle, in Promptfoo - YAML. Its custom provider translates that closed source choice to A2A and - keeps raw origins under AllAgents operator control. -- External network reachability grants access to every available target, - including built-in and gateway-enabled profile targets, plus every retained - Task. Operators treat network policy as authorization and must restrict the - systems and secrets available to the trusted CI runner; AllAgents does not - isolate provider or model-tool descendants within that runner. -- One durable SQLite execution lease enforces one active invocation independent - of consumer concurrency settings. -- GitHub App credentials support private repositories without forcing every - developer to use one identity; GitHub CLI remains a local eligibility fallback - only when no App installation applies. -- Direct repositories and digest-pinned OCI snapshots converge on one validated - immutable-base manifest and evidence contract. Immutable source identities may - reuse a cached base; OCI metadata remains same-origin and only layer blobs may - redirect to exact operator-approved hosts. -- Read-only Tasks may share that base and physical cwd while keeping private - runtime state. Read-write Tasks receive disposable independent writable views - through the selected copy-on-write or copy materializer. -- Docker is a short-lived base-acquisition boundary only when no reusable - validated base exists. The container receives staging plus source credentials, - emits a typed manifest, and is removed before Codex or Pi starts on the host - runner. -- The private Bun workspace root orchestrates `apps/cli`, `apps/gateway`, the - image-only `apps/acquirer`, and the three contract/configuration packages. - CLI-only installs fetch neither the gateway package nor acquisition image. -- CLI and gateway versions and releases remain independent. Gateway releases - verify the exact npm tarball and the exact digest-pinned multi-architecture - acquisition image before publishing. -- Codex and Pi use pinned supported automation surfaces and existing host - authentication through narrow adapters. Explicit environment construction - reduces accidental leakage but cannot hide runner secrets from model-invoked - tools. -- Linux process-group escalation provides bounded best-effort cancellation. - Runner teardown remains the final orphan boundary, and evidence never claims - full descendant quiescence. -- A future deployment configuration becomes justified only when the product - needs multiple worker routes, tenants, credential policies, custom - materializers, centralized storage, or other operator-selected variants. - -## Rejected alternatives - -### Define a second profile registry in `gateway.yaml` - -Rejected because global profiles and launcher identities already belong to -`~/.allagents/workspace.yaml`. A second profile map would drift in client, -model, plugin, MCP, and launcher configuration. - -### Require application authentication for every deployment - -Rejected for the initial trusted-network product. It would add caller identity, -tenant scoping, token lifecycle, and ingress configuration before the expected -users need those boundaries. Tailscale ACLs and firewalls are the initial access -control. - -### Restrict the listener to loopback - -Rejected because developers need to expose the endpoint through Tailscale, -containers, VMs, and private networks. Explicit `0.0.0.0` binding is supported; -the operator owns the surrounding network policy. - -### Execute generated launcher files as the remote protocol - -Rejected because local launchers intentionally preserve cwd and append local -caller arguments. Remote requests must resolve a typed profile adapter and can -never control commands or argv. - -### Let callers provide repository URLs or OCI repositories - -Rejected because workspace configuration already defines trusted source -identities and destinations. Repository requests materialize the configured set -and may override revisions by declared name; snapshot requests select a declared -name and immutable digests. Neither variant introduces a new origin. - -### Let callers provide a host cwd - -Rejected because an absolute or configured destination path would let a caller -select unrelated host content and bypass gateway-owned acquisition. Promptfoo -gets the required runtime control through a logical workspace-root or declared- -repository selector; the gateway maps it into the access-appropriate reusable -or Task-owned base or writable view according to `workspaceAccess`. - -### Always allocate a unique full workspace - -Rejected because read-only Tasks have no mutable project state to isolate, and -copying a large immutable workspace for every trial wastes transfer, storage, -and I/O. They share one validated base. Writable Tasks isolate only their -changes through a disposable copy-on-write view or explicit portable copy. - -### Fall back from a selected GitHub App after runtime failure - -Rejected because it would silently change identity and authorization scope after -selection. GitHub CLI fallback applies only when the App is ineligible. - -### Use mutable OCI tags - -Rejected because the same request could produce different workspaces. Snapshot -selection requires an OCI manifest digest and expected workspace-manifest -digest. - -### Treat provider sessions as durable execution - -Rejected because a resumable provider thread does not prove workspace, -process, cancellation, evidence, or cleanup continuity across gateway restart. - -### Invent a bespoke invocation API - -Rejected because A2A already supplies discovery, Task lifecycle, streaming, -Artifacts, cancellation, and errors. Coding-specific evidence belongs in a -versioned extension. - -### Adopt an evaluator's Job or Trial API - -Rejected because benchmark orchestration, verification, and persisted evaluation -state remain consumer concerns. The gateway executes one coding-agent Task. - -### Build v1 around Rust and kernel containment - -Rejected because the trusted CI job is already the execution boundary. A Rust -gateway plus custom cgroups, pidfds, `openat2` VFS behavior, namespaces, -`nftables`, or spawn mediation would add implementation and release risk without -isolating model-invoked tools from secrets available to that job. Reconsider -native or stronger containment only if hostile-code or in-job secret isolation -becomes a product requirement. - -### Adopt AI SDK Harnesses as the backend abstraction - -Rejected because AllAgents needs a small contract tailored to its A2A Task, -evidence, cancellation, and profile semantics. Depending on a broad -cross-provider abstraction would enlarge the compatibility surface without -removing the need to understand the official Codex and Pi automation APIs. - -### Run shared host provider daemons - -Rejected because a long-lived daemon introduces cross-invocation state, -ownership, cancellation, and authentication ambiguity. V1 starts one direct -provider process for the one active Task and treats provider sessions as -ephemeral. - -### Run providers in per-invocation containers - -Rejected because official Codex and Pi automation should reuse the trusted -runner's existing installation and authentication. Copying or mounting OAuth -state into a provider container complicates ownership without creating a -security boundary against model tools. Docker remains limited to acquisition. - -### Trust ambient unversioned provider binaries - -Rejected because PATH discovery can silently change behavior between runs. -Pinned SDK, RPC, or package surfaces are the default; a global binary override -must pass exact version and capability probes, and runtimes are never downloaded -per request. - -### Couple CLI and gateway versions or publish them together - -Rejected because the products have different dependencies and release cadence. -Compatibility is explicit at the contract boundary; gateway-only work must not -force a CLI release, and CLI-only installation must not fetch gateway or -acquisition artifacts. - -### Bundle the gateway into every CLI installation - -Rejected because plugin/skill-only users do not need the A2A server, SQLite, -provider adapters, or acquisition image. The gateway ships as the separately -installed `allagents-gateway` npm package, and its release independently binds -the digest-pinned GHCR acquisition image. +AllAgents will provide a separately installed gateway that lets trusted tools +start a configured Codex or Pi run remotely and receive its output, usage, file +changes, artifacts, source provenance, and cleanup outcome through A2A. + +AI Evals is the first consumer. The gateway executes one coding-agent run; it +does not own datasets, scoring, assertions, scheduling, retries, or durable +evaluation records. + +Version one serves one AllAgents project workspace on one trusted Linux runner. +Network access controls who can use it, and the runner is the execution and +secret boundary. This is not a sandbox for hostile code or model-invoked tools. + +Implementation details live in the +[coding-agent execution gateway plan](../plans/2026-09-18-0837-feat-coding-execution-gateway-plan.md). +This ADR records the decisions and their impact. + +## What changes for users + +- **CLI users:** installing `allagents` does not install or start the gateway. +- **Operators:** install `allagents-gateway`, select one existing workspace, and + run `allagents-gateway serve`. Existing project and user workspace files remain + the source of truth. +- **Callers:** choose a configured target, declared workspace source, logical + working directory, `readOnly` or `readWrite` access, a bounded deadline, and an + optional result schema. They cannot provide repository URLs, host paths, + commands, credentials, or environment overrides. +- **AI Evals:** owns the Promptfoo provider and evaluation behavior. AllAgents + owns the gateway contract and documentation. + +## Main flow + +1. The operator starts the gateway for one project workspace. +2. A caller sends an A2A Message with one invocation key. Reusing that key with + the same canonical request returns the same Task; a new key starts a new run. +3. The gateway reuses or prepares a workspace from declared Git repositories or + an immutable OCI snapshot. +4. Codex or Pi runs on the trusted host against a validated read-only base, + shared only for exact immutable requests, or a private disposable read-write + workspace. +5. The gateway streams progress, handles cancellation, records observed evidence, + cleans up Task-owned state, and frees the single execution slot. Uncertainty + about source cleanup, an active mount, or a writable Task view stops admission. + If it cannot confirm provider termination, it retains the slot and stays + unready until teardown confirms the process is gone. + +## Important consequences + +### Network reachability grants full access + +The gateway has no application login, caller identity, tenant isolation, or +per-caller privacy. Loopback is the default, but operators may expose it on a +private interface or `0.0.0.0`. + +Every reachable caller can invoke every available target, inspect every retained +Task and Artifact, and request cancellation. Non-loopback exposure requires +operator-managed HTTPS and network access controls such as Tailscale ACLs, +firewalls, or container networking. Direct HTTP is limited to loopback use. + +Providers and model-invoked tools may use the runner's credentials, secrets, and +network access. Explicit environments reduce accidental leakage but do not +create isolation. Application authentication and multi-tenant ownership are +deferred until the service must leave one trusted network. + +### Callers choose logical work, not infrastructure + +The gateway reuses existing workspace configuration; it does not add +`gateway.yaml` or `worker.yaml`. Built-in Codex and Pi targets are available when +ready. Profile-backed targets require explicit gateway enablement. + +A request chooses either the complete configured repository set, with optional +revision overrides by declared name, or one declared OCI snapshot selected by +immutable digests. It never falls back between those modes. The operator owns +origins, destinations, credentials, and registry policy. Credential routing +prefers a proven applicable GitHub App and uses the configured `gh` account only +when the App is absent or positively ineligible. Ambiguity or failure after +selection never falls back to a broader identity. + +The caller selects the workspace root or a directory beneath a declared +repository, never a physical host path. Invalid or escaping paths fail before +provider execution. + +### Read-only is an optimization, not a security boundary + +`readWrite` is the default and gives each Task a private disposable workspace. +`readOnly` uses a validated base: exact immutable requests may share a reusable +base, while mutable revisions get a Task-owned, non-reusable base. Every Task +still gets disposable private provider, temporary, and evidence state. + +Read-only enforcement is cooperative. A provider that writes anyway can +contaminate the shared workspace and later Tasks; the operator must then evict +that workspace before reuse. + +Workspace preparation is isolated from provider execution and receives only the +source credential and network access it needs. That credential and preparation +environment are gone before Codex or Pi starts. + +### Providers run directly on the trusted host + +Codex and Pi use supported, pinned integrations and existing host authentication. +The gateway does not run workspace `setup` shell entries, download a provider +runtime for each request, execute generated launcher files remotely, or expose +arbitrary installed executables. + +If a supported integration lacks a required control, that target is unavailable +rather than silently weakening the public contract. + +### One Task runs at a time + +One execution slot covers workspace preparation, provider execution, evidence, +and cleanup. A second valid request becomes a failed Task with +`execution_capacity_unavailable`; it starts no workspace or provider work. + +Task identity, retries, and visibility are shared across the gateway. Reusing an +invocation key for a different request conflicts. + +Terminal Tasks and evidence are retained within configured limits. Unexpired +Tasks are not deleted to make room for new work. Prompts, output, structured +results, native evidence, and produced Artifacts remain sensitive and +unredacted. +Operational logs exclude credential values, secret-bearing paths, and +unrestricted prompt, output, tool, source, and file content. + +### A2A provides the lifecycle; AllAgents defines coding execution + +A2A 1.0 over HTTP+JSON provides discovery, Messages, Tasks, streaming, Artifacts, +cancellation, and errors. A required, versioned AllAgents extension adds targets, +workspace selection, idempotency, provenance, and evidence. Callers send the A2A +version and activate the extension on every operation that creates, returns, +lists, subscribes to, or mutates profiled Tasks or Artifacts. Missing extension +support and unsupported versions use standard A2A errors. Breaking changes use a +new extension version rather than silent fallback. + +The known conformance question is authentication. A2A 1.0 says servers +authenticate requests and authorization-scope Task operations, while this design +has no application identity and treats every reachable caller as one authority +domain. Agent Card security declarations are optional, so the anonymous case is +not explicit. The implementation feasibility gate must resolve this before the +gateway claims full A2A 1.0 conformance. If it cannot, this ADR must be amended; +the implementation must not silently add authentication or weaken the +conformance claim. + +### The gateway remains a separate product + +`allagents` and `allagents-gateway` have independent versions and release +cadence. Installing the CLI fetches neither the gateway nor its workspace- +preparation image. Compatibility comes from versioned contracts, not matching +package versions. + +Every terminal Task has exactly one versioned, extension-marked +execution-integrity Artifact, plus any produced Artifacts, even after failure or +cancellation. Consumers remain +responsible for evaluation workflows, retries, retention, sharing, and redaction. + +## Failure behavior + +- **Busy:** the new Task fails with `execution_capacity_unavailable`; no work + starts. +- **Malformed source or working-directory input:** admission fails with + `invalid_execution_request`; no Task is created. +- **Accepted source, credential, or logical-directory resolution then fails:** + the Task fails with the corresponding stable error code and does not switch + source mode or credential identity. +- **Gateway restart:** interrupted Tasks fail. Provider work is not resumed or + automatically replayed. +- **Provider cannot be stopped:** the Task reports + `execution_termination_failed` with live-provider and observed termination + evidence, but no filesystem, Git, or produced-Artifact evidence. It retains + its workspace and execution slot and leaves the gateway unready until teardown + confirms the process is gone. +- **Workspace cleanup fails:** the Task reports `workspace_cleanup_failed` and + retains enough state for later cleanup. +- **Durable Task state is unsafe:** the gateway stops admitting work and does not + acknowledge creation or report success it cannot preserve. + +Evidence describes only what the gateway observed. It never presents uncertain +termination, cleanup, provenance, or file state as verified. + +## Deliberate limits + +Version one deliberately avoids: + +- application authentication, tenants, caller-private Tasks, and public-Internet + hardening because the initial product assumes one trusted network; +- queues, concurrent execution, replicas, remote workers, shared provider + daemons, and resumed provider sessions because one durable Task lifecycle is + the initial boundary; +- caller-provided origins, physical host paths, configured destinations, + commands, credentials, environments, or materializers because the gateway is + not a remote shell; +- hostile-code containment and per-provider containers because the runner is + already the execution and secret boundary; +- a bespoke or evaluator-specific API because A2A already owns the remote Task + lifecycle; +- a second configuration registry because workspace files already own sources + and profiles; and +- bundling or version-locking the gateway with the CLI because most CLI users do + not need the service and the products have different release cadence. ## Reconsider when -Revisit this decision when any of these become requirements: - -- callers outside one trusted network must share the endpoint; -- per-caller Task privacy, authorization, or audit identity is required; -- provider or model-tool code must be isolated from runner secrets or treated as - hostile inside the execution environment; -- multiple gateway replicas need transactional shared storage; -- more than one active invocation or shared provider daemons are required; -- execution must route among remote worker pools or sandboxes; -- non-Linux runners need equivalent lifecycle and cancellation semantics; -- custom materializers are needed beyond direct Git and OCI snapshots; -- multiple GitHub hosts, Apps, CLI accounts, or ordered credential policies need - declarative configuration; -- A2A standardizes the required coding-execution evidence without an extension; - or -- a stable cross-vendor automation protocol subsumes the backend adapter seam. +Revisit this decision when: + +- callers outside one trusted network must share the service; +- callers need private Tasks, distinct authorization, or audit identity; +- provider or model-tool code must be isolated from runner secrets; +- the gateway needs concurrency, replicas, shared daemons, or remote workers; +- non-Linux runners, new source materializers, or richer credential routing are + required; +- A2A standardizes the coding-execution fields now carried by the AllAgents + extension; or +- a stable cross-vendor protocol replaces the Codex/Pi adapter seam.