Skip to content

architecture.md

Latest commit

 

History

History
397 lines (338 loc) · 23.1 KB

architecture.md

File metadata and controls

397 lines (338 loc) · 23.1 KB
Raw

Architecture

CodingAgentRunner is the process + protocol layer for coding-agent CLIs: it spawns a CLI as a child process, reads its stream-json output as a typed event stream, classifies the run's outcome, and tracks quota. Everything above that — task lanes, pipelines, review steps, storage — belongs in the application built on top.

Layers

┌────────────────────────────────────────────────────────────────┐
│ Application (task / lane / pipeline orchestration) — out of scope │
└────────────────────────────────────────────────────────────────┘
                 ▲ events · outcomes · quota
┌────────────────────────────────────────────────────────────────┐
│ CodingAgentRunner                                                │
│   Models      known registry · CLI discovery · reasoning ladders  │
│   Quota       remaining-quota cache · escalation · cap/gate       │
│   Lifecycle   stop · process-tree reap · built-in watchdog        │
│   Protocol    stream-json → typed CliRunEvent (incl. completion)  │
│   Hardening   binary resolve (.cmd→.exe) · env · stdin · git      │
│   Abstractions  CliOptions · ILogger · home/log providers          │
└────────────────────────────────────────────────────────────────┘
                 ▲ spawns + reads
        coding-agent CLI (claude · codex · gemini · antigravity)

The public surface

The entry point is CliRunner. From a single CliOptions it builds one ICliDriver per supported CLI — but a driver is not a per-CLI subclass. Each is one parameterized engine, CliRunEngine, constructed from a CliDescriptor resolved through an ICliCatalog. Adding a CLI means registering another descriptor in the catalog, not writing a new class. runner.Get("claude") (or the runner.Claude sugar) returns that descriptor-backed engine as an ICliDriver.

Before a host starts a run, it can call driver.EnsureHealthyAsync(ct). This runs the same descriptor health policy used immediately before spawn, returning a typed PreSpawnHealthResult (Healthy, Repaired, or Failed) without starting an agent turn. Claude's built-in policy probes first and repairs its npm shim only when needed; failures include the repair actions and an actionable error.

var runner = new CliRunner(new CliOptions());
var driver = runner.Get("claude");                 // or runner.Claude

driver.OnRunEvent += (runId, evt) => { /* watchdog / UI */ };

var (run, error) = await driver.StartAsync(new CliRunRequest
{
    RunId            = "task-1",
    Prompt           = "Refactor the parser.",
    WorkingDirectory = repo,
});

Most uses need five public types — CliRunner, CliOptions, CliRunRequest, CliRunInfo, and CliRunEvent. One layer down the descriptor model is also public and inspectable: CliDescriptor, ICliCatalog / CliCatalog, and the launch-time LaunchSpec / CliLaunchContext. The per-CLI stream-json adapters are public too — ClaudeEventAdapter / CodexEventAdapter / GeminiEventAdapter each expose a static Map(line, runId), so you can turn a CLI's output line into typed events without spawning a process (replay, tests, benchmarks). The built-in descriptors and the run engine (CliRunEngine) stay internal — reachable only through CliRunner / the catalog. The spawner, the hardening, and the log stores stay internal too: that machine room is not part of the contract, and there is no "add your own CLI" extension point today — the descriptors are a fixed, inspectable catalog, not a registration hook. CliOptions.Spawner is the process-launch seam (see below); it does not own argument policy.

CliRunRequest.LaunchExtensions is a separate, constrained host seam for a built-in CLI option that must survive a runner upgrade. It is not an argv overlay: the host can currently request exactly one Claude CliLaunchExtension.AppendClaudeSystemPromptFile(absolutePath), and the runner checks the CLI, cardinality, absolute existing file path, and option kind before it spawns. The descriptor places that option while retaining base argv order, structured output, permission and reasoning flags, prompt transport, hardening, events, and outcomes. Raw flags and replacements for runner-owned switches are intentionally not accepted. This seam was added for Agent Studio AGT-2371.

CliRunRequest.Attachments carries durable references from a chat/task message. The host-owned CliOptions.AttachmentResolver resolves them to existing absolute file paths before launch. Codex receives images through its native image option; all descriptors receive the validated files through CliLaunchContext.Attachments, and the prompt includes an absolute-path context block. Resolution is all-or-nothing: an unavailable reference returns a start error before a child process is spawned. See Chat attachments.

Use, don't extend. You consume the library through interfaces and records; you do not subclass it. A CLI is data — a CliDescriptor is a sealed record of fields plus a few pure delegates (BuildLaunch, Parse, InterruptClassifier, Capabilities, optional CleanContext / liveness), not a base class with protected virtual seams. There is exactly one run engine, internal sealed, parameterized by the descriptor; a consumer cannot subclass it or even name it. That is the design's load-bearing invariant: a new CLI is one descriptor's worth of data, and nothing about the engine changes.

Modules

Spawning / hardening

Windows-safe launch: BinaryResolver resolves a .cmd shim to the real .exe (the prompt-truncation fix); environment hardening; stdin default-deny; a handle-scrubbing Win32 spawner that hands the child only its three std pipes. The spawner is pluggable through CliOptions.Spawner (ICliProcessSpawner). To inspect, validate, or decorate CAR's prepared ProcessStartInfo while retaining its curated Windows handle list (and the normal redirected-pipe spawn elsewhere), use CliProcessSpawner:

var runner = new CliRunner(new CliOptions
{
    Spawner = CliProcessSpawner.Decorate(startInfo =>
    {
        if (!Path.IsPathFullyQualified(startInfo.FileName))
            throw new InvalidOperationException("The CLI executable must be absolute.");
        startInfo.Environment["HOST_LAUNCH_ID"] = launchId;
    }),
});

Implement ICliProcessSpawner directly only when the host owns a different launch mechanism, such as a Windows pseudo-terminal. This composition seam was added for Agent Studio AGT-2371. The platform-owns-git guard injects a PATH-front git wrapper that blocks mutating commands; it is defence-in-depth, not a sandbox.

Protocol & completion

Each per-CLI adapter maps the CLI's stream-json frames onto the normalized CliRunEvent contract — a closed sum type, so a switch over it is checked for exhaustiveness by the compiler. Completion comes from the CLI's own result frame plus process exit (surfaced as TurnCompleted and the single terminal RunEnded), not from scraping a [[TASK_DONE]] sentinel, which is a fragile heuristic and stays in the consumer's run protocol, not here.

Codex also preserves typed stderr diagnostics as CliRunEvent.Diagnostic records when a line matches one of the known helper-binary / PATH or plugin-loader warnings. The event keeps the severity, subsystem, stable code and category, summary, remediation, raw line, timestamp, dedupe key, and any plugin ids visible so consumers can group repeated warnings without re-parsing prose. Unknown stderr remains lossless via CliRunEvent.Unknown.RawDetail.

Cross-CLI normalization

The three CLIs speak the same ideas in different dialects: a session start is a system frame in Claude, a thread.started in Codex, an init in Antigravity; the id it carries is session_id in two and thread_id in the third; the cached-token field has three different names. The adapters fold all of that onto one event vocabulary, so a consumer's run logic is written once and never branches on the CLI (or the model). Parsing is by frame type, never by model — the model is metadata, so a new model needs no new parser branch. The per-CLI dialect table, the structural asymmetries the model absorbs, and the two shipped projections (the event stream and the optional render line/span model) are in Cross-CLI normalization.

Interrupt classification

Beyond the terminal RunEnded, a run can surface a typed CliRunEvent.Interrupt(InterruptReason Reason, string Detail, bool IsFatal) when a classifier recognizes a stop-worthy condition in the output. InterruptReason is a closed enum: EnvironmentBlocker (an OS/sandbox error the agent can't self-recover from — continuing only burns the silence budget against the same wall), QuotaExhausted (recoverable after the window resets), Sentinel (a consumer-defined completion marker), SelfReference (the scanner self-reference trap — raised with IsFatal: false so it is not mistaken for a real blocker), NeedsInput (blocking on input that won't arrive unattended), and SilentCompletion (a legacy CLI that stopped without a terminal completion frame). Classifiers implement IInterruptClassifier; the library raises the event, and the consumer keeps Stop() authority — deciding based on IsFatal. The mechanism and a starter EnvironmentBlocker classifier ship; the built-in descriptors leave the classifier unset by default, so a consumer opts in (or supplies its own grammar) rather than getting surprise stops.

Capabilities

A descriptor answers "what can this CLI do?" through CliCapabilities, resolved per model via ICliDriver.Capabilities(model): SupportsCleanContext, SupportsResume, SupportsThinking with the ThinkingLevels / DefaultThinkingLevel it accepts, EmitsHeartbeatDuringThinking (true for Codex's reasoning models, which ping while they think, and false for the others), and an open Knobs dictionary for the rest. The point is that a consumer asks the capability, not the CLI type — so logic written against EmitsHeartbeatDuringThinking (to widen a silence budget, say) covers a new CLI the moment its descriptor is registered, with no if (cliType == …) branch.

Defaults — batteries-included, overridable

The "normal knowledge" lives in the library, not the consumer. Thinking levels normalize against the model (CliThinkingLevels) and reasoning flags are per CLI and model (CliReasoningFlags), so the sensible value is the default and the consumer overrides only its delta. For layering your own defaults the same way, CliScope (CliType, optional Model, optional ThinkingLevel) plus CliDefault<T> resolve a value most-specific-first — CLI + Model + ThinkingLevel, then CLI + Model, then CLI + ThinkingLevel, then CLI, then a global fallback — and Set(scope, value) is the one-line override. (The resolution primitive ships; the library does not yet pre-seed a registry of CliDefault<T> values beyond the thinking/reasoning tables above.)

Model registry and discovery

KnownModels is the stable registry for the Claude and Codex families understood by the library. Each entry carries its id, label, vendor, context window, aliases, and a generation order. Availability is not static metadata: it comes from the CLI installed on the host.

CliRunner.DiscoverModelsAsync(cliType, forceRefresh, ct) returns a CliModelCatalog. For Codex, CodexCliModelDiscovery runs codex debug models with the configured CliOptions.CodexPath, redirected stdout/stderr, no stdin, the normal hardened environment, the configured process spawner, and a timeout. Entries marked visibility=hide are excluded. Public entries retain the CLI's priority, context window, description, speed/service tiers, and exact reasoning ladder. The live list is merged with KnownModels: known entries absent from that CLI version stay in the catalog with Available=false; unknown entries reported by the CLI stay available and carry an explanatory note.

Claude Code has no model-list command. ClaudeCliModelDiscovery probes CLI presence and returns the Claude registry with a note that availability is presence-based. Both implement ICliModelDiscovery, which is also the replacement seam for a future Claude probe. A runner caches completed catalogs in memory for CliOptions.ModelDiscoveryCacheTtl; forceRefresh=true bypasses the cached value.

CliThinkingLevels.For, DefaultFor, and Normalize accept an optional discovered catalog. A CLI-reported ladder wins when present; the static compatibility table is used only when discovery has no ladder metadata for that model.

Lifecycle

Stop a run — reported as RunEnded(Stopped, …), a deliberate stop, never a crash — reap the whole process tree (no orphaned grandchildren holding file handles), and a built-in, phase-aware silence watchdog (RunWatchdog / WatchdogPolicy) you attach in one line.

Quota

A per-CLI IQuotaProbe contract plus a QuotaService that caches the remaining window with an escalation policy (default TTL 10 min; poll harder near the limit, e.g. ≥90 % → every 2 min, ≥97 % → every 30 s — all configurable), a cap/gate to skip a run before it hits the wall, and a free event-harvest (Observe) that keeps the cache warm from RateLimitObserved events (with a precise percent when the CLI reports one).

Two built-in probes ship with the library:

  • ClaudeOAuthUsageProbe — reads the OAuth token Claude Code stores after sign-in and calls the CLI's own usage endpoint (api.anthropic.com/api/oauth/usage): real server-side percent + reset time for the 5-hour and weekly windows, plus model-scoped windows when present. It is the CLI's endpoint, not a documented public API — a CLI update can move it; the probe then degrades to an error snapshot.
  • CodexSessionLogProbe — reads the freshest token_count.rate_limits entry from Codex's rollout logs (~/.codex/sessions/**.jsonl): used_percent and reset per window plus plan_type, without spawning a process or spending quota. Data is as old as the last Codex run; pair it with the event harvest for live figures.

Gemini is deprecated (no probe); Antigravity exposes no quota surface to probe today. The IQuotaProbe seam stays open for consumer-supplied probes.

CliOptions.WaitOnQuota adds an optional wait branch after a typed quota-limit failure. It is disabled by default. When enabled, the engine force-refreshes the selected CLI through the configured QuotaService; a known future reset within the threshold (30 minutes by default) produces QuotaWaitStarted, an asynchronous wait, QuotaWaitEnded, and a restart of the same request. The stopped attempt is not exposed as a terminal run. An unknown/errored snapshot or a reset outside the threshold follows the existing failure path.

FileQuotaCacheStore.Global() gives every process of the user one shared cache file in the OS-native app-data location (%LOCALAPPDATA% / ~/Library/ Application Support / ~/.local/share; CODING_AGENT_RUNNER_CACHE_DIR overrides it). Writes merge per CLI under a best-effort cross-process file lock (freshest FetchedAt wins), and a QuotaService whose entry went stale adopts a snapshot another process refreshed instead of spending a probe. Deliberately per-user, not per-machine — quota belongs to the signed-in CLI account.

Environment diagnostics

A small CodingAgentRunner.Diagnostics namespace (in the core package) answers "is this machine ready to run an agent CLI?". CliRunner.InspectEnvironment() probes each driver's CLI (--version), the known credential locations (a credential file under the user home, or the CLI's API-key env var), and the Node.js/npm runtime, and returns an EnvironmentReport — one CliEnvironmentStatus per CLI plus a ToText() rendering for logs/terminals. The static fix-it knowledge (install commands, sign-in steps, headless-auth options, docs URLs) ships as data in CliSetup / CliSetupInfo, available without probing anything. The library detects and explains; it does not install or authenticate a CLI. See cli-setup.md.

Run metrics

A small CodingAgentRunner.Metrics namespace (in the core package) folds the event stream into a structured summary: TurnMetrics and RunMetrics records accumulated by RunMetricsRecorder, with token figures parsed by UsageSummaryParser from each TurnCompleted usage line. Time-to-first-output, time-to-session-id, per-turn wall-clock and output-tokens/sec are reconstructed from the events' timestamps — it records, it does not poll.

Pricing

A small CodingAgentRunner.Pricing namespace (in the core package) is the one library of per-model API prices, with history, plus a pure cost API over it — so token-cost computations go through one deterministic source instead of a hardcoded table copied into each caller. A ModelListing holds a model's id, aliases, and a price history; each ModelPrice entry carries per-MTok input/output (and optional cache) rates and a ValidFrom UTC instant. ModelPriceCatalog resolves the price valid at a run's timestamp (ResolvePrice) and computes a per-component CostBreakdown (ComputeCost); ModelPriceCatalog.Default is the seeded catalog. Unknown and unpriced models are reported explicitly through PriceStatus (UnknownModel / NoPriceForDate) with a null total — never a silent zero. See pricing.md.

Optional: Rendering (separate package)

CodingAgentRunner.Rendering is a separate, opt-in NuGet package — the core library has no dependency on it; the dependency points one way, Rendering → core. It maps agent output onto a presentation-agnostic span/line model, injects links through a pluggable LinkResolver (so task-refs, file links and web URLs are the consumer's policy, not the library's), and materializes Markdown or HTML (Markdig-backed). The default resolver (LinkExtractor.WebDefault) enforces one safe-URL policy — an http/https/mailto allowlist that rejects javascript: / data: targets — so the default is XSS-safe without the consumer wiring anything. A consumer that only needs the event stream never references the package.

Supported CLIs

CLI Type id Status Context Adapter / stream Notes
Claude Code claude supported clean or shared Claude adapter First-class driver.
OpenAI Codex codex supported clean or shared Codex adapter First-class driver, including reasoning-model liveness metadata.
Google Gemini gemini deprecated ([Obsolete]) shared only Gemini adapter Unsupported. Public surface marked [Obsolete]; the driver still resolves, removal planned before 1.0.
Google Antigravity (agentapi) antigravity driver shipped shared only reuses Gemini adapter Maintained Google path; kept out of CliTypes.All until a consumer migrates.

The GitHub Copilot driver was supported earlier but has been removed: its headless surface was PTY/TUI-dependent and couldn't share the hardened structured spawn/stream engine cleanly.

Context modes

ContextMode describes which persistent CLI home a run uses. It does not describe the repository contents, the prompt size, or whether repo instruction files are active.

  • clean creates a temporary per-run CLI home and seeds only the minimum auth/base config. Claude uses CLAUDE_CONFIG_DIR; Codex uses CODEX_HOME.
  • shared uses the operator's normal signed-in CLI home: settings, cache, memory, session history and any other CLI-level state are visible to the run.

Claude and Codex can redirect their home/config directory, so they support both modes. Gemini and Antigravity currently expose no clean-home redirect in this runner, so they are shared-only. In both modes the working directory remains the same: versioned repo files and repo-local instructions such as AGENTS.md / CLAUDE.md still load from the checkout.

Host-owned clean-context leases

ContextMode=clean creates and releases a home for one attempt. A host that needs to continue the same task across attempts can instead acquire a CliCleanContextLease. The lease is prepared by the runner, including its linked credential files and copied base config; the host must not reproduce that seeding. Pass it in each CliRunRequest.CleanContextLease, including a request with ResumeSessionId.

using var context = runner.Codex.AcquireCleanContext();

var first = await runner.Codex.StartAsync(new CliRunRequest
{
    RunId = "task-1-attempt-1",
    Prompt = "Investigate the failing test.",
    WorkingDirectory = repo,
    CleanContextLease = context,
});

var continuation = await runner.Codex.StartAsync(new CliRunRequest
{
    RunId = "task-1-attempt-2",
    Prompt = "Continue and implement the fix.",
    WorkingDirectory = repo,
    ResumeSessionId = sessionId,
    CleanContextLease = context,
});

The runner never disposes a supplied lease at an attempt's terminal event or after a failed start. The host disposes it once at its task-run boundary; disposal removes the isolated home and is idempotent. TempHome, EnvOverrides, and Sources are exposed for diagnostics. A lease belongs to one CLI and cannot be supplied to another. This contract was added for Agent Studio AGT-2371.

Cheap-subagent delegation

Delegation is run configuration, not an API: the runner writes agent definitions into the CLI's own convention (.claude/agents/*.md, .codex/agents/*.toml) before a run and appends a rule plus the agent inventory to the prompt. The CLI does the spawning.

The per-CLI convention is a SubagentSpec on the CliDescriptor, so a CLI without one materializes nothing. Ownership is established by creating a file (FileMode.CreateNew), never by reading one, so anything already at that path — a repository's own definition, or one an earlier run left behind — is adopted rather than overwritten and is never deleted. Every file a run created is removed before its terminal callbacks fire, so a host that commits from OnFinished cannot see a generated definition. See delegation.md — including the probe showing that codex exec accepts the collab tooling but starts no child thread.

Abstractions the library leans on

  • Logging — Microsoft.Extensions.Logging.Abstractions (ILogger); no concrete sink.
  • Options — a plain CliOptions object instead of ambient IConfiguration.
  • Home / path providers — the caller decides where clean-context homes and logs live (IUserHomeProvider, IRunLogPathProvider); no hard-coded app paths.

Out of scope

Task lanes, pipelines, review steps, workspace/job storage — these belong in the application built on top of CodingAgentRunner.