From 6688d475413f80c862e131f1ebc93cf27a5fc830 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Mon, 14 Sep 2026 23:14:50 +0200 Subject: [PATCH 01/42] Record the shared cycle-runner decision and its plan as the review baseline. --- docs/_plans/2026-09-14-shared-cycle-runner.md | 356 ++++++++++++++++++ docs/decisions.md | 117 +++++- 2 files changed, 472 insertions(+), 1 deletion(-) create mode 100644 docs/_plans/2026-09-14-shared-cycle-runner.md diff --git a/docs/_plans/2026-09-14-shared-cycle-runner.md b/docs/_plans/2026-09-14-shared-cycle-runner.md new file mode 100644 index 0000000..9da2d32 --- /dev/null +++ b/docs/_plans/2026-09-14-shared-cycle-runner.md @@ -0,0 +1,356 @@ +# Plan — one shared Python runner drives one minimal proof cycle on two backends + +Decision record: `docs/decisions.md` — "One shared cycle runner, two backend +adapters, export before cleanup". + +**Baseline:** the SHA of the commit carrying the decision record and this plan. +Leave empty until that commit exists rather than guessing it. + +**Spec:** `/home/paczos/Documents/Warsztat/myDely/docs/projekt-srodowiska-testowego.md` +with `projekt-wspolnej-petli.md`, `projekt-lokalnej-petli-distrobox.md`, +`projekt-lokalnej-petli-vm.md`, and `plan-srodowiska-testowego.md`. Those files +live outside this repository and are not copied into it. + +## Goal + +`experiments/minimal-cycle/run-cycle` accepts one configuration naming +`backend: distrobox|vm`, mints a `run_id`, creates a fresh identifiable +environment, proves that Orca runs inside that environment rather than on the +host, runs exactly one Claude Code worker through Orca on a clean copy of the +project at a pinned revision, runs exactly one independent check, exports +status, elapsed time, diff, logs and result to a host directory, confirms that +export by re-reading every required artifact, and only then destroys per-run +resources while leaving shared images and bases intact. Timeout, cancellation +and error keep the artifacts and classify the run; they never fabricate +success. + +Out of reach in this plan: the separate acceptance test the spec reserves — +a full green cycle on either backend on a host that actually carries Orca, +Claude Code and a built tool image. This host has neither `pulumi` nor an Orca +package inside any container image, so the plan delivers the runner and proves +its rails, not a green backend run. + +## Allowed scope + +``` +experiments/minimal-cycle/** +docs/decisions.md +docs/_plans/2026-09-14-shared-cycle-runner.md +.gitignore +``` + +Command that produced the list: `git ls-files` at baseline showed no +`experiments/` path and no `.gitignore`, so every path under +`experiments/minimal-cycle/` is created by this plan. `docs/decisions.md` is +the repository's durable decision store named by `AGENTS.md`. No colocated +test of an existing source file applies: the repository carries no Python +before this plan (`git ls-files '*.py'` is empty). `tests/contracts.sh` owns +repository shape, not this runner, and yields nothing here. + +## Forbidden scope + +`skills/**` — the delivery contract does not change, so neither plugin +manifest version nor the pin in `tests/contracts.sh` moves. +`tests/contracts.sh` — it checks the dely package's structural contract; this +runner is not part of that contract and adding it would couple the two. +`README.md` — the repository README is the newcomer install path for the dely +plugin; an experiment directory is not an install step. +`.github/workflows/contracts.yml` — the runner's own suite needs a Python +step and a decision about running it in CI that this plan does not take. + +## Execution envelope + +Protected dirty paths: none. `git status --short` was empty at baseline. + +Branch, base, remote, and pull-request target: branch +`kacperpaczos/shared-python-runner-distrobox`, base `main`, remote `origin`, +pull request to `origin/main`. No merge. + +Resolved phase pins: `AGENTS.md` pins `implement` to Cursor Agent CLI +(`cursor-grok-4.6-high`, default effort) and `review` to Codex CLI +(`gpt-5.6-sol`, high). `command -v cursor-agent` is empty on this host, so the +pinned implement harness cannot be launched and the delivery run cannot +dispatch as pinned. This is recorded as the reason the phases were not +dispatched, not worked around by substituting a harness. + +Authority: this plan may branch, commit only its own owned paths, run gates, +push the named branch, and open or update the named pull request. It may not +merge, force-push, stash, reset, clean, or edit anything outside owned scope. + +## Global constraints + +- Python standard library only at runtime. `yaml` is used when importable and + a `.json` configuration is always accepted, so the runner never requires a + package install. +- Repository artifacts are English. +- No file the repository tracks may contain a capital letter immediately + followed by digits at a token boundary, or the string `pace` followed by at + most one character and then `id`. Both are repository closure gates and both + are lexical, so identifiers such as `namespace_id` are forbidden. +- No secret value, and no absolute path to a credential file, is written to a + manifest, receipt, log, image, seed, stack state, or this repository. + +## Tasks + +### 1. The run contract is a typed, validated, secret-free artifact + +**Behaviour.** A configuration file loads into a validated `RunConfig`, a +`run_id` is minted that carries no secret, statuses are a closed set, log text +passes through a pattern redactor, and a manifest is emitted that satisfies +`schema.json`. A configuration naming an absolute allowlist path, an unknown +backend, a non-positive timeout, or an artifact root inside the per-run state +is rejected before anything is created. + +**Direction.** `config.py` holds frozen dataclasses and one `load` that +accepts `.json` always and `.yaml`/`.yml` when `yaml` imports; a missing +`yaml` for a `.yaml` file is a clear error naming the `.json` alternative, not +a silent partial parse. `ids.py` mints +`YYYYMMDDTHHMMSSZ--` where `` is a truncated +hash of the host name rather than the host name itself, and `` comes +from `secrets.token_hex`. `status.py` holds `RunStatus`, `CleanupStatus`, +`ExportStatus`, `PhaseStatus` and the process exit code mapping. +`redact.py` matches by pattern — bearer headers, `sk-`-style keys, `ghp_`, +`xox` Slack forms, compact web-token triples, private-key blocks, and any +`key=value`/`"key": "value"` whose key names a token, secret, password, api +key, cookie, or authorization header — plus any absolute path ending in a +known credential file name. `manifest.py` builds the document and validates it +against `schema.json` with a small in-repo required-field checker, because +`jsonschema` is not in the standard library. + +**Files.** Create `experiments/minimal-cycle/cycle_runner/{__init__,config, +ids,status,redact,hashing,manifest}.py`, `experiments/minimal-cycle/schema.json`, +and their tests under `experiments/minimal-cycle/tests/`. + +**Focused verification.** +`cd experiments/minimal-cycle && python3 -m unittest discover -s tests -t . -v` +fails with the redaction, allowlist and manifest tests red before the module +exists and green after. + +**Document impact.** `experiments/minimal-cycle/README.md` owns the +configuration reference; it is written in task 6 and must match the fields this +task validates. + +### 2. The shared lifecycle exports before it destroys, and classifies rather than fabricates + +**Behaviour.** `run_cycle(config, adapter)` walks prepare, create, bootstrap, +identity, task, check, collect, export, cleanup, close and returns a +`RunResult` carrying the same fields for every backend. Export is confirmed by +re-reading each required artifact from the host path and recomputing its +digest. Cleanup runs only after that confirmation; when export is not +confirmed, no destroy is attempted and `cleanup_status` is `RESIDUE` with the +reason. A task that exceeds `timeout_seconds` collects status and logs, +requests a confirmed stop, verifies it, and settles `TIMEOUT` with every +artifact already exported. An identity verdict that matches the host settles +`BLOCKED` and never runs the task. + +**Direction.** `proc.py` runs argv with a deadline, captures stdout and stderr +separately, and returns a `CommandResult` that records argv, exit code, +timings and whether the deadline fired; every captured stream is redacted +before it reaches disk. `hostinfo.py` snapshots the host before and after. +`probe.py` carries one POSIX-shell probe emitting `key=value` lines, a parser, +and `verdict(host, env, expectations)` which rejects a snapshot that carries +no environment marker and repeats the host's own name, machine identity and +home. `project.py` exports the pinned revision with `git archive`, keeps a +pristine baseline, and computes the patch on the host with `difflib` over the +fetched tree, so no diff tool is required inside the environment. +`export.py` writes, then verifies, then writes the receipt. `cleanup.py` +refuses any target that is not a declared per-run resource and asserts the +declared shared resources still exist afterwards. `lifecycle.py` sequences the +phases and owns the classification. + +**Files.** Create `experiments/minimal-cycle/cycle_runner/{proc,hostinfo,probe, +project,export,cleanup,lifecycle,result}.py` and +`experiments/minimal-cycle/tests/{fakes.py,test_probe.py,test_project.py, +test_export.py,test_cleanup.py,test_lifecycle.py}`. + +**Focused verification.** The suite above, with `test_lifecycle.py` driving a +fake adapter through the settled, blocked, timeout and export-unconfirmed +paths and asserting the order of adapter calls. + +**Document impact.** The README's lifecycle section and `schema.json`'s +`cleanup` and `export` blocks must name exactly the states this task returns. + +### 3. The Distrobox adapter creates a fresh box with its own home and never mounts the host home + +**Behaviour.** `DistroboxAdapter` writes a Distrobox Assemble manifest into the +per-run directory, creates the box from it, executes commands inside it, +moves trees in and out, stops it, removes it, and reports which resources were +per-run. The generated manifest names a per-run home and only the mounts the +configuration lists; it never names the host home directory, and a +configuration that asks for one is rejected. + +**Direction.** Use `distrobox assemble create --file ` and +`distrobox assemble rm --file `, with `distrobox enter --` +for execution and `distrobox list` for existence proof, as those are the +documented general interfaces. Because the per-run home is a host directory, +`put_tree` and `fetch_tree` are host copies and need no transport. Do not emit +`--unshare-all`; record in the README that desktop integration is a deliberate +prototype trade-off rather than a sandbox. + +**Files.** Create `experiments/minimal-cycle/cycle_runner/adapters/{__init__, +base,distrobox}.py` and `experiments/minimal-cycle/tests/test_adapter_distrobox.py`. + +**Focused verification.** Tests assert the rendered manifest text, that a +configuration naming the host home is rejected, and that the destroy plan lists +only per-run paths. A dry-run integration test runs +`distrobox assemble create --dry-run --file ` when `distrobox` is on +`PATH` and skips otherwise. + +**Document impact.** README backend section for Distrobox. + +### 4. The VM adapter builds per-run domain, overlay and seed over an unmodified base + +**Behaviour.** `VmAdapter` renders a Pulumi Python program and stack settings +into a per-run directory, records the pinned provider and version, renders a +cloud-init NoCloud seed that carries no credential, creates the overlay as a +qcow2 image backed by the preserved base, brings the stack up, executes +commands inside the guest over the configured transport, moves trees, destroys +the stack, and reports per-run versus shared resources. The base image is +never opened for writing and never appears in a destroy plan. + +**Direction.** The adapter refuses to run when `pulumi` is absent, when the +base image digest does not match the pinned value, or when the provider +version is unpinned — each as a preflight failure naming the missing fact, not +a guess. Overlay creation uses `qemu-img create -f qcow2 -b -F qcow2`, +which the tool documents, and is never run against an image an active domain +holds. The seed is `user-data` plus `meta-data` on a `cidata` volume. + +**Files.** Create `experiments/minimal-cycle/cycle_runner/adapters/{vm.py, +vm_program.py,vm_seed.py}` and `experiments/minimal-cycle/tests/test_adapter_vm.py`. + +**Focused verification.** Tests assert the seed carries none of the auth +values, that the destroy plan excludes the base image, that a wrong base digest +blocks, and that the overlay command names the base as a backing file rather +than as the output. + +**Document impact.** README backend section for the virtual machine, including +that the tool image is built separately and the runner only consumes it. + +### 5. Auth reaches the environment by one declared method and leaves no secret behind + +**Behaviour.** `bootstrap_auth(config, adapter)` implements exactly the +configured mode, writes an `auth-receipt.json` naming the method, the +non-secret reference, the transfer target and the observed status, and never +records a secret value, length, prefix or absolute source path. `existing_login` +copies only relative allowlist entries into the per-run configuration +directory with owner-only permissions; `short_lived_token` passes a value +through the process environment for the run and never writes it; `api_key_helper` +records the helper contract and passes no value at all. The worker launcher +builds the Orca argv from pinned values, records it, and waits for the +worker's own settling message. + +**Direction.** Auth material is removed from the per-run configuration +directory before cleanup, and the removal is verified. The launcher composes +`orca orchestration run-create`, `worker-start` with `--agent`, `--model`, +`--effort` and `--json`, and `check --wait --types worker_done,escalation,question`, +which are the flags this host's Orca reports. It writes the worker prompt to a +file inside the environment rather than inlining it as a shell argument. + +**Files.** Create `experiments/minimal-cycle/cycle_runner/{auth,worker}.py`, +`experiments/minimal-cycle/fixtures/evidence-task/prompt.md`, and +`experiments/minimal-cycle/tests/{test_auth.py,test_worker.py}`. + +**Focused verification.** Tests feed a known token through every mode and +assert it appears in no receipt, manifest, rendered seed or log, and that the +launcher's argv carries the pinned model and effort. + +**Document impact.** README auth section, stating that the runner verifies the +method and status and never reads a secret into an artifact. + +### 6. One command runs, reports, and proves the rails on this host + +**Behaviour.** `run-cycle preflight` and `run-cycle run` are usable from the +experiment directory, exit with the mapped code, and write the artifact tree +the spec names. A real preflight on this host reports the Distrobox backend +usable and the virtual-machine backend blocked on the absent `pulumi`, and a +real Distrobox cycle creates a fresh box, proves it is not the host, stops at +the Orca identity gate because the image carries no Orca, exports every +artifact, and destroys only the per-run box and home. + +**Direction.** The command-line interface is `argparse` with subcommands +`preflight`, `run`, and `selftest`. Evidence from the real runs is committed +under `experiments/minimal-cycle/evidence/` with digests redacted where +required; the artifact root itself stays untracked through `.gitignore`. + +**Files.** Create `experiments/minimal-cycle/{run-cycle,README.md, +config.distrobox.example.yaml,config.vm.example.yaml}`, +`experiments/minimal-cycle/cycle_runner/cli.py`, +`experiments/minimal-cycle/evidence/**`, and `.gitignore`. + +**Focused verification.** The real runs above, with their exported manifests +quoted in the handoff. + +**Document impact.** `docs/decisions.md` gains the settled record; this plan is +deleted in the release commit. + +## Acceptance + +| Requirement | Instrument | Counterexample | Observed red | +| --- | --- | --- | --- | +| No silent host fallback: a probe result equal to the host's own snapshot is rejected | `tests/test_probe.py::test_host_snapshot_is_rejected` | A verdict that only asserts `command -v orca` returned zero and an exit status of zero — present, runs, returns a pass, and passes identically when the probe ran on the host | | +| Export is confirmed before any destroy | `tests/test_lifecycle.py::test_no_destroy_when_export_unconfirmed` | A lifecycle that destroys in a `finally` block and then reports the export status — it runs, returns a pass on the happy path, and still deletes evidence when export fails | | +| The receipt proves bytes on the host, not bytes in memory | `tests/test_export.py::test_receipt_fails_when_artifact_changes_after_write` | A receipt built from the buffer that was written, which passes every happy-path test while never re-reading the file | | +| Cleanup removes per-run resources only | `tests/test_cleanup.py::test_shared_base_survives_cleanup` | A cleanup that removes the overlay's parent directory, which passes when the base lives elsewhere and destroys it when it does not | | +| Redaction is by pattern, not by known literal | `tests/test_redact.py::test_token_never_seen_before_is_redacted` | A redactor seeded with the tokens the run knows about, which passes every test that reuses those tokens | | +| Timeout keeps the artifacts and classifies the run | `tests/test_lifecycle.py::test_timeout_exports_before_stop_and_settles_timeout` | A lifecycle that raises on the deadline and lets the caller clean up, which passes a test asserting only the returned status | | +| `run_id` carries no secret and stays unique and sortable | `tests/test_ids.py::test_run_id_excludes_auth_reference_and_host_name` | An id that appends `auth.reference` for traceability, which is unique, sortable, and leaks | | +| The manifest carries every required field on every path, including failures | `tests/test_manifest.py::test_required_fields_present_on_blocked_run` | A manifest writer that fills `cleanup` and `check` only when they ran, which validates on the settled path | | +| Both adapters satisfy one result contract | `tests/test_adapter_contract.py::test_every_adapter_implements_the_interface` | An adapter that returns a plain dictionary with the same keys, which reads the same in a manifest and drops the declared per-run resource list | | +| The Distrobox manifest never mounts the host home | `tests/test_adapter_distrobox.py::test_host_home_mount_is_refused` | An adapter that passes the configured mounts through unchanged, which is correct for every configuration that does not ask for the host home | | +| The virtual-machine base image is never written and never destroyed | `tests/test_adapter_vm.py::test_base_image_is_backing_file_and_not_in_destroy_plan` | An overlay command that names the base as the output path, which produces a working disk and silently consumes the shared base | | +| No credential reaches the seed, the stack settings, or the receipt | `tests/test_auth.py::test_no_auth_material_in_any_rendered_artifact` | A receipt that records the token's length and first four characters for debugging, which contains no full secret and still discloses one | | +| Preflight blocks instead of inventing a path | `experiments/minimal-cycle/run-cycle preflight --config config.vm.example.yaml` on this host | A preflight that reports the backend usable because the adapter module imported, which passes on a host with no `pulumi` at all | | +| The whole cycle runs for real and stops at the identity gate rather than on the host | `experiments/minimal-cycle/run-cycle run --config ` on this host | A run that finds no Orca inside the box and uses the host's `orca`, which produces a complete green manifest | | + +**Cannot be observed:** that Orca actually starts, shows a window, and drives a +Claude Code worker inside either backend. No image on this host carries Orca, +`pulumi` is absent, and no tool image has been built, so every instrument here +stops at the identity gate or below it. The separate acceptance test the spec +reserves is the only thing that can observe those, and this plan does not +claim them. + +## Stop conditions + +`BLOCKED` rather than a workaround when: the pinned implement or review +harness is absent and a substitute would have to be chosen; a real cycle would +need a credential to be read; a backend would have to fall back to the host to +produce a green result; or `pulumi`, an Orca package, or a verified base image +would have to be installed on this host to finish a task. + +Assumptions that hold for one environment and are not verified for the others: +the Distrobox path is exercised against rootless Podman with +`docker.io/library/ubuntu:24.04` already present locally; nothing here verifies +Docker as the container backend, another base image, or another architecture. +The virtual-machine path is exercised only through rendering and refusal — +`qemu:///session` answers and `/dev/kvm` exists, but no domain is created. + +## Closure gates + +``` +git diff --check +bash -n tests/contracts.sh +jq -e . plugin.json .claude-plugin/plugin.json .claude-plugin/marketplace.json .codex-plugin/plugin.json .cursor-plugin/plugin.json >/dev/null +bash tests/contracts.sh +test "$(wc -l < tests/contracts.sh)" -le 280 +test ! -e git-hooks/pre-push +test ! -e docs/delivery-log.md +test ! -e docs/findings.md +test ! -e docs/harness-surface.md +test ! -e docs/options.md +test ! -e docs/_plans/2026-08-24-automation-first-dely-design.md +test ! -e bin/delivery-doctor +test ! -e bin/delivery-evidence +test ! -e hooks/hooks.json +test ! -e hooks/grok-hooks.json.template +test ! -e hooks/post-tool-journal.sh +test ! -e hooks/session-start-context.sh +git grep -Ei 'pace.?id' -- . ':!docs/_plans' && exit 1 || true +git grep -E '(^|[^A-Za-z0-9])[A-Z][0-9]+[a-z]?([^A-Za-z0-9]|$)' -- . ':!docs/_plans' && exit 1 || true +``` + +All run from the repository root. The runner's own suite runs from +`experiments/minimal-cycle`: + +``` +python3 -m unittest discover -s tests -t . -v +``` diff --git a/docs/decisions.md b/docs/decisions.md index 5578fcb..50625c7 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -3,12 +3,127 @@ What has been settled, what is still open, and what was rejected and why. Rationale is kept because the reasons are the reusable part. -Last updated 2026-09-06. +Last updated 2026-09-14. --- ## Settled +### 2026-09-14 — One shared cycle runner, two backend adapters, export before cleanup + +#### Context + +The delivery contract has never been exercised against a disposable +environment. Every observation so far comes from this checkout on this host, +so a run that silently used the host's own Orca, the host's own checkout, or +the host's own login would look exactly like a run that used an isolated one. +The design handed to this delivery asks a narrower question than a benchmark +does: can one cycle be created, driven, observed, exported and destroyed +without losing its evidence? + +Two isolation mechanisms were on the table and neither subsumes the other. +Distrobox integrates deliberately with the host — home directory, graphical +sockets, audio, message bus — and its own documentation warns against reading +that as container-grade isolation. A virtual machine under `libvirt` with +`qemu` isolates far more and costs a base image, a tool image, a seed and an +address for reaching into the guest. Choosing one would have settled the +isolation question by deleting the other half of it. + +The failure this record is built around is not a weak backend. It is a green +manifest produced by a run that never entered the environment, or a clean +cleanup that removed the evidence that would have shown it. + +#### Decision + +`experiments/minimal-cycle/` carries one Python runner with one lifecycle and +two backend adapters behind one interface. The runner is a coordinator and a +redactor; it is not a container manager, a virtual-machine manager, a +scheduler, or a second orchestration next to Orca. Each adapter drives the +documented interface of its own tool — Distrobox Assemble and the Distrobox +command line for one, Pulumi with the `libvirt` provider over `qemu` for the +other — and both return the same result contract. + +Four rails are load-bearing and each is proved by an instrument that rejects a +present, running, passing implementation rather than an absent one. + +An identity verdict precedes the task. The runner runs the same probe on the +host and inside the environment and refuses to continue when the environment's +answer carries no environment marker and repeats the host's own name, machine +identity and home. Finding no Orca inside the environment is a blocked run, +never a reason to use the host's. + +Export precedes destruction, and the receipt is a re-read. Every required +artifact is written to the host, then read back from that host path and +digested again; the run is exportable only when every one of them matches. +Cleanup is attempted only after that confirmation. An unconfirmed export +leaves the environment standing and records residue with its reason. + +Cleanup is a declared list, not a path expression. Each adapter names its +per-run resources and its shared ones, and the runner refuses any target +outside the per-run list. A shared base image, a shared tool image and a +shared configuration survive every terminal state by construction rather than +by the destroy command happening to point elsewhere. + +Redaction is by pattern. A token the run has never seen is redacted because of +its shape, not because it was on a list, and an absolute path to a credential +file is replaced before it reaches a log, a manifest, a seed, a stack setting +or this repository. + +#### Alternatives considered + +One backend only, deferring the other. It would have halved the work and +settled by omission the one architectural question actually in play — whether +the lifecycle can be written once. Writing both adapters against one interface +is what tests that claim. + +`libvirt-python` directly instead of Pulumi. It is the lower-level official +binding, and using it would have meant writing domain, disk, state and cleanup +management inside this runner: exactly the second manager the design forbids. + +A single shell script per backend. It is smaller, and it cannot express the +one ordering that matters. Export-then-verify-then-destroy, with a refusal in +the middle, is control flow with state, and the rails above are testable only +because they are functions rather than lines in a script. + +Writing the diff inside the environment with `diff` or `git`. It adds a tool +requirement to every image and makes the patch a property of the environment +rather than of the runner. The tree is fetched and the patch is computed on +the host with the standard library instead, identically for both backends. + +#### Consequences + +The repository now carries Python, which it did not before. It is confined to +`experiments/minimal-cycle/`, depends on the standard library at runtime, and +is not part of the dely plugin surface: no skill, no plugin manifest and no +structural contract check moves because of it. + +The runner is ready for the separate acceptance test the design reserves, and +it has not passed that test. This host carries no `pulumi`, no Orca package +inside any container image, and no built tool image, so nothing here observes +Orca starting in either backend, a window bound to that instance, a Claude +Code worker driven through it, or a login surviving a run. Those remain +unproven, and the runner's rails are written so that the absence shows up as a +blocked run rather than as a green one. + +This decision does not make Distrobox a sandbox and does not claim the two +backends are equivalent. It claims one lifecycle can drive both and that the +evidence survives every way the cycle can end. + +#### Non-goals + +Measuring start-up time, model quality, tester quality, cost, or any task +matrix. Building an image cache engine. Implementing the tester, builder, +reviewer and committer roles. Running the cycle under continuous integration. + +#### Deferred + +The real one-cycle acceptance on each backend, triggered by a host that +carries `pulumi`, a verified base image, a built tool image, and an Orca +package installable into the chosen image. A second fixture beyond the single +marker file, triggered by that acceptance passing. Running the runner's own +suite in continuous integration, triggered by a decision about whether this +repository's workflow should carry a Python step. + ### 2026-09-06 — The dispatch prompt carries the acceptance row as written #### Context From 9322cac6c50e8cd869899a1a31a7a683c9b6ad72 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Mon, 14 Sep 2026 23:25:16 +0200 Subject: [PATCH 02/42] Add the cycle run contract: validated config, run identifiers, closed statuses, pattern redaction, result contract and a schema-validated manifest. --- .gitignore | 4 + docs/_plans/2026-09-14-shared-cycle-runner.md | 3 +- .../minimal-cycle/cycle_runner/__init__.py | 0 .../cycle_runner/adapters/__init__.py | 0 .../minimal-cycle/cycle_runner/config.py | 544 ++++++++++++++++++ .../minimal-cycle/cycle_runner/hashing.py | 46 ++ experiments/minimal-cycle/cycle_runner/ids.py | 59 ++ .../minimal-cycle/cycle_runner/manifest.py | 129 +++++ .../minimal-cycle/cycle_runner/redact.py | 116 ++++ .../minimal-cycle/cycle_runner/result.py | 309 ++++++++++ .../minimal-cycle/cycle_runner/status.py | 68 +++ experiments/minimal-cycle/schema.json | 188 ++++++ experiments/minimal-cycle/tests/__init__.py | 0 .../minimal-cycle/tests/test_config.py | 227 ++++++++ .../minimal-cycle/tests/test_hashing.py | 68 +++ experiments/minimal-cycle/tests/test_ids.py | 78 +++ .../minimal-cycle/tests/test_manifest.py | 149 +++++ .../minimal-cycle/tests/test_redact.py | 120 ++++ .../minimal-cycle/tests/test_result.py | 84 +++ .../minimal-cycle/tests/test_status.py | 61 ++ 20 files changed, 2251 insertions(+), 2 deletions(-) create mode 100644 .gitignore create mode 100644 experiments/minimal-cycle/cycle_runner/__init__.py create mode 100644 experiments/minimal-cycle/cycle_runner/adapters/__init__.py create mode 100644 experiments/minimal-cycle/cycle_runner/config.py create mode 100644 experiments/minimal-cycle/cycle_runner/hashing.py create mode 100644 experiments/minimal-cycle/cycle_runner/ids.py create mode 100644 experiments/minimal-cycle/cycle_runner/manifest.py create mode 100644 experiments/minimal-cycle/cycle_runner/redact.py create mode 100644 experiments/minimal-cycle/cycle_runner/result.py create mode 100644 experiments/minimal-cycle/cycle_runner/status.py create mode 100644 experiments/minimal-cycle/schema.json create mode 100644 experiments/minimal-cycle/tests/__init__.py create mode 100644 experiments/minimal-cycle/tests/test_config.py create mode 100644 experiments/minimal-cycle/tests/test_hashing.py create mode 100644 experiments/minimal-cycle/tests/test_ids.py create mode 100644 experiments/minimal-cycle/tests/test_manifest.py create mode 100644 experiments/minimal-cycle/tests/test_redact.py create mode 100644 experiments/minimal-cycle/tests/test_result.py create mode 100644 experiments/minimal-cycle/tests/test_status.py diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..8664665 --- /dev/null +++ b/.gitignore @@ -0,0 +1,4 @@ +__pycache__/ +*.py[cod] +experiments/minimal-cycle/.artifacts/ +experiments/minimal-cycle/.state/ diff --git a/docs/_plans/2026-09-14-shared-cycle-runner.md b/docs/_plans/2026-09-14-shared-cycle-runner.md index 9da2d32..6c1643c 100644 --- a/docs/_plans/2026-09-14-shared-cycle-runner.md +++ b/docs/_plans/2026-09-14-shared-cycle-runner.md @@ -3,8 +3,7 @@ Decision record: `docs/decisions.md` — "One shared cycle runner, two backend adapters, export before cleanup". -**Baseline:** the SHA of the commit carrying the decision record and this plan. -Leave empty until that commit exists rather than guessing it. +**Baseline:** `6688d475413f80c862e131f1ebc93cf27a5fc830`. **Spec:** `/home/paczos/Documents/Warsztat/myDely/docs/projekt-srodowiska-testowego.md` with `projekt-wspolnej-petli.md`, `projekt-lokalnej-petli-distrobox.md`, diff --git a/experiments/minimal-cycle/cycle_runner/__init__.py b/experiments/minimal-cycle/cycle_runner/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/experiments/minimal-cycle/cycle_runner/adapters/__init__.py b/experiments/minimal-cycle/cycle_runner/adapters/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py new file mode 100644 index 0000000..f1848ef --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -0,0 +1,544 @@ +"""The run configuration. + +One document names the backend, the pinned revisions and versions, the single +task and the single check, where artifacts land, and how the environment +authenticates. It never names a credential value: a key whose name means +"secret" is refused, and so is a value that redaction would change. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass, field, replace +from pathlib import Path +from typing import Any, Mapping, Sequence + +from . import redact + +try: # pragma: no cover - environment dependent + import yaml as _DEFAULT_YAML +except ImportError: # pragma: no cover - environment dependent + _DEFAULT_YAML = None + +BACKENDS = ("distrobox", "vm") +AUTH_MODES = ("existing_login", "short_lived_token", "api_key_helper") + +#: Keys whose name mentions a secret but whose value is the *name* of +#: something rather than the thing itself. +NAME_ONLY_KEYS = frozenset({"token_env"}) + +_UNSET = object() + + +class ConfigError(ValueError): + """The configuration document cannot be used as written.""" + + +def _fail(message: str) -> None: + raise ConfigError(message) + + +def _section(document: Mapping[str, Any], name: str, *, required: bool) -> dict: + value = document.get(name) + if value is None: + if required: + _fail(f"missing required section: {name}") + return {} + if not isinstance(value, Mapping): + _fail(f"section {name} must be a mapping") + return dict(value) + + +def _reject_unknown(document: Mapping[str, Any], allowed: Sequence[str], where: str) -> None: + for key in document: + if key not in allowed: + _fail(f"unknown key in {where}: {key}") + + +def _text(document: Mapping[str, Any], key: str, where: str, default=_UNSET) -> str: + if key not in document: + if default is _UNSET: + _fail(f"missing required field: {where}.{key}" if where else f"missing required field: {key}") + return default + value = document[key] + if not isinstance(value, str) or not value.strip(): + _fail(f"field {where}.{key} must be a non-empty string" if where else f"field {key} must be a non-empty string") + return value + + +def _positive_int(document: Mapping[str, Any], key: str, where: str, default=_UNSET) -> int: + if key not in document: + if default is _UNSET: + _fail(f"missing required field: {key}") + return default + value = document[key] + if isinstance(value, bool) or not isinstance(value, int) or value <= 0: + _fail(f"field {key} must be a positive whole number, got {value!r}") + return value + + +def _absolute(document: Mapping[str, Any], key: str) -> Path: + raw = _text(document, key, "") + path = Path(raw) + if not path.is_absolute(): + _fail(f"field {key} must be an absolute path, got {raw!r}") + return path + + +def _relative(value: str, where: str) -> str: + candidate = Path(value) + if candidate.is_absolute() or ".." in candidate.parts: + _fail(f"{where} must be a relative path that stays inside its root, got {value!r}") + return candidate.as_posix() + + +def _argv(document: Mapping[str, Any], key: str, where: str, default=_UNSET) -> tuple[str, ...]: + if key not in document: + if default is _UNSET: + _fail(f"missing required field: {where}.{key}") + return tuple(default) + value = document[key] + if not isinstance(value, (list, tuple)) or not value: + _fail(f"field {where}.{key} must be a non-empty list of strings") + for item in value: + if not isinstance(item, str): + _fail(f"field {where}.{key} must contain only strings") + return tuple(value) + + +def refuse_secrets(document: Any, trail: str = "") -> None: + """Refuse a document that names a secret or carries a secret-shaped value.""" + if isinstance(document, Mapping): + for key, value in document.items(): + here = f"{trail}.{key}" if trail else str(key) + if ( + isinstance(key, str) + and key not in NAME_ONLY_KEYS + and redact.SENSITIVE_KEY.match(key) + ): + _fail(f"configuration key names a secret and must not exist: {here}") + refuse_secrets(value, here) + elif isinstance(document, (list, tuple)): + for index, item in enumerate(document): + refuse_secrets(item, f"{trail}[{index}]") + elif isinstance(document, str): + if not redact.looks_secret_free(document): + _fail(f"configuration value looks like a secret at {trail or 'the document root'}") + + +@dataclass(frozen=True) +class ProjectConfig: + source: Path + revision: str + environment_path: str = "project" + + def to_document(self) -> dict[str, Any]: + return { + "source": str(self.source), + "revision": self.revision, + "environment_path": self.environment_path, + } + + +@dataclass(frozen=True) +class TaskConfig: + marker: str + relative_path: str + prompt_path: str | None = None + + def to_document(self) -> dict[str, Any]: + document: dict[str, Any] = { + "marker": self.marker, + "relative_path": self.relative_path, + } + if self.prompt_path is not None: + document["prompt_path"] = self.prompt_path + return document + + +@dataclass(frozen=True) +class CheckConfig: + relative_path: str + expected_marker: str + + def to_document(self) -> dict[str, Any]: + return { + "relative_path": self.relative_path, + "expected_marker": self.expected_marker, + } + + +@dataclass(frozen=True) +class AuthConfig: + mode: str + reference: str + allowlist: tuple[str, ...] = () + token_env: str | None = None + helper_argv: tuple[str, ...] = () + + def to_document(self) -> dict[str, Any]: + document: dict[str, Any] = {"mode": self.mode, "reference": self.reference} + if self.allowlist: + document["allowlist"] = list(self.allowlist) + if self.token_env is not None: + document["token_env"] = self.token_env + if self.helper_argv: + document["helper_argv"] = list(self.helper_argv) + return document + + +@dataclass(frozen=True) +class OrcaConfig: + version: str + model: str + effort: str + agent: str = "claude" + status_argv: tuple[str, ...] = ("orca", "status", "--json") + version_argv: tuple[str, ...] = ("orca", "--version") + run_objective: str = "dely minimal cycle" + + def to_document(self) -> dict[str, Any]: + return { + "version": self.version, + "model": self.model, + "effort": self.effort, + "agent": self.agent, + "status_argv": list(self.status_argv), + "version_argv": list(self.version_argv), + "run_objective": self.run_objective, + } + + +@dataclass(frozen=True) +class DistroboxConfig: + image: str + container_prefix: str + extra_mounts: tuple[str, ...] = () + provision: tuple[tuple[str, ...], ...] = () + + def to_document(self) -> dict[str, Any]: + return { + "image": self.image, + "container_prefix": self.container_prefix, + "extra_mounts": list(self.extra_mounts), + "provision": [list(argv) for argv in self.provision], + } + + +@dataclass(frozen=True) +class VmConfig: + provider: str + provider_version: str + stack_prefix: str + base_image: Path + base_image_sha256: str + guest_user: str + pulumi_binary: str = "pulumi" + connect_uri: str = "qemu:///session" + overlay_size: str = "24g" + memory_mb: int = 4096 + vcpus: int = 2 + graphics: str = "spice" + transport: str = "ssh" + provision: tuple[tuple[str, ...], ...] = () + + def to_document(self) -> dict[str, Any]: + return { + "provider": self.provider, + "provider_version": self.provider_version, + "stack_prefix": self.stack_prefix, + "base_image": str(self.base_image), + "base_image_sha256": self.base_image_sha256, + "guest_user": self.guest_user, + "pulumi_binary": self.pulumi_binary, + "connect_uri": self.connect_uri, + "overlay_size": self.overlay_size, + "memory_mb": self.memory_mb, + "vcpus": self.vcpus, + "graphics": self.graphics, + "transport": self.transport, + "provision": [list(argv) for argv in self.provision], + } + + +@dataclass(frozen=True) +class RunConfig: + backend: str + tested_revision: str + dely_revision: str + claude_code_version: str + timeout_seconds: int + artifact_root: Path + state_root: Path + project: ProjectConfig + task: TaskConfig + check: CheckConfig + auth: AuthConfig + orca: OrcaConfig + distrobox: DistroboxConfig | None = None + vm: VmConfig | None = None + source_path: Path | None = field(default=None, compare=False) + + def to_document(self) -> dict[str, Any]: + document: dict[str, Any] = { + "backend": self.backend, + "tested_revision": self.tested_revision, + "dely_revision": self.dely_revision, + "claude_code_version": self.claude_code_version, + "timeout_seconds": self.timeout_seconds, + "artifact_root": str(self.artifact_root), + "state_root": str(self.state_root), + "project": self.project.to_document(), + "task": self.task.to_document(), + "check": self.check.to_document(), + "auth": self.auth.to_document(), + "orca": self.orca.to_document(), + } + if self.distrobox is not None: + document["distrobox"] = self.distrobox.to_document() + if self.vm is not None: + document["vm"] = self.vm.to_document() + return document + + def with_source(self, path: Path) -> "RunConfig": + return replace(self, source_path=path) + + +_TOP_LEVEL = ( + "backend", + "tested_revision", + "dely_revision", + "claude_code_version", + "timeout_seconds", + "artifact_root", + "state_root", + "project", + "task", + "check", + "auth", + "orca", + "distrobox", + "vm", +) + + +def _project(document: Mapping[str, Any]) -> ProjectConfig: + _reject_unknown(document, ("source", "revision", "environment_path"), "project") + source = Path(_text(document, "source", "project")) + if not source.is_absolute(): + _fail("field project.source must be an absolute path") + return ProjectConfig( + source=source, + revision=_text(document, "revision", "project"), + environment_path=_relative( + _text(document, "environment_path", "project", "project"), + "project.environment_path", + ), + ) + + +def _task(document: Mapping[str, Any]) -> TaskConfig: + _reject_unknown(document, ("marker", "relative_path", "prompt_path"), "task") + return TaskConfig( + marker=_text(document, "marker", "task"), + relative_path=_relative( + _text(document, "relative_path", "task"), "task.relative_path" + ), + prompt_path=document.get("prompt_path"), + ) + + +def _check(document: Mapping[str, Any], task: TaskConfig) -> CheckConfig: + _reject_unknown(document, ("relative_path", "expected_marker"), "check") + return CheckConfig( + relative_path=_relative( + _text(document, "relative_path", "check"), "check.relative_path" + ), + expected_marker=_text(document, "expected_marker", "check", task.marker), + ) + + +def _auth(document: Mapping[str, Any]) -> AuthConfig: + _reject_unknown( + document, ("mode", "reference", "allowlist", "token_env", "helper_argv"), "auth" + ) + mode = _text(document, "mode", "auth") + if mode not in AUTH_MODES: + _fail(f"auth.mode must be one of {', '.join(AUTH_MODES)}, got {mode!r}") + allowlist = tuple(document.get("allowlist") or ()) + for entry in allowlist: + if not isinstance(entry, str): + _fail("auth.allowlist must contain only strings") + _relative(entry, "auth.allowlist entry") + token_env = document.get("token_env") + helper_argv = tuple(document.get("helper_argv") or ()) + if mode == "existing_login": + if not allowlist: + _fail("auth.allowlist must name at least one relative path for existing_login") + elif allowlist: + _fail(f"auth.allowlist is only meaningful for existing_login, not {mode}") + if mode == "short_lived_token" and not token_env: + _fail("auth.token_env must name the variable carrying the short-lived token") + if mode == "api_key_helper" and not helper_argv: + _fail("auth.helper_argv must name the helper command") + return AuthConfig( + mode=mode, + reference=_text(document, "reference", "auth"), + allowlist=allowlist, + token_env=token_env, + helper_argv=helper_argv, + ) + + +def _orca(document: Mapping[str, Any]) -> OrcaConfig: + _reject_unknown( + document, + ("version", "model", "effort", "agent", "status_argv", "version_argv", "run_objective"), + "orca", + ) + return OrcaConfig( + version=_text(document, "version", "orca"), + model=_text(document, "model", "orca"), + effort=_text(document, "effort", "orca"), + agent=_text(document, "agent", "orca", "claude"), + status_argv=_argv(document, "status_argv", "orca", ("orca", "status", "--json")), + version_argv=_argv(document, "version_argv", "orca", ("orca", "--version")), + run_objective=_text(document, "run_objective", "orca", "dely minimal cycle"), + ) + + +def _provision(document: Mapping[str, Any], where: str) -> tuple[tuple[str, ...], ...]: + raw = document.get("provision") or () + if not isinstance(raw, (list, tuple)): + _fail(f"field {where}.provision must be a list of argv lists") + steps = [] + for index, argv in enumerate(raw): + if not isinstance(argv, (list, tuple)) or not argv: + _fail(f"field {where}.provision[{index}] must be a non-empty argv list") + for item in argv: + if not isinstance(item, str): + _fail(f"field {where}.provision[{index}] must contain only strings") + steps.append(tuple(argv)) + return tuple(steps) + + +def _distrobox(document: Mapping[str, Any]) -> DistroboxConfig: + _reject_unknown( + document, ("image", "container_prefix", "extra_mounts", "provision"), "distrobox" + ) + mounts = tuple(document.get("extra_mounts") or ()) + for mount in mounts: + if not isinstance(mount, str): + _fail("distrobox.extra_mounts must contain only strings") + return DistroboxConfig( + image=_text(document, "image", "distrobox"), + container_prefix=_text(document, "container_prefix", "distrobox"), + extra_mounts=mounts, + provision=_provision(document, "distrobox"), + ) + + +def _vm(document: Mapping[str, Any]) -> VmConfig: + _reject_unknown( + document, + ( + "provider", + "provider_version", + "stack_prefix", + "base_image", + "base_image_sha256", + "guest_user", + "pulumi_binary", + "connect_uri", + "overlay_size", + "memory_mb", + "vcpus", + "graphics", + "transport", + "provision", + ), + "vm", + ) + base_image = Path(_text(document, "base_image", "vm")) + if not base_image.is_absolute(): + _fail("field vm.base_image must be an absolute path") + digest = _text(document, "base_image_sha256", "vm") + if len(digest) != 64 or any(character not in "0123456789abcdef" for character in digest): + _fail("field vm.base_image_sha256 must be a lowercase content digest") + return VmConfig( + provider=_text(document, "provider", "vm"), + provider_version=_text(document, "provider_version", "vm"), + stack_prefix=_text(document, "stack_prefix", "vm"), + base_image=base_image, + base_image_sha256=digest, + guest_user=_text(document, "guest_user", "vm"), + pulumi_binary=_text(document, "pulumi_binary", "vm", "pulumi"), + connect_uri=_text(document, "connect_uri", "vm", "qemu:///session"), + overlay_size=_text(document, "overlay_size", "vm", "24g"), + memory_mb=_positive_int(document, "memory_mb", "vm", 4096), + vcpus=_positive_int(document, "vcpus", "vm", 2), + graphics=_text(document, "graphics", "vm", "spice"), + transport=_text(document, "transport", "vm", "ssh"), + provision=_provision(document, "vm"), + ) + + +def from_document(document: Mapping[str, Any]) -> RunConfig: + """Validate a loaded document and return the typed configuration.""" + if not isinstance(document, Mapping): + _fail("the configuration must be a mapping") + refuse_secrets(document) + _reject_unknown(document, _TOP_LEVEL, "the configuration") + + backend = _text(document, "backend", "") + if backend not in BACKENDS: + _fail(f"backend must be one of {', '.join(BACKENDS)}, got {backend!r}") + + artifact_root = _absolute(document, "artifact_root") + state_root = _absolute(document, "state_root") + if artifact_root.is_relative_to(state_root): + _fail("artifact_root must not be inside state_root; cleanup would remove the evidence") + if state_root.is_relative_to(artifact_root): + _fail("state_root must not be inside artifact_root; cleanup would remove the evidence") + + task = _task(_section(document, "task", required=True)) + distrobox_document = _section(document, "distrobox", required=backend == "distrobox") + vm_document = _section(document, "vm", required=backend == "vm") + if backend == "distrobox" and not distrobox_document: + _fail("backend distrobox requires a distrobox section") + if backend == "vm" and not vm_document: + _fail("backend vm requires a vm section") + + return RunConfig( + backend=backend, + tested_revision=_text(document, "tested_revision", ""), + dely_revision=_text(document, "dely_revision", ""), + claude_code_version=_text(document, "claude_code_version", ""), + timeout_seconds=_positive_int(document, "timeout_seconds", ""), + artifact_root=artifact_root, + state_root=state_root, + project=_project(_section(document, "project", required=True)), + task=task, + check=_check(_section(document, "check", required=True), task), + auth=_auth(_section(document, "auth", required=True)), + orca=_orca(_section(document, "orca", required=True)), + distrobox=_distrobox(distrobox_document) if distrobox_document else None, + vm=_vm(vm_document) if vm_document else None, + ) + + +def load(path: Path, *, yaml_module: Any = _UNSET) -> RunConfig: + """Load a configuration from a JSON or YAML file.""" + path = Path(path) + raw = path.read_text(encoding="utf-8") + if path.suffix.lower() in (".yaml", ".yml"): + parser = _DEFAULT_YAML if yaml_module is _UNSET else yaml_module + if parser is None: + _fail( + f"{path} needs a yaml parser this host does not have; " + "write the same document as json and pass that instead" + ) + document = parser.safe_load(raw) + else: + document = json.loads(raw) + return from_document(document).with_source(path) diff --git a/experiments/minimal-cycle/cycle_runner/hashing.py b/experiments/minimal-cycle/cycle_runner/hashing.py new file mode 100644 index 0000000..2c3ebeb --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/hashing.py @@ -0,0 +1,46 @@ +"""Content digests and sizes for export receipts and manifests.""" + +from __future__ import annotations + +import hashlib +from pathlib import Path +from typing import Any + +_CHUNK = 1024 * 1024 + + +def digest_file(path: Path) -> str: + """Return the content digest of one file.""" + digest = hashlib.sha256() + with path.open("rb") as handle: + while True: + chunk = handle.read(_CHUNK) + if not chunk: + break + digest.update(chunk) + return digest.hexdigest() + + +def describe_file(path: Path, *, relative_to: Path) -> dict[str, Any]: + """Describe one file by its relative path, size and digest.""" + resolved = Path(path) + if not resolved.is_file(): + raise FileNotFoundError(str(resolved)) + return { + "path": resolved.relative_to(relative_to).as_posix(), + "size_bytes": resolved.stat().st_size, + "sha256": digest_file(resolved), + } + + +def describe_tree(root: Path) -> list[dict[str, Any]]: + """Describe every regular file under a directory, in path order.""" + root = Path(root) + if not root.is_dir(): + return [] + described = [ + describe_file(path, relative_to=root) + for path in sorted(root.rglob("*")) + if path.is_file() and not path.is_symlink() + ] + return sorted(described, key=lambda entry: entry["path"]) diff --git a/experiments/minimal-cycle/cycle_runner/ids.py b/experiments/minimal-cycle/cycle_runner/ids.py new file mode 100644 index 0000000..c6f0934 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/ids.py @@ -0,0 +1,59 @@ +"""Run identifiers. + +A run identifier has to survive in a directory name, a container name, a +domain name and a log line, and it must not disclose anything about the host +or about how the run authenticates. It therefore carries a timestamp, a +digest of the host name rather than the host name, and fresh entropy. +""" + +from __future__ import annotations + +import hashlib +import re +import secrets +from datetime import datetime, timezone + +RUN_ID_PATTERN = re.compile(r"^\d{8}T\d{6}Z-[0-9a-f]{6}-[0-9a-f]{8}$") + +_TIMESTAMP_FORMAT = "%Y%m%dT%H%M%SZ" +_HOST_SEGMENT_LENGTH = 6 +_ENTROPY_BYTES = 4 +_MAX_RESOURCE_NAME = 63 +_PREFIX_PATTERN = re.compile(r"^[a-z][a-z0-9-]*$") + + +def host_segment(host_name: str) -> str: + """Return a stable, non-reversing short segment for a host name.""" + digest = hashlib.sha256(host_name.encode("utf-8")).hexdigest() + return digest[:_HOST_SEGMENT_LENGTH] + + +def mint_run_id(*, now: datetime, host_name: str) -> str: + """Mint one run identifier for the given instant and host.""" + if now.tzinfo is None or now.utcoffset() is None: + raise ValueError("run identifiers need an aware timestamp") + stamp = now.astimezone(timezone.utc).strftime(_TIMESTAMP_FORMAT) + return f"{stamp}-{host_segment(host_name)}-{secrets.token_hex(_ENTROPY_BYTES)}" + + +def is_run_id(candidate: str) -> bool: + """Report whether a string is shaped like a minted run identifier.""" + return bool(RUN_ID_PATTERN.match(candidate)) + + +def resource_name(prefix: str, run_id: str) -> str: + """Return a backend resource name derived from a run identifier. + + The run identifier is kept whole so two runs never collide; the prefix is + what gives way when the combined name would exceed the limit that + container and domain names share. + """ + if not _PREFIX_PATTERN.match(prefix): + raise ValueError(f"resource prefix is not a safe name: {prefix!r}") + if not is_run_id(run_id): + raise ValueError(f"not a run identifier: {run_id!r}") + tail = run_id.lower() + budget = _MAX_RESOURCE_NAME - len(tail) - 1 + if budget < 1: + raise ValueError("run identifier leaves no room for a resource prefix") + return f"{prefix[:budget].rstrip('-')}-{tail}" diff --git a/experiments/minimal-cycle/cycle_runner/manifest.py b/experiments/minimal-cycle/cycle_runner/manifest.py new file mode 100644 index 0000000..c3dc1da --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/manifest.py @@ -0,0 +1,129 @@ +"""Building and validating the run manifest. + +The manifest is the only durable claim the runner makes, so it is redacted +before it is returned and validated before it is written. Validation uses the +restricted subset of the schema vocabulary this package needs, because the +runtime is the standard library alone. +""" + +from __future__ import annotations + +import json +import re +from pathlib import Path +from typing import Any, Mapping + +from . import redact +from .config import RunConfig +from .result import RunResult + +SCHEMA_VERSION = 1 +SCHEMA_PATH = Path(__file__).resolve().parent.parent / "schema.json" + +_TYPE_NAMES = { + "object": dict, + "array": list, + "string": str, + "number": (int, float), + "integer": int, + "boolean": bool, + "null": type(None), +} + + +class ManifestError(ValueError): + """The manifest does not satisfy the schema.""" + + +def load_schema(path: Path | None = None) -> dict[str, Any]: + """Return the manifest schema document.""" + return json.loads(Path(path or SCHEMA_PATH).read_text(encoding="utf-8")) + + +def _matches_type(value: Any, expected: Any) -> bool: + names = expected if isinstance(expected, list) else [expected] + for name in names: + python_type = _TYPE_NAMES.get(name) + if python_type is None: + continue + if name == "integer" and isinstance(value, bool): + continue + if name == "number" and isinstance(value, bool): + continue + if isinstance(value, python_type): + return True + return False + + +def _check(document: Any, schema: Mapping[str, Any], trail: str, problems: list[str]) -> None: + if "const" in schema and document != schema["const"]: + problems.append(f"{trail or 'document'} must be {schema['const']!r}") + return + if "type" in schema and not _matches_type(document, schema["type"]): + problems.append( + f"{trail or 'document'} must be of type {schema['type']!r}, " + f"got {type(document).__name__}" + ) + return + if "enum" in schema and document not in schema["enum"]: + problems.append(f"{trail or 'document'} must be one of {schema['enum']!r}, got {document!r}") + return + if "pattern" in schema and isinstance(document, str): + if not re.match(schema["pattern"], document): + problems.append(f"{trail or 'document'} does not match {schema['pattern']!r}") + return + if isinstance(document, dict): + for name in schema.get("required", ()): + if name not in document: + problems.append(f"missing required field: {f'{trail}.{name}' if trail else name}") + for name, subschema in schema.get("properties", {}).items(): + if name in document: + _check(document[name], subschema, f"{trail}.{name}" if trail else name, problems) + elif isinstance(document, list) and "items" in schema: + for index, item in enumerate(document): + _check(item, schema["items"], f"{trail}[{index}]", problems) + + +def validate(document: Mapping[str, Any], schema: Mapping[str, Any] | None = None) -> None: + """Raise :class:`ManifestError` naming every problem the schema finds.""" + problems: list[str] = [] + _check(document, schema or load_schema(), "", problems) + if problems: + raise ManifestError("; ".join(problems)) + + +def build( + *, + run_result: RunResult, + run_config: RunConfig, + host_before: Mapping[str, Any], + host_after: Mapping[str, Any], + artifacts: list[Mapping[str, Any]], + tool_versions: Mapping[str, Any], + generated_at: str, +) -> dict[str, Any]: + """Return the redacted manifest document for one run.""" + document = dict(run_result.to_document()) + document.update( + { + "schema_version": SCHEMA_VERSION, + "generated_at": generated_at, + "timeout_seconds": run_config.timeout_seconds, + "config": run_config.to_document(), + "artifact_root": str(run_config.artifact_root / run_result.run_id), + "artifacts": [dict(entry) for entry in artifacts], + "host_before": dict(host_before), + "host_after": dict(host_after), + } + ) + document["versions"] = {**dict(run_result.versions), **dict(tool_versions)} + return redact.structure(document) + + +def write(document: Mapping[str, Any], path: Path) -> None: + """Validate, then write the manifest; an invalid manifest reaches no disk.""" + validate(document) + Path(path).parent.mkdir(parents=True, exist_ok=True) + Path(path).write_text( + json.dumps(document, indent=2, sort_keys=True) + "\n", encoding="utf-8" + ) diff --git a/experiments/minimal-cycle/cycle_runner/redact.py b/experiments/minimal-cycle/cycle_runner/redact.py new file mode 100644 index 0000000..ad56d7a --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/redact.py @@ -0,0 +1,116 @@ +"""Redaction. + +Everything the runner writes to the host passes through here. The rules match +on shape rather than on a list of known values, so a credential this run has +never seen is still removed; a list of known values is accepted as an +addition, never as the mechanism. +""" + +from __future__ import annotations + +import re +from typing import Any, Iterable, Sequence + +VALUE_MARK = "" +BEARER_MARK = "" +TOKEN_MARK = "" +KEY_BLOCK_MARK = "" +PATH_MARK = "" + +_SENSITIVE_WORD = ( + r"(?:token|secret|password|passwd|api[_-]?key|apikey|credential|cookie" + r"|authorization|private[_-]?key|session[_-]?key|passphrase)" +) + +SENSITIVE_KEY = re.compile( + rf"^[\"']?[A-Za-z0-9_.\-]*{_SENSITIVE_WORD}[A-Za-z0-9_.\-]*[\"']?$", + re.IGNORECASE, +) + +_PRIVATE_KEY_BLOCK = re.compile( + r"-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----" +) + +_BEARER = re.compile(r"\bBearer\s+[A-Za-z0-9._\-+/=]+", re.IGNORECASE) + +_ASSIGNMENT = re.compile( + rf"(?P[\"']?[A-Za-z0-9_.\-]*{_SENSITIVE_WORD}[A-Za-z0-9_.\-]*[\"']?)" + r"(?P\s*[:=]\s*)" + r"(?P\"[^\"]*\"|'[^']*'|[^\s,;)\]}]+)", + re.IGNORECASE, +) + +_TOKEN_SHAPES = ( + re.compile(r"\bsk-[A-Za-z0-9_\-]{16,}"), + re.compile(r"\bgh[pousr]_[A-Za-z0-9]{20,}"), + re.compile(r"\bgithub_pat_[A-Za-z0-9_]{20,}"), + re.compile(r"\bxox[baprse]-[A-Za-z0-9\-]{10,}"), + re.compile(r"\bey[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}\.[A-Za-z0-9_\-]{8,}"), +) + +_CREDENTIAL_PATH = re.compile( + r"(?:~|/)[^\s:;,'\"]*/" + r"(?:\.credentials\.json|\.netrc|credentials\.json|credentials" + r"|id_[a-z0-9]+|\.ssh/[^\s:;,'\"]*|\.aws/credentials|\.docker/config\.json)" +) + + +def _redact_assignment(match: "re.Match[str]") -> str: + value = match.group("value") + quote = value[0] if value[:1] in ("\"", "'") else "" + return f"{match.group('key')}{match.group('sep')}{quote}{VALUE_MARK}{quote}" + + +def text(value: str, extra_values: Sequence[str] = ()) -> str: + """Return the string with every recognised secret shape removed.""" + if not isinstance(value, str): + raise TypeError("redact.text needs a string") + cleaned = _PRIVATE_KEY_BLOCK.sub(KEY_BLOCK_MARK, value) + cleaned = _BEARER.sub(f"Bearer {BEARER_MARK}", cleaned) + for shape in _TOKEN_SHAPES: + cleaned = shape.sub(TOKEN_MARK, cleaned) + cleaned = _CREDENTIAL_PATH.sub(PATH_MARK, cleaned) + cleaned = _ASSIGNMENT.sub(_redact_assignment, cleaned) + for literal in sorted({v for v in extra_values if v}, key=len, reverse=True): + cleaned = cleaned.replace(literal, VALUE_MARK) + return cleaned + + +def data(raw: bytes, extra_values: Sequence[str] = ()) -> bytes: + """Redact a captured byte stream without requiring it to be valid text.""" + decoded = raw.decode("utf-8", errors="surrogateescape") + return text(decoded, extra_values).encode("utf-8", errors="surrogateescape") + + +def structure(document: Any, extra_values: Sequence[str] = ()) -> Any: + """Return a redacted deep copy of a JSON-shaped document.""" + if isinstance(document, dict): + result = {} + for key, value in document.items(): + if isinstance(key, str) and SENSITIVE_KEY.match(key): + result[key] = VALUE_MARK + else: + result[key] = structure(value, extra_values) + return result + if isinstance(document, (list, tuple)): + return [structure(item, extra_values) for item in document] + if isinstance(document, str): + return text(document, extra_values) + return document + + +def looks_secret_free(value: str) -> bool: + """Report whether redaction would change the string at all.""" + return text(value) == value + + +def sensitive_keys(document: Any) -> Iterable[str]: + """Yield every dictionary key in the document that names a secret.""" + if isinstance(document, dict): + for key, value in document.items(): + if isinstance(key, str) and SENSITIVE_KEY.match(key): + yield key + yield from sensitive_keys(value) + elif isinstance(document, (list, tuple)): + for item in document: + yield from sensitive_keys(item) diff --git a/experiments/minimal-cycle/cycle_runner/result.py b/experiments/minimal-cycle/cycle_runner/result.py new file mode 100644 index 0000000..5c480ff --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/result.py @@ -0,0 +1,309 @@ +"""The result contract both backend adapters settle into. + +Every block exists on every path. A run that never reached the check still +carries a `check` block saying so, because a manifest that omits what did not +happen reads like a manifest that was never asked. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any, Sequence + +from .status import CleanupStatus, ExportStatus, PhaseStatus, RunStatus + +CONTEXTS = ("host", "environment") + +IDENTITY_ENVIRONMENT = "ENVIRONMENT" +IDENTITY_HOST_FALLBACK = "HOST_FALLBACK" +IDENTITY_UNKNOWN = "UNKNOWN" + + +@dataclass +class CommandRecord: + """One executed command, and which side of the boundary it ran on.""" + + argv: Sequence[str] + exit_code: int | None + started_at: str + finished_at: str + elapsed_seconds: float + timed_out: bool + context: str + stdout_path: str | None = None + stderr_path: str | None = None + + def __post_init__(self) -> None: + if self.context not in CONTEXTS: + raise ValueError( + f"a command runs on the host or in the environment, not {self.context!r}" + ) + + def to_document(self) -> dict[str, Any]: + return { + "argv": list(self.argv), + "exit_code": self.exit_code, + "started_at": self.started_at, + "finished_at": self.finished_at, + "elapsed_seconds": self.elapsed_seconds, + "timed_out": self.timed_out, + "context": self.context, + "stdout_path": self.stdout_path, + "stderr_path": self.stderr_path, + } + + +@dataclass +class PhaseRecord: + """One lifecycle phase and the commands that produced its outcome.""" + + name: str + status: PhaseStatus + started_at: str | None = None + finished_at: str | None = None + elapsed_seconds: float | None = None + detail: str = "" + commands: list[CommandRecord] = field(default_factory=list) + + def to_document(self) -> dict[str, Any]: + return { + "name": self.name, + "status": self.status.value, + "started_at": self.started_at, + "finished_at": self.finished_at, + "elapsed_seconds": self.elapsed_seconds, + "detail": self.detail, + "commands": [command.to_document() for command in self.commands], + } + + +@dataclass +class IdentityRecord: + """Whether the environment answered as itself or as the host.""" + + verdict: str = IDENTITY_UNKNOWN + reason: str = "not attempted" + host: dict[str, Any] = field(default_factory=dict) + environment: dict[str, Any] = field(default_factory=dict) + markers: list[str] = field(default_factory=list) + orca_present: bool = False + orca_version: str | None = None + orca_status_exit_code: int | None = None + + def to_document(self) -> dict[str, Any]: + return { + "verdict": self.verdict, + "reason": self.reason, + "host": dict(self.host), + "environment": dict(self.environment), + "markers": list(self.markers), + "orca_present": self.orca_present, + "orca_version": self.orca_version, + "orca_status_exit_code": self.orca_status_exit_code, + } + + +@dataclass +class WorkerRecord: + """The single Claude Code worker Orca was asked to run.""" + + status: PhaseStatus = PhaseStatus.SKIPPED + agent: str | None = None + model: str | None = None + effort: str | None = None + run_id: str | None = None + dispatch_id: str | None = None + outcome: str | None = None + detail: str = "not attempted" + commands: list[CommandRecord] = field(default_factory=list) + + def to_document(self) -> dict[str, Any]: + return { + "status": self.status.value, + "agent": self.agent, + "model": self.model, + "effort": self.effort, + "run_id": self.run_id, + "dispatch_id": self.dispatch_id, + "outcome": self.outcome, + "detail": self.detail, + "commands": [command.to_document() for command in self.commands], + } + + +@dataclass +class CheckRecord: + """The one independent check, as a process rather than as an opinion.""" + + status: PhaseStatus = PhaseStatus.SKIPPED + argv: Sequence[str] = () + exit_code: int | None = None + started_at: str | None = None + finished_at: str | None = None + elapsed_seconds: float | None = None + stdout_path: str | None = None + stderr_path: str | None = None + expected_marker: str | None = None + detail: str = "not attempted" + + def to_document(self) -> dict[str, Any]: + return { + "status": self.status.value, + "argv": list(self.argv), + "exit_code": self.exit_code, + "started_at": self.started_at, + "finished_at": self.finished_at, + "elapsed_seconds": self.elapsed_seconds, + "stdout_path": self.stdout_path, + "stderr_path": self.stderr_path, + "expected_marker": self.expected_marker, + "detail": self.detail, + } + + +@dataclass +class ExportRecord: + """What the host holds, proved by re-reading it.""" + + status: ExportStatus = ExportStatus.FAILED + confirmed_at: str | None = None + receipt_path: str | None = None + artifacts: list[dict[str, Any]] = field(default_factory=list) + missing: list[str] = field(default_factory=list) + detail: str = "not attempted" + + def to_document(self) -> dict[str, Any]: + return { + "status": self.status.value, + "confirmed_at": self.confirmed_at, + "receipt_path": self.receipt_path, + "artifacts": [dict(entry) for entry in self.artifacts], + "missing": list(self.missing), + "detail": self.detail, + } + + +@dataclass +class CleanupRecord: + """What was removed, what was deliberately kept, and how that was checked.""" + + status: CleanupStatus = CleanupStatus.RESIDUE + reason: str = "not attempted" + removed: list[str] = field(default_factory=list) + retained: list[str] = field(default_factory=list) + shared_preserved: list[str] = field(default_factory=list) + verified: bool = False + + def to_document(self) -> dict[str, Any]: + return { + "status": self.status.value, + "reason": self.reason, + "removed": list(self.removed), + "retained": list(self.retained), + "shared_preserved": list(self.shared_preserved), + "verified": self.verified, + } + + +@dataclass +class AuthRecord: + """The auth method and its observed status, never its material.""" + + mode: str = "unknown" + reference: str = "" + status: PhaseStatus = PhaseStatus.SKIPPED + target: str | None = None + entries: list[str] = field(default_factory=list) + removed_after_run: bool = False + detail: str = "not attempted" + + def to_document(self) -> dict[str, Any]: + return { + "mode": self.mode, + "reference": self.reference, + "status": self.status.value, + "target": self.target, + "entries": list(self.entries), + "removed_after_run": self.removed_after_run, + "detail": self.detail, + } + + +@dataclass +class RunResult: + """One cycle, whatever backend produced it.""" + + run_id: str + backend: str + tested_revision: str + dely_revision: str + environment_id: str | None = None + status: RunStatus = RunStatus.UNKNOWN + started_at: str | None = None + finished_at: str | None = None + elapsed_seconds: float | None = None + failure_classification: str = "" + versions: dict[str, Any] = field(default_factory=dict) + environment: dict[str, Any] = field(default_factory=dict) + phases: list[PhaseRecord] = field(default_factory=list) + identity: IdentityRecord = field(default_factory=IdentityRecord) + worker: WorkerRecord = field(default_factory=WorkerRecord) + check: CheckRecord = field(default_factory=CheckRecord) + export: ExportRecord = field(default_factory=ExportRecord) + cleanup: CleanupRecord = field(default_factory=CleanupRecord) + auth: AuthRecord = field(default_factory=AuthRecord) + + def phase(self, name: str) -> PhaseRecord | None: + for record in self.phases: + if record.name == name: + return record + return None + + def to_document(self) -> dict[str, Any]: + return { + "run_id": self.run_id, + "backend": self.backend, + "tested_revision": self.tested_revision, + "dely_revision": self.dely_revision, + "environment_id": self.environment_id, + "status": self.status.value, + "started_at": self.started_at, + "finished_at": self.finished_at, + "elapsed_seconds": self.elapsed_seconds, + "failure_classification": self.failure_classification, + "versions": dict(self.versions), + "environment": dict(self.environment), + "phases": [phase.to_document() for phase in self.phases], + "identity": self.identity.to_document(), + "worker": self.worker.to_document(), + "check": self.check.to_document(), + "export": self.export.to_document(), + "cleanup": self.cleanup.to_document(), + "auth": self.auth.to_document(), + } + + +def blocked_result( + *, + run_id: str, + backend: str, + tested_revision: str, + dely_revision: str, + reason: str, +) -> RunResult: + """Return a result for a run that stopped before it could prove anything.""" + record = RunResult( + run_id=run_id, + backend=backend, + tested_revision=tested_revision, + dely_revision=dely_revision, + status=RunStatus.BLOCKED, + failure_classification=reason, + ) + record.identity.reason = reason + record.cleanup.reason = reason + record.export.detail = reason + record.check.detail = reason + record.worker.detail = reason + record.auth.detail = reason + return record diff --git a/experiments/minimal-cycle/cycle_runner/status.py b/experiments/minimal-cycle/cycle_runner/status.py new file mode 100644 index 0000000..73043b3 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/status.py @@ -0,0 +1,68 @@ +"""The closed sets of states a cycle can end in, and how they leave the process. + +A status name never stands in for evidence: these values only classify what +the manifest already records. +""" + +from __future__ import annotations + +import enum + + +class RunStatus(enum.Enum): + """How the whole cycle ended.""" + + SETTLED = "SETTLED" + ERROR = "ERROR" + TIMEOUT = "TIMEOUT" + CANCELLED = "CANCELLED" + CLEANUP_FAILED = "CLEANUP_FAILED" + UNKNOWN = "UNKNOWN" + BLOCKED = "BLOCKED" + + +class CleanupStatus(enum.Enum): + """What happened to the per-run resources.""" + + DESTROYED = "DESTROYED" + RESIDUE = "RESIDUE" + UNKNOWN = "UNKNOWN" + + +class ExportStatus(enum.Enum): + """Whether the host holds every artifact the run promised.""" + + CONFIRMED = "CONFIRMED" + PARTIAL = "PARTIAL" + FAILED = "FAILED" + + @property + def permits_cleanup(self) -> bool: + """Only a confirmed export releases the run to destroy anything.""" + return self is ExportStatus.CONFIRMED + + +class PhaseStatus(enum.Enum): + """How one lifecycle phase ended.""" + + OK = "OK" + FAILED = "FAILED" + SKIPPED = "SKIPPED" + TIMEOUT = "TIMEOUT" + BLOCKED = "BLOCKED" + + +_EXIT_CODES = { + RunStatus.SETTLED: 0, + RunStatus.ERROR: 1, + RunStatus.TIMEOUT: 2, + RunStatus.CANCELLED: 3, + RunStatus.BLOCKED: 4, + RunStatus.CLEANUP_FAILED: 5, + RunStatus.UNKNOWN: 6, +} + + +def exit_code(run_status: RunStatus) -> int: + """Return the process exit code that reports this run status.""" + return _EXIT_CODES[run_status] diff --git a/experiments/minimal-cycle/schema.json b/experiments/minimal-cycle/schema.json new file mode 100644 index 0000000..fdaab7c --- /dev/null +++ b/experiments/minimal-cycle/schema.json @@ -0,0 +1,188 @@ +{ + "title": "dely minimal cycle manifest", + "description": "Required shape of the manifest one cycle writes to the host. Validated by cycle_runner.manifest, which supports type, required, properties, items, enum, const and pattern.", + "type": "object", + "required": [ + "schema_version", + "generated_at", + "run_id", + "backend", + "tested_revision", + "dely_revision", + "environment_id", + "status", + "started_at", + "finished_at", + "elapsed_seconds", + "timeout_seconds", + "failure_classification", + "versions", + "environment", + "config", + "artifact_root", + "artifacts", + "host_before", + "host_after", + "phases", + "identity", + "worker", + "check", + "export", + "cleanup", + "auth" + ], + "properties": { + "schema_version": {"const": 1}, + "generated_at": {"type": "string"}, + "run_id": {"type": "string", "pattern": "^\\d{8}T\\d{6}Z-[0-9a-f]{6}-[0-9a-f]{8}$"}, + "backend": {"type": "string", "enum": ["distrobox", "vm"]}, + "tested_revision": {"type": "string"}, + "dely_revision": {"type": "string"}, + "environment_id": {"type": ["string", "null"]}, + "status": { + "type": "string", + "enum": [ + "SETTLED", + "ERROR", + "TIMEOUT", + "CANCELLED", + "CLEANUP_FAILED", + "UNKNOWN", + "BLOCKED" + ] + }, + "started_at": {"type": ["string", "null"]}, + "finished_at": {"type": ["string", "null"]}, + "elapsed_seconds": {"type": ["number", "null"]}, + "timeout_seconds": {"type": "integer"}, + "failure_classification": {"type": "string"}, + "versions": {"type": "object"}, + "environment": {"type": "object"}, + "config": {"type": "object"}, + "artifact_root": {"type": "string"}, + "artifacts": { + "type": "array", + "items": { + "type": "object", + "required": ["path", "size_bytes", "sha256"], + "properties": { + "path": {"type": "string"}, + "size_bytes": {"type": "integer"}, + "sha256": {"type": "string", "pattern": "^[0-9a-f]{64}$"} + } + } + }, + "host_before": {"type": "object"}, + "host_after": {"type": "object"}, + "phases": { + "type": "array", + "items": { + "type": "object", + "required": ["name", "status", "commands"], + "properties": { + "name": {"type": "string"}, + "status": { + "type": "string", + "enum": ["OK", "FAILED", "SKIPPED", "TIMEOUT", "BLOCKED"] + }, + "detail": {"type": "string"}, + "commands": {"type": "array"} + } + } + }, + "identity": { + "type": "object", + "required": ["verdict", "reason", "host", "environment", "orca_present"], + "properties": { + "verdict": { + "type": "string", + "enum": ["ENVIRONMENT", "HOST_FALLBACK", "UNKNOWN"] + }, + "reason": {"type": "string"}, + "host": {"type": "object"}, + "environment": {"type": "object"}, + "markers": {"type": "array"}, + "orca_present": {"type": "boolean"}, + "orca_version": {"type": ["string", "null"]}, + "orca_status_exit_code": {"type": ["integer", "null"]} + } + }, + "worker": { + "type": "object", + "required": ["status", "detail", "commands"], + "properties": { + "status": { + "type": "string", + "enum": ["OK", "FAILED", "SKIPPED", "TIMEOUT", "BLOCKED"] + }, + "agent": {"type": ["string", "null"]}, + "model": {"type": ["string", "null"]}, + "effort": {"type": ["string", "null"]}, + "run_id": {"type": ["string", "null"]}, + "dispatch_id": {"type": ["string", "null"]}, + "outcome": {"type": ["string", "null"]}, + "detail": {"type": "string"}, + "commands": {"type": "array"} + } + }, + "check": { + "type": "object", + "required": ["status", "argv", "exit_code", "detail"], + "properties": { + "status": { + "type": "string", + "enum": ["OK", "FAILED", "SKIPPED", "TIMEOUT", "BLOCKED"] + }, + "argv": {"type": "array"}, + "exit_code": {"type": ["integer", "null"]}, + "started_at": {"type": ["string", "null"]}, + "finished_at": {"type": ["string", "null"]}, + "elapsed_seconds": {"type": ["number", "null"]}, + "stdout_path": {"type": ["string", "null"]}, + "stderr_path": {"type": ["string", "null"]}, + "expected_marker": {"type": ["string", "null"]}, + "detail": {"type": "string"} + } + }, + "export": { + "type": "object", + "required": ["status", "artifacts", "missing", "detail"], + "properties": { + "status": {"type": "string", "enum": ["CONFIRMED", "PARTIAL", "FAILED"]}, + "confirmed_at": {"type": ["string", "null"]}, + "receipt_path": {"type": ["string", "null"]}, + "artifacts": {"type": "array"}, + "missing": {"type": "array"}, + "detail": {"type": "string"} + } + }, + "cleanup": { + "type": "object", + "required": ["status", "reason", "removed", "retained", "verified"], + "properties": { + "status": {"type": "string", "enum": ["DESTROYED", "RESIDUE", "UNKNOWN"]}, + "reason": {"type": "string"}, + "removed": {"type": "array"}, + "retained": {"type": "array"}, + "shared_preserved": {"type": "array"}, + "verified": {"type": "boolean"} + } + }, + "auth": { + "type": "object", + "required": ["mode", "reference", "status", "removed_after_run", "detail"], + "properties": { + "mode": {"type": "string"}, + "reference": {"type": "string"}, + "status": { + "type": "string", + "enum": ["OK", "FAILED", "SKIPPED", "TIMEOUT", "BLOCKED"] + }, + "target": {"type": ["string", "null"]}, + "entries": {"type": "array"}, + "removed_after_run": {"type": "boolean"}, + "detail": {"type": "string"} + } + } + } +} diff --git a/experiments/minimal-cycle/tests/__init__.py b/experiments/minimal-cycle/tests/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/experiments/minimal-cycle/tests/test_config.py b/experiments/minimal-cycle/tests/test_config.py new file mode 100644 index 0000000..31c2e20 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_config.py @@ -0,0 +1,227 @@ +"""The run configuration is explicit, backend-aware, and never carries a secret.""" + +import copy +import json +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import config + + +def minimal_document(**overrides): + document = { + "backend": "distrobox", + "tested_revision": "6688d475413f80c862e131f1ebc93cf27a5fc830", + "dely_revision": "6688d475413f80c862e131f1ebc93cf27a5fc830", + "claude_code_version": "pinned-by-the-operator", + "timeout_seconds": 900, + "artifact_root": "/var/tmp/cycle-artifacts", + "state_root": "/var/tmp/cycle-state", + "project": { + "source": "/home/someone/code/under-test", + "revision": "main", + "environment_path": "project", + }, + "task": {"marker": "dely-cycle-marker", "relative_path": "evidence.txt"}, + "check": {"relative_path": "evidence.txt"}, + "auth": { + "mode": "existing_login", + "reference": "workshop-host-login", + "allowlist": [".claude/.credentials.json"], + }, + "orca": { + "version": "1.4.201", + "model": "pinned-model", + "effort": "high", + "agent": "claude", + }, + "distrobox": { + "image": "docker.io/library/ubuntu:24.04", + "container_prefix": "dely-cycle", + }, + } + document.update(overrides) + return document + + +class LoadTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.addCleanup(self._tmp.cleanup) + + def write(self, document, name="config.json"): + path = self.root / name + path.write_text(json.dumps(document), encoding="utf-8") + return path + + def test_a_valid_document_loads_into_a_typed_configuration(self): + loaded = config.load(self.write(minimal_document())) + self.assertEqual(loaded.backend, "distrobox") + self.assertEqual(loaded.timeout_seconds, 900) + self.assertEqual(loaded.artifact_root, Path("/var/tmp/cycle-artifacts")) + self.assertEqual(loaded.project.environment_path, "project") + self.assertEqual(loaded.distrobox.image, "docker.io/library/ubuntu:24.04") + self.assertIsNone(loaded.vm) + + def test_yaml_is_accepted_when_the_parser_is_available(self): + try: + import yaml + except ImportError: # pragma: no cover - environment dependent + self.skipTest("no yaml parser on this host") + path = self.root / "config.yaml" + path.write_text(yaml.safe_dump(minimal_document()), encoding="utf-8") + self.assertEqual(config.load(path).backend, "distrobox") + + def test_yaml_without_a_parser_names_the_json_alternative(self): + path = self.root / "config.yaml" + path.write_text("backend: distrobox\n", encoding="utf-8") + with self.assertRaises(config.ConfigError) as caught: + config.load(path, yaml_module=None) + self.assertIn("json", str(caught.exception).lower()) + + def test_the_check_marker_defaults_to_the_task_marker(self): + loaded = config.load(self.write(minimal_document())) + self.assertEqual(loaded.check.expected_marker, "dely-cycle-marker") + + def test_an_explicit_check_marker_is_kept(self): + document = minimal_document() + document["check"]["expected_marker"] = "a-different-marker" + loaded = config.load(self.write(document)) + self.assertEqual(loaded.check.expected_marker, "a-different-marker") + + +class RejectionTest(unittest.TestCase): + def assert_rejected(self, document, needle): + with self.assertRaises(config.ConfigError) as caught: + config.from_document(document) + self.assertIn(needle, str(caught.exception)) + + def test_an_unknown_backend_is_refused(self): + self.assert_rejected(minimal_document(backend="kubernetes"), "backend") + + def test_the_selected_backend_needs_its_own_section(self): + document = minimal_document(backend="vm") + document.pop("distrobox") + self.assert_rejected(document, "vm") + + def test_an_unknown_top_level_key_is_refused(self): + self.assert_rejected(minimal_document(harbor={}), "harbor") + + def test_an_unknown_nested_key_is_refused(self): + document = minimal_document() + document["orca"]["sandbox"] = "none" + self.assert_rejected(document, "sandbox") + + def test_a_non_positive_timeout_is_refused(self): + self.assert_rejected(minimal_document(timeout_seconds=0), "timeout_seconds") + + def test_a_relative_artifact_root_is_refused(self): + self.assert_rejected(minimal_document(artifact_root="results"), "absolute") + + def test_an_artifact_root_inside_the_per_run_state_is_refused(self): + document = minimal_document( + artifact_root="/var/tmp/cycle-state/results", + state_root="/var/tmp/cycle-state", + ) + self.assert_rejected(document, "artifact_root") + + def test_a_state_root_inside_the_artifact_root_is_refused(self): + document = minimal_document( + artifact_root="/var/tmp/results", + state_root="/var/tmp/results/state", + ) + self.assert_rejected(document, "state_root") + + def test_an_unknown_auth_mode_is_refused(self): + document = minimal_document() + document["auth"]["mode"] = "whatever-works" + self.assert_rejected(document, "mode") + + def test_an_absolute_allowlist_entry_is_refused(self): + document = minimal_document() + document["auth"]["allowlist"] = ["/etc/hosts"] + self.assert_rejected(document, "relative") + + def test_an_escaping_allowlist_entry_is_refused(self): + document = minimal_document() + document["auth"]["allowlist"] = ["../settings.json"] + self.assert_rejected(document, "relative") + + def test_an_absolute_credential_path_is_refused_before_anything_else(self): + document = minimal_document() + document["auth"]["allowlist"] = ["/home/someone/.claude/.credentials.json"] + self.assert_rejected(document, "secret") + + def test_an_allowlist_outside_the_existing_login_mode_is_refused(self): + document = minimal_document() + document["auth"] = { + "mode": "short_lived_token", + "reference": "workshop-token", + "allowlist": [".claude/.credentials.json"], + "token_env": "CLAUDE_CODE_OAUTH_TOKEN", + } + self.assert_rejected(document, "allowlist") + + def test_a_short_lived_token_mode_needs_the_variable_name(self): + document = minimal_document() + document["auth"] = {"mode": "short_lived_token", "reference": "workshop"} + self.assert_rejected(document, "token_env") + + def test_a_helper_mode_needs_the_helper_argv(self): + document = minimal_document() + document["auth"] = {"mode": "api_key_helper", "reference": "workshop"} + self.assert_rejected(document, "helper_argv") + + def test_an_escaping_task_path_is_refused(self): + document = minimal_document() + document["task"]["relative_path"] = "../evidence.txt" + self.assert_rejected(document, "relative") + + def test_a_missing_required_field_is_named(self): + document = minimal_document() + document.pop("tested_revision") + self.assert_rejected(document, "tested_revision") + + +class SecretRefusalTest(unittest.TestCase): + def assert_rejected(self, document, needle): + with self.assertRaises(config.ConfigError) as caught: + config.from_document(document) + self.assertIn(needle, str(caught.exception)) + + def test_a_key_that_names_a_secret_is_refused(self): + document = minimal_document() + document["auth"]["token"] = "irrelevant" + self.assert_rejected(document, "token") + + def test_a_secret_shaped_value_under_a_dull_key_is_refused(self): + document = minimal_document() + document["orca"]["model"] = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyui" + self.assert_rejected(document, "secret") + + def test_the_variable_name_for_a_token_is_allowed_because_it_is_a_name(self): + document = minimal_document() + document["auth"] = { + "mode": "short_lived_token", + "reference": "workshop-token", + "token_env": "CLAUDE_CODE_OAUTH_TOKEN", + } + loaded = config.from_document(document) + self.assertEqual(loaded.auth.token_env, "CLAUDE_CODE_OAUTH_TOKEN") + + +class DocumentRoundTripTest(unittest.TestCase): + def test_the_configuration_renders_back_to_a_json_shaped_document(self): + document = minimal_document() + loaded = config.from_document(copy.deepcopy(document)) + rendered = loaded.to_document() + self.assertEqual(rendered["backend"], "distrobox") + self.assertEqual(rendered["auth"]["mode"], "existing_login") + json.dumps(rendered) + + def test_the_rendered_document_reloads_unchanged(self): + loaded = config.from_document(minimal_document()) + reloaded = config.from_document(loaded.to_document()) + self.assertEqual(reloaded.to_document(), loaded.to_document()) diff --git a/experiments/minimal-cycle/tests/test_hashing.py b/experiments/minimal-cycle/tests/test_hashing.py new file mode 100644 index 0000000..2440243 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_hashing.py @@ -0,0 +1,68 @@ +"""Digests are how an export receipt proves the host holds the bytes.""" + +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import hashing + + +class DigestFileTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.addCleanup(self._tmp.cleanup) + + def test_digest_reports_size_and_content_digest(self): + target = self.root / "artifact.txt" + target.write_bytes(b"marker") + described = hashing.describe_file(target, relative_to=self.root) + self.assertEqual(described["path"], "artifact.txt") + self.assertEqual(described["size_bytes"], 6) + self.assertEqual(len(described["sha256"]), 64) + + def test_an_empty_file_still_describes(self): + target = self.root / "empty.txt" + target.write_bytes(b"") + described = hashing.describe_file(target, relative_to=self.root) + self.assertEqual(described["size_bytes"], 0) + + def test_changing_one_byte_changes_the_digest(self): + target = self.root / "artifact.txt" + target.write_bytes(b"marker") + before = hashing.describe_file(target, relative_to=self.root)["sha256"] + target.write_bytes(b"markeR") + after = hashing.describe_file(target, relative_to=self.root)["sha256"] + self.assertNotEqual(before, after) + + def test_a_missing_file_raises_rather_than_reporting_zero(self): + with self.assertRaises(FileNotFoundError): + hashing.describe_file(self.root / "absent", relative_to=self.root) + + def test_paths_are_reported_with_forward_slashes(self): + nested = self.root / "logs" / "runner.log" + nested.parent.mkdir() + nested.write_bytes(b"line") + described = hashing.describe_file(nested, relative_to=self.root) + self.assertEqual(described["path"], "logs/runner.log") + + +class DescribeTreeTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.addCleanup(self._tmp.cleanup) + + def test_a_tree_is_described_in_a_stable_order(self): + (self.root / "b.txt").write_bytes(b"two") + (self.root / "a.txt").write_bytes(b"one") + (self.root / "nested").mkdir() + (self.root / "nested" / "c.txt").write_bytes(b"three") + described = hashing.describe_tree(self.root) + self.assertEqual( + [entry["path"] for entry in described], + ["a.txt", "b.txt", "nested/c.txt"], + ) + + def test_an_absent_tree_describes_as_empty(self): + self.assertEqual(hashing.describe_tree(self.root / "absent"), []) diff --git a/experiments/minimal-cycle/tests/test_ids.py b/experiments/minimal-cycle/tests/test_ids.py new file mode 100644 index 0000000..0aec58e --- /dev/null +++ b/experiments/minimal-cycle/tests/test_ids.py @@ -0,0 +1,78 @@ +"""The run identifier is durable, sortable, and free of anything secret.""" + +import re +import unittest +from datetime import datetime, timezone + +from cycle_runner import ids + + +class MintRunIdTest(unittest.TestCase): + def test_run_id_matches_the_documented_shape(self): + minted = ids.mint_run_id( + now=datetime(2026, 9, 14, 22, 15, 30, tzinfo=timezone.utc), + host_name="workshop.example", + ) + self.assertRegex(minted, r"^20260914T221530Z-[0-9a-f]{6}-[0-9a-f]{8}$") + + def test_run_id_excludes_auth_reference_and_host_name(self): + minted = ids.mint_run_id( + now=datetime(2026, 9, 14, 22, 15, 30, tzinfo=timezone.utc), + host_name="workshop.example", + ) + self.assertNotIn("workshop", minted) + self.assertNotIn("example", minted) + + def test_same_host_gives_a_stable_short_segment(self): + first = ids.mint_run_id( + now=datetime(2026, 9, 14, 22, 15, 30, tzinfo=timezone.utc), + host_name="workshop.example", + ) + second = ids.mint_run_id( + now=datetime(2026, 9, 14, 22, 15, 31, tzinfo=timezone.utc), + host_name="workshop.example", + ) + self.assertEqual(first.split("-")[1], second.split("-")[1]) + + def test_run_ids_sort_by_time_and_stay_unique(self): + early = ids.mint_run_id( + now=datetime(2026, 9, 14, 22, 15, 30, tzinfo=timezone.utc), + host_name="workshop.example", + ) + late = ids.mint_run_id( + now=datetime(2026, 9, 14, 22, 15, 31, tzinfo=timezone.utc), + host_name="workshop.example", + ) + self.assertLess(early, late) + minted = { + ids.mint_run_id( + now=datetime(2026, 9, 14, 22, 15, 30, tzinfo=timezone.utc), + host_name="workshop.example", + ) + for _ in range(64) + } + self.assertEqual(len(minted), 64) + + def test_naive_timestamps_are_refused(self): + with self.assertRaises(ValueError): + ids.mint_run_id(now=datetime(2026, 9, 14, 22, 15, 30), host_name="h") + + def test_run_id_is_safe_as_a_path_segment(self): + minted = ids.mint_run_id( + now=datetime(2026, 9, 14, 22, 15, 30, tzinfo=timezone.utc), + host_name="../../etc", + ) + self.assertIsNone(re.search(r"[^0-9A-Za-z-]", minted)) + + +class ResourceNameTest(unittest.TestCase): + def test_resource_name_is_prefixed_and_bounded(self): + name = ids.resource_name("dely-cycle", "20260914T221530Z-abc123-0123abcd") + self.assertTrue(name.startswith("dely-cycle-")) + self.assertLessEqual(len(name), 63) + self.assertRegex(name, r"^[a-z0-9][a-z0-9-]*[a-z0-9]$") + + def test_resource_name_keeps_runs_distinct(self): + first = ids.resource_name("dely-cycle", "20260914T221530Z-abc123-0123abcd") + second = ids.resource_name("dely-cycle", "20260914T221530Z-abc123-0123abce") + self.assertNotEqual(first, second) diff --git a/experiments/minimal-cycle/tests/test_manifest.py b/experiments/minimal-cycle/tests/test_manifest.py new file mode 100644 index 0000000..b432212 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_manifest.py @@ -0,0 +1,149 @@ +"""The manifest is the run's durable record and it validates against the schema.""" + +import json +import unittest +from pathlib import Path + +from cycle_runner import manifest, result, status +from tests.test_config import minimal_document + +from cycle_runner import config as config_module + +SCHEMA_PATH = Path(__file__).resolve().parent.parent / "schema.json" + + +def blocked(): + return result.blocked_result( + run_id="20260914T221530Z-abc123-0123abcd", + backend="distrobox", + tested_revision="rev", + dely_revision="rev", + reason="no orca inside the environment", + ) + + +def built(run_result=None, **overrides): + arguments = { + "run_result": run_result or blocked(), + "run_config": config_module.from_document(minimal_document()), + "host_before": {"hostname_digest": "abc", "processes": 1}, + "host_after": {"hostname_digest": "abc", "processes": 1}, + "artifacts": [{"path": "manifest.json", "size_bytes": 2, "sha256": "0" * 64}], + "tool_versions": {"runner": "one", "distrobox": "1.8.2.5"}, + "generated_at": "2026-09-14T22:16:00Z", + } + arguments.update(overrides) + return manifest.build(**arguments) + + +class SchemaTest(unittest.TestCase): + def test_the_schema_file_parses(self): + json.loads(SCHEMA_PATH.read_text(encoding="utf-8")) + + def test_every_required_field_the_schema_names_is_produced_by_the_builder(self): + schema = json.loads(SCHEMA_PATH.read_text(encoding="utf-8")) + document = built() + for name in schema["required"]: + self.assertIn(name, document, f"builder does not produce {name}") + + +class ValidationTest(unittest.TestCase): + def test_a_built_manifest_validates(self): + manifest.validate(built()) + + def test_a_blocked_run_still_validates_with_every_block_present(self): + document = built() + self.assertEqual(document["status"], "BLOCKED") + self.assertEqual(document["cleanup"]["status"], "RESIDUE") + self.assertEqual(document["check"]["status"], "SKIPPED") + manifest.validate(document) + + def test_a_missing_required_field_is_named(self): + document = built() + document.pop("cleanup") + with self.assertRaises(manifest.ManifestError) as caught: + manifest.validate(document) + self.assertIn("cleanup", str(caught.exception)) + + def test_a_missing_nested_required_field_is_named_with_its_path(self): + document = built() + document["cleanup"].pop("verified") + with self.assertRaises(manifest.ManifestError) as caught: + manifest.validate(document) + self.assertIn("cleanup.verified", str(caught.exception)) + + def test_a_wrong_type_is_refused(self): + document = built() + document["timeout_seconds"] = "nine hundred" + with self.assertRaises(manifest.ManifestError) as caught: + manifest.validate(document) + self.assertIn("timeout_seconds", str(caught.exception)) + + def test_a_status_outside_the_closed_set_is_refused(self): + document = built() + document["status"] = "MOSTLY_FINE" + with self.assertRaises(manifest.ManifestError) as caught: + manifest.validate(document) + self.assertIn("status", str(caught.exception)) + + def test_a_run_identifier_of_the_wrong_shape_is_refused(self): + document = built() + document["run_id"] = "yesterdays-run" + with self.assertRaises(manifest.ManifestError) as caught: + manifest.validate(document) + self.assertIn("run_id", str(caught.exception)) + + def test_an_artifact_without_a_digest_is_refused(self): + document = built() + document["artifacts"][0].pop("sha256") + with self.assertRaises(manifest.ManifestError) as caught: + manifest.validate(document) + self.assertIn("sha256", str(caught.exception)) + + def test_validation_reports_every_problem_it_found(self): + document = built() + document.pop("cleanup") + document.pop("export") + with self.assertRaises(manifest.ManifestError) as caught: + manifest.validate(document) + message = str(caught.exception) + self.assertIn("cleanup", message) + self.assertIn("export", message) + + +class RedactionTest(unittest.TestCase): + def test_the_manifest_is_redacted_before_it_is_returned(self): + run_result = blocked() + run_result.failure_classification = ( + "worker refused key sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopas" + ) + document = built(run_result=run_result) + self.assertNotIn("zzqwertyuiopasdfghjklzxcvbnm", json.dumps(document)) + + def test_the_configuration_echo_carries_no_credential(self): + document = built() + self.assertIn("config", document) + self.assertNotIn("/home/someone/.claude", json.dumps(document["config"])) + + +class WriteTest(unittest.TestCase): + def test_writing_validates_before_it_touches_the_disk(self): + import tempfile + + with tempfile.TemporaryDirectory() as directory: + target = Path(directory) / "manifest.json" + document = built() + document.pop("cleanup") + with self.assertRaises(manifest.ManifestError): + manifest.write(document, target) + self.assertFalse(target.exists()) + + def test_a_valid_manifest_is_written_as_indented_json(self): + import tempfile + + with tempfile.TemporaryDirectory() as directory: + target = Path(directory) / "manifest.json" + manifest.write(built(), target) + reloaded = json.loads(target.read_text(encoding="utf-8")) + self.assertEqual(reloaded["backend"], "distrobox") + self.assertIn("\n ", target.read_text(encoding="utf-8")) diff --git a/experiments/minimal-cycle/tests/test_redact.py b/experiments/minimal-cycle/tests/test_redact.py new file mode 100644 index 0000000..7b875cf --- /dev/null +++ b/experiments/minimal-cycle/tests/test_redact.py @@ -0,0 +1,120 @@ +"""Redaction is by shape, so a secret the runner has never seen is still caught.""" + +import json +import unittest + +from cycle_runner import redact + +NEVER_SEEN = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfgh" + + +class PatternRedactionTest(unittest.TestCase): + def test_token_never_seen_before_is_redacted(self): + line = f"launching worker with key {NEVER_SEEN} now" + cleaned = redact.text(line) + self.assertNotIn(NEVER_SEEN, cleaned) + self.assertIn("launching worker with key", cleaned) + self.assertIn("now", cleaned) + + def test_forge_and_chat_tokens_are_redacted(self): + samples = [ + "ghp_qwertyuiopasdfghjklzxcvbnmqwertyui", + "gho_qwertyuiopasdfghjklzxcvbnmqwertyui", + "github_pat_qwertyuiopasdfghjklzxcvbnmqwertyuiopasdfghjklzxcv", + "xoxb-qwerty-asdfgh-zxcvbnmqwertyuiopasdfg", + "xoxp-qwerty-asdfgh-zxcvbnmqwertyuiopasdfg", + ] + for sample in samples: + with self.subTest(sample=sample[:8]): + self.assertNotIn(sample, redact.text(f"value {sample} trailing")) + + def test_bearer_header_value_is_redacted_but_the_header_name_survives(self): + cleaned = redact.text("authorization: Bearer abcdefghijklmnopqrstuvwxyz") + self.assertNotIn("abcdefghijklmnopqrstuvwxyz", cleaned) + self.assertIn("authorization", cleaned) + + def test_compact_web_token_is_redacted(self): + token = "eyjhbgcioijiuzi1nij9.eyjzdwiioiixmjm0nta3odkwin0.qwertyuiopasdfgh" + self.assertNotIn(token, redact.text(f"cookie value {token}")) + + def test_private_key_block_is_redacted_whole(self): + block = ( + "-----BEGIN OPENSSH PRIVATE KEY-----\n" + "qwertyuiopasdfghjklzxcvbnm\n" + "asdfghjklqwertyuiopzxcvbnm\n" + "-----END OPENSSH PRIVATE KEY-----" + ) + cleaned = redact.text(f"before\n{block}\nafter") + self.assertNotIn("qwertyuiopasdfghjklzxcvbnm", cleaned) + self.assertIn("before", cleaned) + self.assertIn("after", cleaned) + + def test_sensitive_assignments_are_redacted_in_every_common_spelling(self): + samples = [ + "anthropic_api_key=hunterhunterhunter", + "CLAUDE_CODE_OAUTH_TOKEN=hunterhunterhunter", + 'password: "hunterhunterhunter"', + '"secret": "hunterhunterhunter"', + "Cookie=hunterhunterhunter", + ] + for sample in samples: + with self.subTest(sample=sample.split("=")[0][:12]): + self.assertNotIn("hunterhunterhunter", redact.text(sample)) + + def test_absolute_credential_paths_are_redacted(self): + line = "reading /home/someone/.claude/.credentials.json for the run" + cleaned = redact.text(line) + self.assertNotIn("/home/someone", cleaned) + self.assertIn("reading", cleaned) + + def test_relative_allowlist_entries_are_left_readable(self): + cleaned = redact.text("allowlist entry .claude/.credentials.json") + self.assertIn(".claude/.credentials.json", cleaned) + + def test_ordinary_text_is_untouched(self): + line = "the worker reported a token budget of two hundred and one units" + self.assertEqual(redact.text(line), line) + + def test_redaction_is_idempotent(self): + once = redact.text(f"key {NEVER_SEEN}") + self.assertEqual(redact.text(once), once) + + def test_bytes_are_redacted_without_decoding_errors(self): + raw = ("prefix " + NEVER_SEEN).encode("utf-8") + b"\xff\xfe tail" + cleaned = redact.data(raw) + self.assertNotIn(NEVER_SEEN.encode("utf-8"), cleaned) + self.assertIn(b"prefix", cleaned) + + +class StructureRedactionTest(unittest.TestCase): + def test_nested_structures_are_redacted_by_key_and_by_shape(self): + document = { + "auth": {"mode": "short_lived_token", "token": "hunterhunterhunter"}, + "argv": ["orca", "--model", "pinned", "--header", f"Bearer {NEVER_SEEN}"], + "notes": ["nothing secret here"], + "count": 3, + } + cleaned = redact.structure(document) + serialized = json.dumps(cleaned) + self.assertNotIn("hunterhunterhunter", serialized) + self.assertNotIn(NEVER_SEEN, serialized) + self.assertEqual(cleaned["auth"]["mode"], "short_lived_token") + self.assertEqual(cleaned["notes"], ["nothing secret here"]) + self.assertEqual(cleaned["count"], 3) + + def test_structure_redaction_does_not_mutate_its_input(self): + document = {"token": "hunterhunterhunter"} + redact.structure(document) + self.assertEqual(document["token"], "hunterhunterhunter") + + def test_known_values_are_redacted_even_when_their_shape_is_dull(self): + cleaned = redact.text("the value is plain", extra_values=["plain"]) + self.assertNotIn("plain", cleaned) + + +class ContainsSecretTest(unittest.TestCase): + def test_a_clean_document_reports_secret_free(self): + self.assertTrue(redact.looks_secret_free("mode existing_login reference host")) + + def test_a_document_carrying_a_token_is_not_secret_free(self): + self.assertFalse(redact.looks_secret_free(f"key {NEVER_SEEN}")) diff --git a/experiments/minimal-cycle/tests/test_result.py b/experiments/minimal-cycle/tests/test_result.py new file mode 100644 index 0000000..20555d7 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_result.py @@ -0,0 +1,84 @@ +"""Both adapters settle into one result contract.""" + +import json +import unittest + +from cycle_runner import result, status + + +def command(): + return result.CommandRecord( + argv=("sh", "-c", "true"), + exit_code=0, + started_at="2026-09-14T22:15:30Z", + finished_at="2026-09-14T22:15:31Z", + elapsed_seconds=1.0, + timed_out=False, + context="environment", + ) + + +class CommandRecordTest(unittest.TestCase): + def test_a_command_records_where_it_ran(self): + document = command().to_document() + self.assertEqual(document["context"], "environment") + self.assertEqual(document["argv"], ["sh", "-c", "true"]) + self.assertFalse(document["timed_out"]) + + def test_a_command_context_outside_the_two_the_runner_knows_is_refused(self): + with self.assertRaises(ValueError): + result.CommandRecord( + argv=("true",), + exit_code=0, + started_at="2026-09-14T22:15:30Z", + finished_at="2026-09-14T22:15:31Z", + elapsed_seconds=1.0, + timed_out=False, + context="somewhere", + ) + + +class RunResultTest(unittest.TestCase): + def test_a_blocked_result_still_carries_every_block(self): + blocked = result.blocked_result( + run_id="20260914T221530Z-abc123-0123abcd", + backend="distrobox", + tested_revision="rev", + dely_revision="rev", + reason="no orca inside the environment", + ) + document = blocked.to_document() + for key in ("identity", "check", "export", "cleanup", "auth", "worker"): + self.assertIn(key, document) + self.assertEqual(document["status"], "BLOCKED") + self.assertEqual(document["failure_classification"], "no orca inside the environment") + json.dumps(document) + + def test_the_result_serialises_enumerations_as_their_names(self): + blocked = result.blocked_result( + run_id="20260914T221530Z-abc123-0123abcd", + backend="vm", + tested_revision="rev", + dely_revision="rev", + reason="pulumi is absent", + ) + document = blocked.to_document() + self.assertEqual(document["cleanup"]["status"], status.CleanupStatus.RESIDUE.value) + self.assertEqual(document["export"]["status"], status.ExportStatus.FAILED.value) + + def test_phases_keep_the_order_they_ran_in(self): + record = result.RunResult( + run_id="20260914T221530Z-abc123-0123abcd", + backend="distrobox", + tested_revision="rev", + dely_revision="rev", + ) + record.phases.append(result.PhaseRecord(name="prepare", status=status.PhaseStatus.OK)) + record.phases.append(result.PhaseRecord(name="create", status=status.PhaseStatus.OK)) + names = [phase["name"] for phase in record.to_document()["phases"]] + self.assertEqual(names, ["prepare", "create"]) + + def test_a_phase_carries_the_commands_that_produced_it(self): + phase = result.PhaseRecord(name="check", status=status.PhaseStatus.OK) + phase.commands.append(command()) + self.assertEqual(phase.to_document()["commands"][0]["exit_code"], 0) diff --git a/experiments/minimal-cycle/tests/test_status.py b/experiments/minimal-cycle/tests/test_status.py new file mode 100644 index 0000000..c1f88b6 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_status.py @@ -0,0 +1,61 @@ +"""Statuses are a closed set and every one maps to exactly one exit code.""" + +import unittest + +from cycle_runner import status + + +class RunStatusTest(unittest.TestCase): + def test_the_documented_statuses_all_exist(self): + names = {member.value for member in status.RunStatus} + self.assertEqual( + names, + { + "SETTLED", + "ERROR", + "TIMEOUT", + "CANCELLED", + "CLEANUP_FAILED", + "UNKNOWN", + "BLOCKED", + }, + ) + + def test_only_a_settled_run_exits_zero(self): + for member in status.RunStatus: + code = status.exit_code(member) + if member is status.RunStatus.SETTLED: + self.assertEqual(code, 0) + else: + self.assertNotEqual(code, 0) + + def test_every_status_has_a_distinct_exit_code(self): + codes = [status.exit_code(member) for member in status.RunStatus] + self.assertEqual(len(codes), len(set(codes))) + + def test_an_unmapped_object_is_refused(self): + with self.assertRaises(KeyError): + status.exit_code("SETTLED") + + +class CleanupStatusTest(unittest.TestCase): + def test_cleanup_states_are_the_three_the_design_names(self): + names = {member.value for member in status.CleanupStatus} + self.assertEqual(names, {"DESTROYED", "RESIDUE", "UNKNOWN"}) + + +class ExportStatusTest(unittest.TestCase): + def test_export_states_separate_confirmed_from_partial(self): + names = {member.value for member in status.ExportStatus} + self.assertEqual(names, {"CONFIRMED", "PARTIAL", "FAILED"}) + + def test_only_confirmed_permits_cleanup(self): + self.assertTrue(status.ExportStatus.CONFIRMED.permits_cleanup) + self.assertFalse(status.ExportStatus.PARTIAL.permits_cleanup) + self.assertFalse(status.ExportStatus.FAILED.permits_cleanup) + + +class PhaseStatusTest(unittest.TestCase): + def test_phase_states_cover_the_lifecycle_outcomes(self): + names = {member.value for member in status.PhaseStatus} + self.assertEqual(names, {"OK", "FAILED", "SKIPPED", "TIMEOUT", "BLOCKED"}) From a3bd46429d82043a50e08519444a2c4b1cd0eb48 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Mon, 14 Sep 2026 23:38:21 +0200 Subject: [PATCH 03/42] Add the runner core: deadline-aware process capture, host snapshots, the identity probe, project export and diff, export receipts, cleanup rails, auth bootstrap and the Orca worker launcher. --- .../cycle_runner/adapters/base.py | 166 +++++++++++ .../minimal-cycle/cycle_runner/auth.py | 161 +++++++++++ .../minimal-cycle/cycle_runner/cleanup.py | 137 +++++++++ .../minimal-cycle/cycle_runner/config.py | 14 +- .../minimal-cycle/cycle_runner/export.py | 151 ++++++++++ .../minimal-cycle/cycle_runner/hostinfo.py | 72 +++++ .../minimal-cycle/cycle_runner/probe.py | 247 +++++++++++++++++ .../minimal-cycle/cycle_runner/proc.py | 165 +++++++++++ .../minimal-cycle/cycle_runner/project.py | 128 +++++++++ .../minimal-cycle/cycle_runner/worker.py | 247 +++++++++++++++++ .../fixtures/evidence-task/prompt.md | 22 ++ experiments/minimal-cycle/tests/fakes.py | 260 ++++++++++++++++++ experiments/minimal-cycle/tests/test_auth.py | 183 ++++++++++++ .../minimal-cycle/tests/test_cleanup.py | 150 ++++++++++ .../minimal-cycle/tests/test_export.py | 130 +++++++++ .../minimal-cycle/tests/test_hostinfo.py | 102 +++++++ experiments/minimal-cycle/tests/test_probe.py | 182 ++++++++++++ experiments/minimal-cycle/tests/test_proc.py | 98 +++++++ .../minimal-cycle/tests/test_project.py | 133 +++++++++ .../minimal-cycle/tests/test_worker.py | 147 ++++++++++ 20 files changed, 2894 insertions(+), 1 deletion(-) create mode 100644 experiments/minimal-cycle/cycle_runner/adapters/base.py create mode 100644 experiments/minimal-cycle/cycle_runner/auth.py create mode 100644 experiments/minimal-cycle/cycle_runner/cleanup.py create mode 100644 experiments/minimal-cycle/cycle_runner/export.py create mode 100644 experiments/minimal-cycle/cycle_runner/hostinfo.py create mode 100644 experiments/minimal-cycle/cycle_runner/probe.py create mode 100644 experiments/minimal-cycle/cycle_runner/proc.py create mode 100644 experiments/minimal-cycle/cycle_runner/project.py create mode 100644 experiments/minimal-cycle/cycle_runner/worker.py create mode 100644 experiments/minimal-cycle/fixtures/evidence-task/prompt.md create mode 100644 experiments/minimal-cycle/tests/fakes.py create mode 100644 experiments/minimal-cycle/tests/test_auth.py create mode 100644 experiments/minimal-cycle/tests/test_cleanup.py create mode 100644 experiments/minimal-cycle/tests/test_export.py create mode 100644 experiments/minimal-cycle/tests/test_hostinfo.py create mode 100644 experiments/minimal-cycle/tests/test_probe.py create mode 100644 experiments/minimal-cycle/tests/test_proc.py create mode 100644 experiments/minimal-cycle/tests/test_project.py create mode 100644 experiments/minimal-cycle/tests/test_worker.py diff --git a/experiments/minimal-cycle/cycle_runner/adapters/base.py b/experiments/minimal-cycle/cycle_runner/adapters/base.py new file mode 100644 index 0000000..101c0af --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/adapters/base.py @@ -0,0 +1,166 @@ +"""The one interface both backends implement. + +Adapters differ in everything except this: they create a fresh environment, +run commands inside it, move trees across the boundary, stop it, destroy only +what belongs to the run, and say which resources are shared so the runner can +refuse to touch them. +""" + +from __future__ import annotations + +import abc +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Mapping, Sequence + +from ..proc import CommandOutcome + +RESOURCE_KINDS = ("path", "container", "domain", "volume", "stack", "image") + + +@dataclass(frozen=True) +class Resource: + """One thing a run created or depends on, named so cleanup can reason.""" + + kind: str + identifier: str + + def __post_init__(self) -> None: + if self.kind not in RESOURCE_KINDS: + raise ValueError(f"unknown resource kind: {self.kind!r}") + + def to_document(self) -> dict[str, str]: + return {"kind": self.kind, "identifier": self.identifier} + + def __str__(self) -> str: + return f"{self.kind}:{self.identifier}" + + +@dataclass(frozen=True) +class EnvironmentHandle: + """The created environment, and what belongs to this run versus everyone.""" + + environment_id: str + home_path: str + project_path: str + per_run_resources: tuple[Resource, ...] = () + shared_resources: tuple[Resource, ...] = () + description: dict[str, Any] = field(default_factory=dict) + + def to_document(self) -> dict[str, Any]: + return { + "environment_id": self.environment_id, + "home_path": self.home_path, + "project_path": self.project_path, + "per_run_resources": [item.to_document() for item in self.per_run_resources], + "shared_resources": [item.to_document() for item in self.shared_resources], + "description": dict(self.description), + } + + +@dataclass(frozen=True) +class Finding: + """One preflight fact, with the evidence that produced it.""" + + name: str + ok: bool + detail: str + + def to_document(self) -> dict[str, Any]: + return {"name": self.name, "ok": self.ok, "detail": self.detail} + + +@dataclass(frozen=True) +class PreflightReport: + """Whether this host can run this backend, and what is missing.""" + + backend: str + findings: tuple[Finding, ...] = () + + @property + def ok(self) -> bool: + return all(finding.ok for finding in self.findings) + + @property + def blockers(self) -> tuple[Finding, ...]: + return tuple(finding for finding in self.findings if not finding.ok) + + def to_document(self) -> dict[str, Any]: + return { + "backend": self.backend, + "ok": self.ok, + "findings": [finding.to_document() for finding in self.findings], + } + + +@dataclass(frozen=True) +class StopReport: + """Whether the environment was actually stopped, not whether it was asked.""" + + confirmed: bool + detail: str + outcomes: tuple[CommandOutcome, ...] = () + + +@dataclass(frozen=True) +class DestroyReport: + """What destroy removed and what it deliberately left alone.""" + + removed: tuple[str, ...] = () + retained: tuple[str, ...] = () + detail: str = "" + outcomes: tuple[CommandOutcome, ...] = () + + +class BackendAdapter(abc.ABC): + """One disposable environment, driven through its own tool's interface.""" + + name: str = "unnamed" + + @abc.abstractmethod + def preflight(self) -> PreflightReport: + """Report whether this host can run this backend at all.""" + + @abc.abstractmethod + def create(self) -> EnvironmentHandle: + """Create the fresh per-run environment and describe it.""" + + @abc.abstractmethod + def execute( + self, + argv: Sequence[str], + *, + timeout: float, + cwd: str | None = None, + env: Mapping[str, str] | None = None, + extra_values: Sequence[str] = (), + ) -> CommandOutcome: + """Run a command inside the environment.""" + + @abc.abstractmethod + def put_tree(self, local_dir: Path, remote_dir: str) -> None: + """Place a host directory inside the environment.""" + + @abc.abstractmethod + def fetch_tree(self, remote_dir: str, local_dir: Path) -> None: + """Bring a directory out of the environment onto the host.""" + + @abc.abstractmethod + def write_file(self, remote_path: str, content: str, *, mode: int = 0o600) -> None: + """Write one file inside the environment with the given permissions.""" + + @abc.abstractmethod + def stop(self) -> StopReport: + """Stop the environment through its documented path and confirm it.""" + + @abc.abstractmethod + def destroy(self) -> DestroyReport: + """Remove only the per-run resources this adapter declared.""" + + @abc.abstractmethod + def resource_exists(self, resource: Resource) -> bool: + """Report whether a resource is still present, for cleanup verification.""" + + def describe(self) -> dict[str, Any]: + """Return backend facts for the manifest.""" + return {"backend": self.name} diff --git a/experiments/minimal-cycle/cycle_runner/auth.py b/experiments/minimal-cycle/cycle_runner/auth.py new file mode 100644 index 0000000..fdf58a3 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/auth.py @@ -0,0 +1,161 @@ +"""Getting the environment authenticated without leaving a secret behind. + +Exactly one of the three declared methods runs. None of them writes a +credential value, a length or a prefix into a receipt, a manifest, a log or +this repository, and the copied material is removed from the environment and +verified gone before cleanup. +""" + +from __future__ import annotations + +import json +import os +from pathlib import Path +from typing import Mapping + +from .adapters.base import BackendAdapter, EnvironmentHandle +from .config import RunConfig +from .result import AuthRecord +from .status import PhaseStatus + +SETTINGS_RELATIVE = ".claude/settings.json" + + +def _environment_path(handle: EnvironmentHandle, relative: str) -> str: + return str(Path(handle.home_path) / relative) + + +def _existing_login( + run_config: RunConfig, + adapter: BackendAdapter, + handle: EnvironmentHandle, + host_home: Path, +) -> tuple[AuthRecord, dict[str, str]]: + record = AuthRecord( + mode="existing_login", + reference=run_config.auth.reference, + target=str(Path(handle.home_path)), + entries=list(run_config.auth.allowlist), + ) + missing = [ + entry for entry in run_config.auth.allowlist if not (host_home / entry).is_file() + ] + if missing: + record.status = PhaseStatus.BLOCKED + record.detail = ( + "the host does not carry every allowlisted entry: " + ", ".join(missing) + ) + return record, {} + for entry in run_config.auth.allowlist: + content = (host_home / entry).read_text(encoding="utf-8") + adapter.write_file(_environment_path(handle, entry), content, mode=0o600) + record.status = PhaseStatus.OK + record.detail = ( + f"{len(record.entries)} allowlisted entry(s) copied into the per-run home " + "with owner-only permissions" + ) + return record, {} + + +def _short_lived_token( + run_config: RunConfig, + handle: EnvironmentHandle, + environ: Mapping[str, str], +) -> tuple[AuthRecord, dict[str, str]]: + variable = run_config.auth.token_env or "" + record = AuthRecord( + mode="short_lived_token", + reference=run_config.auth.reference, + target="the worker process environment", + entries=[variable], + ) + value = environ.get(variable, "") + if not value: + record.status = PhaseStatus.BLOCKED + record.detail = ( + f"the variable {variable} carries no short-lived token on this host; " + "the run stops rather than prompting for an interactive login" + ) + return record, {} + record.status = PhaseStatus.OK + record.removed_after_run = True + record.detail = ( + f"the value of {variable} is passed to the worker process for this run only " + "and is never written to a file, an image, a seed or a log" + ) + return record, {variable: value} + + +def _api_key_helper( + run_config: RunConfig, + adapter: BackendAdapter, + handle: EnvironmentHandle, +) -> tuple[AuthRecord, dict[str, str]]: + helper = " ".join(run_config.auth.helper_argv) + record = AuthRecord( + mode="api_key_helper", + reference=run_config.auth.reference, + target=_environment_path(handle, SETTINGS_RELATIVE), + entries=[SETTINGS_RELATIVE], + ) + adapter.write_file( + _environment_path(handle, SETTINGS_RELATIVE), + json.dumps({"apiKeyHelper": helper}, indent=2) + "\n", + mode=0o600, + ) + record.status = PhaseStatus.OK + record.removed_after_run = False + record.detail = ( + "the helper command is declared in the per-run settings; no credential " + "value passes through the runner" + ) + return record, {} + + +def bootstrap( + *, + run_config: RunConfig, + adapter: BackendAdapter, + handle: EnvironmentHandle, + host_home: Path | None = None, + environ: Mapping[str, str] | None = None, +) -> tuple[AuthRecord, dict[str, str]]: + """Run the one configured auth method and return its record and overlay.""" + host_home = Path(host_home if host_home is not None else Path.home()) + environ = os.environ if environ is None else environ + mode = run_config.auth.mode + if mode == "existing_login": + return _existing_login(run_config, adapter, handle, host_home) + if mode == "short_lived_token": + return _short_lived_token(run_config, handle, environ) + return _api_key_helper(run_config, adapter, handle) + + +def teardown( + *, adapter: BackendAdapter, handle: EnvironmentHandle, record: AuthRecord +) -> AuthRecord: + """Remove copied auth material from the environment and verify it is gone.""" + if record.mode != "existing_login" or record.status is not PhaseStatus.OK: + record.removed_after_run = record.mode == "short_lived_token" + return record + paths = [_environment_path(handle, entry) for entry in record.entries] + adapter.execute(["rm", "-f", *paths], timeout=60) + survey = adapter.execute( + [ + "sh", + "-c", + 'for candidate in "$@"; do if [ -e "$candidate" ]; then ' + 'printf "present:%s\\n" "$candidate"; fi; done', + "auth-teardown", + *paths, + ], + timeout=60, + ) + still_present = [line for line in survey.stdout.splitlines() if line.startswith("present:")] + record.removed_after_run = not still_present + record.detail = ( + "the copied auth material was removed from the per-run home and verified gone" + if record.removed_after_run + else "auth material is still present in the environment: " + ", ".join(still_present) + ) + return record diff --git a/experiments/minimal-cycle/cycle_runner/cleanup.py b/experiments/minimal-cycle/cycle_runner/cleanup.py new file mode 100644 index 0000000..751c34b --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/cleanup.py @@ -0,0 +1,137 @@ +"""Destroying the run, and proving what was destroyed. + +Two rules carry the weight. Destruction happens only after a confirmed +export, so evidence is never traded for tidiness. And destruction targets a +declared list of per-run resources, so a shared base image survives because +nothing was ever allowed to point at it, not because the command happened to +point elsewhere. +""" + +from __future__ import annotations + +import shutil +from pathlib import Path +from typing import Iterable, Sequence + +from .adapters.base import BackendAdapter, EnvironmentHandle, Resource +from .result import CleanupRecord, ExportRecord +from .status import CleanupStatus + +_NEVER_REMOVABLE = (Path("/"), Path("/home"), Path("/usr"), Path("/etc"), Path("/var")) + + +class CleanupContractError(RuntimeError): + """A removal was asked for that the declarations do not permit.""" + + +def assert_declarations_disjoint( + *, per_run: Sequence[Resource], shared: Sequence[Resource] +) -> None: + """Refuse declarations where removing a per-run resource takes a shared one.""" + for owned in per_run: + for common in shared: + if owned.kind == common.kind and owned.identifier == common.identifier: + raise CleanupContractError( + f"{owned} is declared both per-run and shared" + ) + if owned.kind == "path" and common.kind in ("path", "image", "volume"): + owned_path = Path(owned.identifier).resolve() + shared_path = Path(common.identifier).resolve() + if shared_path == owned_path or shared_path.is_relative_to(owned_path): + raise CleanupContractError( + f"the per-run path {owned_path} contains the shared resource " + f"{shared_path}; removing it would take the shared resource" + ) + + +def safe_remove( + path: Path, *, allowed_roots: Iterable[Path], protected: Iterable[Path] +) -> None: + """Remove a path only when it is inside an allowed root and holds nothing shared.""" + target = Path(path) + resolved = target.resolve() if target.exists() else target.absolute() + if resolved in _NEVER_REMOVABLE: + raise CleanupContractError(f"refusing to remove {resolved}") + roots = [Path(root).resolve() for root in allowed_roots] + if not any(resolved.is_relative_to(root) for root in roots): + raise CleanupContractError( + f"{resolved} is outside every allowed root {[str(root) for root in roots]}" + ) + for keep in protected: + keep_path = Path(keep).resolve() if Path(keep).exists() else Path(keep).absolute() + if keep_path == resolved or keep_path.is_relative_to(resolved): + raise CleanupContractError( + f"refusing to remove {resolved}: it holds the protected path {keep_path}" + ) + if not target.exists() and not target.is_symlink(): + return + if target.is_dir() and not target.is_symlink(): + shutil.rmtree(target) + else: + target.unlink() + + +def perform( + *, + adapter: BackendAdapter, + handle: EnvironmentHandle, + export_record: ExportRecord, +) -> CleanupRecord: + """Destroy per-run resources if, and only if, the export was confirmed.""" + per_run = list(handle.per_run_resources) + shared = list(handle.shared_resources) + assert_declarations_disjoint(per_run=per_run, shared=shared) + + if not export_record.status.permits_cleanup: + return CleanupRecord( + status=CleanupStatus.RESIDUE, + reason=( + "the export was not confirmed " + f"({export_record.status.value}), so nothing was destroyed and the " + "environment is left standing for a manual decision" + ), + retained=[str(item) for item in per_run], + shared_preserved=[str(item) for item in shared], + verified=False, + ) + + report = adapter.destroy() + + surviving = [item for item in per_run if adapter.resource_exists(item)] + lost_shared = [item for item in shared if not adapter.resource_exists(item)] + preserved_shared = [item for item in shared if adapter.resource_exists(item)] + + if lost_shared: + return CleanupRecord( + status=CleanupStatus.UNKNOWN, + reason=( + "cleanup removed a shared resource it must never touch: " + + ", ".join(str(item) for item in lost_shared) + ), + removed=list(report.removed), + retained=[str(item) for item in surviving], + shared_preserved=[str(item) for item in preserved_shared], + verified=False, + ) + + if surviving: + return CleanupRecord( + status=CleanupStatus.RESIDUE, + reason=( + "destroy returned but these per-run resources are still present: " + + ", ".join(str(item) for item in surviving) + ), + removed=list(report.removed), + retained=[str(item) for item in surviving], + shared_preserved=[str(item) for item in preserved_shared], + verified=True, + ) + + return CleanupRecord( + status=CleanupStatus.DESTROYED, + reason="every declared per-run resource is gone and every shared one remains", + removed=list(report.removed), + retained=[], + shared_preserved=[str(item) for item in preserved_shared], + verified=True, + ) diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py index f1848ef..72b169b 100644 --- a/experiments/minimal-cycle/cycle_runner/config.py +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -196,6 +196,7 @@ class OrcaConfig: status_argv: tuple[str, ...] = ("orca", "status", "--json") version_argv: tuple[str, ...] = ("orca", "--version") run_objective: str = "dely minimal cycle" + worktree_selector: str = "current" def to_document(self) -> dict[str, Any]: return { @@ -206,6 +207,7 @@ def to_document(self) -> dict[str, Any]: "status_argv": list(self.status_argv), "version_argv": list(self.version_argv), "run_objective": self.run_objective, + "worktree_selector": self.worktree_selector, } @@ -393,7 +395,16 @@ def _auth(document: Mapping[str, Any]) -> AuthConfig: def _orca(document: Mapping[str, Any]) -> OrcaConfig: _reject_unknown( document, - ("version", "model", "effort", "agent", "status_argv", "version_argv", "run_objective"), + ( + "version", + "model", + "effort", + "agent", + "status_argv", + "version_argv", + "run_objective", + "worktree_selector", + ), "orca", ) return OrcaConfig( @@ -404,6 +415,7 @@ def _orca(document: Mapping[str, Any]) -> OrcaConfig: status_argv=_argv(document, "status_argv", "orca", ("orca", "status", "--json")), version_argv=_argv(document, "version_argv", "orca", ("orca", "--version")), run_objective=_text(document, "run_objective", "orca", "dely minimal cycle"), + worktree_selector=_text(document, "worktree_selector", "orca", "current"), ) diff --git a/experiments/minimal-cycle/cycle_runner/export.py b/experiments/minimal-cycle/cycle_runner/export.py new file mode 100644 index 0000000..40740ce --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/export.py @@ -0,0 +1,151 @@ +"""Writing artifacts to the host and proving the host holds them. + +Everything the run wants to keep goes through one exporter. Confirmation +re-reads each artifact from its host path and recomputes its digest, so a +receipt cannot describe bytes that never landed or that changed after the +write. Only a confirmed export releases the run to destroy anything. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Mapping + +from . import hashing, redact +from .proc import utc_now +from .result import ExportRecord +from .status import ExportStatus + +RECEIPT_NAME = "export-receipt.json" + + +class ExportError(ValueError): + """An artifact path is not inside the run's own directory.""" + + +@dataclass +class _Declared: + required: bool + expected_sha256: str | None = None + + +class Exporter: + """Collects the run's artifacts under one host directory.""" + + def __init__(self, root: Path): + self.root = Path(root) + self.root.mkdir(parents=True, exist_ok=True) + self._declared: dict[str, _Declared] = {} + + # -- placing artifacts ------------------------------------------------ + + def _resolve(self, relative: str) -> Path: + candidate = Path(relative) + if candidate.is_absolute() or ".." in candidate.parts: + raise ExportError( + f"an artifact path must stay inside the run directory, got {relative!r}" + ) + return self.root / candidate + + def declare(self, relative: str, *, required: bool = True) -> None: + """Record that an artifact is expected without writing it here.""" + self._resolve(relative) + self._declared[Path(relative).as_posix()] = _Declared(required=required) + + def write_bytes(self, relative: str, data: bytes, *, required: bool = True) -> Path: + """Write a binary artifact and remember the digest it was written with.""" + target = self._resolve(relative) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(data) + self._declared[Path(relative).as_posix()] = _Declared( + required=required, expected_sha256=hashing.digest_file(target) + ) + return target + + def write_text(self, relative: str, text: str, *, required: bool = True) -> Path: + """Write a redacted text artifact.""" + return self.write_bytes( + relative, redact.text(text).encode("utf-8"), required=required + ) + + def write_json( + self, relative: str, document: Mapping[str, Any] | list, *, required: bool = True + ) -> Path: + """Write a redacted document as indented JSON.""" + payload = json.dumps(redact.structure(document), indent=2, sort_keys=True) + "\n" + return self.write_bytes(relative, payload.encode("utf-8"), required=required) + + def adopt_tree(self, relative_dir: str, source: Path, *, required: bool = False) -> None: + """Copy a directory into the run and declare every file it carried.""" + destination = self._resolve(relative_dir) + source = Path(source) + prefix = Path(relative_dir).as_posix() + if not source.is_dir(): + self._declared[f"{prefix}/"] = _Declared(required=required) + return + copied = 0 + for path in sorted(source.rglob("*")): + if path.is_symlink() or not path.is_file(): + continue + relative = path.relative_to(source) + target = destination / relative + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(path.read_bytes()) + self._declared[(Path(prefix) / relative).as_posix()] = _Declared( + required=False, expected_sha256=hashing.digest_file(target) + ) + copied += 1 + if copied == 0: + self._declared[f"{prefix}/"] = _Declared(required=required) + + # -- proving the host holds them -------------------------------------- + + def confirm(self) -> ExportRecord: + """Re-read every declared artifact and write the receipt.""" + artifacts: list[dict[str, Any]] = [] + problems: list[str] = [] + blocking = 0 + for relative in sorted(self._declared): + declared = self._declared[relative] + target = self.root / relative + if not target.is_file(): + problems.append(f"{relative} (absent from the host)") + blocking += int(declared.required) + continue + described = hashing.describe_file(target, relative_to=self.root) + if ( + declared.expected_sha256 is not None + and described["sha256"] != declared.expected_sha256 + ): + problems.append(f"{relative} (digest changed after the write)") + blocking += int(declared.required) + continue + artifacts.append(described) + + if blocking == 0: + state = ExportStatus.CONFIRMED + elif artifacts: + state = ExportStatus.PARTIAL + else: + state = ExportStatus.FAILED + + record = ExportRecord( + status=state, + confirmed_at=utc_now() if state is ExportStatus.CONFIRMED else None, + receipt_path=RECEIPT_NAME, + artifacts=artifacts, + missing=problems, + detail=( + f"{len(artifacts)} artifact(s) re-read from the host; " + f"{len(problems)} unresolved" + ), + ) + receipt = self.root / RECEIPT_NAME + receipt.write_text( + json.dumps(redact.structure(record.to_document()), indent=2, sort_keys=True) + + "\n", + encoding="utf-8", + ) + return record diff --git a/experiments/minimal-cycle/cycle_runner/hostinfo.py b/experiments/minimal-cycle/cycle_runner/hostinfo.py new file mode 100644 index 0000000..507dbed --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/hostinfo.py @@ -0,0 +1,72 @@ +"""What the host looked like before and after the run. + +The comparison is what shows whether anything the run created outlived it. +Host identifiers are stored as digests: the snapshot travels into artifacts a +person may share, and the run never needs the names themselves. +""" + +from __future__ import annotations + +import platform +import socket +from pathlib import Path +from typing import Any, Mapping + +from .probe import digest +from .proc import utc_now + + +def _read_first_line(path: str) -> str: + try: + with open(path, "r", encoding="utf-8", errors="replace") as handle: + return handle.readline().strip() + except OSError: + return "" + + +def _root_view(root: Path) -> dict[str, Any]: + root = Path(root) + if not root.is_dir(): + return {"path": str(root), "exists": False, "entries": []} + return { + "path": str(root), + "exists": True, + "entries": sorted(child.name for child in root.iterdir()), + } + + +def snapshot( + *, + state_root: Path, + artifact_root: Path, + extra: Mapping[str, Any] | None = None, +) -> dict[str, Any]: + """Capture the host facts this run is allowed to compare against.""" + return { + "captured_at": utc_now(), + "kernel": platform.release(), + "architecture": platform.machine(), + "python_version": platform.python_version(), + "hostname_digest": digest(socket.gethostname()), + "machine_id_digest": digest(_read_first_line("/etc/machine-id")), + "boot_id_digest": digest(_read_first_line("/proc/sys/kernel/random/boot_id")), + "user_digest": digest(str(Path.home())), + "state_root": _root_view(state_root), + "artifact_root": _root_view(artifact_root), + "extra": dict(extra or {}), + } + + +def difference(before: Mapping[str, Any], after: Mapping[str, Any]) -> dict[str, Any]: + """Report what changed between two host snapshots.""" + changes: dict[str, Any] = {} + for name in ("state_root", "artifact_root"): + was = set(before.get(name, {}).get("entries", [])) + now = set(after.get(name, {}).get("entries", [])) + added = sorted(now - was) + removed = sorted(was - now) + if added: + changes[f"{name}_added"] = added + if removed: + changes[f"{name}_removed"] = removed + return changes diff --git a/experiments/minimal-cycle/cycle_runner/probe.py b/experiments/minimal-cycle/cycle_runner/probe.py new file mode 100644 index 0000000..0f39d0f --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/probe.py @@ -0,0 +1,247 @@ +"""Proving which side of the boundary a command ran on. + +The same shell probe runs on the host and inside the environment. A verdict +that cannot tell the two apart is a host fallback, and a host fallback stops +the run: it never degrades into using the host's Orca, the host's home or the +host's checkout. +""" + +from __future__ import annotations + +import hashlib +from typing import Any, Mapping + +from .result import ( + IDENTITY_ENVIRONMENT, + IDENTITY_HOST_FALLBACK, + IDENTITY_UNKNOWN, + IdentityRecord, +) + +REQUIRED_FIELDS = ( + "hostname", + "machine_id", + "boot_id", + "home", + "user", + "path", + "pid", + "uname", + "container_marker", + "virt", + "orca_path", + "orca_version", + "project_real", +) + +#: POSIX shell. It is run identically on the host and in the environment, and +#: it reads only what every image can be expected to expose. +PROBE_SCRIPT = r""" +set -u +project_path="${1:-}" + +first_line() { + if [ -r "$1" ]; then + head -n 1 "$1" 2>/dev/null || printf '' + else + printf '' + fi +} + +marker='' +if [ -f /run/.containerenv ]; then marker='containerenv'; fi +if [ -z "$marker" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi +if [ -z "$marker" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi +if [ -z "$marker" ] && [ -n "${container:-}" ]; then marker="${container}"; fi + +virt='unknown' +if command -v systemd-detect-virt >/dev/null 2>&1; then + detected="$(systemd-detect-virt 2>/dev/null | head -n 1)" + if [ -n "$detected" ]; then virt="$detected"; fi +fi + +orca_binary="$(command -v orca 2>/dev/null || printf '')" +orca_release='' +if [ -n "$orca_binary" ]; then + orca_release="$("$orca_binary" --version 2>/dev/null | head -n 1 || printf '')" +fi + +project_resolved='' +if [ -n "$project_path" ] && [ -d "$project_path" ]; then + project_resolved="$(cd "$project_path" 2>/dev/null && pwd -P || printf '')" +fi + +printf 'hostname=%s\n' "$(uname -n 2>/dev/null || printf '')" +printf 'machine_id=%s\n' "$(first_line /etc/machine-id)" +printf 'boot_id=%s\n' "$(first_line /proc/sys/kernel/random/boot_id)" +printf 'home=%s\n' "${HOME:-}" +printf 'user=%s\n' "$(id -un 2>/dev/null || printf '')" +printf 'path=%s\n' "${PATH:-}" +printf 'pid=%s\n' "$$" +printf 'uname=%s\n' "$(uname -srm 2>/dev/null || printf '')" +printf 'container_marker=%s\n' "$marker" +printf 'virt=%s\n' "$virt" +printf 'orca_path=%s\n' "$orca_binary" +printf 'orca_version=%s\n' "$orca_release" +printf 'project_real=%s\n' "$project_resolved" +""" + +_UNINFORMATIVE_VIRT = ("", "none", "unknown") + + +def probe_argv(project_path: str) -> list[str]: + """Return the argv that runs the probe with the project path to resolve.""" + return ["sh", "-c", PROBE_SCRIPT, "cycle-probe", project_path] + + +def parse(text: str) -> dict[str, str]: + """Parse the probe's key-value output; unrecognised lines are dropped.""" + snapshot: dict[str, str] = {} + for line in text.splitlines(): + if "=" not in line: + continue + key, _, value = line.partition("=") + key = key.strip() + if key: + snapshot[key] = value.strip() + return snapshot + + +def digest(value: str) -> str: + """Return a short, non-reversing digest for an identifier.""" + if not value: + return "" + return hashlib.sha256(value.encode("utf-8")).hexdigest()[:16] + + +def _host_view(host: Mapping[str, str]) -> dict[str, Any]: + return { + "hostname_digest": digest(host.get("hostname", "")), + "machine_id_digest": digest(host.get("machine_id", "")), + "boot_id_digest": digest(host.get("boot_id", "")), + "home_digest": digest(host.get("home", "")), + "user_digest": digest(host.get("user", "")), + "project_real_digest": digest(host.get("project_real", "")), + "virt": host.get("virt", ""), + "orca_present": bool(host.get("orca_path")), + } + + +def _environment_view(environment: Mapping[str, str]) -> dict[str, Any]: + return { + key: environment.get(key, "") + for key in ( + "hostname", + "home", + "user", + "path", + "pid", + "uname", + "container_marker", + "virt", + "orca_path", + "orca_version", + "project_real", + ) + } + + +def _markers(host: Mapping[str, str], environment: Mapping[str, str]) -> list[str]: + markers: list[str] = [] + marker = environment.get("container_marker", "") + if marker: + markers.append(f"container-marker:{marker}") + machine = environment.get("machine_id", "") + if machine and machine != host.get("machine_id", ""): + markers.append("distinct-machine-identity") + boot = environment.get("boot_id", "") + if boot and boot != host.get("boot_id", ""): + markers.append("distinct-boot-identity") + name = environment.get("hostname", "") + if name and name != host.get("hostname", ""): + markers.append("distinct-hostname") + virt = environment.get("virt", "") + if virt not in _UNINFORMATIVE_VIRT and virt != host.get("virt", ""): + markers.append(f"reported-virtualisation:{virt}") + return markers + + +def verdict( + *, + host: Mapping[str, str], + environment: Mapping[str, str], + backend: str, + expected_home: str, + expected_project: str, +) -> IdentityRecord: + """Decide whether the environment answered as itself or as the host.""" + record = IdentityRecord( + host=_host_view(host), + environment=_environment_view(environment), + orca_present=bool(environment.get("orca_path")), + orca_version=environment.get("orca_version") or None, + ) + + missing = [field for field in ("hostname", "home") if not environment.get(field)] + if missing: + record.verdict = IDENTITY_UNKNOWN + record.reason = ( + "the environment probe returned no " + + ", ".join(missing) + + "; nothing can be concluded about where the command ran" + ) + return record + + markers = _markers(host, environment) + record.markers = markers + + if not markers: + record.verdict = IDENTITY_HOST_FALLBACK + record.reason = ( + "the environment answered with the host's own identity: no container " + "marker, no distinct machine or boot identity, and the same host name" + ) + return record + + if backend == "vm" and "distinct-boot-identity" not in markers: + record.verdict = IDENTITY_HOST_FALLBACK + record.reason = ( + "the machine backend requires a distinct boot identity; the probe " + "reported the host's own boot identity, so this is not a separate kernel" + ) + return record + + observed_home = environment.get("home", "") + if observed_home != expected_home: + record.verdict = IDENTITY_HOST_FALLBACK + record.reason = ( + f"the environment home is {observed_home!r}, not the per-run home " + f"{expected_home!r}" + ) + return record + + observed_project = environment.get("project_real", "") + if observed_project != expected_project: + record.verdict = IDENTITY_HOST_FALLBACK + record.reason = ( + f"the resolved project path is {observed_project!r}, not the per-run " + f"project copy {expected_project!r}" + ) + return record + + record.verdict = IDENTITY_ENVIRONMENT + record.reason = "the environment answered with its own identity, home and project copy" + return record + + +def may_continue(record: IdentityRecord) -> tuple[bool, str]: + """Report whether the run may proceed past the identity gate, and why not.""" + if record.verdict != IDENTITY_ENVIRONMENT: + return False, record.reason + if not record.orca_present: + return ( + False, + "orca is not present inside the environment; the run stops rather than " + "using the host's installation", + ) + return True, record.reason diff --git a/experiments/minimal-cycle/cycle_runner/proc.py b/experiments/minimal-cycle/cycle_runner/proc.py new file mode 100644 index 0000000..832bbe4 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/proc.py @@ -0,0 +1,165 @@ +"""Running one command with a deadline, and keeping what it said. + +A deadline is an outcome, not an exception: a timed-out command still carries +its partial output, its timings and the fact that the runner had to kill it. +Nothing captured here reaches a caller before redaction. +""" + +from __future__ import annotations + +import os +import signal +import subprocess +import time +from dataclasses import dataclass +from datetime import datetime, timezone +from pathlib import Path +from typing import Mapping, Sequence + +from . import redact +from .result import CommandRecord + +_GRACE_SECONDS = 2.0 + + +def utc_now() -> str: + """Return the current instant as a second-resolution stamp.""" + return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + + +@dataclass(frozen=True) +class CommandOutcome: + """What one command did, already redacted.""" + + argv: tuple[str, ...] + exit_code: int | None + stdout: str + stderr: str + started_at: str + finished_at: str + elapsed_seconds: float + timed_out: bool + context: str + killed_process_group: bool = False + launch_failed: bool = False + + @property + def ok(self) -> bool: + """Whether the command ran to completion and reported success.""" + return self.exit_code == 0 + + def to_record(self, *, stdout_path: str | None = None, stderr_path: str | None = None) -> CommandRecord: + """Return the manifest-shaped record for this command.""" + return CommandRecord( + argv=self.argv, + exit_code=self.exit_code, + started_at=self.started_at, + finished_at=self.finished_at, + elapsed_seconds=self.elapsed_seconds, + timed_out=self.timed_out, + context=self.context, + stdout_path=stdout_path, + stderr_path=stderr_path, + ) + + +def _kill_group(process: subprocess.Popen) -> bool: + """Terminate the child's whole session; report whether the group was signalled.""" + try: + group = os.getpgid(process.pid) + except (ProcessLookupError, PermissionError): + process.kill() + return False + for number in (signal.SIGTERM, signal.SIGKILL): + try: + os.killpg(group, number) + except (ProcessLookupError, PermissionError): + break + try: + process.wait(timeout=_GRACE_SECONDS) + break + except subprocess.TimeoutExpired: + continue + return True + + +def run( + argv: Sequence[str], + *, + timeout: float, + context: str, + cwd: Path | str | None = None, + env: Mapping[str, str] | None = None, + extra_values: Sequence[str] = (), + stdin_text: str | None = None, +) -> CommandOutcome: + """Run one command with a deadline and return its redacted outcome.""" + argv = tuple(str(item) for item in argv) + started_at = utc_now() + started = time.monotonic() + child_env = dict(os.environ) + if env: + child_env.update(env) + + def finish( + *, + exit_code: int | None, + stdout: str, + stderr: str, + timed_out: bool, + killed_process_group: bool = False, + launch_failed: bool = False, + ) -> CommandOutcome: + return CommandOutcome( + argv=argv, + exit_code=exit_code, + stdout=redact.text(stdout, extra_values), + stderr=redact.text(stderr, extra_values), + started_at=started_at, + finished_at=utc_now(), + elapsed_seconds=round(time.monotonic() - started, 3), + timed_out=timed_out, + context=context, + killed_process_group=killed_process_group, + launch_failed=launch_failed, + ) + + try: + process = subprocess.Popen( + argv, + cwd=str(cwd) if cwd is not None else None, + env=child_env, + stdin=subprocess.PIPE if stdin_text is not None else subprocess.DEVNULL, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + errors="replace", + start_new_session=True, + ) + except OSError as error: + return finish( + exit_code=None, + stdout="", + stderr=f"could not launch {argv[0]}: {error}", + timed_out=False, + launch_failed=True, + ) + + try: + stdout, stderr = process.communicate(input=stdin_text, timeout=timeout) + except subprocess.TimeoutExpired: + killed = _kill_group(process) + try: + stdout, stderr = process.communicate(timeout=_GRACE_SECONDS) + except subprocess.TimeoutExpired: + stdout, stderr = "", "" + return finish( + exit_code=None, + stdout=stdout or "", + stderr=(stderr or "") + f"\ndeadline of {timeout} seconds reached", + timed_out=True, + killed_process_group=killed, + ) + return finish( + exit_code=process.returncode, stdout=stdout or "", stderr=stderr or "", timed_out=False + ) diff --git a/experiments/minimal-cycle/cycle_runner/project.py b/experiments/minimal-cycle/cycle_runner/project.py new file mode 100644 index 0000000..c895526 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/project.py @@ -0,0 +1,128 @@ +"""Exporting the project at a pinned revision, and diffing it afterwards. + +The export is `git archive`, so the copy carries the tracked tree at exactly +one revision and none of the repository's own state. The patch is computed on +the host from the fetched tree, so no image has to carry a diff tool and both +backends produce the same artifact. +""" + +from __future__ import annotations + +import difflib +import subprocess +import tarfile +import tempfile +from pathlib import Path + +from . import redact + +_TEXT_SAMPLE = 8192 + + +class ProjectError(RuntimeError): + """The project copy could not be produced from the named revision.""" + + +def _git(source: Path, *arguments: str) -> str: + completed = subprocess.run( + ["git", "-C", str(source), *arguments], + capture_output=True, + text=True, + ) + if completed.returncode != 0: + raise ProjectError( + f"git {' '.join(arguments)} failed in {source}: " + f"{redact.text(completed.stderr.strip())}" + ) + return completed.stdout + + +def resolve_revision(source: Path, revision: str) -> str: + """Resolve a revision to the commit it names.""" + source = Path(source) + if not source.is_dir(): + raise ProjectError(f"project source is not a directory: {source}") + return _git(source, "rev-parse", "--verify", f"{revision}^{{commit}}").strip() + + +def export_revision(source: Path, revision: str, destination: Path) -> str: + """Export the tracked tree at `revision` into an empty destination.""" + source = Path(source) + destination = Path(destination) + if destination.exists() and any(destination.iterdir()): + raise ProjectError(f"project destination is not empty: {destination}") + commit = resolve_revision(source, revision) + destination.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory() as staging: + archive = Path(staging) / "project.tar" + _git(source, "archive", "--format=tar", f"--output={archive}", commit) + with tarfile.open(archive) as bundle: + bundle.extractall(destination, filter="data") + return commit + + +def copy_tree(source: Path, destination: Path) -> None: + """Copy a directory tree verbatim, files only, creating parents as needed.""" + source = Path(source) + destination = Path(destination) + destination.mkdir(parents=True, exist_ok=True) + for path in sorted(source.rglob("*")): + if path.is_symlink() or not path.is_file(): + continue + target = destination / path.relative_to(source) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(path.read_bytes()) + + +def _relative_files(root: Path) -> dict[str, Path]: + root = Path(root) + if not root.is_dir(): + return {} + return { + path.relative_to(root).as_posix(): path + for path in sorted(root.rglob("*")) + if path.is_file() and not path.is_symlink() + } + + +def _is_text(raw: bytes) -> bool: + if b"\x00" in raw[:_TEXT_SAMPLE]: + return False + try: + raw.decode("utf-8") + except UnicodeDecodeError: + return False + return True + + +def _lines(raw: bytes) -> list[str]: + return raw.decode("utf-8").splitlines(keepends=True) + + +def unified_diff(baseline: Path, current: Path) -> str: + """Return one redacted patch describing what the run changed.""" + before = _relative_files(baseline) + after = _relative_files(current) + pieces: list[str] = [] + for relative in sorted(set(before) | set(after)): + old = before.get(relative) + new = after.get(relative) + old_bytes = old.read_bytes() if old else b"" + new_bytes = new.read_bytes() if new else b"" + if old and new and old_bytes == new_bytes: + continue + if not _is_text(old_bytes) or not _is_text(new_bytes): + pieces.append(f"Binary files a/{relative} and b/{relative} differ\n") + continue + from_file = "/dev/null" if old is None else f"a/{relative}" + to_file = "/dev/null" if new is None else f"b/{relative}" + pieces.extend( + difflib.unified_diff( + _lines(old_bytes), + _lines(new_bytes), + fromfile=from_file, + tofile=to_file, + lineterm="\n", + ) + ) + return redact.text("".join(pieces)) diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py new file mode 100644 index 0000000..7bc1de2 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -0,0 +1,247 @@ +"""Launching the one Claude Code worker through Orca, inside the environment. + +The prompt is a file in the environment and the dispatch carries a pointer to +it, because a prompt inlined as a shell argument is mangled by quoting. The +model and the effort are named on every launch, so the worker never runs on a +harness default the manifest cannot report. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Mapping, Sequence + +from . import redact +from .adapters.base import BackendAdapter, EnvironmentHandle +from .config import RunConfig +from .result import WorkerRecord +from .status import PhaseStatus + +SETTLING_TYPES = ("worker_done", "escalation", "question") +PROMPT_NAME = "dispatch-prompt.md" +DEFAULT_PROMPT_PATH = ( + Path(__file__).resolve().parent.parent / "fixtures" / "evidence-task" / "prompt.md" +) + + +@dataclass(frozen=True) +class LaunchPlan: + """The three Orca commands one dispatch needs, as argv.""" + + prompt_path: str + spec: str + run_create_argv: tuple[str, ...] + worker_start_argv: tuple[str, ...] + wait_argv: tuple[str, ...] + + +def build_prompt(run_config: RunConfig, handle: EnvironmentHandle) -> str: + """Render the task prompt for this run's marker and project copy.""" + source = run_config.task.prompt_path + if source: + candidate = Path(source) + if not candidate.is_absolute() and run_config.source_path is not None: + candidate = run_config.source_path.parent / candidate + template = candidate.read_text(encoding="utf-8") + else: + template = DEFAULT_PROMPT_PATH.read_text(encoding="utf-8") + return ( + template.replace("{{project_path}}", handle.project_path) + .replace("{{relative_path}}", run_config.task.relative_path) + .replace("{{marker}}", run_config.task.marker) + ) + + +def build_plan( + *, run_config: RunConfig, handle: EnvironmentHandle, timeout_seconds: int, orca_run_id: str | None +) -> LaunchPlan: + """Compose the argv for run-create, worker-start and the completion wait.""" + prompt_path = str(Path(handle.home_path) / PROMPT_NAME) + spec = ( + f"Read the file {prompt_path} in this environment and do exactly what it says." + ) + deadline = str(int(timeout_seconds) * 1000) + start = [ + "orca", + "orchestration", + "worker-start", + "--spec", + spec, + "--agent", + run_config.orca.agent, + "--model", + run_config.orca.model, + "--effort", + run_config.orca.effort, + "--worktree", + run_config.orca.worktree_selector, + "--timeout-ms", + deadline, + "--json", + ] + wait = [ + "orca", + "orchestration", + "check", + "--wait", + "--types", + ",".join(SETTLING_TYPES), + "--timeout-ms", + deadline, + "--json", + ] + if orca_run_id: + start.extend(["--run", orca_run_id]) + wait.extend(["--run", orca_run_id]) + return LaunchPlan( + prompt_path=prompt_path, + spec=spec, + run_create_argv=( + "orca", + "orchestration", + "run-create", + "--objective", + run_config.orca.run_objective, + "--json", + ), + worker_start_argv=tuple(start), + wait_argv=tuple(wait), + ) + + +def _first_document(text: str) -> dict[str, Any]: + for candidate in (text, *text.splitlines()): + stripped = candidate.strip() + if not stripped.startswith("{"): + continue + try: + parsed = json.loads(stripped) + except json.JSONDecodeError: + continue + if isinstance(parsed, dict): + return parsed + return {} + + +def _identifier(document: Mapping[str, Any], *names: str) -> str | None: + for name in names: + value = document.get(name) + if isinstance(value, str) and value: + return value + for value in document.values(): + if isinstance(value, Mapping): + found = _identifier(value, *names) + if found: + return found + return None + + +def _settling_message(document: Mapping[str, Any]) -> dict[str, Any]: + messages = document.get("messages") + if isinstance(messages, list): + for message in messages: + if isinstance(message, Mapping) and message.get("type") in SETTLING_TYPES: + return dict(message) + if document.get("type") in SETTLING_TYPES: + return dict(document) + return {} + + +def launch( + *, + run_config: RunConfig, + adapter: BackendAdapter, + handle: EnvironmentHandle, + timeout_seconds: int, + env_overlay: Mapping[str, str] | None = None, +) -> WorkerRecord: + """Run exactly one worker and report how it settled.""" + record = WorkerRecord( + agent=run_config.orca.agent, + model=run_config.orca.model, + effort=run_config.orca.effort, + ) + overlay = dict(env_overlay or {}) + secrets: Sequence[str] = tuple(value for value in overlay.values() if value) + + plan = build_plan( + run_config=run_config, + handle=handle, + timeout_seconds=timeout_seconds, + orca_run_id=None, + ) + + created = adapter.execute( + plan.run_create_argv, timeout=timeout_seconds, env=overlay, extra_values=secrets + ) + record.commands.append(created.to_record()) + if not created.ok: + record.status = PhaseStatus.FAILED + record.detail = ( + "orca orchestration run-create did not return a Run: " + + redact.text((created.stderr or created.stdout).strip()[:400], secrets) + ) + return record + record.run_id = _identifier(_first_document(created.stdout), "runId", "run_id", "id") + + plan = build_plan( + run_config=run_config, + handle=handle, + timeout_seconds=timeout_seconds, + orca_run_id=record.run_id, + ) + adapter.write_file(plan.prompt_path, build_prompt(run_config, handle), mode=0o644) + + started = adapter.execute( + plan.worker_start_argv, + timeout=timeout_seconds, + env=overlay, + extra_values=secrets, + ) + record.commands.append(started.to_record()) + if started.timed_out: + record.status = PhaseStatus.TIMEOUT + record.detail = "orca orchestration worker-start reached the run deadline" + return record + if not started.ok: + record.status = PhaseStatus.FAILED + record.detail = ( + "orca orchestration worker-start did not reach ready: " + + redact.text((started.stderr or started.stdout).strip()[:400], secrets) + ) + return record + record.dispatch_id = _identifier( + _first_document(started.stdout), "dispatchId", "dispatch_id", "id" + ) + + settled = adapter.execute( + plan.wait_argv, timeout=timeout_seconds, env=overlay, extra_values=secrets + ) + record.commands.append(settled.to_record()) + if settled.timed_out: + record.status = PhaseStatus.TIMEOUT + record.detail = "the completion wait reached the run deadline before the worker settled" + return record + if not settled.ok: + record.status = PhaseStatus.FAILED + record.detail = "the completion wait failed: " + redact.text( + (settled.stderr or settled.stdout).strip()[:400], secrets + ) + return record + + message = _settling_message(_first_document(settled.stdout)) + record.outcome = message.get("outcome") or message.get("type") or None + if message.get("type") == "worker_done": + record.status = PhaseStatus.OK + record.detail = f"the worker reported worker_done with outcome {record.outcome!r}" + elif message: + record.status = PhaseStatus.FAILED + record.detail = ( + f"the wait settled on {message.get('type')!r} rather than worker_done" + ) + else: + record.status = PhaseStatus.FAILED + record.detail = "the wait returned no settling message for this dispatch" + return record diff --git a/experiments/minimal-cycle/fixtures/evidence-task/prompt.md b/experiments/minimal-cycle/fixtures/evidence-task/prompt.md new file mode 100644 index 0000000..60857c0 --- /dev/null +++ b/experiments/minimal-cycle/fixtures/evidence-task/prompt.md @@ -0,0 +1,22 @@ +# One small task + +You are running inside a disposable environment created for a single proof +cycle. Do exactly this task and nothing else. Do not install anything, do not +change any other file, and do not run the project's own tests. + +## The task + +In the project copy at `{{project_path}}`, create the file +`{{relative_path}}` whose entire content is exactly this marker, with no +trailing newline and no other text: + +{{marker}} + +## What happens next + +An independent check runs outside this session. It reads that file and +compares its content with the marker above, byte for byte. Nothing else about +this session is measured: this is a test of the environment's plumbing, not of +your reasoning. + +When the file exists with exactly that content, report that you are done. diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py new file mode 100644 index 0000000..0c9b863 --- /dev/null +++ b/experiments/minimal-cycle/tests/fakes.py @@ -0,0 +1,260 @@ +"""A backend adapter that records what the lifecycle asked it to do. + +It is a real implementation of the interface over a host directory, not a +mock: the lifecycle's ordering claims are checked against what actually +happened to files, and the recorded call list is what proves the order. +""" + +from __future__ import annotations + +import shutil +from pathlib import Path +from typing import Mapping, Sequence + +from cycle_runner import probe, proc +from cycle_runner.adapters.base import ( + BackendAdapter, + DestroyReport, + EnvironmentHandle, + Finding, + PreflightReport, + Resource, + StopReport, +) + +HOST = { + "hostname": "workshop", + "machine_id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "boot_id": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", + "home": "/home/someone", + "user": "someone", + "path": "/usr/bin:/bin", + "pid": "1", + "uname": "linux workshop amd64", + "container_marker": "", + "virt": "none", + "orca_path": "/usr/bin/orca", + "orca_version": "1.4.201", + "project_real": "/home/someone/code/under-test", +} + + +def render(snapshot: Mapping[str, str]) -> str: + return "".join(f"{key}={value}\n" for key, value in snapshot.items()) + + +class FakeAdapter(BackendAdapter): + """An environment that is really just a directory on the host.""" + + name = "fake" + + def __init__( + self, + root: Path, + *, + preflight_ok: bool = True, + identity: str = "environment", + orca_present: bool = True, + check_exit_code: int = 0, + task_hangs: bool = False, + fetch_fails: bool = False, + stop_confirmed: bool = True, + leave_residue: bool = False, + ): + self.root = Path(root) + self.home = self.root / "home" + self.project = self.home / "project" + self.shared_base = self.root.parent / "shared" / "base.img" + self.calls: list[str] = [] + self.preflight_ok = preflight_ok + self.identity = identity + self.orca_present = orca_present + self.check_exit_code = check_exit_code + self.task_hangs = task_hangs + self.fetch_fails = fetch_fails + self.stop_confirmed = stop_confirmed + self.leave_residue = leave_residue + self.destroyed = False + + # -- lifecycle -------------------------------------------------------- + + def preflight(self) -> PreflightReport: + self.calls.append("preflight") + return PreflightReport( + backend=self.name, + findings=( + Finding( + name="fake backend", + ok=self.preflight_ok, + detail="configured by the test", + ), + ), + ) + + def create(self) -> EnvironmentHandle: + self.calls.append("create") + self.project.mkdir(parents=True, exist_ok=True) + self.shared_base.parent.mkdir(parents=True, exist_ok=True) + self.shared_base.write_bytes(b"shared base image") + return EnvironmentHandle( + environment_id=f"fake-{self.root.name}", + home_path=str(self.home), + project_path=str(self.project), + per_run_resources=(Resource(kind="path", identifier=str(self.home)),), + shared_resources=(Resource(kind="image", identifier=str(self.shared_base)),), + description={"image": "fake"}, + ) + + def _snapshot(self) -> str: + if self.identity == "host": + return render(HOST) + environment = dict(HOST) + environment.update( + { + "hostname": "fake-box", + "machine_id": "cccccccccccccccccccccccccccccccc", + "boot_id": "dddddddd-dddd-dddd-dddd-dddddddddddd", + "home": str(self.home), + "container_marker": "containerenv", + "project_real": str(self.project), + "orca_path": "/usr/bin/orca" if self.orca_present else "", + "orca_version": "1.4.201" if self.orca_present else "", + } + ) + return render(environment) + + def execute( + self, + argv: Sequence[str], + *, + timeout: float, + cwd: str | None = None, + env: Mapping[str, str] | None = None, + extra_values: Sequence[str] = (), + ) -> proc.CommandOutcome: + joined = " ".join(argv) + if probe.PROBE_SCRIPT in joined: + self.calls.append("probe") + return self._outcome(argv, 0, self._snapshot(), "") + if "worker-start" in joined or "run-create" in joined or "check" in joined and "--wait" in joined: + self.calls.append("worker") + if self.task_hangs: + return self._outcome(argv, None, "", "deadline reached", timed_out=True) + self.project.mkdir(parents=True, exist_ok=True) + (self.project / "evidence.txt").write_text( + "dely-cycle-marker", encoding="utf-8" + ) + return self._outcome(argv, 0, '{"outcome":"done"}', "") + if "cycle-check" in joined: + self.calls.append("check") + if self.check_exit_code == 0: + return self._outcome(argv, 0, "marker matched\n", "") + return self._outcome(argv, self.check_exit_code, "", "marker mismatch\n") + self.calls.append(f"execute:{argv[0]}") + return proc.run( + argv, + timeout=timeout, + context="environment", + cwd=cwd, + env=env, + extra_values=extra_values, + ) + + def _outcome(self, argv, exit_code, stdout, stderr, timed_out=False): + return proc.CommandOutcome( + argv=tuple(argv), + exit_code=exit_code, + stdout=stdout, + stderr=stderr, + started_at="2026-09-14T22:15:30Z", + finished_at="2026-09-14T22:15:31Z", + elapsed_seconds=1.0, + timed_out=timed_out, + context="environment", + ) + + def put_tree(self, local_dir: Path, remote_dir: str) -> None: + self.calls.append("put_tree") + destination = Path(remote_dir) + destination.mkdir(parents=True, exist_ok=True) + for path in sorted(Path(local_dir).rglob("*")): + if path.is_file(): + target = destination / path.relative_to(local_dir) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(path.read_bytes()) + + def fetch_tree(self, remote_dir: str, local_dir: Path) -> None: + self.calls.append("fetch_tree") + if self.fetch_fails: + raise OSError("the transport refused to copy the tree out") + source = Path(remote_dir) + Path(local_dir).mkdir(parents=True, exist_ok=True) + for path in sorted(source.rglob("*")): + if path.is_file(): + target = Path(local_dir) / path.relative_to(source) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_bytes(path.read_bytes()) + + def write_file(self, remote_path: str, content: str, *, mode: int = 0o600) -> None: + self.calls.append("write_file") + target = Path(remote_path) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(content, encoding="utf-8") + target.chmod(mode) + + def stop(self) -> StopReport: + self.calls.append("stop") + return StopReport( + confirmed=self.stop_confirmed, + detail="stopped by the fake adapter" + if self.stop_confirmed + else "the fake adapter could not confirm the stop", + ) + + def destroy(self) -> DestroyReport: + self.calls.append("destroy") + if not self.leave_residue and self.home.exists(): + shutil.rmtree(self.home) + self.destroyed = True + return DestroyReport( + removed=() if self.leave_residue else (str(self.home),), + retained=(str(self.shared_base),), + detail="fake destroy", + ) + + def resource_exists(self, resource: Resource) -> bool: + return Path(resource.identifier).exists() + + def describe(self) -> dict: + return {"backend": self.name, "image": "fake"} + + +class ScriptedAdapter(FakeAdapter): + """A fake whose command answers are chosen by matching the argv.""" + + def __init__(self, root: Path, script=None, **options): + super().__init__(root, **options) + self.script = list(script or []) + self.executed: list[tuple[str, ...]] = [] + self.written: list[tuple[str, str, int]] = [] + + def execute( + self, + argv, + *, + timeout: float, + cwd=None, + env=None, + extra_values=(), + ): + argv = tuple(argv) + self.executed.append(argv) + joined = " ".join(argv) + for needle, exit_code, stdout, stderr, timed_out in self.script: + if needle in joined: + return self._outcome(argv, exit_code, stdout, stderr, timed_out) + return self._outcome(argv, 0, "", "") + + def write_file(self, remote_path: str, content: str, *, mode: int = 0o600) -> None: + self.written.append((remote_path, content, mode)) + super().write_file(remote_path, content, mode=mode) diff --git a/experiments/minimal-cycle/tests/test_auth.py b/experiments/minimal-cycle/tests/test_auth.py new file mode 100644 index 0000000..d994fc7 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_auth.py @@ -0,0 +1,183 @@ +"""Auth reaches the environment by one declared method and leaves no trace.""" + +import json +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import auth, config as config_module, status +from tests.fakes import FakeAdapter +from tests.test_config import minimal_document + +TOKEN = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfgh" + + +def make_config(auth_section): + document = minimal_document() + document["auth"] = auth_section + return config_module.from_document(document) + + +class ExistingLoginTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.host_home = self.root / "hosthome" + (self.host_home / ".claude").mkdir(parents=True) + (self.host_home / ".claude" / ".credentials.json").write_text( + json.dumps({"token": TOKEN}), encoding="utf-8" + ) + (self.host_home / ".claude" / "history.jsonl").write_text( + "not allowlisted\n", encoding="utf-8" + ) + self.adapter = FakeAdapter(self.root / "run") + self.handle = self.adapter.create() + self.config = make_config( + { + "mode": "existing_login", + "reference": "workshop-host-login", + "allowlist": [".claude/.credentials.json"], + } + ) + self.addCleanup(self._tmp.cleanup) + + def bootstrap(self): + return auth.bootstrap( + run_config=self.config, + adapter=self.adapter, + handle=self.handle, + host_home=self.host_home, + environ={}, + ) + + def test_the_allowlisted_entry_reaches_the_environment(self): + record, _ = self.bootstrap() + self.assertEqual(record.status, status.PhaseStatus.OK) + copied = Path(self.handle.home_path) / ".claude" / ".credentials.json" + self.assertTrue(copied.is_file()) + + def test_entries_outside_the_allowlist_are_left_on_the_host(self): + self.bootstrap() + self.assertFalse( + (Path(self.handle.home_path) / ".claude" / "history.jsonl").exists() + ) + + def test_the_copied_entry_is_owner_only(self): + self.bootstrap() + copied = Path(self.handle.home_path) / ".claude" / ".credentials.json" + self.assertEqual(copied.stat().st_mode & 0o777, 0o600) + + def test_no_auth_material_in_any_rendered_artifact(self): + record, overlay = self.bootstrap() + rendered = json.dumps(record.to_document()) + self.assertNotIn(TOKEN, rendered) + self.assertNotIn("zzqwertyuiopasdfghjklzxcvbnm", rendered) + self.assertEqual(overlay, {}) + + def test_the_receipt_names_the_method_and_the_relative_entries_only(self): + record, _ = self.bootstrap() + self.assertEqual(record.mode, "existing_login") + self.assertEqual(record.reference, "workshop-host-login") + self.assertEqual(record.entries, [".claude/.credentials.json"]) + self.assertNotIn(str(self.host_home), json.dumps(record.to_document())) + + def test_a_missing_allowlisted_entry_blocks_the_run(self): + (self.host_home / ".claude" / ".credentials.json").unlink() + record, _ = self.bootstrap() + self.assertEqual(record.status, status.PhaseStatus.BLOCKED) + self.assertIn(".claude/.credentials.json", record.detail) + + def test_teardown_removes_the_entry_and_verifies_it(self): + record, _ = self.bootstrap() + after = auth.teardown(adapter=self.adapter, handle=self.handle, record=record) + self.assertTrue(after.removed_after_run) + self.assertFalse( + (Path(self.handle.home_path) / ".claude" / ".credentials.json").exists() + ) + + +class ShortLivedTokenTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.adapter = FakeAdapter(self.root / "run") + self.handle = self.adapter.create() + self.config = make_config( + { + "mode": "short_lived_token", + "reference": "workshop-token", + "token_env": "CLAUDE_CODE_OAUTH_TOKEN", + } + ) + self.addCleanup(self._tmp.cleanup) + + def test_the_value_travels_in_the_process_environment_only(self): + record, overlay = auth.bootstrap( + run_config=self.config, + adapter=self.adapter, + handle=self.handle, + host_home=self.root, + environ={"CLAUDE_CODE_OAUTH_TOKEN": TOKEN}, + ) + self.assertEqual(record.status, status.PhaseStatus.OK) + self.assertEqual(overlay, {"CLAUDE_CODE_OAUTH_TOKEN": TOKEN}) + self.assertNotIn("write_file", self.adapter.calls) + self.assertNotIn(TOKEN, json.dumps(record.to_document())) + self.assertTrue(record.removed_after_run) + + def test_a_missing_variable_blocks_rather_than_prompting(self): + record, overlay = auth.bootstrap( + run_config=self.config, + adapter=self.adapter, + handle=self.handle, + host_home=self.root, + environ={}, + ) + self.assertEqual(record.status, status.PhaseStatus.BLOCKED) + self.assertIn("CLAUDE_CODE_OAUTH_TOKEN", record.detail) + self.assertEqual(overlay, {}) + + def test_the_receipt_records_no_length_and_no_prefix(self): + record, _ = auth.bootstrap( + run_config=self.config, + adapter=self.adapter, + handle=self.handle, + host_home=self.root, + environ={"CLAUDE_CODE_OAUTH_TOKEN": TOKEN}, + ) + rendered = json.dumps(record.to_document()) + self.assertNotIn(str(len(TOKEN)), rendered) + self.assertNotIn(TOKEN[:4], rendered) + + +class HelperTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.adapter = FakeAdapter(self.root / "run") + self.handle = self.adapter.create() + self.config = make_config( + { + "mode": "api_key_helper", + "reference": "workshop-helper", + "helper_argv": ["/usr/local/bin/fetch-key", "--quiet"], + } + ) + self.addCleanup(self._tmp.cleanup) + + def test_the_helper_is_declared_and_no_value_is_passed(self): + record, overlay = auth.bootstrap( + run_config=self.config, + adapter=self.adapter, + handle=self.handle, + host_home=self.root, + environ={}, + ) + self.assertEqual(record.status, status.PhaseStatus.OK) + self.assertEqual(overlay, {}) + settings = Path(self.handle.home_path) / ".claude" / "settings.json" + self.assertTrue(settings.is_file()) + self.assertEqual( + json.loads(settings.read_text(encoding="utf-8"))["apiKeyHelper"], + "/usr/local/bin/fetch-key --quiet", + ) diff --git a/experiments/minimal-cycle/tests/test_cleanup.py b/experiments/minimal-cycle/tests/test_cleanup.py new file mode 100644 index 0000000..d9e81e9 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_cleanup.py @@ -0,0 +1,150 @@ +"""Cleanup runs only after a confirmed export, and only on per-run resources.""" + +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import cleanup, result, status +from cycle_runner.adapters.base import Resource +from tests.fakes import FakeAdapter + + +def export_record(state): + return result.ExportRecord(status=state, detail="from the test") + + +class GateTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) / "run" + self.adapter = FakeAdapter(self.root) + self.handle = self.adapter.create() + self.addCleanup(self._tmp.cleanup) + + def test_no_destroy_when_export_is_not_confirmed(self): + record = cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record(status.ExportStatus.PARTIAL), + ) + self.assertNotIn("destroy", self.adapter.calls) + self.assertEqual(record.status, status.CleanupStatus.RESIDUE) + self.assertIn("export", record.reason.lower()) + self.assertTrue(self.adapter.home.exists()) + + def test_a_confirmed_export_destroys_and_verifies(self): + record = cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record(status.ExportStatus.CONFIRMED), + ) + self.assertIn("destroy", self.adapter.calls) + self.assertEqual(record.status, status.CleanupStatus.DESTROYED) + self.assertTrue(record.verified) + self.assertFalse(self.adapter.home.exists()) + + def test_a_surviving_per_run_resource_is_reported_as_residue(self): + self.adapter.leave_residue = True + record = cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record(status.ExportStatus.CONFIRMED), + ) + self.assertEqual(record.status, status.CleanupStatus.RESIDUE) + self.assertTrue(any(str(self.adapter.home) in item for item in record.retained)) + + def test_shared_base_survives_cleanup(self): + cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record(status.ExportStatus.CONFIRMED), + ) + self.assertTrue(self.adapter.shared_base.is_file()) + self.assertEqual(self.adapter.shared_base.read_bytes(), b"shared base image") + + def test_a_lost_shared_resource_makes_cleanup_unknown(self): + record_before = self.adapter.shared_base + self.adapter.destroy = _destroy_too_much(self.adapter) + record = cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record(status.ExportStatus.CONFIRMED), + ) + self.assertFalse(record_before.exists()) + self.assertEqual(record.status, status.CleanupStatus.UNKNOWN) + self.assertIn("shared", record.reason.lower()) + + +def _destroy_too_much(adapter): + import shutil + + from cycle_runner.adapters.base import DestroyReport + + def destroy(): + adapter.calls.append("destroy") + shutil.rmtree(adapter.root.parent) + return DestroyReport(removed=(str(adapter.home),), detail="removed the world") + + return destroy + + +class DeclarationTest(unittest.TestCase): + def test_a_per_run_path_that_contains_a_shared_path_is_refused(self): + with self.assertRaises(cleanup.CleanupContractError): + cleanup.assert_declarations_disjoint( + per_run=(Resource(kind="path", identifier="/state/run"),), + shared=(Resource(kind="path", identifier="/state/run/base.img"),), + ) + + def test_a_resource_declared_both_ways_is_refused(self): + with self.assertRaises(cleanup.CleanupContractError): + cleanup.assert_declarations_disjoint( + per_run=(Resource(kind="container", identifier="box"),), + shared=(Resource(kind="container", identifier="box"),), + ) + + def test_declarations_that_do_not_overlap_are_accepted(self): + cleanup.assert_declarations_disjoint( + per_run=(Resource(kind="path", identifier="/state/run/home"),), + shared=(Resource(kind="image", identifier="/state/shared/base.img"),), + ) + + +class SafeRemoveTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.addCleanup(self._tmp.cleanup) + + def test_a_file_and_a_directory_are_removed(self): + target_file = self.root / "overlay.img" + target_file.write_bytes(b"x") + target_dir = self.root / "home" + (target_dir / "nested").mkdir(parents=True) + cleanup.safe_remove(target_file, allowed_roots=[self.root], protected=[]) + cleanup.safe_remove(target_dir, allowed_roots=[self.root], protected=[]) + self.assertFalse(target_file.exists()) + self.assertFalse(target_dir.exists()) + + def test_a_path_outside_the_allowed_roots_is_refused(self): + outside = Path(tempfile.mkdtemp()) + self.addCleanup(lambda: outside.rmdir() if outside.exists() else None) + with self.assertRaises(cleanup.CleanupContractError): + cleanup.safe_remove(outside, allowed_roots=[self.root], protected=[]) + self.assertTrue(outside.exists()) + + def test_a_path_containing_a_protected_path_is_refused(self): + home = self.root / "home" + base = home / "shared" / "base.img" + base.parent.mkdir(parents=True) + base.write_bytes(b"shared") + with self.assertRaises(cleanup.CleanupContractError): + cleanup.safe_remove(home, allowed_roots=[self.root], protected=[base]) + self.assertTrue(base.is_file()) + + def test_the_filesystem_root_is_never_removable(self): + with self.assertRaises(cleanup.CleanupContractError): + cleanup.safe_remove(Path("/"), allowed_roots=[Path("/")], protected=[]) + + def test_removing_something_absent_is_not_an_error(self): + cleanup.safe_remove(self.root / "absent", allowed_roots=[self.root], protected=[]) diff --git a/experiments/minimal-cycle/tests/test_export.py b/experiments/minimal-cycle/tests/test_export.py new file mode 100644 index 0000000..db36cb3 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_export.py @@ -0,0 +1,130 @@ +"""The export receipt is a re-read of the host, not a memory of the write.""" + +import json +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import export, status + + +class ExporterTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) / "run" + self.exporter = export.Exporter(self.root) + self.addCleanup(self._tmp.cleanup) + + def test_a_written_artifact_lands_on_the_host(self): + self.exporter.write_text("check.stdout", "marker matched\n") + self.assertEqual( + (self.root / "check.stdout").read_text(encoding="utf-8"), "marker matched\n" + ) + + def test_nested_artifacts_create_their_directories(self): + self.exporter.write_text("logs/runner.log", "line\n") + self.assertTrue((self.root / "logs" / "runner.log").is_file()) + + def test_a_confirmed_export_describes_every_artifact(self): + self.exporter.write_text("check.stdout", "marker matched\n") + self.exporter.write_json("run.json", {"status": "SETTLED"}) + record = self.exporter.confirm() + self.assertEqual(record.status, status.ExportStatus.CONFIRMED) + paths = {entry["path"] for entry in record.artifacts} + self.assertEqual(paths, {"check.stdout", "run.json"}) + for entry in record.artifacts: + self.assertEqual(len(entry["sha256"]), 64) + self.assertGreater(entry["size_bytes"], 0) + + def test_receipt_fails_when_artifact_changes_after_write(self): + self.exporter.write_text("check.stdout", "marker matched\n") + (self.root / "check.stdout").write_text("", encoding="utf-8") + record = self.exporter.confirm() + self.assertNotEqual(record.status, status.ExportStatus.CONFIRMED) + self.assertTrue(any("check.stdout" in entry for entry in record.missing)) + + def test_receipt_fails_when_a_required_artifact_disappears(self): + self.exporter.write_text("check.stdout", "marker matched\n") + (self.root / "check.stdout").unlink() + record = self.exporter.confirm() + self.assertNotEqual(record.status, status.ExportStatus.CONFIRMED) + self.assertTrue(any("check.stdout" in entry for entry in record.missing)) + + def test_an_export_with_some_evidence_left_is_partial_not_failed(self): + self.exporter.write_text("check.stdout", "marker matched\n") + self.exporter.write_text("run.json", "{}\n") + (self.root / "check.stdout").unlink() + record = self.exporter.confirm() + self.assertEqual(record.status, status.ExportStatus.PARTIAL) + + def test_an_export_with_nothing_left_has_failed(self): + self.exporter.write_text("check.stdout", "marker matched\n") + (self.root / "check.stdout").unlink() + record = self.exporter.confirm() + self.assertEqual(record.status, status.ExportStatus.FAILED) + + def test_an_optional_artifact_that_is_absent_does_not_block(self): + self.exporter.write_text("run.json", "{}\n") + self.exporter.declare("task-artifact/evidence.txt", required=False) + record = self.exporter.confirm() + self.assertEqual(record.status, status.ExportStatus.CONFIRMED) + + def test_a_required_artifact_declared_but_never_written_blocks(self): + self.exporter.write_text("run.json", "{}\n") + self.exporter.declare("diff.patch", required=True) + record = self.exporter.confirm() + self.assertNotEqual(record.status, status.ExportStatus.CONFIRMED) + self.assertTrue(any("diff.patch" in entry for entry in record.missing)) + + def test_only_a_confirmed_export_permits_cleanup(self): + self.exporter.write_text("run.json", "{}\n") + self.assertTrue(self.exporter.confirm().status.permits_cleanup) + + def test_the_receipt_is_written_and_reloads(self): + self.exporter.write_text("run.json", "{}\n") + record = self.exporter.confirm() + receipt = self.root / "export-receipt.json" + self.assertTrue(receipt.is_file()) + self.assertEqual(record.receipt_path, "export-receipt.json") + reloaded = json.loads(receipt.read_text(encoding="utf-8")) + self.assertEqual(reloaded["status"], "CONFIRMED") + self.assertIn("artifacts", reloaded) + + def test_written_text_is_redacted_on_the_way_out(self): + secret = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfgh" + self.exporter.write_text("logs/runner.log", f"key {secret}\n") + written = (self.root / "logs" / "runner.log").read_text(encoding="utf-8") + self.assertNotIn(secret, written) + + def test_written_structures_are_redacted_on_the_way_out(self): + self.exporter.write_json("auth-receipt.json", {"token": "hunterhunterhunter"}) + written = (self.root / "auth-receipt.json").read_text(encoding="utf-8") + self.assertNotIn("hunterhunterhunter", written) + + def test_a_path_escaping_the_run_directory_is_refused(self): + with self.assertRaises(export.ExportError): + self.exporter.write_text("../escape.txt", "x") + + def test_an_absolute_path_is_refused(self): + with self.assertRaises(export.ExportError): + self.exporter.write_text("/etc/passwd", "x") + + def test_an_adopted_tree_is_copied_and_described(self): + source = Path(self._tmp.name) / "source" + (source / "nested").mkdir(parents=True) + (source / "nested" / "evidence.txt").write_text("marker\n", encoding="utf-8") + self.exporter.adopt_tree("task-artifact", source, required=True) + record = self.exporter.confirm() + self.assertEqual(record.status, status.ExportStatus.CONFIRMED) + self.assertTrue( + (self.root / "task-artifact" / "nested" / "evidence.txt").is_file() + ) + self.assertTrue( + any(entry["path"].startswith("task-artifact/") for entry in record.artifacts) + ) + + def test_adopting_an_absent_tree_is_reported_rather_than_raised(self): + self.exporter.write_text("run.json", "{}\n") + self.exporter.adopt_tree("task-artifact", Path("/nonexistent"), required=True) + record = self.exporter.confirm() + self.assertNotEqual(record.status, status.ExportStatus.CONFIRMED) diff --git a/experiments/minimal-cycle/tests/test_hostinfo.py b/experiments/minimal-cycle/tests/test_hostinfo.py new file mode 100644 index 0000000..96ab175 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_hostinfo.py @@ -0,0 +1,102 @@ +"""The host baseline is what shows whether the run leaked outside its environment.""" + +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import hostinfo + + +class SnapshotTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.state = self.root / "state" + self.artifacts = self.root / "artifacts" + self.state.mkdir() + self.artifacts.mkdir() + self.addCleanup(self._tmp.cleanup) + + def snapshot(self): + return hostinfo.snapshot(state_root=self.state, artifact_root=self.artifacts) + + def test_the_snapshot_names_the_facts_a_reader_needs(self): + taken = self.snapshot() + for key in ( + "captured_at", + "kernel", + "architecture", + "python_version", + "hostname_digest", + "machine_id_digest", + "boot_id_digest", + "user_digest", + "state_root", + "artifact_root", + ): + self.assertIn(key, taken) + + def test_identifiers_are_digests_rather_than_names(self): + import socket + + taken = self.snapshot() + self.assertNotIn(socket.gethostname(), str(taken)) + self.assertEqual(len(taken["hostname_digest"]), 16) + + def test_the_roots_are_listed_by_their_immediate_entries(self): + (self.state / "run-one").mkdir() + (self.state / "run-two").mkdir() + taken = self.snapshot() + self.assertEqual(taken["state_root"]["entries"], ["run-one", "run-two"]) + self.assertTrue(taken["state_root"]["exists"]) + + def test_an_absent_root_is_recorded_rather_than_raised(self): + taken = hostinfo.snapshot( + state_root=self.root / "absent", artifact_root=self.artifacts + ) + self.assertFalse(taken["state_root"]["exists"]) + self.assertEqual(taken["state_root"]["entries"], []) + + def test_extra_backend_facts_are_carried_through(self): + taken = hostinfo.snapshot( + state_root=self.state, + artifact_root=self.artifacts, + extra={"containers": ["one"]}, + ) + self.assertEqual(taken["extra"]["containers"], ["one"]) + + +class DifferenceTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.state = self.root / "state" + self.artifacts = self.root / "artifacts" + self.state.mkdir() + self.artifacts.mkdir() + self.addCleanup(self._tmp.cleanup) + + def snapshot(self): + return hostinfo.snapshot(state_root=self.state, artifact_root=self.artifacts) + + def test_identical_snapshots_differ_in_nothing(self): + before = self.snapshot() + after = self.snapshot() + self.assertEqual(hostinfo.difference(before, after), {}) + + def test_an_entry_left_behind_is_reported(self): + before = self.snapshot() + (self.state / "leftover").mkdir() + after = self.snapshot() + self.assertEqual( + hostinfo.difference(before, after)["state_root_added"], ["leftover"] + ) + + def test_an_entry_that_disappeared_is_reported(self): + (self.state / "temporary").mkdir() + before = self.snapshot() + (self.state / "temporary").rmdir() + after = self.snapshot() + self.assertEqual( + hostinfo.difference(before, after)["state_root_removed"], ["temporary"] + ) diff --git a/experiments/minimal-cycle/tests/test_probe.py b/experiments/minimal-cycle/tests/test_probe.py new file mode 100644 index 0000000..0547624 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_probe.py @@ -0,0 +1,182 @@ +"""The identity verdict is what stands between a real run and a host fallback.""" + +import unittest + +from cycle_runner import probe, result + + +def host_snapshot(**overrides): + snapshot = { + "hostname": "workshop", + "machine_id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "boot_id": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", + "home": "/home/someone", + "user": "someone", + "path": "/usr/bin:/bin", + "pid": "4242", + "uname": "linux workshop amd64", + "container_marker": "", + "virt": "none", + "orca_path": "/usr/bin/orca", + "orca_version": "1.4.201", + "project_real": "/home/someone/code/under-test", + } + snapshot.update(overrides) + return snapshot + + +def container_snapshot(**overrides): + snapshot = host_snapshot( + hostname="dely-cycle-box", + machine_id="cccccccccccccccccccccccccccccccc", + home="/var/tmp/cycle-state/run/home", + container_marker="containerenv", + project_real="/var/tmp/cycle-state/run/home/project", + ) + snapshot.update(overrides) + return snapshot + + +def machine_snapshot(**overrides): + snapshot = container_snapshot( + container_marker="", + boot_id="dddddddd-dddd-dddd-dddd-dddddddddddd", + virt="kvm", + ) + snapshot.update(overrides) + return snapshot + + +def verdict(environment, *, backend="distrobox", host=None): + return probe.verdict( + host=host or host_snapshot(), + environment=environment, + backend=backend, + expected_home="/var/tmp/cycle-state/run/home", + expected_project="/var/tmp/cycle-state/run/home/project", + ) + + +class ParseTest(unittest.TestCase): + def test_key_value_lines_parse_into_a_snapshot(self): + parsed = probe.parse("hostname=box\nhome=/root\nempty=\n") + self.assertEqual(parsed["hostname"], "box") + self.assertEqual(parsed["home"], "/root") + self.assertEqual(parsed["empty"], "") + + def test_values_containing_an_equals_sign_survive(self): + parsed = probe.parse("path=/usr/bin:/bin\nnote=a=b\n") + self.assertEqual(parsed["note"], "a=b") + + def test_unrecognised_lines_are_ignored_rather_than_crashing(self): + parsed = probe.parse("garbage line\nhostname=box\n") + self.assertEqual(parsed["hostname"], "box") + + def test_the_probe_script_is_posix_shell_and_names_every_field(self): + script = probe.PROBE_SCRIPT + for field in probe.REQUIRED_FIELDS: + self.assertIn(f"{field}=", script) + self.assertNotIn("bash", script) + + +class VerdictTest(unittest.TestCase): + def test_a_container_snapshot_is_accepted(self): + record = verdict(container_snapshot()) + self.assertEqual(record.verdict, result.IDENTITY_ENVIRONMENT) + self.assertIn("container-marker", " ".join(record.markers)) + + def test_host_snapshot_is_rejected(self): + record = verdict(host_snapshot()) + self.assertEqual(record.verdict, result.IDENTITY_HOST_FALLBACK) + self.assertIn("host", record.reason.lower()) + + def test_a_snapshot_with_a_marker_but_the_host_home_is_rejected(self): + record = verdict(container_snapshot(home="/home/someone")) + self.assertEqual(record.verdict, result.IDENTITY_HOST_FALLBACK) + self.assertIn("home", record.reason.lower()) + + def test_a_snapshot_pointing_at_the_host_checkout_is_rejected(self): + record = verdict( + container_snapshot(project_real="/home/someone/code/under-test") + ) + self.assertEqual(record.verdict, result.IDENTITY_HOST_FALLBACK) + self.assertIn("project", record.reason.lower()) + + def test_a_container_marker_does_not_satisfy_the_machine_backend(self): + record = verdict(container_snapshot(), backend="vm") + self.assertEqual(record.verdict, result.IDENTITY_HOST_FALLBACK) + self.assertIn("boot", record.reason.lower()) + + def test_a_machine_snapshot_satisfies_the_machine_backend(self): + record = verdict(machine_snapshot(), backend="vm") + self.assertEqual(record.verdict, result.IDENTITY_ENVIRONMENT) + + def test_an_empty_snapshot_is_unknown_rather_than_accepted(self): + record = verdict({}) + self.assertEqual(record.verdict, result.IDENTITY_UNKNOWN) + + def test_identity_records_whether_orca_is_present_in_the_environment(self): + present = verdict(container_snapshot()) + self.assertTrue(present.orca_present) + self.assertEqual(present.orca_version, "1.4.201") + absent = verdict(container_snapshot(orca_path="", orca_version="")) + self.assertFalse(absent.orca_present) + + def test_the_host_identifiers_are_stored_as_digests_not_as_names(self): + record = verdict(container_snapshot()) + self.assertNotIn("workshop", str(record.host)) + self.assertNotIn("/home/someone", str(record.host)) + self.assertIn("hostname_digest", record.host) + + def test_the_environment_home_and_project_stay_readable_for_a_reader(self): + record = verdict(container_snapshot()) + self.assertEqual(record.environment["home"], "/var/tmp/cycle-state/run/home") + + +class GateTest(unittest.TestCase): + def test_only_an_environment_verdict_with_orca_opens_the_gate(self): + self.assertTrue(probe.may_continue(verdict(container_snapshot()))[0]) + blocked, reason = probe.may_continue(verdict(host_snapshot())) + self.assertFalse(blocked) + self.assertIn("host", reason.lower()) + + def test_a_missing_orca_blocks_instead_of_falling_back(self): + allowed, reason = probe.may_continue( + verdict(container_snapshot(orca_path="", orca_version="")) + ) + self.assertFalse(allowed) + self.assertIn("orca", reason.lower()) + + +class HostProbeIntegrationTest(unittest.TestCase): + """The probe is shell, so the only honest check is running it.""" + + def test_the_probe_emits_exactly_one_line_for_each_field(self): + from pathlib import Path + + from cycle_runner import proc + + outcome = proc.run( + probe.probe_argv(str(Path.cwd())), timeout=30, context="host" + ) + self.assertEqual(outcome.exit_code, 0, outcome.stderr) + lines = [line for line in outcome.stdout.splitlines() if line.strip()] + keys = [line.split("=", 1)[0] for line in lines] + self.assertEqual(sorted(keys), sorted(probe.REQUIRED_FIELDS)) + + def test_the_probe_resolves_the_directory_it_is_given(self): + from pathlib import Path + + from cycle_runner import proc + + here = Path.cwd().resolve() + outcome = proc.run(probe.probe_argv(str(here)), timeout=30, context="host") + self.assertEqual(probe.parse(outcome.stdout)["project_real"], str(here)) + + def test_an_absent_project_directory_resolves_to_nothing(self): + from cycle_runner import proc + + outcome = proc.run( + probe.probe_argv("/nonexistent-cycle-path"), timeout=30, context="host" + ) + self.assertEqual(probe.parse(outcome.stdout)["project_real"], "") diff --git a/experiments/minimal-cycle/tests/test_proc.py b/experiments/minimal-cycle/tests/test_proc.py new file mode 100644 index 0000000..e6c7e8f --- /dev/null +++ b/experiments/minimal-cycle/tests/test_proc.py @@ -0,0 +1,98 @@ +"""Commands are captured with their timings, their deadline and their context.""" + +import sys +import unittest + +from cycle_runner import proc + + +class RunTest(unittest.TestCase): + def test_output_and_exit_code_are_captured(self): + outcome = proc.run( + ["sh", "-c", "echo out; echo err >&2; exit 3"], + timeout=10, + context="host", + ) + self.assertEqual(outcome.exit_code, 3) + self.assertIn("out", outcome.stdout) + self.assertIn("err", outcome.stderr) + self.assertFalse(outcome.timed_out) + + def test_the_command_records_its_argv_and_context(self): + outcome = proc.run(["true"], timeout=10, context="environment") + self.assertEqual(outcome.argv, ("true",)) + self.assertEqual(outcome.context, "environment") + + def test_timings_are_recorded_and_ordered(self): + outcome = proc.run(["true"], timeout=10, context="host") + self.assertLessEqual(outcome.started_at, outcome.finished_at) + self.assertGreaterEqual(outcome.elapsed_seconds, 0.0) + + def test_a_deadline_marks_the_command_rather_than_raising(self): + outcome = proc.run( + [sys.executable, "-c", "import time; time.sleep(30)"], + timeout=0.4, + context="host", + ) + self.assertTrue(outcome.timed_out) + self.assertIsNone(outcome.exit_code) + self.assertLess(outcome.elapsed_seconds, 20) + + def test_a_deadline_leaves_no_child_behind(self): + outcome = proc.run( + [ + sys.executable, + "-c", + "import subprocess, sys, time;" + "subprocess.Popen([sys.executable, '-c', 'import time; time.sleep(30)']);" + "time.sleep(30)", + ], + timeout=0.6, + context="host", + ) + self.assertTrue(outcome.timed_out) + self.assertTrue(outcome.killed_process_group) + + def test_output_is_redacted_before_it_is_kept(self): + secret = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfgh" + outcome = proc.run( + ["sh", "-c", f"echo {secret}"], timeout=10, context="host" + ) + self.assertNotIn(secret, outcome.stdout) + + def test_a_missing_executable_is_reported_rather_than_raised(self): + outcome = proc.run( + ["this-command-does-not-exist-anywhere"], timeout=10, context="host" + ) + self.assertIsNone(outcome.exit_code) + self.assertFalse(outcome.timed_out) + self.assertIn("this-command-does-not-exist-anywhere", outcome.stderr) + self.assertTrue(outcome.launch_failed) + + def test_the_outcome_converts_to_a_command_record(self): + record = proc.run(["true"], timeout=10, context="host").to_record() + document = record.to_document() + self.assertEqual(document["argv"], ["true"]) + self.assertEqual(document["context"], "host") + + def test_extra_values_are_redacted_too(self): + outcome = proc.run( + ["sh", "-c", "echo plainvalue"], + timeout=10, + context="host", + extra_values=["plainvalue"], + ) + self.assertNotIn("plainvalue", outcome.stdout) + + def test_an_environment_overlay_reaches_the_child(self): + outcome = proc.run( + ["sh", "-c", "echo $CYCLE_PROBE"], + timeout=10, + context="host", + env={"CYCLE_PROBE": "visible"}, + ) + self.assertIn("visible", outcome.stdout) + + def test_the_command_succeeded_helper_reads_the_exit_code(self): + self.assertTrue(proc.run(["true"], timeout=10, context="host").ok) + self.assertFalse(proc.run(["false"], timeout=10, context="host").ok) diff --git a/experiments/minimal-cycle/tests/test_project.py b/experiments/minimal-cycle/tests/test_project.py new file mode 100644 index 0000000..96eb2b2 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_project.py @@ -0,0 +1,133 @@ +"""The project copy is exported at a pinned revision and diffed on the host.""" + +import subprocess +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import project + + +def git(repo: Path, *arguments): + subprocess.run( + ["git", "-C", str(repo), *arguments], + check=True, + capture_output=True, + text=True, + ) + + +def make_repo(root: Path) -> Path: + repo = root / "under-test" + repo.mkdir() + git(repo, "init", "-q", "-b", "main") + git(repo, "config", "user.email", "cycle@example.invalid") + git(repo, "config", "user.name", "cycle") + (repo / "readme.md").write_text("first\n", encoding="utf-8") + (repo / "src").mkdir() + (repo / "src" / "app.py").write_text("value = 1\n", encoding="utf-8") + git(repo, "add", "-A") + git(repo, "commit", "-q", "-m", "initial") + return repo + + +class ExportTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.repo = make_repo(self.root) + self.addCleanup(self._tmp.cleanup) + + def test_a_revision_resolves_to_a_commit_digest(self): + resolved = project.resolve_revision(self.repo, "main") + self.assertRegex(resolved, r"^[0-9a-f]{40}$") + + def test_an_unknown_revision_is_refused(self): + with self.assertRaises(project.ProjectError): + project.resolve_revision(self.repo, "no-such-ref") + + def test_the_export_carries_the_tracked_files(self): + destination = self.root / "copy" + project.export_revision(self.repo, "main", destination) + self.assertEqual((destination / "readme.md").read_text(encoding="utf-8"), "first\n") + self.assertTrue((destination / "src" / "app.py").is_file()) + + def test_the_export_carries_no_repository_metadata(self): + destination = self.root / "copy" + project.export_revision(self.repo, "main", destination) + self.assertFalse((destination / ".git").exists()) + + def test_the_export_ignores_uncommitted_work(self): + (self.repo / "scratch.txt").write_text("not committed\n", encoding="utf-8") + destination = self.root / "copy" + project.export_revision(self.repo, "main", destination) + self.assertFalse((destination / "scratch.txt").exists()) + + def test_exporting_into_an_occupied_directory_is_refused(self): + destination = self.root / "copy" + destination.mkdir() + (destination / "leftover").write_text("x", encoding="utf-8") + with self.assertRaises(project.ProjectError): + project.export_revision(self.repo, "main", destination) + + def test_a_source_that_is_not_a_repository_is_refused(self): + with self.assertRaises(project.ProjectError): + project.resolve_revision(self.root, "main") + + +class DiffTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.baseline = self.root / "baseline" + self.current = self.root / "current" + for tree in (self.baseline, self.current): + tree.mkdir() + (tree / "readme.md").write_text("first\n", encoding="utf-8") + self.addCleanup(self._tmp.cleanup) + + def test_identical_trees_produce_no_patch(self): + self.assertEqual(project.unified_diff(self.baseline, self.current), "") + + def test_an_added_file_appears_as_an_addition(self): + (self.current / "evidence.txt").write_text("dely-cycle-marker\n", encoding="utf-8") + patch = project.unified_diff(self.baseline, self.current) + self.assertIn("--- /dev/null", patch) + self.assertIn("+++ b/evidence.txt", patch) + self.assertIn("+dely-cycle-marker", patch) + + def test_a_removed_file_appears_as_a_removal(self): + (self.current / "readme.md").unlink() + patch = project.unified_diff(self.baseline, self.current) + self.assertIn("--- a/readme.md", patch) + self.assertIn("+++ /dev/null", patch) + + def test_a_modified_file_appears_as_a_hunk(self): + (self.current / "readme.md").write_text("second\n", encoding="utf-8") + patch = project.unified_diff(self.baseline, self.current) + self.assertIn("-first", patch) + self.assertIn("+second", patch) + + def test_a_changed_binary_file_is_named_without_its_content(self): + (self.baseline / "blob.bin").write_bytes(b"\x00\x01\x02") + (self.current / "blob.bin").write_bytes(b"\x00\x09\x02") + patch = project.unified_diff(self.baseline, self.current) + self.assertIn("Binary files a/blob.bin and b/blob.bin differ", patch) + self.assertNotIn("\x09\x02", patch) + + def test_nested_paths_are_reported_with_forward_slashes(self): + nested = self.current / "src" + nested.mkdir() + (nested / "new.py").write_text("value = 2\n", encoding="utf-8") + patch = project.unified_diff(self.baseline, self.current) + self.assertIn("+++ b/src/new.py", patch) + + def test_the_patch_is_redacted(self): + secret = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfgh" + (self.current / "leak.txt").write_text(secret + "\n", encoding="utf-8") + patch = project.unified_diff(self.baseline, self.current) + self.assertNotIn(secret, patch) + + def test_a_missing_current_tree_reports_every_file_as_removed(self): + patch = project.unified_diff(self.baseline, self.root / "absent") + self.assertIn("--- a/readme.md", patch) diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py new file mode 100644 index 0000000..e5d26ef --- /dev/null +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -0,0 +1,147 @@ +"""The single Claude Code worker is launched through Orca, inside the environment.""" + +import json +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import config as config_module, status, worker +from tests.fakes import ScriptedAdapter +from tests.test_config import minimal_document + +READY = json.dumps({"runId": "run-abcdef"}) +STARTED = json.dumps({"dispatchId": "dispatch-abcdef", "status": "ready"}) +SETTLED = json.dumps({"messages": [{"type": "worker_done", "outcome": "DONE"}]}) + +DEFAULT_SCRIPT = [ + ("run-create", 0, READY, "", False), + ("worker-start", 0, STARTED, "", False), + ("check --wait", 0, SETTLED, "", False), +] + + +class WorkerTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.config = config_module.from_document(minimal_document()) + self.addCleanup(self._tmp.cleanup) + + def launch(self, script=None, **options): + adapter = ScriptedAdapter(self.root / "run", script=script or DEFAULT_SCRIPT, **options) + handle = adapter.create() + record = worker.launch( + run_config=self.config, + adapter=adapter, + handle=handle, + timeout_seconds=self.config.timeout_seconds, + env_overlay={}, + ) + return adapter, handle, record + + def test_the_launch_names_the_agent_model_and_effort(self): + adapter, _, record = self.launch() + start = next(argv for argv in adapter.executed if "worker-start" in argv) + self.assertIn("--agent", start) + self.assertIn("claude", start) + self.assertIn("--model", start) + self.assertIn("pinned-model", start) + self.assertIn("--effort", start) + self.assertIn("high", start) + self.assertEqual(record.model, "pinned-model") + self.assertEqual(record.effort, "high") + + def test_the_prompt_is_a_file_and_the_spec_only_points_at_it(self): + adapter, handle, _ = self.launch() + self.assertEqual(len(adapter.written), 1) + path, content, _mode = adapter.written[0] + self.assertTrue(path.startswith(handle.home_path)) + self.assertIn("dely-cycle-marker", content) + start = next(argv for argv in adapter.executed if "worker-start" in argv) + spec = start[start.index("--spec") + 1] + self.assertIn(path, spec) + self.assertNotIn("dely-cycle-marker", spec) + + def test_the_spec_stays_a_single_plain_line(self): + adapter, _, _ = self.launch() + start = next(argv for argv in adapter.executed if "worker-start" in argv) + spec = start[start.index("--spec") + 1] + self.assertNotIn("\n", spec) + self.assertNotIn("`", spec) + + def test_the_prompt_names_the_marker_and_the_relative_path(self): + adapter, _, _ = self.launch() + _path, content, _mode = adapter.written[0] + self.assertIn("evidence.txt", content) + self.assertIn("dely-cycle-marker", content) + + def test_the_wait_asks_for_the_settling_types_with_a_millisecond_deadline(self): + adapter, _, _ = self.launch() + wait = next(argv for argv in adapter.executed if "--wait" in argv) + self.assertIn("--types", wait) + types = wait[wait.index("--types") + 1] + self.assertEqual(sorted(types.split(",")), ["escalation", "question", "worker_done"]) + self.assertEqual(wait[wait.index("--timeout-ms") + 1], "900000") + + def test_a_settled_worker_is_recorded_with_its_identifiers(self): + _, _, record = self.launch() + self.assertEqual(record.status, status.PhaseStatus.OK) + self.assertEqual(record.run_id, "run-abcdef") + self.assertEqual(record.dispatch_id, "dispatch-abcdef") + self.assertEqual(record.outcome, "DONE") + + def test_a_failed_launch_is_not_reported_as_done(self): + script = [ + ("run-create", 0, READY, "", False), + ("worker-start", 1, "", "the runtime refused the launch", False), + ] + _, _, record = self.launch(script=script) + self.assertEqual(record.status, status.PhaseStatus.FAILED) + self.assertIn("worker-start", record.detail) + + def test_a_failed_run_create_stops_before_the_launch(self): + script = [("run-create", 1, "", "no runtime", False)] + adapter, _, record = self.launch(script=script) + self.assertEqual(record.status, status.PhaseStatus.FAILED) + self.assertFalse(any("worker-start" in argv for argv in adapter.executed)) + + def test_a_wait_that_reaches_the_deadline_is_a_timeout(self): + script = [ + ("run-create", 0, READY, "", False), + ("worker-start", 0, STARTED, "", False), + ("check --wait", None, "", "deadline", True), + ] + _, _, record = self.launch(script=script) + self.assertEqual(record.status, status.PhaseStatus.TIMEOUT) + + def test_an_escalation_settles_the_wait_without_claiming_success(self): + script = [ + ("run-create", 0, READY, "", False), + ("worker-start", 0, STARTED, "", False), + ( + "check --wait", + 0, + json.dumps({"messages": [{"type": "escalation", "outcome": "BLOCKED"}]}), + "", + False, + ), + ] + _, _, record = self.launch(script=script) + self.assertEqual(record.status, status.PhaseStatus.FAILED) + self.assertEqual(record.outcome, "BLOCKED") + + def test_every_command_the_worker_ran_is_recorded(self): + _, _, record = self.launch() + contexts = {command.context for command in record.commands} + self.assertEqual(contexts, {"environment"}) + self.assertGreaterEqual(len(record.commands), 3) + + def test_no_secret_reaches_the_worker_record(self): + secret = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfgh" + script = [ + ("run-create", 0, READY, "", False), + ("worker-start", 0, STARTED, f"key {secret}", False), + ("check --wait", 0, SETTLED, "", False), + ] + _, _, record = self.launch(script=script) + self.assertNotIn(secret, json.dumps(record.to_document())) From 115b5437310ce2b0e5e0432fa16f6bea4fc60be0 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Mon, 14 Sep 2026 23:43:20 +0200 Subject: [PATCH 04/42] Add the shared cycle lifecycle: identity gate before the task, collect and export before stop, and destroy only after both are confirmed. --- .../minimal-cycle/cycle_runner/cleanup.py | 15 +- .../minimal-cycle/cycle_runner/lifecycle.py | 537 ++++++++++++++++++ .../minimal-cycle/cycle_runner/worker.py | 4 + experiments/minimal-cycle/tests/fakes.py | 39 +- .../minimal-cycle/tests/test_cleanup.py | 30 + .../minimal-cycle/tests/test_lifecycle.py | 258 +++++++++ .../minimal-cycle/tests/test_worker.py | 6 + 7 files changed, 877 insertions(+), 12 deletions(-) create mode 100644 experiments/minimal-cycle/cycle_runner/lifecycle.py create mode 100644 experiments/minimal-cycle/tests/test_lifecycle.py diff --git a/experiments/minimal-cycle/cycle_runner/cleanup.py b/experiments/minimal-cycle/cycle_runner/cleanup.py index 751c34b..d0554d5 100644 --- a/experiments/minimal-cycle/cycle_runner/cleanup.py +++ b/experiments/minimal-cycle/cycle_runner/cleanup.py @@ -76,8 +76,9 @@ def perform( adapter: BackendAdapter, handle: EnvironmentHandle, export_record: ExportRecord, + stop_confirmed: bool = True, ) -> CleanupRecord: - """Destroy per-run resources if, and only if, the export was confirmed.""" + """Destroy per-run resources only after a confirmed export and a confirmed stop.""" per_run = list(handle.per_run_resources) shared = list(handle.shared_resources) assert_declarations_disjoint(per_run=per_run, shared=shared) @@ -95,6 +96,18 @@ def perform( verified=False, ) + if not stop_confirmed: + return CleanupRecord( + status=CleanupStatus.RESIDUE, + reason=( + "the stop was not confirmed, so nothing was destroyed; a resource " + "whose state is unknown is left standing for a manual decision" + ), + retained=[str(item) for item in per_run], + shared_preserved=[str(item) for item in shared], + verified=False, + ) + report = adapter.destroy() surviving = [item for item in per_run if adapter.resource_exists(item)] diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py new file mode 100644 index 0000000..2bc9617 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -0,0 +1,537 @@ +"""The shared cycle: one lifecycle, whichever backend is behind it. + +Two orderings carry the contract. Evidence is collected and exported before +anything is stopped or destroyed, so no terminal state loses the run. And the +task runs only after the identity gate has shown that the environment answers +as itself, so a missing Orca is a blocked run rather than a quiet fall back +onto the host's installation. +""" + +from __future__ import annotations + +import json +import tempfile +import time +from contextlib import contextmanager +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Mapping + +from . import auth, cleanup, hostinfo, manifest, probe, proc, project, redact, worker +from .adapters.base import BackendAdapter, EnvironmentHandle +from .config import RunConfig +from .export import Exporter +from .result import CheckRecord, CleanupRecord, PhaseRecord, RunResult +from .status import CleanupStatus, ExportStatus, PhaseStatus, RunStatus, exit_code + +PHASE_ORDER = ( + "prepare", + "create", + "bootstrap", + "identity", + "task", + "check", + "collect", + "export", + "cleanup", + "close", +) + +CHECK_SCRIPT = r""" +set -u +target="$1" +expected="$2" +if [ ! -f "$target" ]; then + printf 'missing: %s\n' "$target" >&2 + exit 3 +fi +actual="$(cat "$target")" +if [ "$actual" = "$expected" ]; then + printf 'marker matched at %s\n' "$target" + exit 0 +fi +printf 'marker mismatch at %s\n' "$target" >&2 +exit 4 +""" + + +def check_argv(project_path: str, relative_path: str, marker: str) -> list[str]: + """Return the independent check as one command the environment can run.""" + return [ + "sh", + "-c", + CHECK_SCRIPT, + "cycle-check", + str(Path(project_path) / relative_path), + marker, + ] + + +class RunLog: + """The runner's own narration, redacted and kept beside the artifacts.""" + + def __init__(self) -> None: + self.lines: list[str] = [] + self.commands: list[dict[str, Any]] = [] + + def say(self, message: str) -> None: + self.lines.append(f"{proc.utc_now()} {redact.text(message)}") + + def record(self, outcome: proc.CommandOutcome) -> proc.CommandOutcome: + self.commands.append(outcome.to_record().to_document()) + self.say( + f"ran {outcome.argv[0]} in the {outcome.context}: " + f"exit={outcome.exit_code} timed_out={outcome.timed_out} " + f"elapsed={outcome.elapsed_seconds}s" + ) + return outcome + + def as_text(self) -> str: + return "\n".join(self.lines) + "\n" + + def as_json_lines(self) -> str: + return "".join(json.dumps(entry, sort_keys=True) + "\n" for entry in self.commands) + + +@dataclass +class CycleOutcome: + """What one call to :func:`run_cycle` produced.""" + + run_result: RunResult + artifact_dir: Path + manifest_path: Path | None = None + exit_code: int = 0 + blockers: list[str] = field(default_factory=list) + + +def _classify( + *, + export_status: ExportStatus, + blocked_reason: str, + timed_out: bool, + error_reason: str, + cleanup_record: CleanupRecord, +) -> tuple[RunStatus, str]: + if export_status is not ExportStatus.CONFIRMED: + return ( + RunStatus.UNKNOWN, + f"the export was not confirmed ({export_status.value}); " + "no claim about this run rests on complete evidence", + ) + if blocked_reason: + return RunStatus.BLOCKED, blocked_reason + if timed_out: + return RunStatus.TIMEOUT, "the task reached the configured deadline" + if error_reason: + return RunStatus.ERROR, error_reason + if cleanup_record.status is not CleanupStatus.DESTROYED: + return RunStatus.CLEANUP_FAILED, cleanup_record.reason + return RunStatus.SETTLED, "" + + +class _Cycle: + """One run, held together so the phases can share their state.""" + + def __init__( + self, + *, + run_config: RunConfig, + adapter: BackendAdapter, + run_id: str, + host_home: Path | None, + environ: Mapping[str, str] | None, + tool_versions: Mapping[str, Any] | None, + ): + self.config = run_config + self.adapter = adapter + self.run_id = run_id + self.host_home = host_home + self.environ = environ + self.tool_versions = dict(tool_versions or {}) + self.artifact_dir = run_config.artifact_root / run_id + self.exporter = Exporter(self.artifact_dir) + self.log = RunLog() + self.result = RunResult( + run_id=run_id, + backend=run_config.backend, + tested_revision=run_config.tested_revision, + dely_revision=run_config.dely_revision, + ) + self.handle: EnvironmentHandle | None = None + self.blocked_reason = "" + self.error_reason = "" + self.timed_out = False + self.task_ran = False + self.stop_confirmed = True + self.env_overlay: dict[str, str] = {} + self.host_before: dict[str, Any] = {} + + # -- plumbing --------------------------------------------------------- + + @contextmanager + def phase(self, name: str): + record = PhaseRecord(name=name, status=PhaseStatus.OK, started_at=proc.utc_now()) + started = time.monotonic() + self.result.phases.append(record) + self.log.say(f"phase {name} started") + try: + yield record + except Exception as error: # a phase failure is data, not a crash + record.status = PhaseStatus.FAILED + record.detail = redact.text(f"{type(error).__name__}: {error}") + self.error_reason = self.error_reason or record.detail + self.log.say(f"phase {name} failed: {record.detail}") + finally: + record.finished_at = proc.utc_now() + record.elapsed_seconds = round(time.monotonic() - started, 3) + self.log.say(f"phase {name} ended as {record.status.value}") + + def skip(self, name: str, detail: str) -> None: + self.result.phases.append( + PhaseRecord(name=name, status=PhaseStatus.SKIPPED, detail=detail) + ) + self.log.say(f"phase {name} skipped: {detail}") + + def _snapshot_host(self) -> dict[str, Any]: + return hostinfo.snapshot( + state_root=self.config.state_root, + artifact_root=self.config.artifact_root, + extra={"backend": self.adapter.name}, + ) + + def execute(self, argv, *, timeout=None, **options) -> proc.CommandOutcome: + return self.log.record( + self.adapter.execute( + argv, + timeout=self.config.timeout_seconds if timeout is None else timeout, + extra_values=tuple(self.env_overlay.values()), + **options, + ) + ) + + # -- phases ----------------------------------------------------------- + + def run(self, staging: Path) -> CycleOutcome: + baseline = staging / "baseline" + fetched = staging / "fetched" + self.result.started_at = proc.utc_now() + started = time.monotonic() + + self._prepare(baseline) + if not self.blocked_reason and not self.error_reason: + self._create() + else: + self.skip("create", self.blocked_reason or self.error_reason) + if self.handle is not None: + self._bootstrap(baseline) + self._identity() + else: + self.skip("bootstrap", "no environment was created") + self.skip("identity", "no environment was created") + if self.handle is not None and not self.blocked_reason and not self.error_reason: + self._task() + else: + self.skip("task", self.blocked_reason or self.error_reason or "no environment") + if self.task_ran and not self.timed_out and not self.error_reason: + self._check() + else: + self.skip("check", self.blocked_reason or self.error_reason or "the task did not settle") + + self._collect(baseline, fetched) + export_record = self._export() + cleanup_record = self._cleanup(export_record) + return self._close(export_record, cleanup_record, started) + + def _prepare(self, baseline: Path) -> None: + with self.phase("prepare"): + self.host_before = self._snapshot_host() + self.exporter.write_json("host-before.json", self.host_before) + commit = project.resolve_revision( + self.config.project.source, self.config.project.revision + ) + self.result.versions["project_revision"] = commit + self.result.versions["configured_revision"] = self.config.project.revision + project.export_revision(self.config.project.source, commit, baseline) + self.log.say(f"exported the project at {commit} for the environment copy") + report = self.adapter.preflight() + self.exporter.write_json("preflight.json", report.to_document()) + if not report.ok: + self.blocked_reason = "preflight refused this backend on this host: " + "; ".join( + f"{finding.name}: {finding.detail}" for finding in report.blockers + ) + + def _create(self) -> None: + with self.phase("create"): + self.handle = self.adapter.create() + self.result.environment_id = self.handle.environment_id + self.result.environment = { + **self.handle.to_document(), + "backend": self.adapter.describe(), + } + self.exporter.write_json("backend-status.json", self.result.environment) + + def _bootstrap(self, baseline: Path) -> None: + assert self.handle is not None + with self.phase("bootstrap") as record: + self.adapter.put_tree(baseline, self.handle.project_path) + for argv in self._provision_steps(): + record.commands.append(self.execute(list(argv)).to_record()) + auth_record, overlay = auth.bootstrap( + run_config=self.config, + adapter=self.adapter, + handle=self.handle, + host_home=self.host_home, + environ=self.environ, + ) + self.result.auth = auth_record + self.env_overlay = dict(overlay) + self.exporter.write_json("auth-receipt.json", auth_record.to_document()) + if auth_record.status is PhaseStatus.BLOCKED: + record.status = PhaseStatus.BLOCKED + self.blocked_reason = auth_record.detail + + def _provision_steps(self): + section = ( + self.config.distrobox if self.config.backend == "distrobox" else self.config.vm + ) + return getattr(section, "provision", ()) if section is not None else () + + def _identity(self) -> None: + assert self.handle is not None + with self.phase("identity") as record: + host_probe = self.log.record( + proc.run( + probe.probe_argv(str(self.config.project.source)), + timeout=min(120, self.config.timeout_seconds), + context="host", + ) + ) + environment_probe = self.execute( + probe.probe_argv(self.handle.project_path), + timeout=min(120, self.config.timeout_seconds), + ) + self.exporter.write_text("identity/host-probe.txt", host_probe.stdout) + self.exporter.write_text( + "identity/environment-probe.txt", environment_probe.stdout + ) + record.commands.extend( + [host_probe.to_record(), environment_probe.to_record()] + ) + identity = probe.verdict( + host=probe.parse(host_probe.stdout), + environment=probe.parse(environment_probe.stdout), + backend=self.config.backend, + expected_home=self.handle.home_path, + expected_project=self.handle.project_path, + ) + orca_status = self.execute( + list(self.config.orca.status_argv), timeout=min(120, self.config.timeout_seconds) + ) + identity.orca_status_exit_code = orca_status.exit_code + self.exporter.write_text( + "identity/orca-status.json", orca_status.stdout or "{}\n" + ) + record.commands.append(orca_status.to_record()) + self.result.identity = identity + allowed, reason = probe.may_continue(identity) + if not allowed: + record.status = PhaseStatus.BLOCKED + record.detail = reason + self.blocked_reason = self.blocked_reason or reason + + def _task(self) -> None: + assert self.handle is not None + with self.phase("task") as record: + self.task_ran = True + worker_record = worker.launch( + run_config=self.config, + adapter=self.adapter, + handle=self.handle, + timeout_seconds=self.config.timeout_seconds, + env_overlay=self.env_overlay, + ) + self.result.worker = worker_record + record.status = worker_record.status + record.detail = worker_record.detail + if worker_record.status is PhaseStatus.TIMEOUT: + self.timed_out = True + elif worker_record.status is not PhaseStatus.OK: + self.error_reason = self.error_reason or worker_record.detail + + def _check(self) -> None: + assert self.handle is not None + with self.phase("check") as record: + argv = check_argv( + self.handle.project_path, + self.config.check.relative_path, + self.config.check.expected_marker, + ) + outcome = self.execute(argv, timeout=min(300, self.config.timeout_seconds)) + self.exporter.write_text("check.stdout", outcome.stdout) + self.exporter.write_text("check.stderr", outcome.stderr) + record.commands.append( + outcome.to_record(stdout_path="check.stdout", stderr_path="check.stderr") + ) + self.result.check = CheckRecord( + status=PhaseStatus.OK if outcome.ok else PhaseStatus.FAILED, + argv=outcome.argv, + exit_code=outcome.exit_code, + started_at=outcome.started_at, + finished_at=outcome.finished_at, + elapsed_seconds=outcome.elapsed_seconds, + stdout_path="check.stdout", + stderr_path="check.stderr", + expected_marker=self.config.check.expected_marker, + detail=( + "the independent check observed the marker" + if outcome.ok + else f"the independent check exited {outcome.exit_code}" + ), + ) + record.status = self.result.check.status + if not outcome.ok: + self.error_reason = self.error_reason or self.result.check.detail + + def _collect(self, baseline: Path, fetched: Path) -> None: + if self.handle is None: + self.skip("collect", "no environment was created") + return + with self.phase("collect") as record: + try: + self.adapter.fetch_tree(self.handle.project_path, fetched) + except Exception as error: + record.status = PhaseStatus.FAILED + record.detail = redact.text( + f"the project tree could not be brought out: {error}" + ) + self.exporter.declare("diff.patch", required=self.task_ran) + self.log.say(record.detail) + return + patch = project.unified_diff(baseline, fetched) + self.exporter.write_text("diff.patch", patch, required=self.task_ran) + artifact = fetched / self.config.task.relative_path + if artifact.is_file(): + self.exporter.write_bytes( + f"task-artifact/{self.config.task.relative_path}", + artifact.read_bytes(), + required=False, + ) + else: + self.exporter.declare( + f"task-artifact/{self.config.task.relative_path}", required=False + ) + record.detail = f"{len(patch.splitlines())} patch line(s) collected" + + def _export(self): + with self.phase("export") as record: + self.exporter.write_json( + "run-before-cleanup.json", self.result.to_document() + ) + self.exporter.write_text("logs/runner.log", self.log.as_text()) + self.exporter.write_text("logs/commands.jsonl", self.log.as_json_lines()) + export_record = self.exporter.confirm() + self.result.export = export_record + record.status = ( + PhaseStatus.OK + if export_record.status is ExportStatus.CONFIRMED + else PhaseStatus.FAILED + ) + record.detail = export_record.detail + return export_record + + def _cleanup(self, export_record) -> CleanupRecord: + if self.handle is None: + record = CleanupRecord( + status=CleanupStatus.DESTROYED, + reason="no environment was created, so no per-run resource existed", + verified=True, + ) + self.result.cleanup = record + self.skip("cleanup", record.reason) + return record + with self.phase("cleanup") as phase_record: + self.result.auth = auth.teardown( + adapter=self.adapter, handle=self.handle, record=self.result.auth + ) + stop_report = self.adapter.stop() + self.stop_confirmed = stop_report.confirmed + self.log.say(f"stop confirmed={stop_report.confirmed}: {stop_report.detail}") + record = cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record, + stop_confirmed=self.stop_confirmed, + ) + self.result.cleanup = record + phase_record.status = ( + PhaseStatus.OK + if record.status is CleanupStatus.DESTROYED + else PhaseStatus.FAILED + ) + phase_record.detail = record.reason + return record + + def _close(self, export_record, cleanup_record, started: float) -> CycleOutcome: + with self.phase("close"): + self.result.finished_at = proc.utc_now() + self.result.elapsed_seconds = round(time.monotonic() - started, 3) + run_status, classification = _classify( + export_status=export_record.status, + blocked_reason=self.blocked_reason, + timed_out=self.timed_out, + error_reason=self.error_reason, + cleanup_record=cleanup_record, + ) + self.result.status = run_status + self.result.failure_classification = classification + host_after = self._snapshot_host() + self.exporter.write_json("host-after.json", host_after, required=False) + self.exporter.write_json( + "cleanup.json", + { + **cleanup_record.to_document(), + "stop_confirmed": self.stop_confirmed, + "host_difference": hostinfo.difference(self.host_before, host_after), + }, + required=False, + ) + document = manifest.build( + run_result=self.result, + run_config=self.config, + host_before=self.host_before, + host_after=host_after, + artifacts=export_record.artifacts, + tool_versions=self.tool_versions, + generated_at=proc.utc_now(), + ) + manifest_path = self.artifact_dir / "manifest.json" + manifest.write(document, manifest_path) + return CycleOutcome( + run_result=self.result, + artifact_dir=self.artifact_dir, + manifest_path=manifest_path, + exit_code=exit_code(self.result.status), + blockers=[self.blocked_reason] if self.blocked_reason else [], + ) + + +def run_cycle( + *, + run_config: RunConfig, + adapter: BackendAdapter, + run_id: str, + host_home: Path | None = None, + environ: Mapping[str, str] | None = None, + tool_versions: Mapping[str, Any] | None = None, +) -> CycleOutcome: + """Run one minimal proof cycle and return its result and artifacts.""" + cycle = _Cycle( + run_config=run_config, + adapter=adapter, + run_id=run_id, + host_home=host_home, + environ=environ, + tool_versions=tool_versions, + ) + with tempfile.TemporaryDirectory(prefix="dely-cycle-") as staging: + return cycle.run(Path(staging)) diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index 7bc1de2..1a892e6 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -177,6 +177,10 @@ def launch( plan.run_create_argv, timeout=timeout_seconds, env=overlay, extra_values=secrets ) record.commands.append(created.to_record()) + if created.timed_out: + record.status = PhaseStatus.TIMEOUT + record.detail = "orca orchestration run-create reached the run deadline" + return record if not created.ok: record.status = PhaseStatus.FAILED record.detail = ( diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index 0c9b863..1c5b6cd 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -39,6 +39,12 @@ } +ORCA_DISPATCH_REPLY = ( + '{"runId": "run-fake", "dispatchId": "dispatch-fake", ' + '"messages": [{"type": "worker_done", "outcome": "DONE"}]}' +) + + def render(snapshot: Mapping[str, str]) -> str: return "".join(f"{key}={value}\n" for key, value in snapshot.items()) @@ -60,6 +66,9 @@ def __init__( fetch_fails: bool = False, stop_confirmed: bool = True, leave_residue: bool = False, + marker: str = "dely-cycle-marker", + task_writes_nothing: bool = False, + host_project: Path | None = None, ): self.root = Path(root) self.home = self.root / "home" @@ -74,6 +83,9 @@ def __init__( self.fetch_fails = fetch_fails self.stop_confirmed = stop_confirmed self.leave_residue = leave_residue + self.marker = marker + self.task_writes_nothing = task_writes_nothing + self.host_project = host_project self.destroyed = False # -- lifecycle -------------------------------------------------------- @@ -107,7 +119,12 @@ def create(self) -> EnvironmentHandle: def _snapshot(self) -> str: if self.identity == "host": - return render(HOST) + # The counterexample: the command really did run on the host, so the + # probe answers with the host's own values rather than a stand-in. + target = str(self.host_project or self.project) + return proc.run( + probe.probe_argv(target), timeout=30, context="host" + ).stdout environment = dict(HOST) environment.update( { @@ -136,21 +153,21 @@ def execute( if probe.PROBE_SCRIPT in joined: self.calls.append("probe") return self._outcome(argv, 0, self._snapshot(), "") - if "worker-start" in joined or "run-create" in joined or "check" in joined and "--wait" in joined: + if "orchestration" in joined: self.calls.append("worker") if self.task_hangs: return self._outcome(argv, None, "", "deadline reached", timed_out=True) - self.project.mkdir(parents=True, exist_ok=True) - (self.project / "evidence.txt").write_text( - "dely-cycle-marker", encoding="utf-8" - ) - return self._outcome(argv, 0, '{"outcome":"done"}', "") + if not self.task_writes_nothing: + self.project.mkdir(parents=True, exist_ok=True) + (self.project / "evidence.txt").write_text(self.marker, encoding="utf-8") + return self._outcome(argv, 0, ORCA_DISPATCH_REPLY, "") + if tuple(argv[:2]) == ("orca", "status"): + self.calls.append("orca-status") + return self._outcome(argv, 0, '{"ready": true}', "") if "cycle-check" in joined: self.calls.append("check") - if self.check_exit_code == 0: - return self._outcome(argv, 0, "marker matched\n", "") - return self._outcome(argv, self.check_exit_code, "", "marker mismatch\n") - self.calls.append(f"execute:{argv[0]}") + else: + self.calls.append(f"execute:{argv[0]}") return proc.run( argv, timeout=timeout, diff --git a/experiments/minimal-cycle/tests/test_cleanup.py b/experiments/minimal-cycle/tests/test_cleanup.py index d9e81e9..722cc12 100644 --- a/experiments/minimal-cycle/tests/test_cleanup.py +++ b/experiments/minimal-cycle/tests/test_cleanup.py @@ -148,3 +148,33 @@ def test_the_filesystem_root_is_never_removable(self): def test_removing_something_absent_is_not_an_error(self): cleanup.safe_remove(self.root / "absent", allowed_roots=[self.root], protected=[]) + + +class StopGateTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) / "run" + self.adapter = FakeAdapter(self.root) + self.handle = self.adapter.create() + self.addCleanup(self._tmp.cleanup) + + def test_an_unconfirmed_stop_prevents_destroy(self): + record = cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record(status.ExportStatus.CONFIRMED), + stop_confirmed=False, + ) + self.assertNotIn("destroy", self.adapter.calls) + self.assertEqual(record.status, status.CleanupStatus.RESIDUE) + self.assertIn("stop", record.reason.lower()) + self.assertTrue(self.adapter.home.exists()) + + def test_a_confirmed_stop_lets_destroy_run(self): + record = cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record(status.ExportStatus.CONFIRMED), + stop_confirmed=True, + ) + self.assertEqual(record.status, status.CleanupStatus.DESTROYED) diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py new file mode 100644 index 0000000..073585b --- /dev/null +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -0,0 +1,258 @@ +"""The shared lifecycle: export before destroy, and never a silent host fallback.""" + +import json +import subprocess +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import config as config_module, ids, lifecycle, status +from tests.fakes import FakeAdapter +from tests.test_config import minimal_document + +RUN_ID = "20260914T221530Z-abc123-0123abcd" +MARKER = "dely-cycle-marker" + + +def make_source_repo(root: Path) -> Path: + repo = root / "under-test" + repo.mkdir(parents=True) + for arguments in ( + ("init", "-q", "-b", "main"), + ("config", "user.email", "cycle@example.invalid"), + ("config", "user.name", "cycle"), + ): + subprocess.run(["git", "-C", str(repo), *arguments], check=True, capture_output=True) + (repo / "readme.md").write_text("project under test\n", encoding="utf-8") + subprocess.run(["git", "-C", str(repo), "add", "-A"], check=True, capture_output=True) + subprocess.run( + ["git", "-C", str(repo), "commit", "-q", "-m", "initial"], + check=True, + capture_output=True, + ) + return repo + + +class CycleTestCase(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.repo = make_source_repo(self.root / "src") + self.artifacts = self.root / "artifacts" + self.state = self.root / "state" + self.host_home = self.root / "hosthome" + (self.host_home / ".claude").mkdir(parents=True) + (self.host_home / ".claude" / ".credentials.json").write_text( + '{"note": "stand-in"}', encoding="utf-8" + ) + self.addCleanup(self._tmp.cleanup) + + def make_config(self, **overrides): + document = minimal_document() + document["artifact_root"] = str(self.artifacts) + document["state_root"] = str(self.state) + document["project"]["source"] = str(self.repo) + document["project"]["revision"] = "main" + document["task"]["marker"] = MARKER + document.update(overrides) + return config_module.from_document(document) + + def run_cycle(self, **adapter_options): + run_config = self.make_config() + adapter = FakeAdapter( + self.state / RUN_ID, host_project=self.repo, **adapter_options + ) + outcome = lifecycle.run_cycle( + run_config=run_config, + adapter=adapter, + run_id=RUN_ID, + host_home=self.host_home, + environ={}, + tool_versions={"runner": "one"}, + ) + return adapter, outcome + + def artifact(self, relative): + return (self.artifacts / RUN_ID / relative) + + +class SettledCycleTest(CycleTestCase): + def test_a_full_cycle_settles(self): + _, outcome = self.run_cycle() + self.assertEqual(outcome.run_result.status, status.RunStatus.SETTLED, outcome.run_result.failure_classification) + self.assertEqual(outcome.exit_code, 0) + + def test_the_phases_run_in_the_documented_order(self): + adapter, outcome = self.run_cycle() + names = [phase.name for phase in outcome.run_result.phases] + self.assertEqual( + names, + [ + "prepare", + "create", + "bootstrap", + "identity", + "task", + "check", + "collect", + "export", + "cleanup", + "close", + ], + ) + + def test_the_export_happens_before_the_environment_is_destroyed(self): + adapter, outcome = self.run_cycle() + self.assertIn("fetch_tree", adapter.calls) + self.assertIn("destroy", adapter.calls) + self.assertLess(adapter.calls.index("fetch_tree"), adapter.calls.index("destroy")) + self.assertLess(adapter.calls.index("stop"), adapter.calls.index("destroy")) + + def test_every_required_artifact_is_on_the_host(self): + _, outcome = self.run_cycle() + for relative in ( + "manifest.json", + "host-before.json", + "host-after.json", + "backend-status.json", + "auth-receipt.json", + "preflight.json", + "identity/host-probe.txt", + "identity/environment-probe.txt", + "identity/orca-status.json", + "check.stdout", + "check.stderr", + "diff.patch", + "logs/runner.log", + "logs/commands.jsonl", + "run-before-cleanup.json", + "export-receipt.json", + "cleanup.json", + ): + self.assertTrue(self.artifact(relative).is_file(), f"missing {relative}") + + def test_the_task_artifact_is_exported(self): + _, outcome = self.run_cycle() + self.assertEqual( + self.artifact("task-artifact/evidence.txt").read_text(encoding="utf-8"), + MARKER, + ) + + def test_the_diff_shows_what_the_task_changed(self): + _, outcome = self.run_cycle() + patch = self.artifact("diff.patch").read_text(encoding="utf-8") + self.assertIn("+++ b/evidence.txt", patch) + self.assertIn(f"+{MARKER}", patch) + + def test_the_manifest_validates_and_names_the_run(self): + _, outcome = self.run_cycle() + document = json.loads(self.artifact("manifest.json").read_text(encoding="utf-8")) + self.assertEqual(document["run_id"], RUN_ID) + self.assertEqual(document["status"], "SETTLED") + self.assertEqual(document["cleanup"]["status"], "DESTROYED") + self.assertEqual(document["export"]["status"], "CONFIRMED") + self.assertTrue(document["artifacts"]) + + def test_the_environment_is_gone_and_the_shared_base_is_not(self): + adapter, _ = self.run_cycle() + self.assertFalse(adapter.home.exists()) + self.assertTrue(adapter.shared_base.is_file()) + + def test_the_check_really_ran_against_the_environment_copy(self): + _, outcome = self.run_cycle() + check = outcome.run_result.check + self.assertEqual(check.exit_code, 0) + self.assertEqual(check.status, status.PhaseStatus.OK) + self.assertIn("marker matched", self.artifact("check.stdout").read_text(encoding="utf-8")) + + +class HostFallbackTest(CycleTestCase): + def test_a_host_identity_blocks_before_the_worker_runs(self): + adapter, outcome = self.run_cycle(identity="host") + self.assertEqual(outcome.run_result.status, status.RunStatus.BLOCKED) + self.assertNotIn("worker", adapter.calls) + self.assertEqual( + outcome.run_result.identity.verdict, "HOST_FALLBACK" + ) + + def test_a_blocked_run_still_exports_and_cleans_up(self): + adapter, outcome = self.run_cycle(identity="host") + self.assertEqual(outcome.run_result.export.status, status.ExportStatus.CONFIRMED) + self.assertEqual(outcome.run_result.cleanup.status, status.CleanupStatus.DESTROYED) + self.assertTrue(self.artifact("manifest.json").is_file()) + + def test_a_missing_orca_blocks_rather_than_using_the_host(self): + adapter, outcome = self.run_cycle(orca_present=False) + self.assertEqual(outcome.run_result.status, status.RunStatus.BLOCKED) + self.assertIn("orca", outcome.run_result.failure_classification.lower()) + self.assertNotIn("worker", adapter.calls) + + def test_a_blocked_run_exits_with_its_own_code(self): + _, outcome = self.run_cycle(identity="host") + self.assertEqual(outcome.exit_code, status.exit_code(status.RunStatus.BLOCKED)) + + +class ExportGateTest(CycleTestCase): + def test_no_destroy_when_export_unconfirmed(self): + adapter, outcome = self.run_cycle(fetch_fails=True) + self.assertNotIn("destroy", adapter.calls) + self.assertEqual(outcome.run_result.cleanup.status, status.CleanupStatus.RESIDUE) + self.assertNotEqual(outcome.run_result.export.status, status.ExportStatus.CONFIRMED) + self.assertEqual(outcome.run_result.status, status.RunStatus.UNKNOWN) + self.assertTrue(adapter.home.exists()) + + def test_an_unconfirmed_stop_prevents_destroy(self): + adapter, outcome = self.run_cycle(stop_confirmed=False) + self.assertNotIn("destroy", adapter.calls) + self.assertEqual(outcome.run_result.cleanup.status, status.CleanupStatus.RESIDUE) + self.assertIn("stop", outcome.run_result.cleanup.reason.lower()) + + def test_surviving_resources_are_reported_as_cleanup_failed(self): + _, outcome = self.run_cycle(leave_residue=True) + self.assertEqual(outcome.run_result.status, status.RunStatus.CLEANUP_FAILED) + self.assertEqual(outcome.run_result.cleanup.status, status.CleanupStatus.RESIDUE) + + +class TimeoutTest(CycleTestCase): + def test_timeout_exports_before_stop_and_settles_timeout(self): + adapter, outcome = self.run_cycle(task_hangs=True) + self.assertEqual(outcome.run_result.status, status.RunStatus.TIMEOUT) + self.assertLess(adapter.calls.index("fetch_tree"), adapter.calls.index("stop")) + self.assertEqual(outcome.run_result.export.status, status.ExportStatus.CONFIRMED) + self.assertTrue(self.artifact("diff.patch").is_file()) + self.assertTrue(self.artifact("manifest.json").is_file()) + + def test_a_timed_out_run_never_reports_a_check_result(self): + _, outcome = self.run_cycle(task_hangs=True) + self.assertEqual(outcome.run_result.check.status, status.PhaseStatus.SKIPPED) + + +class CheckFailureTest(CycleTestCase): + def test_a_missing_marker_is_an_error_not_a_settled_run(self): + _, outcome = self.run_cycle(task_writes_nothing=True) + self.assertEqual(outcome.run_result.status, status.RunStatus.ERROR) + self.assertEqual(outcome.run_result.check.status, status.PhaseStatus.FAILED) + self.assertNotEqual(outcome.run_result.check.exit_code, 0) + + def test_a_wrong_marker_is_an_error(self): + _, outcome = self.run_cycle(marker="not-the-marker") + self.assertEqual(outcome.run_result.status, status.RunStatus.ERROR) + self.assertIn( + "mismatch", self.artifact("check.stderr").read_text(encoding="utf-8") + ) + + +class PreflightTest(CycleTestCase): + def test_a_failed_preflight_blocks_without_creating_anything(self): + adapter, outcome = self.run_cycle(preflight_ok=False) + self.assertEqual(outcome.run_result.status, status.RunStatus.BLOCKED) + self.assertNotIn("create", adapter.calls) + self.assertTrue(self.artifact("preflight.json").is_file()) + self.assertTrue(self.artifact("manifest.json").is_file()) + + +class RunIdentifierTest(CycleTestCase): + def test_the_artifact_directory_is_named_by_the_run(self): + _, outcome = self.run_cycle() + self.assertEqual(outcome.artifact_dir, self.artifacts / RUN_ID) + self.assertTrue(ids.is_run_id(outcome.run_result.run_id)) diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py index e5d26ef..dca401f 100644 --- a/experiments/minimal-cycle/tests/test_worker.py +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -145,3 +145,9 @@ def test_no_secret_reaches_the_worker_record(self): ] _, _, record = self.launch(script=script) self.assertNotIn(secret, json.dumps(record.to_document())) + + def test_a_run_create_that_reaches_the_deadline_is_a_timeout(self): + script = [("run-create", None, "", "deadline", True)] + _, _, record = self.launch(script=script) + self.assertEqual(record.status, status.PhaseStatus.TIMEOUT) + self.assertIn("deadline", record.detail) From 0f0c35bd7e783eec7cc2a517f72ad88a1cb34f86 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Mon, 14 Sep 2026 23:48:24 +0200 Subject: [PATCH 05/42] Add the Distrobox adapter, with a preflight that asks Distrobox what it would mount and blocks on an unacknowledged host home. --- .../cycle_runner/adapters/distrobox.py | 428 ++++++++++++++++++ .../minimal-cycle/cycle_runner/config.py | 24 +- .../tests/test_adapter_distrobox.py | 262 +++++++++++ 3 files changed, 713 insertions(+), 1 deletion(-) create mode 100644 experiments/minimal-cycle/cycle_runner/adapters/distrobox.py create mode 100644 experiments/minimal-cycle/tests/test_adapter_distrobox.py diff --git a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py new file mode 100644 index 0000000..a5ccda7 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py @@ -0,0 +1,428 @@ +"""The Distrobox backend. + +Distrobox is an interface to a container, not a virtual machine and not a +sandbox: its own documentation describes tight host integration and warns +against expecting container-grade isolation. What this adapter buys is a +fresh box with its own home and its own copy of the project, created and +destroyed through Distrobox Assemble rather than through a container manager +this runner would have to become. + +The host home is never mounted, and neither is any directory that holds host +credentials. That is a refusal, not a default. +""" + +from __future__ import annotations + +import shutil +from pathlib import Path +from typing import Callable, Mapping, Sequence + +from .. import cleanup, ids, proc +from ..config import RunConfig +from .base import ( + BackendAdapter, + DestroyReport, + EnvironmentHandle, + Finding, + PreflightReport, + Resource, + StopReport, +) + +CONTAINER_MANAGERS = ("podman", "docker") + +#: Directories under the host home that carry credentials for the tools this +#: run uses. None of them may be mounted into the environment. +CREDENTIAL_DIRECTORIES = ( + ".claude", + ".ssh", + ".aws", + ".docker", + ".config/gh", + ".config/orca", +) + + +class DistroboxContractError(ValueError): + """The requested box would reach into the host in a way the design refuses.""" + + +class DistroboxAdapter(BackendAdapter): + """One disposable Distrobox, driven through Assemble and the command line.""" + + name = "distrobox" + + def __init__( + self, + *, + run_config: RunConfig, + run_id: str, + host_home: Path | None = None, + binary: str = "distrobox", + runner: Callable[..., proc.CommandOutcome] = proc.run, + which: Callable[[str], str | None] = shutil.which, + ): + if run_config.distrobox is None: + raise DistroboxContractError("the configuration carries no distrobox section") + self.config = run_config + self.settings = run_config.distrobox + self.run_id = run_id + self.binary = binary + self.runner = runner + self.which = which + self.host_home = Path(host_home if host_home is not None else Path.home()) + self.container_name = ids.resource_name(self.settings.container_prefix, run_id) + self.run_state = run_config.state_root / run_id + self.home_path = self.run_state / "home" + self.project_path = self.home_path / run_config.project.environment_path + self.manifest_path = self.run_state / "distrobox.ini" + self.mounts = tuple(self._checked_mount(mount) for mount in self.settings.extra_mounts) + + # -- refusals --------------------------------------------------------- + + def _checked_mount(self, mount: str) -> str: + host_side = Path(mount.split(":", 1)[0]) + if not host_side.is_absolute(): + raise DistroboxContractError( + f"an extra mount must name an absolute host path, got {mount!r}" + ) + resolved = host_side.absolute() + home = self.host_home.absolute() + if resolved == Path("/") or home == resolved or home.is_relative_to(resolved): + raise DistroboxContractError( + f"refusing to mount {resolved}: it carries the whole host home" + ) + for relative in CREDENTIAL_DIRECTORIES: + credential = (home / relative).absolute() + if resolved == credential or resolved.is_relative_to(credential): + raise DistroboxContractError( + f"refusing to mount {resolved}: it carries host credentials " + f"({relative})" + ) + return mount + + # -- the declared environment ---------------------------------------- + + def render_manifest(self) -> str: + """Return the Distrobox Assemble manifest for this run.""" + lines = [ + f"[{self.container_name}]", + f"image={self.settings.image}", + f"home={self.home_path}", + f"hostname={self.container_name}", + "entry=false", + "pull=false", + "init=false", + "nvidia=false", + "start_now=true", + ] + lines.extend(f"volume={mount}" for mount in self.mounts) + return "\n".join(lines) + "\n" + + def write_manifest(self) -> Path: + """Write the manifest into the per-run state directory.""" + self.run_state.mkdir(parents=True, exist_ok=True) + self.manifest_path.write_text(self.render_manifest(), encoding="utf-8") + return self.manifest_path + + def plan_handle(self) -> EnvironmentHandle: + """Describe the environment this adapter will create.""" + return EnvironmentHandle( + environment_id=self.container_name, + home_path=str(self.home_path), + project_path=str(self.project_path), + per_run_resources=( + Resource(kind="container", identifier=self.container_name), + Resource(kind="path", identifier=str(self.run_state)), + ), + shared_resources=(Resource(kind="image", identifier=self.settings.image),), + description={ + "image": self.settings.image, + "manifest": str(self.manifest_path), + "container_manager": self._manager() or "unknown", + "desktop_integration": ( + "Distrobox integrates with the host desktop by design; this run " + "limits state with a separate home and project copy, and does not " + "claim container-grade isolation" + ), + "host_home_mount_acknowledged": self.settings.accept_host_home_mount, + }, + ) + + # -- host facts ------------------------------------------------------- + + def _manager(self) -> str | None: + for candidate in CONTAINER_MANAGERS: + if self.which(candidate): + return candidate + return None + + def host_home_mount_finding(self) -> Finding: + """Ask Distrobox what it would really mount, rather than assuming. + + Distrobox mounts the invoking user's home at its own path and offers + no flag to suppress it, so a separate per-run home limits state + without making host credentials unreachable. The rendered command is + the evidence; the configuration has to acknowledge what it shows. + """ + name = "host home reachable from the environment" + self.write_manifest() + rendered = self.runner( + [self.binary, "assemble", "create", "--dry-run", "--file", str(self.manifest_path)], + timeout=300, + context="host", + ) + home = str(self.host_home) + mounted = ( + f'--volume "{home}":"{home}"' in rendered.stdout + or f"--volume {home}:{home}" in rendered.stdout + ) + if not mounted: + return Finding( + name=name, + ok=True, + detail="the rendered container command mounts no host home", + ) + if self.settings.accept_host_home_mount: + return Finding( + name=name, + ok=True, + detail=( + f"distrobox mounts {home} into the box and offers no flag to " + "suppress it; the configuration acknowledged this, so the run " + "records a declared compromise rather than claiming isolation" + ), + ) + return Finding( + name=name, + ok=False, + detail=( + f"distrobox mounts {home} into the box, so host credentials stay " + "reachable from inside it. Set distrobox.accept_host_home_mount to " + "true to record that deliberately, or use the machine backend" + ), + ) + + def preflight(self) -> PreflightReport: + """Report whether this host can create the declared box.""" + findings = [] + distrobox_path = self.which(self.binary) + findings.append( + Finding( + name="distrobox on the path", + ok=bool(distrobox_path), + detail=distrobox_path or f"{self.binary} was not found on PATH", + ) + ) + manager = self._manager() + findings.append( + Finding( + name="container manager on the path", + ok=bool(manager), + detail=manager or "neither podman nor docker was found on PATH", + ) + ) + if manager: + inspected = self.runner( + [manager, "image", "inspect", self.settings.image], + timeout=120, + context="host", + ) + findings.append( + Finding( + name="image present locally", + ok=inspected.ok, + detail=( + f"{self.settings.image} is present" + if inspected.ok + else f"{self.settings.image} is not present locally; pull it " + "deliberately rather than letting a run fetch it" + ), + ) + ) + for label, root in ( + ("state root writable", self.config.state_root), + ("artifact root writable", self.config.artifact_root), + ): + findings.append(Finding(name=label, ok=_writable(root), detail=str(root))) + if distrobox_path: + findings.append(self.host_home_mount_finding()) + return PreflightReport(backend=self.name, findings=tuple(findings)) + + # -- lifecycle -------------------------------------------------------- + + def create(self) -> EnvironmentHandle: + """Create the box from the manifest and prove it exists.""" + self.home_path.mkdir(parents=True, exist_ok=True) + self.write_manifest() + created = self.runner( + [self.binary, "assemble", "create", "--file", str(self.manifest_path)], + timeout=self.config.timeout_seconds, + context="host", + ) + if not created.ok: + raise RuntimeError( + "distrobox assemble create did not create the box: " + + (created.stderr or created.stdout).strip()[:400] + ) + handle = self.plan_handle() + if not self.resource_exists(Resource(kind="container", identifier=self.container_name)): + raise RuntimeError( + f"distrobox list does not report {self.container_name} after assemble" + ) + return handle + + def enter_argv(self, argv: Sequence[str], *, env_names: Sequence[str] = ()) -> list[str]: + """Return the argv that runs a command inside the box.""" + command = [self.binary, "enter", "--name", self.container_name, "-T"] + if env_names: + command.extend( + ["--additional-flags", " ".join(f"--env {name}" for name in env_names)] + ) + command.append("--") + command.extend(str(item) for item in argv) + return command + + def execute( + self, + argv: Sequence[str], + *, + timeout: float, + cwd: str | None = None, + env: Mapping[str, str] | None = None, + extra_values: Sequence[str] = (), + ) -> proc.CommandOutcome: + """Run a command inside the box, forwarding values by name only.""" + overlay = dict(env or {}) + inside = list(argv) + if cwd: + inside = ["sh", "-c", 'cd "$1" || exit 1; shift; exec "$@"', "cycle-cd", cwd, *inside] + return self.runner( + self.enter_argv(inside, env_names=tuple(overlay)), + timeout=timeout, + context="environment", + env=overlay, + extra_values=extra_values, + ) + + def put_tree(self, local_dir: Path, remote_dir: str) -> None: + """The per-run home is a host directory, so this is a host copy.""" + from .. import project + + project.copy_tree(Path(local_dir), Path(remote_dir)) + + def fetch_tree(self, remote_dir: str, local_dir: Path) -> None: + """The per-run home is a host directory, so this is a host copy.""" + from .. import project + + project.copy_tree(Path(remote_dir), Path(local_dir)) + + def write_file(self, remote_path: str, content: str, *, mode: int = 0o600) -> None: + """Write a file into the per-run home with the given permissions.""" + target = Path(remote_path) + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(content, encoding="utf-8") + target.chmod(mode) + + def stop(self) -> StopReport: + """Stop the box and confirm the container manager no longer runs it.""" + stopped = self.runner( + [self.binary, "stop", "--yes", self.container_name], + timeout=300, + context="host", + ) + running = self._running() + return StopReport( + confirmed=not running, + detail=( + f"{self.container_name} is no longer running" + if not running + else f"{self.container_name} is still reported as running" + ), + outcomes=(stopped,), + ) + + def _running(self) -> bool: + manager = self._manager() + if manager is None: + return True + listed = self.runner( + [manager, "ps", "--filter", f"name={self.container_name}", "--format", "{{.Names}}"], + timeout=120, + context="host", + ) + return self.container_name in listed.stdout + + def destroy(self) -> DestroyReport: + """Remove the box and the per-run state, and nothing else.""" + removed: list[str] = [] + outcome = self.runner( + [self.binary, "rm", "--force", self.container_name], + timeout=600, + context="host", + ) + if outcome.ok: + removed.append(f"container:{self.container_name}") + cleanup.safe_remove( + self.run_state, + allowed_roots=[self.config.state_root], + protected=[Path(self.settings.image)] if Path(self.settings.image).is_absolute() else [], + ) + if not self.run_state.exists(): + removed.append(f"path:{self.run_state}") + return DestroyReport( + removed=tuple(removed), + retained=(f"image:{self.settings.image}",), + detail="the box and the per-run state were removed; the image was not", + outcomes=(outcome,), + ) + + def resource_exists(self, resource: Resource) -> bool: + """Report whether a declared resource is still on this host.""" + if resource.kind == "path": + return Path(resource.identifier).exists() + if resource.kind == "container": + listed = self.runner( + [self.binary, "list", "--no-color"], timeout=120, context="host" + ) + return any( + resource.identifier in line.split("|")[1] + for line in listed.stdout.splitlines() + if "|" in line + ) + if resource.kind == "image": + manager = self._manager() + if manager is None: + return False + return self.runner( + [manager, "image", "inspect", resource.identifier], + timeout=120, + context="host", + ).ok + return False + + def describe(self) -> dict: + """Return backend facts for the manifest.""" + version = self.runner([self.binary, "--version"], timeout=60, context="host") + return { + "backend": self.name, + "distrobox_version": version.stdout.strip().splitlines()[-1] + if version.stdout.strip() + else "unknown", + "container_manager": self._manager() or "unknown", + "image": self.settings.image, + "container": self.container_name, + "manifest": str(self.manifest_path), + } + + +def _writable(root: Path) -> bool: + probe = Path(root) + try: + probe.mkdir(parents=True, exist_ok=True) + marker = probe / ".cycle-write-probe" + marker.write_text("", encoding="utf-8") + marker.unlink() + return True + except OSError: + return False diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py index 72b169b..a5d0150 100644 --- a/experiments/minimal-cycle/cycle_runner/config.py +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -66,6 +66,15 @@ def _text(document: Mapping[str, Any], key: str, where: str, default=_UNSET) -> return value +def _flag(document: Mapping[str, Any], key: str, where: str, default: bool) -> bool: + if key not in document: + return default + value = document[key] + if not isinstance(value, bool): + _fail(f"field {where}.{key} must be true or false, got {value!r}") + return value + + def _positive_int(document: Mapping[str, Any], key: str, where: str, default=_UNSET) -> int: if key not in document: if default is _UNSET: @@ -217,6 +226,7 @@ class DistroboxConfig: container_prefix: str extra_mounts: tuple[str, ...] = () provision: tuple[tuple[str, ...], ...] = () + accept_host_home_mount: bool = False def to_document(self) -> dict[str, Any]: return { @@ -224,6 +234,7 @@ def to_document(self) -> dict[str, Any]: "container_prefix": self.container_prefix, "extra_mounts": list(self.extra_mounts), "provision": [list(argv) for argv in self.provision], + "accept_host_home_mount": self.accept_host_home_mount, } @@ -436,7 +447,15 @@ def _provision(document: Mapping[str, Any], where: str) -> tuple[tuple[str, ...] def _distrobox(document: Mapping[str, Any]) -> DistroboxConfig: _reject_unknown( - document, ("image", "container_prefix", "extra_mounts", "provision"), "distrobox" + document, + ( + "image", + "container_prefix", + "extra_mounts", + "provision", + "accept_host_home_mount", + ), + "distrobox", ) mounts = tuple(document.get("extra_mounts") or ()) for mount in mounts: @@ -447,6 +466,9 @@ def _distrobox(document: Mapping[str, Any]) -> DistroboxConfig: container_prefix=_text(document, "container_prefix", "distrobox"), extra_mounts=mounts, provision=_provision(document, "distrobox"), + accept_host_home_mount=_flag( + document, "accept_host_home_mount", "distrobox", False + ), ) diff --git a/experiments/minimal-cycle/tests/test_adapter_distrobox.py b/experiments/minimal-cycle/tests/test_adapter_distrobox.py new file mode 100644 index 0000000..6292b18 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_adapter_distrobox.py @@ -0,0 +1,262 @@ +"""The Distrobox adapter declares a fresh box and never reaches for the host home.""" + +import shutil +import subprocess +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import config as config_module, proc +from cycle_runner.adapters import distrobox +from tests.test_config import minimal_document + +RUN_ID = "20260914T221530Z-abc123-0123abcd" + + +class StubRunner: + """Answers commands from a table so the adapter can be tested without podman.""" + + def __init__(self, table=None): + self.table = list(table or []) + self.seen: list[tuple[str, ...]] = [] + + def __call__(self, argv, *, timeout, context, cwd=None, env=None, extra_values=(), stdin_text=None): + argv = tuple(str(item) for item in argv) + self.seen.append(argv) + joined = " ".join(argv) + for needle, code, out, err in self.table: + if needle in joined: + return self._outcome(argv, code, out, err, context) + return self._outcome(argv, 0, "", "", context) + + @staticmethod + def _outcome(argv, code, out, err, context): + return proc.CommandOutcome( + argv=argv, + exit_code=code, + stdout=out, + stderr=err, + started_at="2026-09-14T22:15:30Z", + finished_at="2026-09-14T22:15:31Z", + elapsed_seconds=0.1, + timed_out=False, + context=context, + ) + + +class AdapterTestCase(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.host_home = self.root / "hosthome" + (self.host_home / ".claude").mkdir(parents=True) + self.addCleanup(self._tmp.cleanup) + + def make( + self, + *, + extra_mounts=(), + runner=None, + present=("distrobox", "podman"), + accept_host_home_mount=False, + host_home=None, + ): + document = minimal_document() + document["state_root"] = str(self.root / "state") + document["artifact_root"] = str(self.root / "artifacts") + document["distrobox"]["extra_mounts"] = list(extra_mounts) + document["distrobox"]["accept_host_home_mount"] = accept_host_home_mount + run_config = config_module.from_document(document) + return distrobox.DistroboxAdapter( + run_config=run_config, + run_id=RUN_ID, + host_home=host_home or self.host_home, + runner=runner or StubRunner(), + which=lambda name: f"/usr/bin/{name}" if name in present else None, + ) + + +class ManifestTest(AdapterTestCase): + def test_the_manifest_names_the_image_the_home_and_a_distinct_hostname(self): + adapter = self.make() + text = adapter.render_manifest() + self.assertIn(f"[{adapter.container_name}]", text) + self.assertIn("image=docker.io/library/ubuntu:24.04", text) + self.assertIn(f"home={adapter.home_path}", text) + self.assertIn(f"hostname={adapter.container_name}", text) + + def test_the_manifest_disables_the_desktop_entry_and_the_pull(self): + text = self.make().render_manifest() + self.assertIn("entry=false", text) + self.assertIn("pull=false", text) + + def test_the_manifest_never_names_the_host_home(self): + text = self.make().render_manifest() + self.assertNotIn(str(self.host_home), text) + + def test_an_ordinary_extra_mount_is_kept(self): + adapter = self.make(extra_mounts=["/opt/shared:/opt/shared:ro"]) + self.assertIn("volume=/opt/shared:/opt/shared:ro", adapter.render_manifest()) + + def test_host_home_mount_is_refused(self): + with self.assertRaises(distrobox.DistroboxContractError) as caught: + self.make(extra_mounts=[f"{self.host_home}:{self.host_home}"]) + self.assertIn("home", str(caught.exception)) + + def test_a_mount_above_the_host_home_is_refused(self): + with self.assertRaises(distrobox.DistroboxContractError): + self.make(extra_mounts=[f"{self.host_home.parent}:/mnt/host"]) + + def test_a_credential_directory_mount_is_refused(self): + with self.assertRaises(distrobox.DistroboxContractError) as caught: + self.make(extra_mounts=[f"{self.host_home / '.claude'}:/root/.claude"]) + self.assertIn(".claude", str(caught.exception)) + + def test_the_manifest_carries_no_unshare_flag_it_was_not_asked_for(self): + text = self.make().render_manifest() + self.assertNotIn("unshare", text) + + +class CommandTest(AdapterTestCase): + def test_the_enter_argv_is_non_interactive_and_names_the_container(self): + adapter = self.make() + argv = adapter.enter_argv(["true"], env_names=()) + self.assertEqual(argv[:2], ["distrobox", "enter"]) + self.assertIn("--name", argv) + self.assertIn(adapter.container_name, argv) + self.assertIn("-T", argv) + self.assertEqual(argv[argv.index("--") + 1 :], ["true"]) + + def test_an_environment_overlay_passes_names_not_values(self): + adapter = self.make() + argv = adapter.enter_argv(["true"], env_names=("CLAUDE_CODE_OAUTH_TOKEN",)) + flags = argv[argv.index("--additional-flags") + 1] + self.assertEqual(flags, "--env CLAUDE_CODE_OAUTH_TOKEN") + self.assertNotIn("=", flags) + + def test_executing_forwards_the_values_through_the_process_environment(self): + runner = StubRunner() + adapter = self.make(runner=runner) + adapter.execute(["true"], timeout=5, env={"CLAUDE_CODE_OAUTH_TOKEN": "value"}) + argv = runner.seen[-1] + self.assertIn("--env CLAUDE_CODE_OAUTH_TOKEN", argv) + self.assertNotIn("value", " ".join(argv)) + + def test_stop_asks_without_a_prompt_and_verifies(self): + runner = StubRunner([("ps --filter", 0, "", "")]) + adapter = self.make(runner=runner) + report = adapter.stop() + self.assertTrue(any("stop" in " ".join(argv) for argv in runner.seen)) + self.assertTrue(any("--yes" in argv for argv in runner.seen)) + self.assertTrue(report.confirmed) + + def test_a_container_still_running_is_not_a_confirmed_stop(self): + adapter = self.make() + runner = StubRunner([("ps --filter", 0, adapter.container_name + "\n", "")]) + adapter = self.make(runner=runner) + self.assertFalse(adapter.stop().confirmed) + + +class ResourceTest(AdapterTestCase): + def test_the_declared_resources_separate_the_run_from_the_image(self): + adapter = self.make() + handle = adapter.plan_handle() + per_run = {str(item) for item in handle.per_run_resources} + shared = {str(item) for item in handle.shared_resources} + self.assertIn(f"container:{adapter.container_name}", per_run) + self.assertIn(f"path:{adapter.run_state}", per_run) + self.assertEqual(shared, {"image:docker.io/library/ubuntu:24.04"}) + + def test_the_project_copy_lives_under_the_per_run_home(self): + adapter = self.make() + handle = adapter.plan_handle() + self.assertTrue(Path(handle.project_path).is_relative_to(Path(handle.home_path))) + + def test_the_declarations_do_not_overlap(self): + from cycle_runner import cleanup + + handle = self.make().plan_handle() + cleanup.assert_declarations_disjoint( + per_run=handle.per_run_resources, shared=handle.shared_resources + ) + + +class PreflightTest(AdapterTestCase): + def test_a_missing_distrobox_blocks(self): + adapter = self.make(present=("podman",)) + report = adapter.preflight() + self.assertFalse(report.ok) + self.assertTrue(any("distrobox" in finding.name for finding in report.blockers)) + + def test_a_missing_container_manager_blocks(self): + adapter = self.make(present=("distrobox",)) + report = adapter.preflight() + self.assertFalse(report.ok) + self.assertTrue( + any("container manager" in finding.name for finding in report.blockers) + ) + + def test_an_absent_image_blocks_rather_than_pulling_silently(self): + runner = StubRunner([("image inspect", 1, "", "no such image")]) + adapter = self.make(runner=runner) + report = adapter.preflight() + self.assertFalse(report.ok) + self.assertTrue(any("image" in finding.name for finding in report.blockers)) + + def test_a_ready_host_passes(self): + adapter = self.make() + self.assertTrue(adapter.preflight().ok) + + +@unittest.skipUnless(shutil.which("distrobox"), "distrobox is not on this host") +class DryRunTest(AdapterTestCase): + """Distrobox itself is the instrument: the rendered command is the evidence.""" + + def make(self, **options): + options.setdefault("host_home", Path.home()) + options.setdefault("runner", proc.run) + return super().make(**options) + + def test_the_rendered_manifest_is_accepted_by_distrobox(self): + adapter = self.make() + adapter.write_manifest() + completed = subprocess.run( + ["distrobox", "assemble", "create", "--dry-run", "--file", str(adapter.manifest_path)], + capture_output=True, + text=True, + ) + self.assertEqual(completed.returncode, 0, completed.stderr) + self.assertIn(adapter.container_name, completed.stdout) + self.assertIn(f'HOME={adapter.home_path}', completed.stdout) + self.assertIn(f"{adapter.home_path}:{adapter.home_path}", completed.stdout) + + def test_distrobox_really_does_mount_the_host_home(self): + """The property the next test blocks on is observed, not assumed.""" + adapter = self.make() + adapter.write_manifest() + completed = subprocess.run( + ["distrobox", "assemble", "create", "--dry-run", "--file", str(adapter.manifest_path)], + capture_output=True, + text=True, + ) + real_home = str(Path.home()) + self.assertIn(f'--volume "{real_home}":"{real_home}"', completed.stdout) + + def test_the_unacknowledged_host_home_mount_blocks_this_host(self): + adapter = self.make() + report = adapter.preflight() + blocker_names = [finding.name for finding in report.blockers] + self.assertIn("host home reachable from the environment", blocker_names) + self.assertFalse(report.ok) + + def test_acknowledging_the_mount_records_it_as_a_declared_compromise(self): + adapter = self.make(accept_host_home_mount=True) + report = adapter.preflight() + finding = next( + item + for item in report.findings + if item.name == "host home reachable from the environment" + ) + self.assertTrue(finding.ok) + self.assertIn("acknowledged", finding.detail) + self.assertTrue(report.ok, [item.detail for item in report.blockers]) From e631b5cb38209a41cb5bd3ba77d8a284a8380ab8 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Mon, 14 Sep 2026 23:52:19 +0200 Subject: [PATCH 06/42] Add the machine adapter: a per-run domain declared with Pulumi over an overlay whose base is only a backing file, a seed carrying one public key, and a preflight that refuses an unverified provider schema. --- .../minimal-cycle/cycle_runner/adapters/vm.py | 625 ++++++++++++++++++ .../minimal-cycle/cycle_runner/config.py | 14 + .../tests/test_adapter_distrobox.py | 13 +- .../minimal-cycle/tests/test_adapter_vm.py | 240 +++++++ 4 files changed, 891 insertions(+), 1 deletion(-) create mode 100644 experiments/minimal-cycle/cycle_runner/adapters/vm.py create mode 100644 experiments/minimal-cycle/tests/test_adapter_vm.py diff --git a/experiments/minimal-cycle/cycle_runner/adapters/vm.py b/experiments/minimal-cycle/cycle_runner/adapters/vm.py new file mode 100644 index 0000000..3a431db --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/adapters/vm.py @@ -0,0 +1,625 @@ +"""The virtual-machine backend: Pulumi Python over the libvirt provider. + +The isolation this backend buys is a separate kernel, so the identity gate can +ask for a distinct boot identity rather than a container marker. What it costs +is a preserved base image, a rendered program, a seed and a transport, and +every one of those is a place a credential could end up. None of them carries +one: the seed authorises one per-run public key, the private half never leaves +the per-run state directory, and the base image is opened only as a backing +file. + +The provider's resource schema is not assumed. This runner renders a program +against a pinned provider version and refuses to run until the configuration +records that the schema was actually checked against that version. +""" + +from __future__ import annotations + +import hashlib +import json +import shutil +from pathlib import Path +from typing import Callable, Mapping, Sequence + +from .. import cleanup, ids, proc +from ..config import RunConfig +from .base import ( + BackendAdapter, + DestroyReport, + EnvironmentHandle, + Finding, + PreflightReport, + Resource, + StopReport, +) + +REQUIRED_TOOLS = ("pulumi", "qemu-img", "virsh", "ssh", "scp") + +PROGRAM_TEMPLATE = '''"""Per-run domain for one dely minimal cycle. + +Rendered by cycle_runner.adapters.vm. The resource fields below belong to the +pinned provider version named in Pulumi.yaml; the runner refuses to apply this +program until the configuration records that the schema was checked against +that version. +""" + +import pulumi +import pulumi_libvirt as libvirt + +DOMAIN_NAME = {domain_name!r} +OVERLAY_PATH = {overlay_path!r} +USER_DATA = {user_data!r} +META_DATA = {meta_data!r} +GRAPHICS = {graphics!r} +MEMORY_MB = {memory_mb!r} +VCPUS = {vcpus!r} +NETWORK = {network!r} + +seed = libvirt.CloudInItDisk( + "seed", + name=DOMAIN_NAME + "-seed.iso", + user_data=USER_DATA, + meta_data=META_DATA, +) + +domain = libvirt.Domain( + "domain", + name=DOMAIN_NAME, + memory=MEMORY_MB, + vcpu=VCPUS, + cloudinit=seed.id, + disks=[libvirt.DomainDiskArgs(file=OVERLAY_PATH)], + network_interfaces=[ + libvirt.DomainNetworkInterfaceArgs(network_name=NETWORK, wait_for_lease=True) + ], + graphics=libvirt.DomainGraphicsArgs(type=GRAPHICS, listen_type="address"), +) + +pulumi.export("domain_name", domain.name) +pulumi.export("address", domain.network_interfaces[0].addresses[0]) +''' + +USER_DATA_TEMPLATE = """#cloud-config +hostname: {hostname} +preserve_hostname: false +users: + - name: {user} + shell: /bin/bash + sudo: ALL=(ALL) NOPASSWD:ALL + lock_passwd: true + ssh_authorized_keys: + - {public_key} +write_files: + - path: /etc/dely-cycle-run + permissions: '0444' + content: | + run_id={run_id} + domain={hostname} +runcmd: + - [ install, -d, -o, {user}, -g, {user}, -m, '0750', {home} ] +""" + +META_DATA_TEMPLATE = """instance-id: {run_id} +local-hostname: {hostname} +""" + + +class VmContractError(RuntimeError): + """The machine backend was asked for something it refuses to do.""" + + +class VmAdapter(BackendAdapter): + """One disposable domain, declared with Pulumi and destroyed with it.""" + + name = "vm" + + def __init__( + self, + *, + run_config: RunConfig, + run_id: str, + runner: Callable[..., proc.CommandOutcome] = proc.run, + which: Callable[[str], str | None] = shutil.which, + ): + if run_config.vm is None: + raise VmContractError("the configuration carries no vm section") + self.config = run_config + self.settings = run_config.vm + self.run_id = run_id + self.runner = runner + self.which = which + self.domain_name = ids.resource_name(self.settings.stack_prefix, run_id) + self.stack_name = self.domain_name + self.run_state = run_config.state_root / run_id + self.stack_dir = self.run_state / "stack" + self.overlay_path = self.run_state / "overlay.qcow2" + self.private_key_path = self.run_state / "id_cycle" + self.public_key_path = self.run_state / "id_cycle.pub" + self.home_path = f"/home/{self.settings.guest_user}" + self.project_path = f"{self.home_path}/{run_config.project.environment_path}" + self.address: str | None = None + + # -- per-run identity ------------------------------------------------- + + def prepare_identity(self) -> None: + """Generate the per-run transport key pair inside the per-run state.""" + self.run_state.mkdir(parents=True, exist_ok=True) + if self.private_key_path.exists(): + return + generated = self.runner( + [ + "ssh-keygen", + "-t", + "ed25519", + "-N", + "", + "-C", + f"dely-cycle-{self.run_id}", + "-f", + str(self.private_key_path), + ], + timeout=120, + context="host", + ) + if not self.private_key_path.exists(): + raise VmContractError( + "the per-run transport key could not be generated: " + + (generated.stderr or generated.stdout).strip()[:300] + ) + self.private_key_path.chmod(0o600) + + # -- rendered declarations ------------------------------------------- + + def render_user_data(self) -> str: + """Render the cloud-init user data; it carries no credential.""" + self.prepare_identity() + return USER_DATA_TEMPLATE.format( + hostname=self.domain_name, + user=self.settings.guest_user, + public_key=self.public_key_path.read_text(encoding="utf-8").strip(), + run_id=self.run_id, + home=self.home_path, + ) + + def render_meta_data(self) -> str: + """Render the cloud-init metadata naming this run and this domain.""" + return META_DATA_TEMPLATE.format( + run_id=self.run_id.lower(), hostname=self.domain_name + ) + + def render_program(self) -> str: + """Render the Pulumi program for this run's domain.""" + return PROGRAM_TEMPLATE.format( + domain_name=self.domain_name, + overlay_path=str(self.overlay_path), + user_data=self.render_user_data(), + meta_data=self.render_meta_data(), + graphics=self.settings.graphics, + memory_mb=self.settings.memory_mb, + vcpus=self.settings.vcpus, + network=self.settings.network, + ) + + def render_project(self) -> str: + """Render Pulumi.yaml, pinning the provider and its version.""" + return ( + f"name: {self.settings.stack_prefix}\n" + "runtime: python\n" + f"description: one dely minimal cycle domain for run {self.run_id}\n" + "packages:\n" + f" libvirt: {self.settings.provider}@{self.settings.provider_version}\n" + ) + + def render_stack_settings(self) -> str: + """Render the stack settings; nothing here is a secret.""" + return ( + "config:\n" + f" {self.settings.stack_prefix}:runId: {self.run_id}\n" + f" {self.settings.stack_prefix}:domain: {self.domain_name}\n" + f" {self.settings.stack_prefix}:connectUri: {self.settings.connect_uri}\n" + ) + + def write_declarations(self) -> None: + """Write the program, the project file and the stack settings.""" + self.stack_dir.mkdir(parents=True, exist_ok=True) + (self.stack_dir / "__main__.py").write_text(self.render_program(), encoding="utf-8") + (self.stack_dir / "Pulumi.yaml").write_text(self.render_project(), encoding="utf-8") + (self.stack_dir / f"Pulumi.{self.stack_name}.yaml").write_text( + self.render_stack_settings(), encoding="utf-8" + ) + + def overlay_argv(self) -> list[str]: + """Return the command that creates the overlay over the preserved base.""" + return [ + "qemu-img", + "create", + "-f", + "qcow2", + "-b", + str(self.settings.base_image), + "-F", + "qcow2", + str(self.overlay_path), + self.settings.overlay_size, + ] + + def plan_handle(self) -> EnvironmentHandle: + """Describe the environment this adapter will create.""" + return EnvironmentHandle( + environment_id=self.domain_name, + home_path=self.home_path, + project_path=self.project_path, + per_run_resources=( + Resource(kind="domain", identifier=self.domain_name), + Resource(kind="path", identifier=str(self.run_state)), + ), + shared_resources=( + Resource(kind="image", identifier=str(self.settings.base_image)), + ), + description={ + "domain": self.domain_name, + "overlay": str(self.overlay_path), + "stack": str(self.stack_dir), + "connect_uri": self.settings.connect_uri, + "provider": f"{self.settings.provider}@{self.settings.provider_version}", + "base_image": str(self.settings.base_image), + }, + ) + + # -- host facts ------------------------------------------------------- + + def _base_digest(self) -> str | None: + base = Path(self.settings.base_image) + if not base.is_file(): + return None + digest = hashlib.sha256() + with base.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + def preflight(self) -> PreflightReport: + """Report whether this host can declare and run the domain.""" + findings = [] + for tool in REQUIRED_TOOLS: + located = self.which(tool) + findings.append( + Finding( + name=f"{tool} on the path", + ok=bool(located), + detail=located or f"{tool} was not found on PATH", + ) + ) + findings.append( + Finding( + name="provider schema verified", + ok=self.settings.provider_schema_verified, + detail=( + f"{self.settings.provider}@{self.settings.provider_version} was " + "recorded as checked against the rendered program" + if self.settings.provider_schema_verified + else ( + "the rendered program's resource fields have not been checked " + f"against {self.settings.provider}@" + f"{self.settings.provider_version}; set " + "vm.provider_schema_verified once they have been" + ) + ), + ) + ) + base_present = Path(self.settings.base_image).is_file() + findings.append( + Finding( + name="base image present", + ok=base_present, + detail=str(self.settings.base_image) + if base_present + else f"{self.settings.base_image} is not a file on this host", + ) + ) + if base_present: + observed = self._base_digest() + findings.append( + Finding( + name="base image digest matches the pin", + ok=observed == self.settings.base_image_sha256, + detail=( + "the preserved base matches the pinned digest" + if observed == self.settings.base_image_sha256 + else f"the base image digest is {observed}, not the pinned value" + ), + ) + ) + accelerator = Path("/dev/kvm") + findings.append( + Finding( + name="hardware acceleration device", + ok=accelerator.exists(), + detail=str(accelerator) + if accelerator.exists() + else "/dev/kvm is absent; this host cannot run the domain natively", + ) + ) + if self.which("virsh"): + listed = self.runner( + ["virsh", "--connect", self.settings.connect_uri, "list", "--all"], + timeout=120, + context="host", + ) + findings.append( + Finding( + name="libvirt connection answers", + ok=listed.ok, + detail=self.settings.connect_uri + if listed.ok + else (listed.stderr or listed.stdout).strip()[:200], + ) + ) + return PreflightReport(backend=self.name, findings=tuple(findings)) + + # -- lifecycle -------------------------------------------------------- + + def _pulumi(self, arguments: Sequence[str], *, timeout: float) -> proc.CommandOutcome: + return self.runner( + [self.settings.pulumi_binary, *arguments], + timeout=timeout, + context="host", + cwd=str(self.stack_dir), + env={ + "PULUMI_BACKEND_URL": f"file://{self.run_state}", + "PULUMI_SKIP_UPDATE_CHECK": "true", + "LIBVIRT_DEFAULT_URI": self.settings.connect_uri, + }, + ) + + def create(self) -> EnvironmentHandle: + """Create the overlay, the seed and the domain, then find its address.""" + self.prepare_identity() + overlay = self.runner(self.overlay_argv(), timeout=600, context="host") + if not overlay.ok: + raise VmContractError( + "the overlay could not be created over the preserved base: " + + (overlay.stderr or overlay.stdout).strip()[:300] + ) + self.write_declarations() + self._pulumi(["stack", "init", self.stack_name, "--non-interactive"], timeout=600) + applied = self._pulumi( + ["up", "--yes", "--non-interactive", "--stack", self.stack_name], + timeout=self.config.timeout_seconds, + ) + if not applied.ok: + raise VmContractError( + "pulumi up did not bring the domain up: " + + (applied.stderr or applied.stdout).strip()[:400] + ) + outputs = self._pulumi( + ["stack", "output", "--json", "--stack", self.stack_name], timeout=300 + ) + try: + self.address = json.loads(outputs.stdout or "{}").get("address") + except json.JSONDecodeError: + self.address = None + if not self.address: + raise VmContractError( + "the stack reported no address for the domain, so nothing can be run in it" + ) + handle = self.plan_handle() + return EnvironmentHandle( + environment_id=handle.environment_id, + home_path=handle.home_path, + project_path=handle.project_path, + per_run_resources=handle.per_run_resources, + shared_resources=handle.shared_resources, + description={**handle.description, "address": self.address}, + ) + + def ssh_argv(self, argv: Sequence[str]) -> list[str]: + """Return the argv that runs a command inside the domain.""" + if not self.address: + raise VmContractError( + "the domain has no known address yet; nothing may be run in it" + ) + return [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + f"UserKnownHostsFile={self.run_state / 'known_hosts'}", + "-p", + str(self.settings.ssh_port), + "-i", + str(self.private_key_path), + f"{self.settings.guest_user}@{self.address}", + "--", + *[str(item) for item in argv], + ] + + def execute( + self, + argv: Sequence[str], + *, + timeout: float, + cwd: str | None = None, + env: Mapping[str, str] | None = None, + extra_values: Sequence[str] = (), + ) -> proc.CommandOutcome: + """Run a command inside the domain over the per-run transport.""" + inside = list(argv) + if env: + inside = [ + "env", + *[f"{name}={value}" for name, value in env.items()], + *inside, + ] + if cwd: + inside = ["sh", "-c", 'cd "$1" || exit 1; shift; exec "$@"', "cycle-cd", cwd, *inside] + return self.runner( + self.ssh_argv(inside), + timeout=timeout, + context="environment", + extra_values=tuple(extra_values) + tuple((env or {}).values()), + ) + + def _scp_argv(self, source: str, destination: str) -> list[str]: + return [ + "scp", + "-r", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + f"UserKnownHostsFile={self.run_state / 'known_hosts'}", + "-P", + str(self.settings.ssh_port), + "-i", + str(self.private_key_path), + source, + destination, + ] + + def put_tree(self, local_dir: Path, remote_dir: str) -> None: + """Copy a host directory into the domain.""" + self.execute(["mkdir", "-p", remote_dir], timeout=300) + outcome = self.runner( + self._scp_argv( + f"{Path(local_dir)}/.", + f"{self.settings.guest_user}@{self.address}:{remote_dir}/", + ), + timeout=1800, + context="host", + ) + if not outcome.ok: + raise VmContractError( + "the project copy could not be placed in the domain: " + + (outcome.stderr or outcome.stdout).strip()[:300] + ) + + def fetch_tree(self, remote_dir: str, local_dir: Path) -> None: + """Bring a directory out of the domain onto the host.""" + Path(local_dir).mkdir(parents=True, exist_ok=True) + outcome = self.runner( + self._scp_argv( + f"{self.settings.guest_user}@{self.address}:{remote_dir}/.", + f"{Path(local_dir)}/", + ), + timeout=1800, + context="host", + ) + if not outcome.ok: + raise VmContractError( + "the project tree could not be brought out of the domain: " + + (outcome.stderr or outcome.stdout).strip()[:300] + ) + + def write_file(self, remote_path: str, content: str, *, mode: int = 0o600) -> None: + """Write a file inside the domain with the given permissions.""" + self.execute(["mkdir", "-p", str(Path(remote_path).parent)], timeout=300) + written = self.runner( + self.ssh_argv( + [ + "sh", + "-c", + 'umask 077; cat > "$1"; chmod "$2" "$1"', + "cycle-write", + remote_path, + format(mode, "04o"), + ] + ), + timeout=300, + context="environment", + stdin_text=content, + extra_values=(content,), + ) + if not written.ok: + raise VmContractError(f"could not write {remote_path} inside the domain") + + def stop(self) -> StopReport: + """Shut the domain down and confirm libvirt no longer runs it.""" + outcome = self.runner( + ["virsh", "--connect", self.settings.connect_uri, "shutdown", self.domain_name], + timeout=300, + context="host", + ) + running = self._domain_running() + return StopReport( + confirmed=not running, + detail=( + f"{self.domain_name} is no longer running" + if not running + else f"{self.domain_name} is still reported as running" + ), + outcomes=(outcome,), + ) + + def _domain_running(self) -> bool: + listed = self.runner( + ["virsh", "--connect", self.settings.connect_uri, "domstate", self.domain_name], + timeout=120, + context="host", + ) + return "running" in listed.stdout.lower() + + def destroy(self) -> DestroyReport: + """Destroy the stack and the per-run state; never the preserved base.""" + removed: list[str] = [] + outcomes = [] + if self.stack_dir.exists(): + destroyed = self._pulumi( + ["destroy", "--yes", "--non-interactive", "--stack", self.stack_name], + timeout=self.config.timeout_seconds, + ) + outcomes.append(destroyed) + if destroyed.ok: + removed.append(f"domain:{self.domain_name}") + outcomes.append( + self._pulumi( + ["stack", "rm", "--yes", "--non-interactive", self.stack_name], + timeout=600, + ) + ) + cleanup.safe_remove( + self.run_state, + allowed_roots=[self.config.state_root], + protected=[Path(self.settings.base_image)], + ) + if not self.run_state.exists(): + removed.append(f"path:{self.run_state}") + return DestroyReport( + removed=tuple(removed), + retained=(f"image:{self.settings.base_image}",), + detail="the domain, the overlay, the seed and the stack were removed; the base was not", + outcomes=tuple(outcomes), + ) + + def resource_exists(self, resource: Resource) -> bool: + """Report whether a declared resource is still on this host.""" + if resource.kind == "path": + return Path(resource.identifier).exists() + if resource.kind == "image": + return Path(resource.identifier).is_file() + if resource.kind == "domain": + listed = self.runner( + ["virsh", "--connect", self.settings.connect_uri, "list", "--all", "--name"], + timeout=120, + context="host", + ) + return resource.identifier in listed.stdout.split() + return False + + def describe(self) -> dict: + """Return backend facts for the manifest.""" + return { + "backend": self.name, + "domain": self.domain_name, + "stack": self.stack_name, + "provider": f"{self.settings.provider}@{self.settings.provider_version}", + "connect_uri": self.settings.connect_uri, + "base_image": str(self.settings.base_image), + "base_image_sha256": self.settings.base_image_sha256, + "overlay": str(self.overlay_path), + "graphics": self.settings.graphics, + "provider_schema_verified": self.settings.provider_schema_verified, + } diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py index a5d0150..7a302a6 100644 --- a/experiments/minimal-cycle/cycle_runner/config.py +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -253,6 +253,9 @@ class VmConfig: vcpus: int = 2 graphics: str = "spice" transport: str = "ssh" + network: str = "default" + ssh_port: int = 22 + provider_schema_verified: bool = False provision: tuple[tuple[str, ...], ...] = () def to_document(self) -> dict[str, Any]: @@ -270,6 +273,9 @@ def to_document(self) -> dict[str, Any]: "vcpus": self.vcpus, "graphics": self.graphics, "transport": self.transport, + "network": self.network, + "ssh_port": self.ssh_port, + "provider_schema_verified": self.provider_schema_verified, "provision": [list(argv) for argv in self.provision], } @@ -489,6 +495,9 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: "vcpus", "graphics", "transport", + "network", + "ssh_port", + "provider_schema_verified", "provision", ), "vm", @@ -513,6 +522,11 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: vcpus=_positive_int(document, "vcpus", "vm", 2), graphics=_text(document, "graphics", "vm", "spice"), transport=_text(document, "transport", "vm", "ssh"), + network=_text(document, "network", "vm", "default"), + ssh_port=_positive_int(document, "ssh_port", "vm", 22), + provider_schema_verified=_flag( + document, "provider_schema_verified", "vm", False + ), provision=_provision(document, "vm"), ) diff --git a/experiments/minimal-cycle/tests/test_adapter_distrobox.py b/experiments/minimal-cycle/tests/test_adapter_distrobox.py index 6292b18..64599b7 100644 --- a/experiments/minimal-cycle/tests/test_adapter_distrobox.py +++ b/experiments/minimal-cycle/tests/test_adapter_distrobox.py @@ -16,8 +16,9 @@ class StubRunner: """Answers commands from a table so the adapter can be tested without podman.""" - def __init__(self, table=None): + def __init__(self, table=None, *, passthrough=False): self.table = list(table or []) + self.passthrough = passthrough self.seen: list[tuple[str, ...]] = [] def __call__(self, argv, *, timeout, context, cwd=None, env=None, extra_values=(), stdin_text=None): @@ -27,6 +28,16 @@ def __call__(self, argv, *, timeout, context, cwd=None, env=None, extra_values=( for needle, code, out, err in self.table: if needle in joined: return self._outcome(argv, code, out, err, context) + if self.passthrough: + return proc.run( + argv, + timeout=timeout, + context=context, + cwd=cwd, + env=env, + extra_values=extra_values, + stdin_text=stdin_text, + ) return self._outcome(argv, 0, "", "", context) @staticmethod diff --git a/experiments/minimal-cycle/tests/test_adapter_vm.py b/experiments/minimal-cycle/tests/test_adapter_vm.py new file mode 100644 index 0000000..cb98d4a --- /dev/null +++ b/experiments/minimal-cycle/tests/test_adapter_vm.py @@ -0,0 +1,240 @@ +"""The machine adapter renders a per-run domain over a base it never writes.""" + +import hashlib +import shutil +import subprocess +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import cleanup, config as config_module, proc +from cycle_runner.adapters import vm +from tests.test_adapter_distrobox import StubRunner +from tests.test_config import minimal_document + +RUN_ID = "20260914T221530Z-abc123-0123abcd" +TOKEN = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfgh" + + +class VmTestCase(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.shared = self.root / "shared" + self.shared.mkdir() + self.base = self.shared / "tool-image.qcow2" + self.base.write_bytes(b"pretend this is a prepared tool image") + self.base_digest = hashlib.sha256(self.base.read_bytes()).hexdigest() + self.addCleanup(self._tmp.cleanup) + + def make(self, *, runner=None, present=("pulumi", "qemu-img", "virsh", "ssh", "scp"), **overrides): + document = minimal_document(backend="vm") + document.pop("distrobox") + document["state_root"] = str(self.root / "state") + document["artifact_root"] = str(self.root / "artifacts") + section = { + "provider": "pulumi-libvirt", + "provider_version": "0.5.3", + "stack_prefix": "dely-cycle", + "base_image": str(self.base), + "base_image_sha256": self.base_digest, + "guest_user": "cycle", + "provider_schema_verified": True, + } + section.update(overrides) + document["vm"] = section + run_config = config_module.from_document(document) + return vm.VmAdapter( + run_config=run_config, + run_id=RUN_ID, + runner=runner or StubRunner(passthrough=True), + which=lambda name: f"/usr/bin/{name}" if name in present else None, + ) + + +class SeedTest(VmTestCase): + def test_the_seed_carries_no_credential(self): + adapter = self.make() + adapter.prepare_identity() + user_data = adapter.render_user_data() + self.assertNotIn(TOKEN, user_data) + self.assertNotIn("PRIVATE KEY", user_data) + self.assertNotIn(adapter.private_key_path.read_text(encoding="utf-8"), user_data) + self.assertNotIn("password", user_data.lower()) + + def test_the_seed_authorises_only_the_per_run_public_key(self): + adapter = self.make() + adapter.prepare_identity() + public_key = adapter.public_key_path.read_text(encoding="utf-8").strip() + user_data = adapter.render_user_data() + self.assertIn(public_key, user_data) + self.assertEqual(user_data.count("ssh-"), user_data.count(public_key.split()[0])) + + def test_the_seed_names_the_run_and_a_distinct_host_name(self): + adapter = self.make() + meta = adapter.render_meta_data() + self.assertIn(RUN_ID.lower(), meta) + self.assertIn(adapter.domain_name, meta) + + def test_the_private_key_never_leaves_the_per_run_state(self): + adapter = self.make() + adapter.prepare_identity() + self.assertTrue( + adapter.private_key_path.is_relative_to(adapter.run_state) + ) + self.assertEqual(adapter.private_key_path.stat().st_mode & 0o777, 0o600) + + +class OverlayTest(VmTestCase): + def test_the_overlay_command_names_the_base_as_a_backing_file(self): + adapter = self.make() + argv = adapter.overlay_argv() + self.assertEqual(argv[:3], ["qemu-img", "create", "-f"]) + self.assertIn("-b", argv) + self.assertEqual(argv[argv.index("-b") + 1], str(self.base)) + self.assertEqual(argv[-2], str(adapter.overlay_path)) + + def test_the_base_is_never_the_output_path(self): + adapter = self.make() + argv = adapter.overlay_argv() + self.assertNotEqual(argv[-2], str(self.base)) + self.assertFalse(Path(argv[-2]).is_relative_to(self.shared)) + + def test_a_wrong_base_digest_blocks(self): + adapter = self.make(base_image_sha256="0" * 64) + report = adapter.preflight() + self.assertFalse(report.ok) + self.assertTrue(any("digest" in finding.name for finding in report.blockers)) + + def test_a_missing_base_blocks(self): + self.base.unlink() + adapter = self.make() + report = adapter.preflight() + self.assertFalse(report.ok) + self.assertTrue(any("base image" in finding.name for finding in report.blockers)) + + +class ProgramTest(VmTestCase): + def test_the_rendered_program_is_valid_python(self): + compile(self.make().render_program(), "__main__.py", "exec") + + def test_the_program_pins_the_provider_version(self): + adapter = self.make() + project_file = adapter.render_project() + self.assertIn("0.5.3", project_file) + self.assertIn("pulumi-libvirt", project_file) + + def test_the_program_names_the_overlay_the_seed_and_the_graphics(self): + program = self.make().render_program() + for needle in ("OVERLAY_PATH", "USER_DATA", "META_DATA", "GRAPHICS"): + self.assertIn(needle, program) + + def test_the_stack_settings_carry_no_secret(self): + adapter = self.make() + settings = adapter.render_stack_settings() + self.assertNotIn(TOKEN, settings) + self.assertNotIn("PRIVATE KEY", settings) + + def test_the_program_never_writes_the_base_image(self): + adapter = self.make() + program = adapter.render_program() + self.assertIn(str(adapter.overlay_path), program) + self.assertNotIn(f'"{self.base}"', program) + + +class ResourceTest(VmTestCase): + def test_the_base_image_is_declared_shared_and_stays_out_of_the_destroy_plan(self): + adapter = self.make() + handle = adapter.plan_handle() + per_run = {str(item) for item in handle.per_run_resources} + shared = {str(item) for item in handle.shared_resources} + self.assertIn(f"image:{self.base}", shared) + self.assertNotIn(f"image:{self.base}", per_run) + for item in per_run: + self.assertNotIn(str(self.base), item) + + def test_the_declarations_do_not_overlap(self): + handle = self.make().plan_handle() + cleanup.assert_declarations_disjoint( + per_run=handle.per_run_resources, shared=handle.shared_resources + ) + + def test_the_per_run_resources_name_the_domain_the_overlay_and_the_stack(self): + handle = self.make().plan_handle() + kinds = {item.kind for item in handle.per_run_resources} + self.assertEqual(kinds, {"domain", "path"}) + + +class PreflightTest(VmTestCase): + def test_a_missing_pulumi_blocks(self): + adapter = self.make(present=("qemu-img", "virsh", "ssh", "scp")) + report = adapter.preflight() + self.assertFalse(report.ok) + self.assertTrue(any("pulumi" in finding.name for finding in report.blockers)) + + def test_an_unverified_provider_schema_blocks(self): + adapter = self.make(provider_schema_verified=False) + report = adapter.preflight() + self.assertFalse(report.ok) + self.assertTrue( + any("provider schema" in finding.name for finding in report.blockers) + ) + + def test_a_missing_hardware_acceleration_device_is_reported(self): + adapter = self.make() + report = adapter.preflight() + names = [finding.name for finding in report.findings] + self.assertIn("hardware acceleration device", names) + + def test_a_ready_host_passes(self): + runner = StubRunner([("virsh", 0, " Id Name State\n", "")]) + adapter = self.make(runner=runner) + report = adapter.preflight() + if Path("/dev/kvm").exists(): + self.assertTrue(report.ok, [item.detail for item in report.blockers]) + + +class TransportTest(VmTestCase): + def test_the_transport_argv_is_non_interactive_and_uses_the_per_run_key(self): + adapter = self.make() + adapter.prepare_identity() + adapter.address = "192.0.2.10" + argv = adapter.ssh_argv(["true"]) + self.assertEqual(argv[0], "ssh") + self.assertIn("BatchMode=yes", " ".join(argv)) + self.assertIn(str(adapter.private_key_path), argv) + self.assertIn("cycle@192.0.2.10", argv) + self.assertEqual(argv[-1], "true") + + def test_the_transport_refuses_to_run_before_an_address_is_known(self): + adapter = self.make() + with self.assertRaises(vm.VmContractError): + adapter.ssh_argv(["true"]) + + +@unittest.skipUnless(shutil.which("qemu-img"), "qemu-img is not on this host") +class OverlayIntegrationTest(VmTestCase): + def test_the_overlay_is_backed_by_the_base_and_the_base_is_unchanged(self): + base = self.shared / "real-base.qcow2" + subprocess.run( + ["qemu-img", "create", "-f", "qcow2", str(base), "16M"], + check=True, + capture_output=True, + ) + before = hashlib.sha256(base.read_bytes()).hexdigest() + adapter = self.make( + base_image=str(base), + base_image_sha256=before, + runner=proc.run, + ) + adapter.run_state.mkdir(parents=True, exist_ok=True) + outcome = proc.run(adapter.overlay_argv(), timeout=120, context="host") + self.assertEqual(outcome.exit_code, 0, outcome.stderr) + chain = subprocess.run( + ["qemu-img", "info", "--backing-chain", str(adapter.overlay_path)], + capture_output=True, + text=True, + check=True, + ) + self.assertIn(str(base), chain.stdout) + self.assertEqual(hashlib.sha256(base.read_bytes()).hexdigest(), before) From 7b503bcb94309487948dbd7e3d330f783fa4aa54 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 00:03:08 +0200 Subject: [PATCH 07/42] Add the command line, the runner's documentation, the counterexample harness, and the evidence one host produced; require an orca the environment actually carries. --- experiments/minimal-cycle/README.md | 185 ++++++ .../config.distrobox.example.yaml | 65 +++ .../minimal-cycle/config.vm.example.yaml | 68 +++ experiments/minimal-cycle/counterexamples.py | 292 ++++++++++ .../cycle_runner/adapters/__init__.py | 23 + .../cycle_runner/adapters/distrobox.py | 15 +- experiments/minimal-cycle/cycle_runner/cli.py | 160 ++++++ .../minimal-cycle/cycle_runner/probe.py | 32 +- .../minimal-cycle/cycle_runner/redact.py | 13 + .../minimal-cycle/cycle_runner/result.py | 2 + experiments/minimal-cycle/evidence/README.md | 78 +++ .../evidence/counterexamples.txt | 18 + .../evidence/preflight-distrobox.txt | 8 + .../minimal-cycle/evidence/preflight-vm.txt | 11 + .../auth-receipt.json | 11 + .../run-blocked-on-host-orca/cleanup.json | 15 + .../export-receipt.json | 66 +++ .../identity-environment-probe.txt | 13 + .../identity-host-probe.txt | 13 + .../identity-orca-status.json | 1 + .../run-blocked-on-host-orca/manifest.json | 532 ++++++++++++++++++ .../run-blocked-on-host-orca/preflight.json | 36 ++ experiments/minimal-cycle/run-cycle | 12 + experiments/minimal-cycle/schema.json | 1 + experiments/minimal-cycle/tests/fakes.py | 20 +- .../tests/test_adapter_distrobox.py | 27 + experiments/minimal-cycle/tests/test_cli.py | 132 +++++ .../minimal-cycle/tests/test_config.py | 31 + .../tests/test_counterexamples.py | 29 + .../minimal-cycle/tests/test_lifecycle.py | 24 + experiments/minimal-cycle/tests/test_probe.py | 59 +- .../minimal-cycle/tests/test_redact.py | 30 + 32 files changed, 2006 insertions(+), 16 deletions(-) create mode 100644 experiments/minimal-cycle/README.md create mode 100644 experiments/minimal-cycle/config.distrobox.example.yaml create mode 100644 experiments/minimal-cycle/config.vm.example.yaml create mode 100755 experiments/minimal-cycle/counterexamples.py create mode 100644 experiments/minimal-cycle/cycle_runner/cli.py create mode 100644 experiments/minimal-cycle/evidence/README.md create mode 100644 experiments/minimal-cycle/evidence/counterexamples.txt create mode 100644 experiments/minimal-cycle/evidence/preflight-distrobox.txt create mode 100644 experiments/minimal-cycle/evidence/preflight-vm.txt create mode 100644 experiments/minimal-cycle/evidence/run-blocked-on-host-orca/auth-receipt.json create mode 100644 experiments/minimal-cycle/evidence/run-blocked-on-host-orca/cleanup.json create mode 100644 experiments/minimal-cycle/evidence/run-blocked-on-host-orca/export-receipt.json create mode 100644 experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-environment-probe.txt create mode 100644 experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-host-probe.txt create mode 100644 experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-orca-status.json create mode 100644 experiments/minimal-cycle/evidence/run-blocked-on-host-orca/manifest.json create mode 100644 experiments/minimal-cycle/evidence/run-blocked-on-host-orca/preflight.json create mode 100755 experiments/minimal-cycle/run-cycle create mode 100644 experiments/minimal-cycle/tests/test_cli.py create mode 100644 experiments/minimal-cycle/tests/test_counterexamples.py diff --git a/experiments/minimal-cycle/README.md b/experiments/minimal-cycle/README.md new file mode 100644 index 0000000..c61168a --- /dev/null +++ b/experiments/minimal-cycle/README.md @@ -0,0 +1,185 @@ +# Minimal cycle runner + +One Python runner drives one minimal proof cycle on a disposable environment, +through one of two backends, and answers a single narrow question: + +> Can a fresh environment be created, driven through Orca to one Claude Code +> worker on a clean copy of a project, checked independently, exported to the +> host, and destroyed — without losing the evidence and without quietly falling +> back onto the host? + +It is not a benchmark, not an evaluation of a model or of a tester, and not a +task matrix. It measures plumbing. + +## What is here + +``` +run-cycle the command line +schema.json the required shape of a run manifest +config.distrobox.example.yaml an example configuration for the container backend +config.vm.example.yaml an example configuration for the machine backend +cycle_runner/ the runner +cycle_runner/adapters/ one adapter per backend, behind one interface +fixtures/evidence-task/ the one small task the worker is given +tests/ the suite, including the counterexamples +evidence/ what the runner actually did on one host +``` + +The runner uses the Python standard library at runtime. A configuration may be +written as JSON always, or as YAML when a parser is importable; the error names +the alternative rather than parsing half a document. + +## Running it + +```bash +cd experiments/minimal-cycle +./run-cycle preflight --config /path/to/config.yaml +./run-cycle run --config /path/to/config.yaml +``` + +`preflight` reports, fact by fact, whether this host can run the configured +backend, and exits non-zero when it cannot. `run` mints a run identifier, +performs the whole cycle, and exits with the code its status maps to. Pass +`--run-id` to pin the identifier, and `--json` to get the machine-readable form +of either command. + +The suite needs nothing installed: + +```bash +python3 -m unittest discover -s tests -t . -v +``` + +## The cycle + +``` +prepare -> create -> bootstrap -> identity -> task -> check + -> collect -> export -> cleanup -> close +``` + +Two orderings are the point of the whole runner. + +**Evidence is exported before anything is destroyed.** `collect` brings the +project tree back to the host, `export` writes every artifact and then re-reads +each one from its host path and recomputes its digest. Only an export where +every required artifact matches releases the run to stop and destroy anything. +An unconfirmed export, or an unconfirmed stop, leaves the environment standing +and records residue with the reason. + +**The task runs only after the identity gate.** The same shell probe runs on the +host and inside the environment. A result that carries no environment marker and +repeats the host's own name, machine identity and home is a host fallback, and a +host fallback stops the run. So does finding no Orca inside the environment: the +run is blocked, never redirected to the host's installation. + +## Statuses + +| Status | Exit | Meaning | +| --- | --- | --- | +| `SETTLED` | 0 | every phase ran, the check observed the marker, the export was confirmed and the per-run resources are gone | +| `ERROR` | 1 | the run completed but something it promised did not hold — usually the check | +| `TIMEOUT` | 2 | the task reached the configured deadline; the artifacts were still exported | +| `CANCELLED` | 3 | the run was stopped deliberately | +| `BLOCKED` | 4 | a gate refused: preflight, auth, the identity verdict, or a missing Orca | +| `CLEANUP_FAILED` | 5 | everything else held, but a per-run resource survived | +| `UNKNOWN` | 6 | the export was not confirmed, so nothing about the run rests on complete evidence | + +A status name is a classification of the manifest, never a substitute for it. + +## Artifacts + +``` +$artifact_root// + manifest.json the whole run, validated against schema.json + preflight.json what the host could and could not do + backend-status.json the environment and its declared resources + auth-receipt.json the method and its status, never its material + host-before.json the host before the run + host-after.json the host after it, and what changed + identity/host-probe.txt the probe as the host answered it + identity/environment-probe.txt the probe as the environment answered it + identity/orca-status.json what Orca reported inside the environment + check.stdout / check.stderr the independent check, as a process + diff.patch what the task changed, computed on the host + task-artifact/ the file the task was asked to produce + logs/runner.log the runner's own narration + logs/commands.jsonl every command, with timings and exit codes + run-before-cleanup.json the result as it stood when the export was taken + export-receipt.json each artifact re-read from the host, with its digest + cleanup.json what was removed, what was kept, and how that was checked +``` + +Every stream is redacted on the way out. Redaction matches shapes — bearer +headers, key prefixes, compact web tokens, private-key blocks, absolute paths to +credential files, and any assignment whose key names a secret — so a credential +this run has never seen is still removed. + +## The two backends + +### Distrobox + +A fresh box from a declared Distrobox Assemble manifest, with its own home and +its own copy of the project, created and destroyed through Assemble rather than +through a container manager this runner would have to become. + +**What Distrobox does not give you.** Distrobox exists to integrate with the +host, and it mounts the invoking user's home directory into every box at its own +path. There is no flag that suppresses it. A separate per-run home limits which +state the run writes; it does not make the host's credentials unreachable from +inside the box. + +The runner does not assert this either way. Preflight asks Distrobox what it +would actually mount, with `distrobox assemble create --dry-run`, and reads the +rendered container command. If the host home is mounted, the run is blocked +until `distrobox.accept_host_home_mount` records that the compromise is +deliberate — and the manifest then carries that acknowledgement. + +Extra mounts are checked separately: a mount naming the host home, a directory +above it, or any known credential directory under it is refused outright. + +`--unshare-all` is not written for you. It exists, it may conflict with the +graphical and message-bus access Orca needs, and choosing it is a decision to +make against a real Orca launch rather than a default to inherit. + +### Virtual machine + +A per-run libvirt domain declared with Pulumi in Python, over a qcow2 overlay +whose backing file is a preserved tool image, with a cloud-init NoCloud seed and +an ssh transport. + +The base image is opened only as a backing file and is declared a shared +resource, so it can never appear in a destroy plan. The seed carries one per-run +public key and nothing else; the private half is generated into the per-run +state directory with owner-only permissions and is removed with it. + +The provider's resource schema is not assumed. The rendered program names fields +that belong to the pinned provider version, and preflight refuses to run until +`vm.provider_schema_verified` records that someone checked them against that +version. Building the tool image is a separate job — the runner consumes an +image, it does not build one, and it is not an image cache. + +## Auth + +Exactly one declared method runs, and none of them writes a credential value, a +length, or a prefix into any artifact. + +- `existing_login` copies only the relative allowlist entries from the host home + into the per-run home with owner-only permissions, and removes them before + cleanup. An entry that is absent blocks the run. On Distrobox this buys less + than it looks like, for the reason above. +- `short_lived_token` passes the value of one named variable to the worker + process for the duration of the run. On Distrobox the name alone is forwarded + to the container manager, so the value never appears in a command line. It is + never written to a file, an image, a seed, stack state, or a log. +- `api_key_helper` declares a helper command in the per-run settings and passes + no value at all. + +An absolute path to a credential file is refused in the configuration outright, +and a configuration key whose name means "secret" is refused with it. + +## What this does not prove + +Nothing here observes Orca starting inside either backend, a window bound to +that instance, a Claude Code worker driven through it, or a login surviving a +run. Those need a host that carries Orca in the chosen image, a built tool image, +and a pinned libvirt provider. `evidence/` records what one host actually did, +including where it stopped. diff --git a/experiments/minimal-cycle/config.distrobox.example.yaml b/experiments/minimal-cycle/config.distrobox.example.yaml new file mode 100644 index 0000000..5d8ff1f --- /dev/null +++ b/experiments/minimal-cycle/config.distrobox.example.yaml @@ -0,0 +1,65 @@ +# One minimal proof cycle on the Distrobox backend. +# +# Every path here is an example. Replace them before running, and read the +# README's "What Distrobox does not give you" section before trusting the +# isolation this backend provides. + +backend: distrobox + +# The revision of the system under test, and of dely itself. Both are recorded +# in the manifest so a result can be tied to what produced it. +tested_revision: replace-with-the-revision-under-test +dely_revision: replace-with-the-dely-revision +claude_code_version: replace-with-the-pinned-claude-code-version + +timeout_seconds: 1800 + +# Both roots are absolute, and neither may contain the other: cleanup removes +# per-run state, and the artifacts have to outlive it. +artifact_root: /var/tmp/dely-cycle/artifacts +state_root: /var/tmp/dely-cycle/state + +project: + # A git repository on the host. The runner exports one revision with + # `git archive`, so the copy carries no repository state and no uncommitted work. + source: /var/tmp/dely-cycle/project-source + revision: main + # Where the copy lands inside the environment, relative to the per-run home. + environment_path: project + +task: + marker: dely-cycle-marker + relative_path: evidence.txt + +check: + relative_path: evidence.txt + +auth: + # existing_login | short_lived_token | api_key_helper + # + # existing_login copies only the relative entries listed below into the + # per-run home. On Distrobox that buys less than it appears to: see the + # README, because Distrobox mounts the host home regardless. + mode: api_key_helper + reference: workshop-helper + helper_argv: + - /usr/local/bin/replace-with-your-key-helper + +orca: + version: replace-with-the-pinned-orca-version + model: replace-with-the-pinned-model + effort: replace-with-the-pinned-effort + agent: claude + worktree_selector: current + +distrobox: + # Pull the image deliberately before the run; the runner refuses to fetch it. + image: docker.io/library/ubuntu:24.04 + container_prefix: dely-cycle + extra_mounts: [] + # Commands run inside the fresh box before the identity gate. This is where + # Orca and Claude Code get installed if the image does not already carry them. + provision: [] + # Distrobox mounts the host home into every box and offers no flag to suppress + # it. Preflight blocks until this records that the compromise is deliberate. + accept_host_home_mount: false diff --git a/experiments/minimal-cycle/config.vm.example.yaml b/experiments/minimal-cycle/config.vm.example.yaml new file mode 100644 index 0000000..b71eb24 --- /dev/null +++ b/experiments/minimal-cycle/config.vm.example.yaml @@ -0,0 +1,68 @@ +# One minimal proof cycle on the virtual-machine backend. +# +# This backend needs a prepared tool image, a pinned libvirt provider, and a +# provider schema someone has actually checked. Preflight refuses each of those +# rather than guessing. + +backend: vm + +tested_revision: replace-with-the-revision-under-test +dely_revision: replace-with-the-dely-revision +claude_code_version: replace-with-the-pinned-claude-code-version + +timeout_seconds: 3600 + +artifact_root: /var/tmp/dely-cycle/artifacts +state_root: /var/tmp/dely-cycle/state + +project: + source: /var/tmp/dely-cycle/project-source + revision: main + environment_path: project + +task: + marker: dely-cycle-marker + relative_path: evidence.txt + +check: + relative_path: evidence.txt + +auth: + mode: short_lived_token + reference: workshop-token + # The variable name only. Its value is read from the runner's environment for + # the duration of the run and is never written to the seed, the stack state, + # the manifest or a log. + token_env: CLAUDE_CODE_OAUTH_TOKEN + +orca: + version: replace-with-the-pinned-orca-version + model: replace-with-the-pinned-model + effort: replace-with-the-pinned-effort + agent: claude + worktree_selector: current + +vm: + provider: pulumi-libvirt + provider_version: replace-with-the-version-you-checked + stack_prefix: dely-cycle + # The preserved tool image built separately with Packer. The run opens it only + # as a backing file and never destroys it. + base_image: /var/lib/dely-cycle/images/tool-image.qcow2 + # A digest of the right shape that matches nothing: preflight blocks on it + # until it is replaced with the real one. + base_image_sha256: '0000000000000000000000000000000000000000000000000000000000000000' + guest_user: cycle + pulumi_binary: pulumi + connect_uri: qemu:///session + overlay_size: 24g + memory_mb: 4096 + vcpus: 2 + graphics: spice + transport: ssh + network: default + ssh_port: 22 + # The rendered Pulumi program names resource fields that belong to the pinned + # provider version. Preflight blocks until someone has checked them. + provider_schema_verified: false + provision: [] diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py new file mode 100755 index 0000000..94b28ed --- /dev/null +++ b/experiments/minimal-cycle/counterexamples.py @@ -0,0 +1,292 @@ +#!/usr/bin/env python3 +"""Run every acceptance counterexample and check its instrument goes red. + +Each entry replaces a rail with an implementation that is present, runs, and +returns a pass — the wrong implementation a green suite would otherwise +accept — and then runs the tests that are supposed to reject it. A case that +stays green is a row whose instrument proves nothing. + + python3 counterexamples.py run them all + python3 counterexamples.py --list print the table + python3 counterexamples.py --only + +The edit is applied to the working tree and restored afterwards, including on +failure, so nothing is left mutated. +""" + +from __future__ import annotations + +import argparse +import subprocess +import sys +from dataclasses import dataclass +from pathlib import Path + +ROOT = Path(__file__).resolve().parent + + +@dataclass(frozen=True) +class Counterexample: + """One wrong-but-passing implementation, and what should reject it.""" + + name: str + requirement: str + path: str + original: str + replacement: str + instruments: tuple[str, ...] + + +CASES: tuple[Counterexample, ...] = ( + Counterexample( + name="no-host-fallback", + requirement="A probe result equal to the host's own snapshot is rejected", + path="cycle_runner/probe.py", + original=""" markers = _markers(host, environment) + record.markers = markers + + if not markers:""", + replacement=""" markers = _markers(host, environment) + record.markers = markers + record.verdict = IDENTITY_ENVIRONMENT + record.reason = "the probe answered, so the command ran in the environment" + return record + + if not markers:""", + instruments=( + "tests.test_probe.VerdictTest", + "tests.test_lifecycle.HostFallbackTest", + ), + ), + Counterexample( + name="orca-is-the-environments-own", + requirement="An orca that resolves to the host's installation is not present", + path="cycle_runner/probe.py", + original=""" orca_present=bool(environment_orca) + and not is_host_installation + and bool(environment_version),""", + replacement=""" orca_present=bool(environment_orca),""", + instruments=( + "tests.test_probe.OrcaPresenceTest", + "tests.test_lifecycle.HostOrcaTest", + ), + ), + Counterexample( + name="export-before-destroy", + requirement="Cleanup runs only after a confirmed export", + path="cycle_runner/cleanup.py", + original=" if not export_record.status.permits_cleanup:", + replacement=" if False:", + instruments=("tests.test_cleanup.GateTest", "tests.test_lifecycle.ExportGateTest"), + ), + Counterexample( + name="stop-must-be-confirmed", + requirement="Cleanup runs only after a confirmed stop", + path="cycle_runner/cleanup.py", + original=" if not stop_confirmed:", + replacement=" if False:", + instruments=( + "tests.test_cleanup.StopGateTest", + "tests.test_lifecycle.ExportGateTest", + ), + ), + Counterexample( + name="destroy-after-export", + requirement="Evidence is collected and exported before anything is destroyed", + path="cycle_runner/lifecycle.py", + original=""" self._collect(baseline, fetched) + export_record = self._export() + cleanup_record = self._cleanup(export_record)""", + replacement=""" from .result import ExportRecord as _AssumedExport + from .status import ExportStatus as _AssumedStatus + + cleanup_record = self._cleanup( + _AssumedExport(status=_AssumedStatus.CONFIRMED, detail="assumed") + ) + self._collect(baseline, fetched) + export_record = self._export()""", + instruments=( + "tests.test_lifecycle.TimeoutTest", + "tests.test_lifecycle.SettledCycleTest", + "tests.test_lifecycle.ExportGateTest", + ), + ), + Counterexample( + name="receipt-is-a-re-read", + requirement="The receipt proves bytes on the host, not bytes in memory", + path="cycle_runner/export.py", + original=" described = hashing.describe_file(target, relative_to=self.root)", + replacement=""" described = { + "path": relative, + "size_bytes": 1, + "sha256": declared.expected_sha256 or "0" * 64, + }""", + instruments=("tests.test_export",), + ), + Counterexample( + name="cleanup-only-per-run", + requirement="A per-run path that contains a shared resource is refused", + path="cycle_runner/cleanup.py", + original=" if shared_path == owned_path or shared_path.is_relative_to(owned_path):", + replacement=" if False:", + instruments=("tests.test_cleanup.DeclarationTest",), + ), + Counterexample( + name="redaction-by-shape", + requirement="Redaction catches a credential this run has never seen", + path="cycle_runner/redact.py", + original=""" for shape in _TOKEN_SHAPES: + cleaned = shape.sub(TOKEN_MARK, cleaned)""", + replacement=""" for known in ("a-token-the-runner-was-told-about",): + cleaned = cleaned.replace(known, TOKEN_MARK)""", + instruments=("tests.test_redact",), + ), + Counterexample( + name="manifest-required-fields", + requirement="The manifest carries every required field on every path", + path="cycle_runner/manifest.py", + original=""" if name not in document: + problems.append(f"missing required field: {f'{trail}.{name}' if trail else name}")""", + replacement=""" if False: + problems.append("never")""", + instruments=("tests.test_manifest.ValidationTest",), + ), + Counterexample( + name="run-id-carries-no-name", + requirement="The run identifier discloses neither the host name nor the auth reference", + path="cycle_runner/ids.py", + original=' return f"{stamp}-{host_segment(host_name)}-{secrets.token_hex(_ENTROPY_BYTES)}"', + replacement=' return f"{stamp}-{host_name}-{secrets.token_hex(_ENTROPY_BYTES)}"', + instruments=("tests.test_ids",), + ), + Counterexample( + name="config-refuses-a-secret", + requirement="A secret-shaped value under a dull key is refused", + path="cycle_runner/config.py", + original=" refuse_secrets(document)", + replacement=" pass", + instruments=("tests.test_config.SecretRefusalTest",), + ), + Counterexample( + name="auth-leaves-no-material", + requirement="No credential value, length or prefix reaches a receipt", + path="cycle_runner/auth.py", + original=""" record.detail = ( + f"the value of {variable} is passed to the worker process for this run only " + "and is never written to a file, an image, a seed or a log" + )""", + replacement=""" record.detail = ( + f"the value of {variable} (length {len(value)}, starting {value[:4]}) is " + "passed to the worker process for this run only" + )""", + instruments=("tests.test_auth.ShortLivedTokenTest",), + ), + Counterexample( + name="host-home-mount-acknowledged", + requirement="An unacknowledged host-home mount blocks the run", + path="cycle_runner/adapters/distrobox.py", + original=" if self.settings.accept_host_home_mount:", + replacement=" if True:", + instruments=("tests.test_adapter_distrobox.DryRunTest",), + ), + Counterexample( + name="preflight-leaves-no-residue", + requirement="Preflight inspects without creating the run's state", + path="cycle_runner/adapters/distrobox.py", + original=""" probe_manifest = Path(staging) / "distrobox.ini" + probe_manifest.write_text(self.render_manifest(), encoding="utf-8")""", + replacement=" probe_manifest = self.write_manifest()", + instruments=("tests.test_adapter_distrobox.PreflightResidueTest",), + ), + Counterexample( + name="base-is-only-a-backing-file", + requirement="The preserved base image is never the overlay's output path", + path="cycle_runner/adapters/vm.py", + original=""" "-F", + "qcow2", + str(self.overlay_path),""", + replacement=""" "-F", + "qcow2", + str(self.settings.base_image),""", + instruments=("tests.test_adapter_vm.OverlayTest",), + ), + Counterexample( + name="provider-schema-verified", + requirement="An unverified provider schema blocks the machine backend", + path="cycle_runner/adapters/vm.py", + original=" ok=self.settings.provider_schema_verified,", + replacement=" ok=True,", + instruments=("tests.test_adapter_vm.PreflightTest",), + ), +) + + +def applicable(case: Counterexample) -> bool: + """Whether this case's target text is still present in the tree.""" + return case.original in (ROOT / case.path).read_text(encoding="utf-8") + + +def run(case: Counterexample) -> tuple[bool, str]: + """Apply one counterexample, run its instruments, and restore the tree.""" + target = ROOT / case.path + backup = target.read_text(encoding="utf-8") + if case.original not in backup: + return False, "the counterexample no longer applies to this code" + target.write_text(backup.replace(case.original, case.replacement, 1), encoding="utf-8") + try: + completed = subprocess.run( + [sys.executable, "-m", "unittest", *case.instruments], + cwd=ROOT, + capture_output=True, + text=True, + ) + finally: + target.write_text(backup, encoding="utf-8") + if completed.returncode == 0: + return False, "the instruments stayed green: this row proves nothing" + reasons = [ + line + for line in completed.stderr.splitlines() + if line.startswith(("FAIL:", "ERROR:")) + ] + return True, "; ".join(reasons[:2]) or "red" + + +def main(argv: list[str] | None = None) -> int: + """Run the selected counterexamples and report which discriminated.""" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--list", action="store_true", help="print the table and exit") + parser.add_argument("--only", help="run one counterexample by name") + arguments = parser.parse_args(argv) + + cases = CASES + if arguments.only: + cases = tuple(case for case in CASES if case.name == arguments.only) + if not cases: + print(f"no counterexample named {arguments.only!r}", file=sys.stderr) + return 2 + + if arguments.list: + for case in CASES: + print(f"{case.name}\n {case.requirement}\n {', '.join(case.instruments)}") + return 0 + + width = max(len(case.name) for case in cases) + failures = [] + for case in cases: + discriminated, detail = run(case) + verdict = "RED " if discriminated else "GREEN" + print(f"{verdict} {case.name.ljust(width)} {case.requirement}") + if not discriminated: + failures.append(case.name) + print(f" {detail}") + print() + if failures: + print(f"{len(failures)} counterexample(s) did not discriminate: {', '.join(failures)}") + return 1 + print(f"all {len(cases)} counterexamples discriminated") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/experiments/minimal-cycle/cycle_runner/adapters/__init__.py b/experiments/minimal-cycle/cycle_runner/adapters/__init__.py index e69de29..7e0ce98 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/__init__.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/__init__.py @@ -0,0 +1,23 @@ +"""Backend adapters behind one interface.""" + +from __future__ import annotations + +from typing import Any + +from ..config import RunConfig +from .base import BackendAdapter + +__all__ = ["BackendAdapter", "build"] + + +def build(*, run_config: RunConfig, run_id: str, **options: Any) -> BackendAdapter: + """Return the adapter the configuration selected.""" + if run_config.backend == "distrobox": + from .distrobox import DistroboxAdapter + + return DistroboxAdapter(run_config=run_config, run_id=run_id, **options) + if run_config.backend == "vm": + from .vm import VmAdapter + + return VmAdapter(run_config=run_config, run_id=run_id, **options) + raise ValueError(f"no adapter for backend {run_config.backend!r}") diff --git a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py index a5ccda7..ea593be 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py @@ -14,6 +14,7 @@ from __future__ import annotations import shutil +import tempfile from pathlib import Path from typing import Callable, Mapping, Sequence @@ -166,12 +167,14 @@ def host_home_mount_finding(self) -> Finding: the evidence; the configuration has to acknowledge what it shows. """ name = "host home reachable from the environment" - self.write_manifest() - rendered = self.runner( - [self.binary, "assemble", "create", "--dry-run", "--file", str(self.manifest_path)], - timeout=300, - context="host", - ) + with tempfile.TemporaryDirectory(prefix="dely-cycle-preflight-") as staging: + probe_manifest = Path(staging) / "distrobox.ini" + probe_manifest.write_text(self.render_manifest(), encoding="utf-8") + rendered = self.runner( + [self.binary, "assemble", "create", "--dry-run", "--file", str(probe_manifest)], + timeout=300, + context="host", + ) home = str(self.host_home) mounted = ( f'--volume "{home}":"{home}"' in rendered.stdout diff --git a/experiments/minimal-cycle/cycle_runner/cli.py b/experiments/minimal-cycle/cycle_runner/cli.py new file mode 100644 index 0000000..95ac297 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/cli.py @@ -0,0 +1,160 @@ +"""The command line: preflight a backend, or run one cycle. + +Every failure this tool can reasonably meet — an absent configuration, an +invalid one, a backend this host cannot run — is reported as a sentence and an +exit code rather than as a traceback. +""" + +from __future__ import annotations + +import argparse +import json +import socket +import sys +from datetime import datetime, timezone +from pathlib import Path +from typing import Sequence, TextIO + +from . import adapters, config as config_module, ids, lifecycle, manifest, proc +from .status import RunStatus, exit_code + +USAGE_EXIT = 2 + + +def _load(path: Path) -> config_module.RunConfig: + if not Path(path).is_file(): + raise config_module.ConfigError(f"no configuration at {path}") + return config_module.load(Path(path)) + + +def _tool_versions() -> dict[str, str]: + versions = {"runner": "minimal-cycle", "python": sys.version.split()[0]} + for name, argv in ( + ("distrobox", ["distrobox", "--version"]), + ("podman", ["podman", "--version"]), + ("qemu_img", ["qemu-img", "--version"]), + ("virsh", ["virsh", "--version"]), + ("pulumi", ["pulumi", "version"]), + ("git", ["git", "--version"]), + ): + outcome = proc.run(argv, timeout=60, context="host") + if outcome.ok and outcome.stdout.strip(): + versions[name] = outcome.stdout.strip().splitlines()[-1] + return versions + + +def _preflight(arguments, stdout: TextIO, stderr: TextIO) -> int: + run_config = _load(arguments.config) + adapter = adapters.build( + run_config=run_config, + run_id=arguments.run_id or _mint(), + ) + report = adapter.preflight() + if arguments.json: + print(json.dumps(report.to_document(), indent=2, sort_keys=True), file=stdout) + else: + print(f"backend: {report.backend}", file=stdout) + for finding in report.findings: + mark = "ok " if finding.ok else "BLOCK " + print(f" {mark} {finding.name}: {finding.detail}", file=stdout) + print( + "usable on this host" if report.ok else "not usable on this host", + file=stdout, + ) + return 0 if report.ok else exit_code(RunStatus.BLOCKED) + + +def _mint() -> str: + return ids.mint_run_id(now=datetime.now(timezone.utc), host_name=socket.gethostname()) + + +def _run(arguments, stdout: TextIO, stderr: TextIO) -> int: + run_config = _load(arguments.config) + run_id = arguments.run_id or _mint() + if not ids.is_run_id(run_id): + raise config_module.ConfigError( + f"{run_id!r} is not a run identifier of the documented shape" + ) + adapter = adapters.build(run_config=run_config, run_id=run_id) + outcome = lifecycle.run_cycle( + run_config=run_config, + adapter=adapter, + run_id=run_id, + tool_versions=_tool_versions(), + ) + result = outcome.run_result + if arguments.json: + print( + json.dumps(result.to_document(), indent=2, sort_keys=True), file=stdout + ) + else: + print(f"run_id: {result.run_id}", file=stdout) + print(f"backend: {result.backend}", file=stdout) + print(f"status: {result.status.value}", file=stdout) + print(f"identity: {result.identity.verdict} — {result.identity.reason}", file=stdout) + print(f"check: {result.check.status.value} (exit {result.check.exit_code})", file=stdout) + print(f"export: {result.export.status.value}", file=stdout) + print(f"cleanup: {result.cleanup.status.value} — {result.cleanup.reason}", file=stdout) + print(f"artifacts: {outcome.artifact_dir}", file=stdout) + if result.failure_classification: + print(f"why: {result.failure_classification}", file=stdout) + return outcome.exit_code + + +def _schema(arguments, stdout: TextIO, stderr: TextIO) -> int: + print( + json.dumps(manifest.load_schema(), indent=2, sort_keys=True), file=stdout + ) + return 0 + + +def build_parser() -> argparse.ArgumentParser: + """Return the argument parser for the runner.""" + parser = argparse.ArgumentParser( + prog="run-cycle", + description="Run one minimal proof cycle on a disposable environment.", + ) + subcommands = parser.add_subparsers(dest="command") + + preflight = subcommands.add_parser( + "preflight", help="report whether this host can run the configured backend" + ) + preflight.add_argument("--config", required=True, type=Path) + preflight.add_argument("--run-id") + preflight.add_argument("--json", action="store_true") + preflight.set_defaults(handler=_preflight) + + run = subcommands.add_parser("run", help="run one cycle end to end") + run.add_argument("--config", required=True, type=Path) + run.add_argument("--run-id", help="pin the run identifier instead of minting one") + run.add_argument("--json", action="store_true") + run.set_defaults(handler=_run) + + schema = subcommands.add_parser("schema", help="print the manifest schema") + schema.set_defaults(handler=_schema) + + return parser + + +def main( + argv: Sequence[str] | None = None, + *, + stdout: TextIO | None = None, + stderr: TextIO | None = None, +) -> int: + """Run the command line and return the process exit code.""" + stdout = stdout or sys.stdout + stderr = stderr or sys.stderr + parser = build_parser() + arguments = parser.parse_args(list(argv) if argv is not None else None) + if not getattr(arguments, "handler", None): + parser.print_usage(stderr) + return USAGE_EXIT + try: + return arguments.handler(arguments, stdout, stderr) + except (config_module.ConfigError, manifest.ManifestError) as error: + print(f"error: {error}", file=stderr) + return USAGE_EXIT + except (OSError, RuntimeError, ValueError) as error: + print(f"error: {type(error).__name__}: {error}", file=stderr) + return exit_code(RunStatus.ERROR) diff --git a/experiments/minimal-cycle/cycle_runner/probe.py b/experiments/minimal-cycle/cycle_runner/probe.py index 0f39d0f..a13d3b2 100644 --- a/experiments/minimal-cycle/cycle_runner/probe.py +++ b/experiments/minimal-cycle/cycle_runner/probe.py @@ -175,11 +175,21 @@ def verdict( expected_project: str, ) -> IdentityRecord: """Decide whether the environment answered as itself or as the host.""" + environment_orca = environment.get("orca_path", "") + host_orca = host.get("orca_path", "") + environment_version = environment.get("orca_version", "") + is_host_installation = bool(environment_orca) and environment_orca == host_orca record = IdentityRecord( host=_host_view(host), environment=_environment_view(environment), - orca_present=bool(environment.get("orca_path")), - orca_version=environment.get("orca_version") or None, + # A path that resolves is not an installation the environment has: a + # Distrobox box inherits the host PATH and sees the host home, so the + # host's own launcher answers `command -v orca` inside it. + orca_present=bool(environment_orca) + and not is_host_installation + and bool(environment_version), + orca_is_host_installation=is_host_installation, + orca_version=environment_version or None, ) missing = [field for field in ("hostname", "home") if not environment.get(field)] @@ -238,10 +248,24 @@ def may_continue(record: IdentityRecord) -> tuple[bool, str]: """Report whether the run may proceed past the identity gate, and why not.""" if record.verdict != IDENTITY_ENVIRONMENT: return False, record.reason - if not record.orca_present: + if record.orca_present: + return True, record.reason + found = record.environment.get("orca_path", "") + if record.orca_is_host_installation: + return ( + False, + f"the environment resolved orca to the host's own installation at " + f"{found}; the run stops rather than driving the host from inside a " + "disposable environment", + ) + if not found: return ( False, "orca is not present inside the environment; the run stops rather than " "using the host's installation", ) - return True, record.reason + return ( + False, + f"orca is at {found} inside the environment but reported no version there, " + "so it cannot be driven from inside it", + ) diff --git a/experiments/minimal-cycle/cycle_runner/redact.py b/experiments/minimal-cycle/cycle_runner/redact.py index ad56d7a..45aa3a9 100644 --- a/experiments/minimal-cycle/cycle_runner/redact.py +++ b/experiments/minimal-cycle/cycle_runner/redact.py @@ -99,6 +99,19 @@ def structure(document: Any, extra_values: Sequence[str] = ()) -> Any: return document +def carries_credential_shape(value: str) -> bool: + """Report whether the text carries a credential *value*. + + Narrower than :func:`looks_secret_free`, which also flags a sensitive key + name next to any value at all. This asks only whether something shaped + like a secret is present, so a configuration naming the variable that will + carry a token is not mistaken for one that carries the token. + """ + if _PRIVATE_KEY_BLOCK.search(value) or _BEARER.search(value): + return True + return any(shape.search(value) for shape in _TOKEN_SHAPES) + + def looks_secret_free(value: str) -> bool: """Report whether redaction would change the string at all.""" return text(value) == value diff --git a/experiments/minimal-cycle/cycle_runner/result.py b/experiments/minimal-cycle/cycle_runner/result.py index 5c480ff..c8bf2ae 100644 --- a/experiments/minimal-cycle/cycle_runner/result.py +++ b/experiments/minimal-cycle/cycle_runner/result.py @@ -87,6 +87,7 @@ class IdentityRecord: environment: dict[str, Any] = field(default_factory=dict) markers: list[str] = field(default_factory=list) orca_present: bool = False + orca_is_host_installation: bool = False orca_version: str | None = None orca_status_exit_code: int | None = None @@ -98,6 +99,7 @@ def to_document(self) -> dict[str, Any]: "environment": dict(self.environment), "markers": list(self.markers), "orca_present": self.orca_present, + "orca_is_host_installation": self.orca_is_host_installation, "orca_version": self.orca_version, "orca_status_exit_code": self.orca_status_exit_code, } diff --git a/experiments/minimal-cycle/evidence/README.md b/experiments/minimal-cycle/evidence/README.md new file mode 100644 index 0000000..480d28d --- /dev/null +++ b/experiments/minimal-cycle/evidence/README.md @@ -0,0 +1,78 @@ +# What this runner actually did on one host + +Everything here was produced by the committed code on a Fedora host with +rootless Podman, Distrobox 1.8.2.5, `qemu-img`, `virsh` answering +`qemu:///session`, `/dev/kvm` present, no `pulumi`, and no prepared tool image. +Nothing here was written by hand. + +## What was scrubbed, and what was not + +Three substitutions were applied to the files in this directory, and only +these: the host's home directory became ``, the host user name +became ``, and the raw `machine_id` and `boot_id` lines in the probe +captures became ``. Digests, sizes, timings, exit codes, statuses and +reasons are untouched. The manifests already store host identifiers as digests +rather than as names, so nothing there needed scrubbing. + +## `preflight-distrobox.txt` + +The container backend on this host, usable. The last finding is the one that +matters: Distrobox mounts the host home into every box and has no flag to +suppress it, so the run is blocked until the configuration records that +compromise deliberately. That finding is not a claim — it is the result of +reading what `distrobox assemble create --dry-run` actually renders. + +## `preflight-vm.txt` + +The machine backend on this host, blocked, exit code 4. Three findings refuse: +`pulumi` is not installed, the provider schema has not been checked against a +pinned version, and there is no prepared tool image. The backend reports what is +missing rather than inventing a path around it. + +## `run-blocked-on-host-orca/` + +One real cycle. A fresh box was created from `docker.io/library/ubuntu:24.04` +with its own home, the project was exported at a pinned revision and copied in, +the identity probe ran on both sides, and the run stopped at the identity gate. +Every artifact was exported and confirmed, and the box and its per-run state +were then destroyed while the image was kept. + + status: BLOCKED + identity: ENVIRONMENT + export: CONFIRMED + cleanup: DESTROYED + why: the environment resolved orca to the host's own installation at + /.config/orca/linux-orca-cli-shim/orca; the run stops + rather than driving the host from inside a disposable environment + +Compare the two probe captures. The box answered with its own host name, its own +container marker and the per-run home — a real environment. It also answered +with the *host's* `orca_path` and an empty `orca_version`: Distrobox mounts the +host home and preserves `PATH`, so the host's launcher resolves inside the box +and cannot run there. + +**This is a defect the first run found.** Before it, the gate asked only whether +`command -v orca` resolved. It did, so the run proceeded and failed later at +`worker-start` with `/opt/Orca/resources/bin/orca-ide: No such file or +directory`. Had the container carried a working launcher pointing back at the +host, that run would have produced a complete green manifest for work the host +did. The gate now requires an Orca that is not the host's binary and that +reports a version inside the environment; `counterexamples.py` case +`orca-is-the-environments-own` shows the old rule letting a host-driven run +settle `SETTLED`. + +## `counterexamples.txt` + +`python3 counterexamples.py`. Each line replaces one rail with an +implementation that is present, runs and returns a pass, then runs the tests +that are supposed to reject it. `RED` means the instrument rejected the wrong +implementation. A row that reported `GREEN` would be a row proving nothing. + +## What none of this shows + +No Orca has started inside either backend on this host, no window has been bound +to such an instance, no Claude Code worker has been driven through one, and no +login has survived a run. The machine backend has never created a domain here. +Those need a host carrying Orca in the chosen image, a built tool image and a +pinned libvirt provider, and they are the separate acceptance test this work +does not claim to have passed. diff --git a/experiments/minimal-cycle/evidence/counterexamples.txt b/experiments/minimal-cycle/evidence/counterexamples.txt new file mode 100644 index 0000000..e58ca8d --- /dev/null +++ b/experiments/minimal-cycle/evidence/counterexamples.txt @@ -0,0 +1,18 @@ +RED no-host-fallback A probe result equal to the host's own snapshot is rejected +RED orca-is-the-environments-own An orca that resolves to the host's installation is not present +RED export-before-destroy Cleanup runs only after a confirmed export +RED stop-must-be-confirmed Cleanup runs only after a confirmed stop +RED destroy-after-export Evidence is collected and exported before anything is destroyed +RED receipt-is-a-re-read The receipt proves bytes on the host, not bytes in memory +RED cleanup-only-per-run A per-run path that contains a shared resource is refused +RED redaction-by-shape Redaction catches a credential this run has never seen +RED manifest-required-fields The manifest carries every required field on every path +RED run-id-carries-no-name The run identifier discloses neither the host name nor the auth reference +RED config-refuses-a-secret A secret-shaped value under a dull key is refused +RED auth-leaves-no-material No credential value, length or prefix reaches a receipt +RED host-home-mount-acknowledged An unacknowledged host-home mount blocks the run +RED preflight-leaves-no-residue Preflight inspects without creating the run's state +RED base-is-only-a-backing-file The preserved base image is never the overlay's output path +RED provider-schema-verified An unverified provider schema blocks the machine backend + +all 16 counterexamples discriminated diff --git a/experiments/minimal-cycle/evidence/preflight-distrobox.txt b/experiments/minimal-cycle/evidence/preflight-distrobox.txt new file mode 100644 index 0000000..c89635c --- /dev/null +++ b/experiments/minimal-cycle/evidence/preflight-distrobox.txt @@ -0,0 +1,8 @@ +backend: distrobox + ok distrobox on the path: /usr/bin/distrobox + ok container manager on the path: podman + ok image present locally: docker.io/library/ubuntu:24.04 is present + ok state root writable: /var/tmp/dely-cycle/state + ok artifact root writable: /var/tmp/dely-cycle/artifacts + ok host home reachable from the environment: distrobox mounts into the box and offers no flag to suppress it; the configuration acknowledged this, so the run records a declared compromise rather than claiming isolation +usable on this host diff --git a/experiments/minimal-cycle/evidence/preflight-vm.txt b/experiments/minimal-cycle/evidence/preflight-vm.txt new file mode 100644 index 0000000..46307c6 --- /dev/null +++ b/experiments/minimal-cycle/evidence/preflight-vm.txt @@ -0,0 +1,11 @@ +backend: vm + BLOCK pulumi on the path: pulumi was not found on PATH + ok qemu-img on the path: /usr/bin/qemu-img + ok virsh on the path: /usr/bin/virsh + ok ssh on the path: /usr/bin/ssh + ok scp on the path: /usr/bin/scp + BLOCK provider schema verified: the rendered program's resource fields have not been checked against pulumi-libvirt@unchecked-on-this-host; set vm.provider_schema_verified once they have been + BLOCK base image present: /var/lib/dely-cycle/images/tool-image.qcow2 is not a file on this host + ok hardware acceleration device: /dev/kvm + ok libvirt connection answers: qemu:///session +not usable on this host diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/auth-receipt.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/auth-receipt.json new file mode 100644 index 0000000..baaa5e7 --- /dev/null +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/auth-receipt.json @@ -0,0 +1,11 @@ +{ + "detail": "the helper command is declared in the per-run settings; no credential value passes through the runner", + "entries": [ + ".claude/settings.json" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only", + "removed_after_run": false, + "status": "OK", + "target": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/.claude/settings.json" +} diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/cleanup.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/cleanup.json new file mode 100644 index 0000000..ff4828c --- /dev/null +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/cleanup.json @@ -0,0 +1,15 @@ +{ + "host_difference": {}, + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "container:dely-cycle-20260914t231500z-aaaaaa-00000002", + "path:/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002" + ], + "retained": [], + "shared_preserved": [ + "image:docker.io/library/ubuntu:24.04" + ], + "status": "DESTROYED", + "stop_confirmed": true, + "verified": true +} diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/export-receipt.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/export-receipt.json new file mode 100644 index 0000000..39ee41c --- /dev/null +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/export-receipt.json @@ -0,0 +1,66 @@ +{ + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "5749227c7c4ae0b40ba0ee854e5fec5ad3c26ad4e62c102e544ee69c762c7de4", + "size_bytes": 393 + }, + { + "path": "backend-status.json", + "sha256": "dc441d86db5592190f1cad2c6afb752ca54517e8af11297a9d963a777561bf51", + "size_bytes": 1374 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "ef32560cde0b04d1c08f01cb393f6c7ebca37c0e7535aad07a1d646002d1ea13", + "size_bytes": 690 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "97ca15bcc81dd9d799fd944618db9ba23dcaace049dda83b6366cf716cacf515", + "size_bytes": 1567 + }, + { + "path": "identity/host-probe.txt", + "sha256": "2e9f36ceaa4d492831fa0bf78cdf8130f6827a3704fe940f5467bbee994a2787", + "size_bytes": 1358 + }, + { + "path": "identity/orca-status.json", + "sha256": "ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356", + "size_bytes": 3 + }, + { + "path": "logs/commands.jsonl", + "sha256": "aa57ac6dc2cfd82b53bfa6957d2a15fe8612290a2ec0c26c8ecbb0a01c700e1d", + "size_bytes": 4460 + }, + { + "path": "logs/runner.log", + "sha256": "72d408da85002bb1b3732db8e255d0ea0bc16efb89951f09288a114ccd6cedbe", + "size_bytes": 1354 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "be30779cb0960bb07162a4e9e4008ee81719d31b2d8849d6cb40da030ea348b6", + "size_bytes": 13097 + } + ], + "confirmed_at": "2026-09-14T22:00:35Z", + "detail": "11 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" +} diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-environment-probe.txt b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-environment-probe.txt new file mode 100644 index 0000000..567ed97 --- /dev/null +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-environment-probe.txt @@ -0,0 +1,13 @@ +hostname=dely-cycle-20260914t231500z-aaaaaa-00000002 +machine_id= +boot_id= +home=/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin +pid=3749127 +uname=Linux 7.2.4-200.fc44.x86_64 x86_64 +container_marker=containerenv +virt=docker +orca_path=/.config/orca/linux-orca-cli-shim/orca +orca_version= +project_real=/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/project diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-host-probe.txt b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-host-probe.txt new file mode 100644 index 0000000..06d88de --- /dev/null +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-host-probe.txt @@ -0,0 +1,13 @@ +hostname=system +machine_id= +boot_id= +home= +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin +pid=3748998 +uname=Linux 7.2.4-200.fc44.x86_64 x86_64 +container_marker= +virt=none +orca_path=/.config/orca/linux-orca-cli-shim/orca +orca_version=1.4.201 +project_real=/var/tmp/dely-cycle/project-source diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-orca-status.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-orca-status.json new file mode 100644 index 0000000..0967ef4 --- /dev/null +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-orca-status.json @@ -0,0 +1 @@ +{} diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/manifest.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/manifest.json new file mode 100644 index 0000000..305e274 --- /dev/null +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/manifest.json @@ -0,0 +1,532 @@ +{ + "artifact_root": "/var/tmp/dely-cycle/artifacts/20260914T231500Z-aaaaaa-00000002", + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "5749227c7c4ae0b40ba0ee854e5fec5ad3c26ad4e62c102e544ee69c762c7de4", + "size_bytes": 393 + }, + { + "path": "backend-status.json", + "sha256": "dc441d86db5592190f1cad2c6afb752ca54517e8af11297a9d963a777561bf51", + "size_bytes": 1374 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "ef32560cde0b04d1c08f01cb393f6c7ebca37c0e7535aad07a1d646002d1ea13", + "size_bytes": 690 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "97ca15bcc81dd9d799fd944618db9ba23dcaace049dda83b6366cf716cacf515", + "size_bytes": 1567 + }, + { + "path": "identity/host-probe.txt", + "sha256": "2e9f36ceaa4d492831fa0bf78cdf8130f6827a3704fe940f5467bbee994a2787", + "size_bytes": 1358 + }, + { + "path": "identity/orca-status.json", + "sha256": "ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356", + "size_bytes": 3 + }, + { + "path": "logs/commands.jsonl", + "sha256": "aa57ac6dc2cfd82b53bfa6957d2a15fe8612290a2ec0c26c8ecbb0a01c700e1d", + "size_bytes": 4460 + }, + { + "path": "logs/runner.log", + "sha256": "72d408da85002bb1b3732db8e255d0ea0bc16efb89951f09288a114ccd6cedbe", + "size_bytes": 1354 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "be30779cb0960bb07162a4e9e4008ee81719d31b2d8849d6cb40da030ea348b6", + "size_bytes": 13097 + } + ], + "auth": { + "detail": "the helper command is declared in the per-run settings; no credential value passes through the runner", + "entries": [ + ".claude/settings.json" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only", + "removed_after_run": false, + "status": "OK", + "target": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/.claude/settings.json" + }, + "backend": "distrobox", + "check": { + "argv": [], + "detail": "not attempted", + "elapsed_seconds": null, + "exit_code": null, + "expected_marker": null, + "finished_at": null, + "started_at": null, + "status": "SKIPPED", + "stderr_path": null, + "stdout_path": null + }, + "cleanup": { + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "container:dely-cycle-20260914t231500z-aaaaaa-00000002", + "path:/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002" + ], + "retained": [], + "shared_preserved": [ + "image:docker.io/library/ubuntu:24.04" + ], + "status": "DESTROYED", + "verified": true + }, + "config": { + "artifact_root": "/var/tmp/dely-cycle/artifacts", + "auth": { + "helper_argv": [ + "/usr/local/bin/dely-cycle-key-helper" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only" + }, + "backend": "distrobox", + "check": { + "expected_marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "claude_code_version": "not-installed-in-this-image", + "dely_revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", + "distrobox": { + "accept_host_home_mount": true, + "container_prefix": "dely-cycle", + "extra_mounts": [], + "image": "docker.io/library/ubuntu:24.04", + "provision": [] + }, + "orca": { + "agent": "claude", + "effort": "high", + "model": "claude-opus-5", + "run_objective": "dely minimal cycle", + "status_argv": [ + "orca", + "status", + "--json" + ], + "version": "1.4.201", + "version_argv": [ + "orca", + "--version" + ], + "worktree_selector": "current" + }, + "project": { + "environment_path": "project", + "revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", + "source": "/var/tmp/dely-cycle/project-source" + }, + "state_root": "/var/tmp/dely-cycle/state", + "task": { + "marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "tested_revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", + "timeout_seconds": 900 + }, + "dely_revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", + "elapsed_seconds": 54.556, + "environment": { + "backend": { + "backend": "distrobox", + "container": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "container_manager": "podman", + "distrobox_version": "distrobox: 1.8.2.5", + "image": "docker.io/library/ubuntu:24.04", + "manifest": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/distrobox.ini" + }, + "description": { + "container_manager": "podman", + "desktop_integration": "Distrobox integrates with the host desktop by design; this run limits state with a separate home and project copy, and does not claim container-grade isolation", + "host_home_mount_acknowledged": true, + "image": "docker.io/library/ubuntu:24.04", + "manifest": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/distrobox.ini" + }, + "environment_id": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "home_path": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home", + "per_run_resources": [ + { + "identifier": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "kind": "container" + }, + { + "identifier": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002", + "kind": "path" + } + ], + "project_path": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/project", + "shared_resources": [ + { + "identifier": "docker.io/library/ubuntu:24.04", + "kind": "image" + } + ] + }, + "environment_id": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "export": { + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "5749227c7c4ae0b40ba0ee854e5fec5ad3c26ad4e62c102e544ee69c762c7de4", + "size_bytes": 393 + }, + { + "path": "backend-status.json", + "sha256": "dc441d86db5592190f1cad2c6afb752ca54517e8af11297a9d963a777561bf51", + "size_bytes": 1374 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "ef32560cde0b04d1c08f01cb393f6c7ebca37c0e7535aad07a1d646002d1ea13", + "size_bytes": 690 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "97ca15bcc81dd9d799fd944618db9ba23dcaace049dda83b6366cf716cacf515", + "size_bytes": 1567 + }, + { + "path": "identity/host-probe.txt", + "sha256": "2e9f36ceaa4d492831fa0bf78cdf8130f6827a3704fe940f5467bbee994a2787", + "size_bytes": 1358 + }, + { + "path": "identity/orca-status.json", + "sha256": "ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356", + "size_bytes": 3 + }, + { + "path": "logs/commands.jsonl", + "sha256": "aa57ac6dc2cfd82b53bfa6957d2a15fe8612290a2ec0c26c8ecbb0a01c700e1d", + "size_bytes": 4460 + }, + { + "path": "logs/runner.log", + "sha256": "72d408da85002bb1b3732db8e255d0ea0bc16efb89951f09288a114ccd6cedbe", + "size_bytes": 1354 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "be30779cb0960bb07162a4e9e4008ee81719d31b2d8849d6cb40da030ea348b6", + "size_bytes": 13097 + } + ], + "confirmed_at": "2026-09-14T22:00:35Z", + "detail": "11 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" + }, + "failure_classification": "the environment resolved orca to the host's own installation at /.config/orca/linux-orca-cli-shim/orca; the run stops rather than driving the host from inside a disposable environment", + "finished_at": "2026-09-14T22:00:36Z", + "generated_at": "2026-09-14T22:00:36Z", + "host_after": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "bc21f149eed21643", + "captured_at": "2026-09-14T22:00:36Z", + "extra": { + "backend": "distrobox" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.4-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [ + "20260914T215536Z-bbc5e6-026d6468" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "host_before": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "bc21f149eed21643", + "captured_at": "2026-09-14T21:59:42Z", + "extra": { + "backend": "distrobox" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.4-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [ + "20260914T215536Z-bbc5e6-026d6468" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "identity": { + "environment": { + "container_marker": "containerenv", + "home": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home", + "hostname": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "orca_path": "/.config/orca/linux-orca-cli-shim/orca", + "orca_version": "", + "path": "/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin", + "pid": "3749127", + "project_real": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/project", + "uname": "Linux 7.2.4-200.fc44.x86_64 x86_64", + "user": "", + "virt": "docker" + }, + "host": { + "boot_id_digest": "bc21f149eed21643", + "home_digest": "8fda3821c06d512d", + "hostname_digest": "bbc5e661e106c6dc", + "machine_id_digest": "907fa710d6d1c501", + "orca_present": true, + "project_real_digest": "0d03c9e72dddfdae", + "user_digest": "3cc9cee71b0b967f", + "virt": "none" + }, + "markers": [ + "container-marker:containerenv", + "distinct-hostname", + "reported-virtualisation:docker" + ], + "orca_is_host_installation": true, + "orca_present": false, + "orca_status_exit_code": 127, + "orca_version": null, + "reason": "the environment answered with its own identity, home and project copy", + "verdict": "ENVIRONMENT" + }, + "phases": [ + { + "commands": [], + "detail": "", + "elapsed_seconds": 0.778, + "finished_at": "2026-09-14T21:59:42Z", + "name": "prepare", + "started_at": "2026-09-14T21:59:42Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 51.609, + "finished_at": "2026-09-14T22:00:34Z", + "name": "create", + "started_at": "2026-09-14T21:59:42Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 0.009, + "finished_at": "2026-09-14T22:00:34Z", + "name": "bootstrap", + "started_at": "2026-09-14T22:00:34Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v orca 2>/dev/null || printf '')\"\norca_release=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id=%s\\n' \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/project-source" + ], + "context": "host", + "elapsed_seconds": 0.122, + "exit_code": 0, + "finished_at": "2026-09-14T22:00:34Z", + "started_at": "2026-09-14T22:00:34Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260914t231500z-aaaaaa-00000002", + "-T", + "--", + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v orca 2>/dev/null || printf '')\"\norca_release=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id=%s\\n' \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/project" + ], + "context": "environment", + "elapsed_seconds": 0.234, + "exit_code": 0, + "finished_at": "2026-09-14T22:00:34Z", + "started_at": "2026-09-14T22:00:34Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260914t231500z-aaaaaa-00000002", + "-T", + "--", + "orca", + "status", + "--json" + ], + "context": "environment", + "elapsed_seconds": 0.208, + "exit_code": 127, + "finished_at": "2026-09-14T22:00:35Z", + "started_at": "2026-09-14T22:00:34Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "the environment resolved orca to the host's own installation at /.config/orca/linux-orca-cli-shim/orca; the run stops rather than driving the host from inside a disposable environment", + "elapsed_seconds": 0.567, + "finished_at": "2026-09-14T22:00:35Z", + "name": "identity", + "started_at": "2026-09-14T22:00:34Z", + "status": "BLOCKED" + }, + { + "commands": [], + "detail": "the environment resolved orca to the host's own installation at /.config/orca/linux-orca-cli-shim/orca; the run stops rather than driving the host from inside a disposable environment", + "elapsed_seconds": null, + "finished_at": null, + "name": "task", + "started_at": null, + "status": "SKIPPED" + }, + { + "commands": [], + "detail": "the environment resolved orca to the host's own installation at /.config/orca/linux-orca-cli-shim/orca; the run stops rather than driving the host from inside a disposable environment", + "elapsed_seconds": null, + "finished_at": null, + "name": "check", + "started_at": null, + "status": "SKIPPED" + }, + { + "commands": [], + "detail": "0 patch line(s) collected", + "elapsed_seconds": 0.012, + "finished_at": "2026-09-14T22:00:35Z", + "name": "collect", + "started_at": "2026-09-14T22:00:35Z", + "status": "OK" + }, + { + "commands": [], + "detail": "11 artifact(s) re-read from the host; 1 unresolved", + "elapsed_seconds": 0.009, + "finished_at": "2026-09-14T22:00:35Z", + "name": "export", + "started_at": "2026-09-14T22:00:35Z", + "status": "OK" + }, + { + "commands": [], + "detail": "every declared per-run resource is gone and every shared one remains", + "elapsed_seconds": 1.57, + "finished_at": "2026-09-14T22:00:36Z", + "name": "cleanup", + "started_at": "2026-09-14T22:00:35Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": null, + "finished_at": null, + "name": "close", + "started_at": "2026-09-14T22:00:36Z", + "status": "OK" + } + ], + "run_id": "20260914T231500Z-aaaaaa-00000002", + "schema_version": 1, + "started_at": "2026-09-14T21:59:42Z", + "status": "BLOCKED", + "tested_revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", + "timeout_seconds": 900, + "versions": { + "configured_revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", + "distrobox": "distrobox: 1.8.2.5", + "git": "git version 2.55.0", + "podman": "podman version 5.8.4", + "project_revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", + "python": "3.14.7", + "qemu_img": "Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers", + "runner": "minimal-cycle", + "virsh": "12.0.0" + }, + "worker": { + "agent": null, + "commands": [], + "detail": "not attempted", + "dispatch_id": null, + "effort": null, + "model": null, + "outcome": null, + "run_id": null, + "status": "SKIPPED" + } +} diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/preflight.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/preflight.json new file mode 100644 index 0000000..78f504e --- /dev/null +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/preflight.json @@ -0,0 +1,36 @@ +{ + "backend": "distrobox", + "findings": [ + { + "detail": "/usr/bin/distrobox", + "name": "distrobox on the path", + "ok": true + }, + { + "detail": "podman", + "name": "container manager on the path", + "ok": true + }, + { + "detail": "docker.io/library/ubuntu:24.04 is present", + "name": "image present locally", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/state", + "name": "state root writable", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/artifacts", + "name": "artifact root writable", + "ok": true + }, + { + "detail": "distrobox mounts into the box and offers no flag to suppress it; the configuration acknowledged this, so the run records a declared compromise rather than claiming isolation", + "name": "host home reachable from the environment", + "ok": true + } + ], + "ok": true +} diff --git a/experiments/minimal-cycle/run-cycle b/experiments/minimal-cycle/run-cycle new file mode 100755 index 0000000..c484165 --- /dev/null +++ b/experiments/minimal-cycle/run-cycle @@ -0,0 +1,12 @@ +#!/usr/bin/env python3 +"""Entry point for the dely minimal-cycle runner.""" + +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +if __name__ == "__main__": + from cycle_runner.cli import main + + raise SystemExit(main()) diff --git a/experiments/minimal-cycle/schema.json b/experiments/minimal-cycle/schema.json index fdaab7c..5f05b76 100644 --- a/experiments/minimal-cycle/schema.json +++ b/experiments/minimal-cycle/schema.json @@ -103,6 +103,7 @@ "environment": {"type": "object"}, "markers": {"type": "array"}, "orca_present": {"type": "boolean"}, + "orca_is_host_installation": {"type": "boolean"}, "orca_version": {"type": ["string", "null"]}, "orca_status_exit_code": {"type": ["integer", "null"]} } diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index 1c5b6cd..7226fd2 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -67,6 +67,7 @@ def __init__( stop_confirmed: bool = True, leave_residue: bool = False, marker: str = "dely-cycle-marker", + orca_path_from_host: bool = False, task_writes_nothing: bool = False, host_project: Path | None = None, ): @@ -84,6 +85,7 @@ def __init__( self.stop_confirmed = stop_confirmed self.leave_residue = leave_residue self.marker = marker + self.orca_path_from_host = orca_path_from_host self.task_writes_nothing = task_writes_nothing self.host_project = host_project self.destroyed = False @@ -117,14 +119,15 @@ def create(self) -> EnvironmentHandle: description={"image": "fake"}, ) + def _host_snapshot_text(self) -> str: + target = str(self.host_project or self.project) + return proc.run(probe.probe_argv(target), timeout=30, context="host").stdout + def _snapshot(self) -> str: if self.identity == "host": # The counterexample: the command really did run on the host, so the # probe answers with the host's own values rather than a stand-in. - target = str(self.host_project or self.project) - return proc.run( - probe.probe_argv(target), timeout=30, context="host" - ).stdout + return self._host_snapshot_text() environment = dict(HOST) environment.update( { @@ -134,12 +137,19 @@ def _snapshot(self) -> str: "home": str(self.home), "container_marker": "containerenv", "project_real": str(self.project), - "orca_path": "/usr/bin/orca" if self.orca_present else "", + "orca_path": self._orca_path(), "orca_version": "1.4.201" if self.orca_present else "", } ) return render(environment) + def _orca_path(self) -> str: + if not self.orca_present: + return "" + if self.orca_path_from_host: + return probe.parse(self._host_snapshot_text()).get("orca_path", "") + return "/usr/bin/orca" + def execute( self, argv: Sequence[str], diff --git a/experiments/minimal-cycle/tests/test_adapter_distrobox.py b/experiments/minimal-cycle/tests/test_adapter_distrobox.py index 64599b7..41c9755 100644 --- a/experiments/minimal-cycle/tests/test_adapter_distrobox.py +++ b/experiments/minimal-cycle/tests/test_adapter_distrobox.py @@ -271,3 +271,30 @@ def test_acknowledging_the_mount_records_it_as_a_declared_compromise(self): self.assertTrue(finding.ok) self.assertIn("acknowledged", finding.detail) self.assertTrue(report.ok, [item.detail for item in report.blockers]) + + +class PreflightResidueTest(AdapterTestCase): + """Preflight inspects; it does not create the run's state.""" + + def test_preflight_leaves_no_per_run_state_behind(self): + adapter = self.make() + adapter.preflight() + self.assertFalse( + adapter.run_state.exists(), + f"preflight created {adapter.run_state}", + ) + + def test_preflight_leaves_no_state_root_entry_behind(self): + adapter = self.make() + before = ( + sorted(p.name for p in adapter.config.state_root.iterdir()) + if adapter.config.state_root.is_dir() + else [] + ) + adapter.preflight() + after = ( + sorted(p.name for p in adapter.config.state_root.iterdir()) + if adapter.config.state_root.is_dir() + else [] + ) + self.assertEqual(before, after) diff --git a/experiments/minimal-cycle/tests/test_cli.py b/experiments/minimal-cycle/tests/test_cli.py new file mode 100644 index 0000000..be94f79 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_cli.py @@ -0,0 +1,132 @@ +"""One command runs the cycle, reports it, and exits with the mapped code.""" + +import io +import json +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import cli, status +from tests.test_config import minimal_document +from tests.test_lifecycle import make_source_repo + + +class CliTestCase(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.repo = make_source_repo(self.root / "src") + self.addCleanup(self._tmp.cleanup) + + def write_config(self, **overrides): + document = minimal_document() + document["artifact_root"] = str(self.root / "artifacts") + document["state_root"] = str(self.root / "state") + document["project"]["source"] = str(self.repo) + document["project"]["revision"] = "main" + document.update(overrides) + path = self.root / "config.json" + path.write_text(json.dumps(document), encoding="utf-8") + return path + + def invoke(self, arguments): + out, err = io.StringIO(), io.StringIO() + code = cli.main(arguments, stdout=out, stderr=err) + return code, out.getvalue(), err.getvalue() + + +class PreflightCommandTest(CliTestCase): + def test_preflight_prints_every_finding(self): + code, out, _ = self.invoke(["preflight", "--config", str(self.write_config())]) + self.assertIn("distrobox on the path", out) + self.assertIn("container manager on the path", out) + self.assertIn( + code, {0, status.exit_code(status.RunStatus.BLOCKED)}, out + ) + + def test_preflight_reports_machine_readable_findings(self): + code, out, _ = self.invoke( + ["preflight", "--config", str(self.write_config()), "--json"] + ) + document = json.loads(out) + self.assertEqual(document["backend"], "distrobox") + self.assertIn("findings", document) + self.assertTrue(all("ok" in finding for finding in document["findings"])) + + def test_a_blocked_backend_exits_with_the_blocked_code(self): + path = self.write_config(backend="vm") + document = json.loads(path.read_text(encoding="utf-8")) + document.pop("distrobox") + document["vm"] = { + "provider": "pulumi-libvirt", + "provider_version": "0.5.3", + "stack_prefix": "dely-cycle", + "base_image": str(self.root / "absent.qcow2"), + "base_image_sha256": "0" * 64, + "guest_user": "cycle", + } + path.write_text(json.dumps(document), encoding="utf-8") + code, out, _ = self.invoke(["preflight", "--config", str(path)]) + self.assertEqual(code, status.exit_code(status.RunStatus.BLOCKED)) + self.assertIn("base image present", out) + + +class ErrorReportingTest(CliTestCase): + def test_a_missing_configuration_is_reported_without_a_traceback(self): + code, _, err = self.invoke(["preflight", "--config", str(self.root / "absent.json")]) + self.assertNotEqual(code, 0) + self.assertNotIn("Traceback", err) + self.assertIn("absent.json", err) + + def test_an_invalid_configuration_names_the_field(self): + path = self.root / "bad.json" + path.write_text(json.dumps({"backend": "kubernetes"}), encoding="utf-8") + code, _, err = self.invoke(["preflight", "--config", str(path)]) + self.assertNotEqual(code, 0) + self.assertIn("backend", err) + self.assertNotIn("Traceback", err) + + def test_no_subcommand_prints_usage(self): + code, _, err = self.invoke([]) + self.assertNotEqual(code, 0) + self.assertIn("usage", err.lower()) + + +class RunCommandTest(CliTestCase): + def test_a_run_identifier_can_be_pinned_for_a_reproducible_directory(self): + path = self.write_config() + code, out, err = self.invoke( + [ + "run", + "--config", + str(path), + "--run-id", + "20260914T221530Z-abc123-0123abcd", + ] + ) + self.assertTrue( + (self.root / "artifacts" / "20260914T221530Z-abc123-0123abcd").is_dir(), + err, + ) + self.assertIn("20260914T221530Z-abc123-0123abcd", out) + + def test_a_malformed_run_identifier_is_refused(self): + code, _, err = self.invoke( + ["run", "--config", str(self.write_config()), "--run-id", "yesterday"] + ) + self.assertNotEqual(code, 0) + self.assertIn("run identifier", err) + + def test_the_run_reports_its_status_and_artifact_directory(self): + code, out, _ = self.invoke( + [ + "run", + "--config", + str(self.write_config()), + "--run-id", + "20260914T221531Z-abc123-0123abcd", + ] + ) + self.assertIn("status", out.lower()) + self.assertIn("artifacts", out) + self.assertNotEqual(code, 0) diff --git a/experiments/minimal-cycle/tests/test_config.py b/experiments/minimal-cycle/tests/test_config.py index 31c2e20..aea5582 100644 --- a/experiments/minimal-cycle/tests/test_config.py +++ b/experiments/minimal-cycle/tests/test_config.py @@ -1,6 +1,7 @@ """The run configuration is explicit, backend-aware, and never carries a secret.""" import copy +import importlib.util import json import tempfile import unittest @@ -225,3 +226,33 @@ def test_the_rendered_document_reloads_unchanged(self): loaded = config.from_document(minimal_document()) reloaded = config.from_document(loaded.to_document()) self.assertEqual(reloaded.to_document(), loaded.to_document()) + + +class ExampleConfigurationTest(unittest.TestCase): + """The shipped examples have to satisfy the validator they document.""" + + def examples(self): + root = Path(__file__).resolve().parent.parent + return sorted(root.glob("config.*.example.yaml")) + + def test_both_backends_ship_an_example(self): + names = {path.name for path in self.examples()} + self.assertEqual( + names, {"config.distrobox.example.yaml", "config.vm.example.yaml"} + ) + + def test_every_example_loads(self): + for path in self.examples(): + with self.subTest(path=path.name): + if importlib.util.find_spec("yaml") is None: # pragma: no cover + self.skipTest("no yaml parser on this host") + self.assertIn(config.load(path).backend, config.BACKENDS) + + def test_no_example_carries_a_credential(self): + for path in self.examples(): + with self.subTest(path=path.name): + from cycle_runner import redact + + self.assertFalse( + redact.carries_credential_shape(path.read_text(encoding="utf-8")) + ) diff --git a/experiments/minimal-cycle/tests/test_counterexamples.py b/experiments/minimal-cycle/tests/test_counterexamples.py new file mode 100644 index 0000000..ca8fa05 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_counterexamples.py @@ -0,0 +1,29 @@ +"""The counterexample table has to still apply to the code it describes.""" + +import unittest + +import counterexamples + + +class TableTest(unittest.TestCase): + def test_every_counterexample_still_applies_to_the_code(self): + for case in counterexamples.CASES: + with self.subTest(case=case.name): + self.assertTrue( + counterexamples.applicable(case), + f"{case.name} no longer matches {case.path}", + ) + + def test_every_counterexample_names_at_least_one_instrument(self): + for case in counterexamples.CASES: + with self.subTest(case=case.name): + self.assertTrue(case.instruments) + + def test_every_counterexample_changes_something(self): + for case in counterexamples.CASES: + with self.subTest(case=case.name): + self.assertNotEqual(case.original, case.replacement) + + def test_the_names_are_unique(self): + names = [case.name for case in counterexamples.CASES] + self.assertEqual(len(names), len(set(names))) diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index 073585b..c978b5c 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -1,6 +1,7 @@ """The shared lifecycle: export before destroy, and never a silent host fallback.""" import json +import shutil import subprocess import tempfile import unittest @@ -256,3 +257,26 @@ def test_the_artifact_directory_is_named_by_the_run(self): _, outcome = self.run_cycle() self.assertEqual(outcome.artifact_dir, self.artifacts / RUN_ID) self.assertTrue(ids.is_run_id(outcome.run_result.run_id)) + + +class HostOrcaTest(CycleTestCase): + """Reproduces a real Distrobox run: the box saw the host's own orca. + + Distrobox mounts the host home and preserves PATH, so `command -v orca` + resolved to the host launcher. The environment verdict was correct and the + run still had no Orca of its own. + """ + + @unittest.skipUnless(shutil.which("orca"), "no orca on this host to resolve to") + def test_an_orca_that_is_the_hosts_own_blocks_before_the_worker(self): + adapter, outcome = self.run_cycle(orca_path_from_host=True) + self.assertEqual(outcome.run_result.status, status.RunStatus.BLOCKED) + self.assertNotIn("worker", adapter.calls) + self.assertTrue(outcome.run_result.identity.orca_is_host_installation) + self.assertIn("host", outcome.run_result.failure_classification.lower()) + + @unittest.skipUnless(shutil.which("orca"), "no orca on this host to resolve to") + def test_that_run_still_exports_and_cleans_up(self): + adapter, outcome = self.run_cycle(orca_path_from_host=True) + self.assertEqual(outcome.run_result.export.status, status.ExportStatus.CONFIRMED) + self.assertEqual(outcome.run_result.cleanup.status, status.CleanupStatus.DESTROYED) diff --git a/experiments/minimal-cycle/tests/test_probe.py b/experiments/minimal-cycle/tests/test_probe.py index 0547624..434dcaf 100644 --- a/experiments/minimal-cycle/tests/test_probe.py +++ b/experiments/minimal-cycle/tests/test_probe.py @@ -17,7 +17,7 @@ def host_snapshot(**overrides): "uname": "linux workshop amd64", "container_marker": "", "virt": "none", - "orca_path": "/usr/bin/orca", + "orca_path": "/opt/host/bin/orca", "orca_version": "1.4.201", "project_real": "/home/someone/code/under-test", } @@ -32,6 +32,7 @@ def container_snapshot(**overrides): home="/var/tmp/cycle-state/run/home", container_marker="containerenv", project_real="/var/tmp/cycle-state/run/home/project", + orca_path="/usr/bin/orca", ) snapshot.update(overrides) return snapshot @@ -180,3 +181,59 @@ def test_an_absent_project_directory_resolves_to_nothing(self): probe.probe_argv("/nonexistent-cycle-path"), timeout=30, context="host" ) self.assertEqual(probe.parse(outcome.stdout)["project_real"], "") + + +class OrcaPresenceTest(unittest.TestCase): + """Observed on a real Distrobox run: `command -v orca` found the host's own. + + Distrobox mounts the host home and preserves PATH, so a shim on the host + resolves inside the box. It is not an Orca the environment has. + """ + + def test_an_orca_resolved_to_the_host_installation_is_not_present(self): + record = verdict( + container_snapshot( + orca_path=host_snapshot()["orca_path"], orca_version="" + ) + ) + self.assertFalse(record.orca_present) + self.assertTrue(record.orca_is_host_installation) + + def test_an_orca_that_reports_no_version_inside_is_not_present(self): + record = verdict( + container_snapshot(orca_path="/usr/bin/orca", orca_version="") + ) + self.assertFalse(record.orca_present) + self.assertFalse(record.orca_is_host_installation) + + def test_an_orca_of_its_own_is_present(self): + record = verdict( + container_snapshot(orca_path="/usr/bin/orca", orca_version="1.4.201") + ) + self.assertTrue(record.orca_present) + self.assertFalse(record.orca_is_host_installation) + + def test_the_gate_names_the_host_installation_when_that_is_what_it_found(self): + allowed, reason = probe.may_continue( + verdict( + container_snapshot( + orca_path=host_snapshot()["orca_path"], orca_version="" + ) + ) + ) + self.assertFalse(allowed) + self.assertIn("host", reason.lower()) + self.assertIn(host_snapshot()["orca_path"], reason) + + def test_the_gate_names_a_version_that_never_came_back(self): + allowed, reason = probe.may_continue( + verdict(container_snapshot(orca_path="/usr/bin/orca", orca_version="")) + ) + self.assertFalse(allowed) + self.assertIn("version", reason.lower()) + + def test_the_gate_opens_for_an_orca_the_environment_carries(self): + allowed, _ = probe.may_continue( + verdict(container_snapshot(orca_path="/usr/bin/orca", orca_version="1.4.201")) + ) + self.assertTrue(allowed) diff --git a/experiments/minimal-cycle/tests/test_redact.py b/experiments/minimal-cycle/tests/test_redact.py index 7b875cf..8523479 100644 --- a/experiments/minimal-cycle/tests/test_redact.py +++ b/experiments/minimal-cycle/tests/test_redact.py @@ -118,3 +118,33 @@ def test_a_clean_document_reports_secret_free(self): def test_a_document_carrying_a_token_is_not_secret_free(self): self.assertFalse(redact.looks_secret_free(f"key {NEVER_SEEN}")) + + +class CredentialShapeTest(unittest.TestCase): + """A narrower question than redaction: does this text carry a value?""" + + def test_a_token_is_a_credential_shape(self): + self.assertTrue(redact.carries_credential_shape(f"key {NEVER_SEEN}")) + + def test_a_private_key_block_is_a_credential_shape(self): + block = ( + "-----BEGIN OPENSSH PRIVATE KEY-----\n" + "qwertyuiopasdfghjklzxcvbnm\n" + "-----END OPENSSH PRIVATE KEY-----" + ) + self.assertTrue(redact.carries_credential_shape(block)) + + def test_a_variable_name_is_not_a_credential_shape(self): + self.assertFalse( + redact.carries_credential_shape("token_env: CLAUDE_CODE_OAUTH_TOKEN") + ) + + def test_a_bearer_header_is_a_credential_shape(self): + self.assertTrue( + redact.carries_credential_shape("authorization: Bearer abcdefghijklmnop") + ) + + def test_ordinary_configuration_text_is_not_a_credential_shape(self): + self.assertFalse( + redact.carries_credential_shape("mode: existing_login\nreference: workshop") + ) From e02505111d69f6e6baa0439047481d63640f6fdd Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 00:04:17 +0200 Subject: [PATCH 08/42] Reconcile the acceptance table into the runner's own documentation and delete the transient plan. --- docs/_plans/2026-09-14-shared-cycle-runner.md | 355 ------------------ docs/decisions.md | 27 ++ experiments/minimal-cycle/README.md | 43 ++- 3 files changed, 64 insertions(+), 361 deletions(-) delete mode 100644 docs/_plans/2026-09-14-shared-cycle-runner.md diff --git a/docs/_plans/2026-09-14-shared-cycle-runner.md b/docs/_plans/2026-09-14-shared-cycle-runner.md deleted file mode 100644 index 6c1643c..0000000 --- a/docs/_plans/2026-09-14-shared-cycle-runner.md +++ /dev/null @@ -1,355 +0,0 @@ -# Plan — one shared Python runner drives one minimal proof cycle on two backends - -Decision record: `docs/decisions.md` — "One shared cycle runner, two backend -adapters, export before cleanup". - -**Baseline:** `6688d475413f80c862e131f1ebc93cf27a5fc830`. - -**Spec:** `/home/paczos/Documents/Warsztat/myDely/docs/projekt-srodowiska-testowego.md` -with `projekt-wspolnej-petli.md`, `projekt-lokalnej-petli-distrobox.md`, -`projekt-lokalnej-petli-vm.md`, and `plan-srodowiska-testowego.md`. Those files -live outside this repository and are not copied into it. - -## Goal - -`experiments/minimal-cycle/run-cycle` accepts one configuration naming -`backend: distrobox|vm`, mints a `run_id`, creates a fresh identifiable -environment, proves that Orca runs inside that environment rather than on the -host, runs exactly one Claude Code worker through Orca on a clean copy of the -project at a pinned revision, runs exactly one independent check, exports -status, elapsed time, diff, logs and result to a host directory, confirms that -export by re-reading every required artifact, and only then destroys per-run -resources while leaving shared images and bases intact. Timeout, cancellation -and error keep the artifacts and classify the run; they never fabricate -success. - -Out of reach in this plan: the separate acceptance test the spec reserves — -a full green cycle on either backend on a host that actually carries Orca, -Claude Code and a built tool image. This host has neither `pulumi` nor an Orca -package inside any container image, so the plan delivers the runner and proves -its rails, not a green backend run. - -## Allowed scope - -``` -experiments/minimal-cycle/** -docs/decisions.md -docs/_plans/2026-09-14-shared-cycle-runner.md -.gitignore -``` - -Command that produced the list: `git ls-files` at baseline showed no -`experiments/` path and no `.gitignore`, so every path under -`experiments/minimal-cycle/` is created by this plan. `docs/decisions.md` is -the repository's durable decision store named by `AGENTS.md`. No colocated -test of an existing source file applies: the repository carries no Python -before this plan (`git ls-files '*.py'` is empty). `tests/contracts.sh` owns -repository shape, not this runner, and yields nothing here. - -## Forbidden scope - -`skills/**` — the delivery contract does not change, so neither plugin -manifest version nor the pin in `tests/contracts.sh` moves. -`tests/contracts.sh` — it checks the dely package's structural contract; this -runner is not part of that contract and adding it would couple the two. -`README.md` — the repository README is the newcomer install path for the dely -plugin; an experiment directory is not an install step. -`.github/workflows/contracts.yml` — the runner's own suite needs a Python -step and a decision about running it in CI that this plan does not take. - -## Execution envelope - -Protected dirty paths: none. `git status --short` was empty at baseline. - -Branch, base, remote, and pull-request target: branch -`kacperpaczos/shared-python-runner-distrobox`, base `main`, remote `origin`, -pull request to `origin/main`. No merge. - -Resolved phase pins: `AGENTS.md` pins `implement` to Cursor Agent CLI -(`cursor-grok-4.6-high`, default effort) and `review` to Codex CLI -(`gpt-5.6-sol`, high). `command -v cursor-agent` is empty on this host, so the -pinned implement harness cannot be launched and the delivery run cannot -dispatch as pinned. This is recorded as the reason the phases were not -dispatched, not worked around by substituting a harness. - -Authority: this plan may branch, commit only its own owned paths, run gates, -push the named branch, and open or update the named pull request. It may not -merge, force-push, stash, reset, clean, or edit anything outside owned scope. - -## Global constraints - -- Python standard library only at runtime. `yaml` is used when importable and - a `.json` configuration is always accepted, so the runner never requires a - package install. -- Repository artifacts are English. -- No file the repository tracks may contain a capital letter immediately - followed by digits at a token boundary, or the string `pace` followed by at - most one character and then `id`. Both are repository closure gates and both - are lexical, so identifiers such as `namespace_id` are forbidden. -- No secret value, and no absolute path to a credential file, is written to a - manifest, receipt, log, image, seed, stack state, or this repository. - -## Tasks - -### 1. The run contract is a typed, validated, secret-free artifact - -**Behaviour.** A configuration file loads into a validated `RunConfig`, a -`run_id` is minted that carries no secret, statuses are a closed set, log text -passes through a pattern redactor, and a manifest is emitted that satisfies -`schema.json`. A configuration naming an absolute allowlist path, an unknown -backend, a non-positive timeout, or an artifact root inside the per-run state -is rejected before anything is created. - -**Direction.** `config.py` holds frozen dataclasses and one `load` that -accepts `.json` always and `.yaml`/`.yml` when `yaml` imports; a missing -`yaml` for a `.yaml` file is a clear error naming the `.json` alternative, not -a silent partial parse. `ids.py` mints -`YYYYMMDDTHHMMSSZ--` where `` is a truncated -hash of the host name rather than the host name itself, and `` comes -from `secrets.token_hex`. `status.py` holds `RunStatus`, `CleanupStatus`, -`ExportStatus`, `PhaseStatus` and the process exit code mapping. -`redact.py` matches by pattern — bearer headers, `sk-`-style keys, `ghp_`, -`xox` Slack forms, compact web-token triples, private-key blocks, and any -`key=value`/`"key": "value"` whose key names a token, secret, password, api -key, cookie, or authorization header — plus any absolute path ending in a -known credential file name. `manifest.py` builds the document and validates it -against `schema.json` with a small in-repo required-field checker, because -`jsonschema` is not in the standard library. - -**Files.** Create `experiments/minimal-cycle/cycle_runner/{__init__,config, -ids,status,redact,hashing,manifest}.py`, `experiments/minimal-cycle/schema.json`, -and their tests under `experiments/minimal-cycle/tests/`. - -**Focused verification.** -`cd experiments/minimal-cycle && python3 -m unittest discover -s tests -t . -v` -fails with the redaction, allowlist and manifest tests red before the module -exists and green after. - -**Document impact.** `experiments/minimal-cycle/README.md` owns the -configuration reference; it is written in task 6 and must match the fields this -task validates. - -### 2. The shared lifecycle exports before it destroys, and classifies rather than fabricates - -**Behaviour.** `run_cycle(config, adapter)` walks prepare, create, bootstrap, -identity, task, check, collect, export, cleanup, close and returns a -`RunResult` carrying the same fields for every backend. Export is confirmed by -re-reading each required artifact from the host path and recomputing its -digest. Cleanup runs only after that confirmation; when export is not -confirmed, no destroy is attempted and `cleanup_status` is `RESIDUE` with the -reason. A task that exceeds `timeout_seconds` collects status and logs, -requests a confirmed stop, verifies it, and settles `TIMEOUT` with every -artifact already exported. An identity verdict that matches the host settles -`BLOCKED` and never runs the task. - -**Direction.** `proc.py` runs argv with a deadline, captures stdout and stderr -separately, and returns a `CommandResult` that records argv, exit code, -timings and whether the deadline fired; every captured stream is redacted -before it reaches disk. `hostinfo.py` snapshots the host before and after. -`probe.py` carries one POSIX-shell probe emitting `key=value` lines, a parser, -and `verdict(host, env, expectations)` which rejects a snapshot that carries -no environment marker and repeats the host's own name, machine identity and -home. `project.py` exports the pinned revision with `git archive`, keeps a -pristine baseline, and computes the patch on the host with `difflib` over the -fetched tree, so no diff tool is required inside the environment. -`export.py` writes, then verifies, then writes the receipt. `cleanup.py` -refuses any target that is not a declared per-run resource and asserts the -declared shared resources still exist afterwards. `lifecycle.py` sequences the -phases and owns the classification. - -**Files.** Create `experiments/minimal-cycle/cycle_runner/{proc,hostinfo,probe, -project,export,cleanup,lifecycle,result}.py` and -`experiments/minimal-cycle/tests/{fakes.py,test_probe.py,test_project.py, -test_export.py,test_cleanup.py,test_lifecycle.py}`. - -**Focused verification.** The suite above, with `test_lifecycle.py` driving a -fake adapter through the settled, blocked, timeout and export-unconfirmed -paths and asserting the order of adapter calls. - -**Document impact.** The README's lifecycle section and `schema.json`'s -`cleanup` and `export` blocks must name exactly the states this task returns. - -### 3. The Distrobox adapter creates a fresh box with its own home and never mounts the host home - -**Behaviour.** `DistroboxAdapter` writes a Distrobox Assemble manifest into the -per-run directory, creates the box from it, executes commands inside it, -moves trees in and out, stops it, removes it, and reports which resources were -per-run. The generated manifest names a per-run home and only the mounts the -configuration lists; it never names the host home directory, and a -configuration that asks for one is rejected. - -**Direction.** Use `distrobox assemble create --file ` and -`distrobox assemble rm --file `, with `distrobox enter --` -for execution and `distrobox list` for existence proof, as those are the -documented general interfaces. Because the per-run home is a host directory, -`put_tree` and `fetch_tree` are host copies and need no transport. Do not emit -`--unshare-all`; record in the README that desktop integration is a deliberate -prototype trade-off rather than a sandbox. - -**Files.** Create `experiments/minimal-cycle/cycle_runner/adapters/{__init__, -base,distrobox}.py` and `experiments/minimal-cycle/tests/test_adapter_distrobox.py`. - -**Focused verification.** Tests assert the rendered manifest text, that a -configuration naming the host home is rejected, and that the destroy plan lists -only per-run paths. A dry-run integration test runs -`distrobox assemble create --dry-run --file ` when `distrobox` is on -`PATH` and skips otherwise. - -**Document impact.** README backend section for Distrobox. - -### 4. The VM adapter builds per-run domain, overlay and seed over an unmodified base - -**Behaviour.** `VmAdapter` renders a Pulumi Python program and stack settings -into a per-run directory, records the pinned provider and version, renders a -cloud-init NoCloud seed that carries no credential, creates the overlay as a -qcow2 image backed by the preserved base, brings the stack up, executes -commands inside the guest over the configured transport, moves trees, destroys -the stack, and reports per-run versus shared resources. The base image is -never opened for writing and never appears in a destroy plan. - -**Direction.** The adapter refuses to run when `pulumi` is absent, when the -base image digest does not match the pinned value, or when the provider -version is unpinned — each as a preflight failure naming the missing fact, not -a guess. Overlay creation uses `qemu-img create -f qcow2 -b -F qcow2`, -which the tool documents, and is never run against an image an active domain -holds. The seed is `user-data` plus `meta-data` on a `cidata` volume. - -**Files.** Create `experiments/minimal-cycle/cycle_runner/adapters/{vm.py, -vm_program.py,vm_seed.py}` and `experiments/minimal-cycle/tests/test_adapter_vm.py`. - -**Focused verification.** Tests assert the seed carries none of the auth -values, that the destroy plan excludes the base image, that a wrong base digest -blocks, and that the overlay command names the base as a backing file rather -than as the output. - -**Document impact.** README backend section for the virtual machine, including -that the tool image is built separately and the runner only consumes it. - -### 5. Auth reaches the environment by one declared method and leaves no secret behind - -**Behaviour.** `bootstrap_auth(config, adapter)` implements exactly the -configured mode, writes an `auth-receipt.json` naming the method, the -non-secret reference, the transfer target and the observed status, and never -records a secret value, length, prefix or absolute source path. `existing_login` -copies only relative allowlist entries into the per-run configuration -directory with owner-only permissions; `short_lived_token` passes a value -through the process environment for the run and never writes it; `api_key_helper` -records the helper contract and passes no value at all. The worker launcher -builds the Orca argv from pinned values, records it, and waits for the -worker's own settling message. - -**Direction.** Auth material is removed from the per-run configuration -directory before cleanup, and the removal is verified. The launcher composes -`orca orchestration run-create`, `worker-start` with `--agent`, `--model`, -`--effort` and `--json`, and `check --wait --types worker_done,escalation,question`, -which are the flags this host's Orca reports. It writes the worker prompt to a -file inside the environment rather than inlining it as a shell argument. - -**Files.** Create `experiments/minimal-cycle/cycle_runner/{auth,worker}.py`, -`experiments/minimal-cycle/fixtures/evidence-task/prompt.md`, and -`experiments/minimal-cycle/tests/{test_auth.py,test_worker.py}`. - -**Focused verification.** Tests feed a known token through every mode and -assert it appears in no receipt, manifest, rendered seed or log, and that the -launcher's argv carries the pinned model and effort. - -**Document impact.** README auth section, stating that the runner verifies the -method and status and never reads a secret into an artifact. - -### 6. One command runs, reports, and proves the rails on this host - -**Behaviour.** `run-cycle preflight` and `run-cycle run` are usable from the -experiment directory, exit with the mapped code, and write the artifact tree -the spec names. A real preflight on this host reports the Distrobox backend -usable and the virtual-machine backend blocked on the absent `pulumi`, and a -real Distrobox cycle creates a fresh box, proves it is not the host, stops at -the Orca identity gate because the image carries no Orca, exports every -artifact, and destroys only the per-run box and home. - -**Direction.** The command-line interface is `argparse` with subcommands -`preflight`, `run`, and `selftest`. Evidence from the real runs is committed -under `experiments/minimal-cycle/evidence/` with digests redacted where -required; the artifact root itself stays untracked through `.gitignore`. - -**Files.** Create `experiments/minimal-cycle/{run-cycle,README.md, -config.distrobox.example.yaml,config.vm.example.yaml}`, -`experiments/minimal-cycle/cycle_runner/cli.py`, -`experiments/minimal-cycle/evidence/**`, and `.gitignore`. - -**Focused verification.** The real runs above, with their exported manifests -quoted in the handoff. - -**Document impact.** `docs/decisions.md` gains the settled record; this plan is -deleted in the release commit. - -## Acceptance - -| Requirement | Instrument | Counterexample | Observed red | -| --- | --- | --- | --- | -| No silent host fallback: a probe result equal to the host's own snapshot is rejected | `tests/test_probe.py::test_host_snapshot_is_rejected` | A verdict that only asserts `command -v orca` returned zero and an exit status of zero — present, runs, returns a pass, and passes identically when the probe ran on the host | | -| Export is confirmed before any destroy | `tests/test_lifecycle.py::test_no_destroy_when_export_unconfirmed` | A lifecycle that destroys in a `finally` block and then reports the export status — it runs, returns a pass on the happy path, and still deletes evidence when export fails | | -| The receipt proves bytes on the host, not bytes in memory | `tests/test_export.py::test_receipt_fails_when_artifact_changes_after_write` | A receipt built from the buffer that was written, which passes every happy-path test while never re-reading the file | | -| Cleanup removes per-run resources only | `tests/test_cleanup.py::test_shared_base_survives_cleanup` | A cleanup that removes the overlay's parent directory, which passes when the base lives elsewhere and destroys it when it does not | | -| Redaction is by pattern, not by known literal | `tests/test_redact.py::test_token_never_seen_before_is_redacted` | A redactor seeded with the tokens the run knows about, which passes every test that reuses those tokens | | -| Timeout keeps the artifacts and classifies the run | `tests/test_lifecycle.py::test_timeout_exports_before_stop_and_settles_timeout` | A lifecycle that raises on the deadline and lets the caller clean up, which passes a test asserting only the returned status | | -| `run_id` carries no secret and stays unique and sortable | `tests/test_ids.py::test_run_id_excludes_auth_reference_and_host_name` | An id that appends `auth.reference` for traceability, which is unique, sortable, and leaks | | -| The manifest carries every required field on every path, including failures | `tests/test_manifest.py::test_required_fields_present_on_blocked_run` | A manifest writer that fills `cleanup` and `check` only when they ran, which validates on the settled path | | -| Both adapters satisfy one result contract | `tests/test_adapter_contract.py::test_every_adapter_implements_the_interface` | An adapter that returns a plain dictionary with the same keys, which reads the same in a manifest and drops the declared per-run resource list | | -| The Distrobox manifest never mounts the host home | `tests/test_adapter_distrobox.py::test_host_home_mount_is_refused` | An adapter that passes the configured mounts through unchanged, which is correct for every configuration that does not ask for the host home | | -| The virtual-machine base image is never written and never destroyed | `tests/test_adapter_vm.py::test_base_image_is_backing_file_and_not_in_destroy_plan` | An overlay command that names the base as the output path, which produces a working disk and silently consumes the shared base | | -| No credential reaches the seed, the stack settings, or the receipt | `tests/test_auth.py::test_no_auth_material_in_any_rendered_artifact` | A receipt that records the token's length and first four characters for debugging, which contains no full secret and still discloses one | | -| Preflight blocks instead of inventing a path | `experiments/minimal-cycle/run-cycle preflight --config config.vm.example.yaml` on this host | A preflight that reports the backend usable because the adapter module imported, which passes on a host with no `pulumi` at all | | -| The whole cycle runs for real and stops at the identity gate rather than on the host | `experiments/minimal-cycle/run-cycle run --config ` on this host | A run that finds no Orca inside the box and uses the host's `orca`, which produces a complete green manifest | | - -**Cannot be observed:** that Orca actually starts, shows a window, and drives a -Claude Code worker inside either backend. No image on this host carries Orca, -`pulumi` is absent, and no tool image has been built, so every instrument here -stops at the identity gate or below it. The separate acceptance test the spec -reserves is the only thing that can observe those, and this plan does not -claim them. - -## Stop conditions - -`BLOCKED` rather than a workaround when: the pinned implement or review -harness is absent and a substitute would have to be chosen; a real cycle would -need a credential to be read; a backend would have to fall back to the host to -produce a green result; or `pulumi`, an Orca package, or a verified base image -would have to be installed on this host to finish a task. - -Assumptions that hold for one environment and are not verified for the others: -the Distrobox path is exercised against rootless Podman with -`docker.io/library/ubuntu:24.04` already present locally; nothing here verifies -Docker as the container backend, another base image, or another architecture. -The virtual-machine path is exercised only through rendering and refusal — -`qemu:///session` answers and `/dev/kvm` exists, but no domain is created. - -## Closure gates - -``` -git diff --check -bash -n tests/contracts.sh -jq -e . plugin.json .claude-plugin/plugin.json .claude-plugin/marketplace.json .codex-plugin/plugin.json .cursor-plugin/plugin.json >/dev/null -bash tests/contracts.sh -test "$(wc -l < tests/contracts.sh)" -le 280 -test ! -e git-hooks/pre-push -test ! -e docs/delivery-log.md -test ! -e docs/findings.md -test ! -e docs/harness-surface.md -test ! -e docs/options.md -test ! -e docs/_plans/2026-08-24-automation-first-dely-design.md -test ! -e bin/delivery-doctor -test ! -e bin/delivery-evidence -test ! -e hooks/hooks.json -test ! -e hooks/grok-hooks.json.template -test ! -e hooks/post-tool-journal.sh -test ! -e hooks/session-start-context.sh -git grep -Ei 'pace.?id' -- . ':!docs/_plans' && exit 1 || true -git grep -E '(^|[^A-Za-z0-9])[A-Z][0-9]+[a-z]?([^A-Za-z0-9]|$)' -- . ':!docs/_plans' && exit 1 || true -``` - -All run from the repository root. The runner's own suite runs from -`experiments/minimal-cycle`: - -``` -python3 -m unittest discover -s tests -t . -v -``` diff --git a/docs/decisions.md b/docs/decisions.md index 50625c7..2e96b8d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -69,6 +69,17 @@ its shape, not because it was on a list, and an absolute path to a credential file is replaced before it reaches a log, a manifest, a seed, a stack setting or this repository. +Where a backend's own behaviour contradicts one of those rails, the runner +neither hides it nor works around it. It asks the tool what it would really do, +and turns the answer into a preflight finding that blocks until the +configuration records the compromise deliberately. Two exist. Distrobox mounts +the invoking user's home into every box and offers no flag to suppress it, so +`distrobox.accept_host_home_mount` has to say so before a run starts. The +libvirt provider's resource schema is not assumed from an example, so +`vm.provider_schema_verified` has to record that someone checked the rendered +program against the pinned version. An acknowledgement is recorded in the +manifest; it is not a way of making the fact go away. + #### Alternatives considered One backend only, deferring the other. It would have halved the work and @@ -109,6 +120,22 @@ This decision does not make Distrobox a sandbox and does not claim the two backends are equivalent. It claims one lifecycle can drive both and that the evidence survives every way the cycle can end. +Running it already moved one rail. The first real Distrobox cycle reached the +identity gate, was correctly judged to be in the environment, and was allowed +through because `command -v orca` resolved inside the box. It resolved to the +*host's* launcher: Distrobox mounts the host home and preserves `PATH`, so a +host installation answers from inside a container that has none. That run failed +later, at the launch, for an unrelated reason. Had the launcher been one that +works from inside a container, it would have driven the host and produced a +complete green manifest for work the environment never did. + +Presence of a command is therefore not presence of an installation. Orca counts +as present only when the path found inside differs from the one the host probe +found and the binary reports a version there. The point generalises past Orca: +anything a disposable environment inherits from the host through a mount or +through `PATH` can answer a presence check without being present, and a check +that only asks whether a name resolves cannot tell the two apart. + #### Non-goals Measuring start-up time, model quality, tester quality, cost, or any task diff --git a/experiments/minimal-cycle/README.md b/experiments/minimal-cycle/README.md index c61168a..0deab78 100644 --- a/experiments/minimal-cycle/README.md +++ b/experiments/minimal-cycle/README.md @@ -176,10 +176,41 @@ length, or a prefix into any artifact. An absolute path to a credential file is refused in the configuration outright, and a configuration key whose name means "secret" is refused with it. -## What this does not prove +## Acceptance -Nothing here observes Orca starting inside either backend, a window bound to -that instance, a Claude Code worker driven through it, or a login surviving a -run. Those need a host that carries Orca in the chosen image, a built tool image, -and a pinned libvirt provider. `evidence/` records what one host actually did, -including where it stopped. +Each rail below is proved by an instrument that rejects an implementation which +is present, runs, and returns a pass — not one that is merely absent. +`python3 counterexamples.py` applies each wrong implementation in turn and +checks that its instrument goes red; `python3 counterexamples.py --list` prints +the table with the tests each row runs. The recorded sweep is in +`evidence/counterexamples.txt`. + +| Requirement | Instrument | Evidence | +| --- | --- | --- | +| A probe result equal to the host's own snapshot is rejected | `counterexamples.py` case `no-host-fallback` | `evidence/counterexamples.txt` | +| An orca resolving to the host's installation does not count as present | case `orca-is-the-environments-own` | `evidence/run-blocked-on-host-orca/` | +| Cleanup runs only after a confirmed export | case `export-before-destroy` | `evidence/counterexamples.txt` | +| Cleanup runs only after a confirmed stop | case `stop-must-be-confirmed` | `evidence/counterexamples.txt` | +| Evidence is collected and exported before anything is destroyed | case `destroy-after-export` | `evidence/counterexamples.txt` | +| The receipt proves bytes on the host, not bytes in memory | case `receipt-is-a-re-read` | `evidence/run-blocked-on-host-orca/export-receipt.json` | +| A per-run path containing a shared resource is refused | case `cleanup-only-per-run` | `evidence/counterexamples.txt` | +| Redaction catches a credential this run has never seen | case `redaction-by-shape` | `evidence/counterexamples.txt` | +| The manifest carries every required field on every path | case `manifest-required-fields` | `evidence/run-blocked-on-host-orca/manifest.json` | +| The run identifier discloses neither the host name nor the auth reference | case `run-id-carries-no-name` | `evidence/counterexamples.txt` | +| A secret-shaped value under a dull key is refused in the configuration | case `config-refuses-a-secret` | `evidence/counterexamples.txt` | +| No credential value, length or prefix reaches a receipt | case `auth-leaves-no-material` | `evidence/run-blocked-on-host-orca/auth-receipt.json` | +| An unacknowledged host-home mount blocks the run | case `host-home-mount-acknowledged` | `evidence/preflight-distrobox.txt` | +| Preflight inspects without creating the run's state | case `preflight-leaves-no-residue` | `evidence/counterexamples.txt` | +| The preserved base image is never the overlay's output path | case `base-is-only-a-backing-file`, and a real `qemu-img` backing-chain test | `tests/test_adapter_vm.py` | +| An unverified provider schema blocks the machine backend | case `provider-schema-verified` | `evidence/preflight-vm.txt` | +| Preflight blocks rather than inventing a path | `./run-cycle preflight` on a host with no `pulumi` and no tool image | `evidence/preflight-vm.txt` | +| A real cycle stops at the identity gate rather than using the host | `./run-cycle run` on this host | `evidence/run-blocked-on-host-orca/manifest.json` | + +## What no instrument here observes + +Orca starting inside either backend, a window bound to that instance, a Claude +Code worker driven through it, a login surviving a run, or any domain being +created at all. Those need a host that carries Orca in the chosen image, a built +tool image, and a pinned libvirt provider. They are the separate acceptance test +this work does not claim to have passed; `evidence/README.md` says exactly where +this host stopped. From 9d1708fd782ee615e09d22b8a550a97bee09c73f Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 00:08:11 +0200 Subject: [PATCH 09/42] Redact values forwarded over the machine transport, and state the exposure the ssh path still has. --- experiments/minimal-cycle/README.md | 14 ++++++++++ experiments/minimal-cycle/counterexamples.py | 8 ++++++ .../evidence/counterexamples.txt | 3 ++- .../tests/test_adapter_distrobox.py | 16 ++++++----- .../minimal-cycle/tests/test_adapter_vm.py | 27 +++++++++++++++++++ 5 files changed, 61 insertions(+), 7 deletions(-) diff --git a/experiments/minimal-cycle/README.md b/experiments/minimal-cycle/README.md index 0deab78..50d2c7d 100644 --- a/experiments/minimal-cycle/README.md +++ b/experiments/minimal-cycle/README.md @@ -108,6 +108,11 @@ $artifact_root// cleanup.json what was removed, what was kept, and how that was checked ``` +`manifest.json`, `cleanup.json` and `host-after.json` are written after cleanup, +so they are deliberately outside the export receipt: the receipt proves what the +host held *before* anything was destroyed, which is the only moment at which +that proof is worth anything. + Every stream is redacted on the way out. Redaction matches shapes — bearer headers, key prefixes, compact web tokens, private-key blocks, absolute paths to credential files, and any assignment whose key names a secret — so a credential @@ -176,6 +181,14 @@ length, or a prefix into any artifact. An absolute path to a credential file is refused in the configuration outright, and a configuration key whose name means "secret" is refused with it. +One limit is worth stating rather than glossing. On Distrobox a forwarded value +never appears in a command line: the container manager is given the variable's +*name* and reads the value from the runner's own environment. On the machine +backend the transport is ssh, and the value is placed on the guest command line, +so it is visible in the guest's process table for the life of that command. It +is redacted from everything the runner captures, and the guest is destroyed +after the run, but that is a bounded exposure and not an absence of one. + ## Acceptance Each rail below is proved by an instrument that rejects an implementation which @@ -202,6 +215,7 @@ the table with the tests each row runs. The recorded sweep is in | An unacknowledged host-home mount blocks the run | case `host-home-mount-acknowledged` | `evidence/preflight-distrobox.txt` | | Preflight inspects without creating the run's state | case `preflight-leaves-no-residue` | `evidence/counterexamples.txt` | | The preserved base image is never the overlay's output path | case `base-is-only-a-backing-file`, and a real `qemu-img` backing-chain test | `tests/test_adapter_vm.py` | +| A value forwarded into the guest is redacted from captured output | case `forwarded-value-is-redacted` | `evidence/counterexamples.txt` | | An unverified provider schema blocks the machine backend | case `provider-schema-verified` | `evidence/preflight-vm.txt` | | Preflight blocks rather than inventing a path | `./run-cycle preflight` on a host with no `pulumi` and no tool image | `evidence/preflight-vm.txt` | | A real cycle stops at the identity gate rather than using the host | `./run-cycle run` on this host | `evidence/run-blocked-on-host-orca/manifest.json` | diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 94b28ed..ce1e7f6 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -210,6 +210,14 @@ class Counterexample: str(self.settings.base_image),""", instruments=("tests.test_adapter_vm.OverlayTest",), ), + Counterexample( + name="forwarded-value-is-redacted", + requirement="A value forwarded into the guest is redacted from captured output", + path="cycle_runner/adapters/vm.py", + original=" extra_values=tuple(extra_values) + tuple((env or {}).values()),", + replacement=" extra_values=tuple(extra_values),", + instruments=("tests.test_adapter_vm.TransportRedactionTest",), + ), Counterexample( name="provider-schema-verified", requirement="An unverified provider schema blocks the machine backend", diff --git a/experiments/minimal-cycle/evidence/counterexamples.txt b/experiments/minimal-cycle/evidence/counterexamples.txt index e58ca8d..d91f2ea 100644 --- a/experiments/minimal-cycle/evidence/counterexamples.txt +++ b/experiments/minimal-cycle/evidence/counterexamples.txt @@ -13,6 +13,7 @@ RED auth-leaves-no-material No credential value, length or prefix reache RED host-home-mount-acknowledged An unacknowledged host-home mount blocks the run RED preflight-leaves-no-residue Preflight inspects without creating the run's state RED base-is-only-a-backing-file The preserved base image is never the overlay's output path +RED forwarded-value-is-redacted A value forwarded into the guest is redacted from captured output RED provider-schema-verified An unverified provider schema blocks the machine backend -all 16 counterexamples discriminated +all 17 counterexamples discriminated diff --git a/experiments/minimal-cycle/tests/test_adapter_distrobox.py b/experiments/minimal-cycle/tests/test_adapter_distrobox.py index 41c9755..9940fc0 100644 --- a/experiments/minimal-cycle/tests/test_adapter_distrobox.py +++ b/experiments/minimal-cycle/tests/test_adapter_distrobox.py @@ -6,7 +6,7 @@ import unittest from pathlib import Path -from cycle_runner import config as config_module, proc +from cycle_runner import config as config_module, proc, redact from cycle_runner.adapters import distrobox from tests.test_config import minimal_document @@ -20,14 +20,16 @@ def __init__(self, table=None, *, passthrough=False): self.table = list(table or []) self.passthrough = passthrough self.seen: list[tuple[str, ...]] = [] + self.extra_values_seen: list[tuple[str, ...]] = [] def __call__(self, argv, *, timeout, context, cwd=None, env=None, extra_values=(), stdin_text=None): argv = tuple(str(item) for item in argv) self.seen.append(argv) + self.extra_values_seen.append(tuple(extra_values)) joined = " ".join(argv) for needle, code, out, err in self.table: if needle in joined: - return self._outcome(argv, code, out, err, context) + return self._outcome(argv, code, out, err, context, extra_values) if self.passthrough: return proc.run( argv, @@ -38,15 +40,17 @@ def __call__(self, argv, *, timeout, context, cwd=None, env=None, extra_values=( extra_values=extra_values, stdin_text=stdin_text, ) - return self._outcome(argv, 0, "", "", context) + return self._outcome(argv, 0, "", "", context, extra_values) @staticmethod - def _outcome(argv, code, out, err, context): + def _outcome(argv, code, out, err, context, extra_values=()): + # Redact exactly as the real runner does, so a test cannot pass only + # because the double skipped the step the adapter relies on. return proc.CommandOutcome( argv=argv, exit_code=code, - stdout=out, - stderr=err, + stdout=redact.text(out, tuple(extra_values)), + stderr=redact.text(err, tuple(extra_values)), started_at="2026-09-14T22:15:30Z", finished_at="2026-09-14T22:15:31Z", elapsed_seconds=0.1, diff --git a/experiments/minimal-cycle/tests/test_adapter_vm.py b/experiments/minimal-cycle/tests/test_adapter_vm.py index cb98d4a..c9f5102 100644 --- a/experiments/minimal-cycle/tests/test_adapter_vm.py +++ b/experiments/minimal-cycle/tests/test_adapter_vm.py @@ -238,3 +238,30 @@ def test_the_overlay_is_backed_by_the_base_and_the_base_is_unchanged(self): ) self.assertIn(str(base), chain.stdout) self.assertEqual(hashlib.sha256(base.read_bytes()).hexdigest(), before) + + +class TransportRedactionTest(VmTestCase): + """A token forwarded into the guest must not survive into captured output.""" + + def test_a_forwarded_value_is_redacted_from_what_is_captured(self): + runner = StubRunner( + [("echo", 0, f"the value was {TOKEN}", "")], passthrough=True + ) + adapter = self.make(runner=runner) + adapter.prepare_identity() + adapter.address = "192.0.2.10" + outcome = adapter.execute( + ["echo", "x"], timeout=5, env={"CLAUDE_CODE_OAUTH_TOKEN": TOKEN} + ) + self.assertNotIn(TOKEN, outcome.stdout) + self.assertIn(TOKEN, runner.extra_values_seen[-1]) + + def test_the_forwarded_name_and_value_reach_the_guest_command(self): + runner = StubRunner(passthrough=True) + adapter = self.make(runner=runner) + adapter.prepare_identity() + adapter.address = "192.0.2.10" + adapter.execute(["true"], timeout=5, env={"CLAUDE_CODE_OAUTH_TOKEN": TOKEN}) + argv = runner.seen[-1] + self.assertIn("env", argv) + self.assertIn(f"CLAUDE_CODE_OAUTH_TOKEN={TOKEN}", argv) From 69a0aebe70b6dcca499b73bbc987f0420148f280 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 00:09:32 +0200 Subject: [PATCH 10/42] Record the resources a failed create may have left, instead of reporting a clean run. --- experiments/minimal-cycle/README.md | 1 + experiments/minimal-cycle/counterexamples.py | 8 ++++ .../cycle_runner/adapters/base.py | 9 +++++ .../minimal-cycle/cycle_runner/lifecycle.py | 29 ++++++++++++--- .../evidence/counterexamples.txt | 37 ++++++++++--------- experiments/minimal-cycle/tests/fakes.py | 16 ++++++++ .../minimal-cycle/tests/test_lifecycle.py | 23 ++++++++++++ 7 files changed, 100 insertions(+), 23 deletions(-) diff --git a/experiments/minimal-cycle/README.md b/experiments/minimal-cycle/README.md index 50d2c7d..be96599 100644 --- a/experiments/minimal-cycle/README.md +++ b/experiments/minimal-cycle/README.md @@ -215,6 +215,7 @@ the table with the tests each row runs. The recorded sweep is in | An unacknowledged host-home mount blocks the run | case `host-home-mount-acknowledged` | `evidence/preflight-distrobox.txt` | | Preflight inspects without creating the run's state | case `preflight-leaves-no-residue` | `evidence/counterexamples.txt` | | The preserved base image is never the overlay's output path | case `base-is-only-a-backing-file`, and a real `qemu-img` backing-chain test | `tests/test_adapter_vm.py` | +| A create that fails partway records what it may have left behind | case `failed-create-names-its-residue` | `evidence/counterexamples.txt` | | A value forwarded into the guest is redacted from captured output | case `forwarded-value-is-redacted` | `evidence/counterexamples.txt` | | An unverified provider schema blocks the machine backend | case `provider-schema-verified` | `evidence/preflight-vm.txt` | | Preflight blocks rather than inventing a path | `./run-cycle preflight` on a host with no `pulumi` and no tool image | `evidence/preflight-vm.txt` | diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index ce1e7f6..a8090a9 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -210,6 +210,14 @@ class Counterexample: str(self.settings.base_image),""", instruments=("tests.test_adapter_vm.OverlayTest",), ), + Counterexample( + name="failed-create-names-its-residue", + requirement="A create that fails partway records what it may have left behind", + path="cycle_runner/lifecycle.py", + original=" planned = self.create_attempted and self.adapter.plan_handle()", + replacement=" planned = None", + instruments=("tests.test_lifecycle.CreateFailureTest",), + ), Counterexample( name="forwarded-value-is-redacted", requirement="A value forwarded into the guest is redacted from captured output", diff --git a/experiments/minimal-cycle/cycle_runner/adapters/base.py b/experiments/minimal-cycle/cycle_runner/adapters/base.py index 101c0af..d930ac2 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/base.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/base.py @@ -161,6 +161,15 @@ def destroy(self) -> DestroyReport: def resource_exists(self, resource: Resource) -> bool: """Report whether a resource is still present, for cleanup verification.""" + def plan_handle(self) -> EnvironmentHandle | None: + """Describe the environment this adapter *would* create, before it does. + + The runner uses it when `create` fails partway: the resources it names + may already exist, and a run that cannot confirm their state records + them as residue rather than destroying them blind. + """ + return None + def describe(self) -> dict[str, Any]: """Return backend facts for the manifest.""" return {"backend": self.name} diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index 2bc9617..fda5592 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -158,6 +158,7 @@ def __init__( dely_revision=run_config.dely_revision, ) self.handle: EnvironmentHandle | None = None + self.create_attempted = False self.blocked_reason = "" self.error_reason = "" self.timed_out = False @@ -262,6 +263,7 @@ def _prepare(self, baseline: Path) -> None: def _create(self) -> None: with self.phase("create"): + self.create_attempted = True self.handle = self.adapter.create() self.result.environment_id = self.handle.environment_id self.result.environment = { @@ -441,11 +443,7 @@ def _export(self): def _cleanup(self, export_record) -> CleanupRecord: if self.handle is None: - record = CleanupRecord( - status=CleanupStatus.DESTROYED, - reason="no environment was created, so no per-run resource existed", - verified=True, - ) + record = self._cleanup_without_a_handle() self.result.cleanup = record self.skip("cleanup", record.reason) return record @@ -471,6 +469,27 @@ def _cleanup(self, export_record) -> CleanupRecord: phase_record.detail = record.reason return record + def _cleanup_without_a_handle(self) -> CleanupRecord: + """Report what a run that never got a handle may still have left.""" + planned = self.create_attempted and self.adapter.plan_handle() + if not planned: + return CleanupRecord( + status=CleanupStatus.DESTROYED, + reason="no environment was created, so no per-run resource existed", + verified=True, + ) + return CleanupRecord( + status=CleanupStatus.RESIDUE, + reason=( + "create did not return a handle, so the state of these per-run " + "resources is unknown; nothing was destroyed blind and they need a " + "manual decision" + ), + retained=[str(item) for item in planned.per_run_resources], + shared_preserved=[str(item) for item in planned.shared_resources], + verified=False, + ) + def _close(self, export_record, cleanup_record, started: float) -> CycleOutcome: with self.phase("close"): self.result.finished_at = proc.utc_now() diff --git a/experiments/minimal-cycle/evidence/counterexamples.txt b/experiments/minimal-cycle/evidence/counterexamples.txt index d91f2ea..98e7e26 100644 --- a/experiments/minimal-cycle/evidence/counterexamples.txt +++ b/experiments/minimal-cycle/evidence/counterexamples.txt @@ -1,19 +1,20 @@ -RED no-host-fallback A probe result equal to the host's own snapshot is rejected -RED orca-is-the-environments-own An orca that resolves to the host's installation is not present -RED export-before-destroy Cleanup runs only after a confirmed export -RED stop-must-be-confirmed Cleanup runs only after a confirmed stop -RED destroy-after-export Evidence is collected and exported before anything is destroyed -RED receipt-is-a-re-read The receipt proves bytes on the host, not bytes in memory -RED cleanup-only-per-run A per-run path that contains a shared resource is refused -RED redaction-by-shape Redaction catches a credential this run has never seen -RED manifest-required-fields The manifest carries every required field on every path -RED run-id-carries-no-name The run identifier discloses neither the host name nor the auth reference -RED config-refuses-a-secret A secret-shaped value under a dull key is refused -RED auth-leaves-no-material No credential value, length or prefix reaches a receipt -RED host-home-mount-acknowledged An unacknowledged host-home mount blocks the run -RED preflight-leaves-no-residue Preflight inspects without creating the run's state -RED base-is-only-a-backing-file The preserved base image is never the overlay's output path -RED forwarded-value-is-redacted A value forwarded into the guest is redacted from captured output -RED provider-schema-verified An unverified provider schema blocks the machine backend +RED no-host-fallback A probe result equal to the host's own snapshot is rejected +RED orca-is-the-environments-own An orca that resolves to the host's installation is not present +RED export-before-destroy Cleanup runs only after a confirmed export +RED stop-must-be-confirmed Cleanup runs only after a confirmed stop +RED destroy-after-export Evidence is collected and exported before anything is destroyed +RED receipt-is-a-re-read The receipt proves bytes on the host, not bytes in memory +RED cleanup-only-per-run A per-run path that contains a shared resource is refused +RED redaction-by-shape Redaction catches a credential this run has never seen +RED manifest-required-fields The manifest carries every required field on every path +RED run-id-carries-no-name The run identifier discloses neither the host name nor the auth reference +RED config-refuses-a-secret A secret-shaped value under a dull key is refused +RED auth-leaves-no-material No credential value, length or prefix reaches a receipt +RED host-home-mount-acknowledged An unacknowledged host-home mount blocks the run +RED preflight-leaves-no-residue Preflight inspects without creating the run's state +RED base-is-only-a-backing-file The preserved base image is never the overlay's output path +RED failed-create-names-its-residue A create that fails partway records what it may have left behind +RED forwarded-value-is-redacted A value forwarded into the guest is redacted from captured output +RED provider-schema-verified An unverified provider schema blocks the machine backend -all 17 counterexamples discriminated +all 18 counterexamples discriminated diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index 7226fd2..3d0f40c 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -68,6 +68,7 @@ def __init__( leave_residue: bool = False, marker: str = "dely-cycle-marker", orca_path_from_host: bool = False, + create_fails: bool = False, task_writes_nothing: bool = False, host_project: Path | None = None, ): @@ -86,6 +87,7 @@ def __init__( self.leave_residue = leave_residue self.marker = marker self.orca_path_from_host = orca_path_from_host + self.create_fails = create_fails self.task_writes_nothing = task_writes_nothing self.host_project = host_project self.destroyed = False @@ -105,9 +107,23 @@ def preflight(self) -> PreflightReport: ), ) + def plan_handle(self) -> EnvironmentHandle: + return EnvironmentHandle( + environment_id=f"fake-{self.root.name}", + home_path=str(self.home), + project_path=str(self.project), + per_run_resources=(Resource(kind="path", identifier=str(self.home)),), + shared_resources=(Resource(kind="image", identifier=str(self.shared_base)),), + description={"image": "fake"}, + ) + def create(self) -> EnvironmentHandle: self.calls.append("create") self.project.mkdir(parents=True, exist_ok=True) + if self.create_fails: + self.shared_base.parent.mkdir(parents=True, exist_ok=True) + self.shared_base.write_bytes(b"shared base image") + raise RuntimeError("the container manager refused halfway through") self.shared_base.parent.mkdir(parents=True, exist_ok=True) self.shared_base.write_bytes(b"shared base image") return EnvironmentHandle( diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index c978b5c..284ff05 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -280,3 +280,26 @@ def test_that_run_still_exports_and_cleans_up(self): adapter, outcome = self.run_cycle(orca_path_from_host=True) self.assertEqual(outcome.run_result.export.status, status.ExportStatus.CONFIRMED) self.assertEqual(outcome.run_result.cleanup.status, status.CleanupStatus.DESTROYED) + + +class CreateFailureTest(CycleTestCase): + """A create that fails partway may already have made resources.""" + + def test_a_failed_create_records_the_planned_resources_as_residue(self): + adapter, outcome = self.run_cycle(create_fails=True) + record = outcome.run_result.cleanup + self.assertEqual(record.status, status.CleanupStatus.RESIDUE) + self.assertTrue( + any(str(adapter.home) in item for item in record.retained), + record.retained, + ) + self.assertIn("create", record.reason.lower()) + + def test_a_failed_create_does_not_destroy_blind(self): + adapter, _ = self.run_cycle(create_fails=True) + self.assertNotIn("destroy", adapter.calls) + + def test_a_failed_create_still_writes_a_manifest(self): + _, outcome = self.run_cycle(create_fails=True) + self.assertTrue(self.artifact("manifest.json").is_file()) + self.assertEqual(outcome.run_result.status, status.RunStatus.ERROR) From fc9d31b3b6004126e91230fdaf4b89c06fcd29aa Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 00:11:07 +0200 Subject: [PATCH 11/42] Refresh the committed evidence from a cycle run by the final code. --- experiments/minimal-cycle/evidence/README.md | 2 +- .../auth-receipt.json | 2 +- .../run-blocked-on-host-orca/cleanup.json | 4 +- .../export-receipt.json | 26 +-- .../identity-environment-probe.txt | 8 +- .../identity-host-probe.txt | 2 +- .../run-blocked-on-host-orca/manifest.json | 182 +++++++++--------- 7 files changed, 117 insertions(+), 109 deletions(-) diff --git a/experiments/minimal-cycle/evidence/README.md b/experiments/minimal-cycle/evidence/README.md index 480d28d..599845c 100644 --- a/experiments/minimal-cycle/evidence/README.md +++ b/experiments/minimal-cycle/evidence/README.md @@ -31,7 +31,7 @@ missing rather than inventing a path around it. ## `run-blocked-on-host-orca/` -One real cycle. A fresh box was created from `docker.io/library/ubuntu:24.04` +One real cycle, `20260915T001500Z-aaaaaa-00000003`, produced by the committed code. A fresh box was created from `docker.io/library/ubuntu:24.04` with its own home, the project was exported at a pinned revision and copied in, the identity probe ran on both sides, and the run stopped at the identity gate. Every artifact was exported and confirmed, and the box and its per-run state diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/auth-receipt.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/auth-receipt.json index baaa5e7..e183368 100644 --- a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/auth-receipt.json +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/auth-receipt.json @@ -7,5 +7,5 @@ "reference": "workshop-helper-declared-only", "removed_after_run": false, "status": "OK", - "target": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/.claude/settings.json" + "target": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home/.claude/settings.json" } diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/cleanup.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/cleanup.json index ff4828c..f014573 100644 --- a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/cleanup.json +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/cleanup.json @@ -2,8 +2,8 @@ "host_difference": {}, "reason": "every declared per-run resource is gone and every shared one remains", "removed": [ - "container:dely-cycle-20260914t231500z-aaaaaa-00000002", - "path:/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002" + "container:dely-cycle-20260915t001500z-aaaaaa-00000003", + "path:/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003" ], "retained": [], "shared_preserved": [ diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/export-receipt.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/export-receipt.json index 39ee41c..8583971 100644 --- a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/export-receipt.json +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/export-receipt.json @@ -2,12 +2,12 @@ "artifacts": [ { "path": "auth-receipt.json", - "sha256": "5749227c7c4ae0b40ba0ee854e5fec5ad3c26ad4e62c102e544ee69c762c7de4", + "sha256": "50cc0f37db49dec707da7ca2e365558a7602e991f111af15a104ad548a38b272", "size_bytes": 393 }, { "path": "backend-status.json", - "sha256": "dc441d86db5592190f1cad2c6afb752ca54517e8af11297a9d963a777561bf51", + "sha256": "e1605917b138840dc212ce5b7433594e2d3b9ce4c8d48a1558f14e77ab962b1b", "size_bytes": 1374 }, { @@ -17,17 +17,17 @@ }, { "path": "host-before.json", - "sha256": "ef32560cde0b04d1c08f01cb393f6c7ebca37c0e7535aad07a1d646002d1ea13", - "size_bytes": 690 + "sha256": "fd8bb03f41347f198a358a30d3d1f58b08a1d03267416ca7d4148ff6eb1c98aa", + "size_bytes": 858 }, { "path": "identity/environment-probe.txt", - "sha256": "97ca15bcc81dd9d799fd944618db9ba23dcaace049dda83b6366cf716cacf515", + "sha256": "06a23500e289eeffebaeb5341b59fa337e0db446c834ef0647bb89b82494401a", "size_bytes": 1567 }, { "path": "identity/host-probe.txt", - "sha256": "2e9f36ceaa4d492831fa0bf78cdf8130f6827a3704fe940f5467bbee994a2787", + "sha256": "cc6085fbb2f901482257afc671013e34b8b0de0591cd5dc6b94312d62ec95713", "size_bytes": 1358 }, { @@ -37,13 +37,13 @@ }, { "path": "logs/commands.jsonl", - "sha256": "aa57ac6dc2cfd82b53bfa6957d2a15fe8612290a2ec0c26c8ecbb0a01c700e1d", - "size_bytes": 4460 + "sha256": "4f9312a919653c8b438d29efb6ac41f06ade34e50b628f338906336112cc8bd1", + "size_bytes": 4459 }, { "path": "logs/runner.log", - "sha256": "72d408da85002bb1b3732db8e255d0ea0bc16efb89951f09288a114ccd6cedbe", - "size_bytes": 1354 + "sha256": "f4df0f2a664487a704ae47cb19d39925c39c058b813fa6b6a8318dd74d7f4f9b", + "size_bytes": 1353 }, { "path": "preflight.json", @@ -52,11 +52,11 @@ }, { "path": "run-before-cleanup.json", - "sha256": "be30779cb0960bb07162a4e9e4008ee81719d31b2d8849d6cb40da030ea348b6", - "size_bytes": 13097 + "sha256": "b3064423e01f51a41dc40dedc976bb412085acd269a3836884d29c70b6c1db02", + "size_bytes": 13095 } ], - "confirmed_at": "2026-09-14T22:00:35Z", + "confirmed_at": "2026-09-14T22:10:35Z", "detail": "11 artifact(s) re-read from the host; 1 unresolved", "missing": [ "task-artifact/evidence.txt (absent from the host)" diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-environment-probe.txt b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-environment-probe.txt index 567ed97..70d309a 100644 --- a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-environment-probe.txt +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-environment-probe.txt @@ -1,13 +1,13 @@ -hostname=dely-cycle-20260914t231500z-aaaaaa-00000002 +hostname=dely-cycle-20260915t001500z-aaaaaa-00000003 machine_id= boot_id= -home=/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home +home=/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home user= path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin -pid=3749127 +pid=3838940 uname=Linux 7.2.4-200.fc44.x86_64 x86_64 container_marker=containerenv virt=docker orca_path=/.config/orca/linux-orca-cli-shim/orca orca_version= -project_real=/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/project +project_real=/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home/project diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-host-probe.txt b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-host-probe.txt index 06d88de..0f11ab5 100644 --- a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-host-probe.txt +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/identity-host-probe.txt @@ -4,7 +4,7 @@ boot_id= home= user= path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin -pid=3748998 +pid=3838811 uname=Linux 7.2.4-200.fc44.x86_64 x86_64 container_marker= virt=none diff --git a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/manifest.json b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/manifest.json index 305e274..e4ab0f8 100644 --- a/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/manifest.json +++ b/experiments/minimal-cycle/evidence/run-blocked-on-host-orca/manifest.json @@ -1,14 +1,14 @@ { - "artifact_root": "/var/tmp/dely-cycle/artifacts/20260914T231500Z-aaaaaa-00000002", + "artifact_root": "/var/tmp/dely-cycle/artifacts/20260915T001500Z-aaaaaa-00000003", "artifacts": [ { "path": "auth-receipt.json", - "sha256": "5749227c7c4ae0b40ba0ee854e5fec5ad3c26ad4e62c102e544ee69c762c7de4", + "sha256": "50cc0f37db49dec707da7ca2e365558a7602e991f111af15a104ad548a38b272", "size_bytes": 393 }, { "path": "backend-status.json", - "sha256": "dc441d86db5592190f1cad2c6afb752ca54517e8af11297a9d963a777561bf51", + "sha256": "e1605917b138840dc212ce5b7433594e2d3b9ce4c8d48a1558f14e77ab962b1b", "size_bytes": 1374 }, { @@ -18,17 +18,17 @@ }, { "path": "host-before.json", - "sha256": "ef32560cde0b04d1c08f01cb393f6c7ebca37c0e7535aad07a1d646002d1ea13", - "size_bytes": 690 + "sha256": "fd8bb03f41347f198a358a30d3d1f58b08a1d03267416ca7d4148ff6eb1c98aa", + "size_bytes": 858 }, { "path": "identity/environment-probe.txt", - "sha256": "97ca15bcc81dd9d799fd944618db9ba23dcaace049dda83b6366cf716cacf515", + "sha256": "06a23500e289eeffebaeb5341b59fa337e0db446c834ef0647bb89b82494401a", "size_bytes": 1567 }, { "path": "identity/host-probe.txt", - "sha256": "2e9f36ceaa4d492831fa0bf78cdf8130f6827a3704fe940f5467bbee994a2787", + "sha256": "cc6085fbb2f901482257afc671013e34b8b0de0591cd5dc6b94312d62ec95713", "size_bytes": 1358 }, { @@ -38,13 +38,13 @@ }, { "path": "logs/commands.jsonl", - "sha256": "aa57ac6dc2cfd82b53bfa6957d2a15fe8612290a2ec0c26c8ecbb0a01c700e1d", - "size_bytes": 4460 + "sha256": "4f9312a919653c8b438d29efb6ac41f06ade34e50b628f338906336112cc8bd1", + "size_bytes": 4459 }, { "path": "logs/runner.log", - "sha256": "72d408da85002bb1b3732db8e255d0ea0bc16efb89951f09288a114ccd6cedbe", - "size_bytes": 1354 + "sha256": "f4df0f2a664487a704ae47cb19d39925c39c058b813fa6b6a8318dd74d7f4f9b", + "size_bytes": 1353 }, { "path": "preflight.json", @@ -53,8 +53,8 @@ }, { "path": "run-before-cleanup.json", - "sha256": "be30779cb0960bb07162a4e9e4008ee81719d31b2d8849d6cb40da030ea348b6", - "size_bytes": 13097 + "sha256": "b3064423e01f51a41dc40dedc976bb412085acd269a3836884d29c70b6c1db02", + "size_bytes": 13095 } ], "auth": { @@ -66,7 +66,7 @@ "reference": "workshop-helper-declared-only", "removed_after_run": false, "status": "OK", - "target": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/.claude/settings.json" + "target": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home/.claude/settings.json" }, "backend": "distrobox", "check": { @@ -84,8 +84,8 @@ "cleanup": { "reason": "every declared per-run resource is gone and every shared one remains", "removed": [ - "container:dely-cycle-20260914t231500z-aaaaaa-00000002", - "path:/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002" + "container:dely-cycle-20260915t001500z-aaaaaa-00000003", + "path:/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003" ], "retained": [], "shared_preserved": [ @@ -148,36 +148,36 @@ "timeout_seconds": 900 }, "dely_revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", - "elapsed_seconds": 54.556, + "elapsed_seconds": 56.322, "environment": { "backend": { "backend": "distrobox", - "container": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "container": "dely-cycle-20260915t001500z-aaaaaa-00000003", "container_manager": "podman", "distrobox_version": "distrobox: 1.8.2.5", "image": "docker.io/library/ubuntu:24.04", - "manifest": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/distrobox.ini" + "manifest": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/distrobox.ini" }, "description": { "container_manager": "podman", "desktop_integration": "Distrobox integrates with the host desktop by design; this run limits state with a separate home and project copy, and does not claim container-grade isolation", "host_home_mount_acknowledged": true, "image": "docker.io/library/ubuntu:24.04", - "manifest": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/distrobox.ini" + "manifest": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/distrobox.ini" }, - "environment_id": "dely-cycle-20260914t231500z-aaaaaa-00000002", - "home_path": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home", + "environment_id": "dely-cycle-20260915t001500z-aaaaaa-00000003", + "home_path": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home", "per_run_resources": [ { - "identifier": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "identifier": "dely-cycle-20260915t001500z-aaaaaa-00000003", "kind": "container" }, { - "identifier": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002", + "identifier": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003", "kind": "path" } ], - "project_path": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/project", + "project_path": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home/project", "shared_resources": [ { "identifier": "docker.io/library/ubuntu:24.04", @@ -185,17 +185,17 @@ } ] }, - "environment_id": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "environment_id": "dely-cycle-20260915t001500z-aaaaaa-00000003", "export": { "artifacts": [ { "path": "auth-receipt.json", - "sha256": "5749227c7c4ae0b40ba0ee854e5fec5ad3c26ad4e62c102e544ee69c762c7de4", + "sha256": "50cc0f37db49dec707da7ca2e365558a7602e991f111af15a104ad548a38b272", "size_bytes": 393 }, { "path": "backend-status.json", - "sha256": "dc441d86db5592190f1cad2c6afb752ca54517e8af11297a9d963a777561bf51", + "sha256": "e1605917b138840dc212ce5b7433594e2d3b9ce4c8d48a1558f14e77ab962b1b", "size_bytes": 1374 }, { @@ -205,17 +205,17 @@ }, { "path": "host-before.json", - "sha256": "ef32560cde0b04d1c08f01cb393f6c7ebca37c0e7535aad07a1d646002d1ea13", - "size_bytes": 690 + "sha256": "fd8bb03f41347f198a358a30d3d1f58b08a1d03267416ca7d4148ff6eb1c98aa", + "size_bytes": 858 }, { "path": "identity/environment-probe.txt", - "sha256": "97ca15bcc81dd9d799fd944618db9ba23dcaace049dda83b6366cf716cacf515", + "sha256": "06a23500e289eeffebaeb5341b59fa337e0db446c834ef0647bb89b82494401a", "size_bytes": 1567 }, { "path": "identity/host-probe.txt", - "sha256": "2e9f36ceaa4d492831fa0bf78cdf8130f6827a3704fe940f5467bbee994a2787", + "sha256": "cc6085fbb2f901482257afc671013e34b8b0de0591cd5dc6b94312d62ec95713", "size_bytes": 1358 }, { @@ -225,13 +225,13 @@ }, { "path": "logs/commands.jsonl", - "sha256": "aa57ac6dc2cfd82b53bfa6957d2a15fe8612290a2ec0c26c8ecbb0a01c700e1d", - "size_bytes": 4460 + "sha256": "4f9312a919653c8b438d29efb6ac41f06ade34e50b628f338906336112cc8bd1", + "size_bytes": 4459 }, { "path": "logs/runner.log", - "sha256": "72d408da85002bb1b3732db8e255d0ea0bc16efb89951f09288a114ccd6cedbe", - "size_bytes": 1354 + "sha256": "f4df0f2a664487a704ae47cb19d39925c39c058b813fa6b6a8318dd74d7f4f9b", + "size_bytes": 1353 }, { "path": "preflight.json", @@ -240,11 +240,11 @@ }, { "path": "run-before-cleanup.json", - "sha256": "be30779cb0960bb07162a4e9e4008ee81719d31b2d8849d6cb40da030ea348b6", - "size_bytes": 13097 + "sha256": "b3064423e01f51a41dc40dedc976bb412085acd269a3836884d29c70b6c1db02", + "size_bytes": 13095 } ], - "confirmed_at": "2026-09-14T22:00:35Z", + "confirmed_at": "2026-09-14T22:10:35Z", "detail": "11 artifact(s) re-read from the host; 1 unresolved", "missing": [ "task-artifact/evidence.txt (absent from the host)" @@ -253,20 +253,21 @@ "status": "CONFIRMED" }, "failure_classification": "the environment resolved orca to the host's own installation at /.config/orca/linux-orca-cli-shim/orca; the run stops rather than driving the host from inside a disposable environment", - "finished_at": "2026-09-14T22:00:36Z", - "generated_at": "2026-09-14T22:00:36Z", + "finished_at": "2026-09-14T22:10:37Z", + "generated_at": "2026-09-14T22:10:37Z", "host_after": { "architecture": "x86_64", "artifact_root": { "entries": [ "20260914T230000Z-aaaaaa-00000001", - "20260914T231500Z-aaaaaa-00000002" + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003" ], "exists": true, "path": "/var/tmp/dely-cycle/artifacts" }, "boot_id_digest": "bc21f149eed21643", - "captured_at": "2026-09-14T22:00:36Z", + "captured_at": "2026-09-14T22:10:37Z", "extra": { "backend": "distrobox" }, @@ -276,7 +277,10 @@ "python_version": "3.14.7", "state_root": { "entries": [ - "20260914T215536Z-bbc5e6-026d6468" + "20260914T215536Z-bbc5e6-026d6468", + "20260914T220203Z-bbc5e6-69fb8e45", + "20260914T220249Z-bbc5e6-43aa4a13", + "20260914T220452Z-bbc5e6-fc8971cf" ], "exists": true, "path": "/var/tmp/dely-cycle/state" @@ -288,13 +292,14 @@ "artifact_root": { "entries": [ "20260914T230000Z-aaaaaa-00000001", - "20260914T231500Z-aaaaaa-00000002" + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003" ], "exists": true, "path": "/var/tmp/dely-cycle/artifacts" }, "boot_id_digest": "bc21f149eed21643", - "captured_at": "2026-09-14T21:59:42Z", + "captured_at": "2026-09-14T22:09:41Z", "extra": { "backend": "distrobox" }, @@ -304,7 +309,10 @@ "python_version": "3.14.7", "state_root": { "entries": [ - "20260914T215536Z-bbc5e6-026d6468" + "20260914T215536Z-bbc5e6-026d6468", + "20260914T220203Z-bbc5e6-69fb8e45", + "20260914T220249Z-bbc5e6-43aa4a13", + "20260914T220452Z-bbc5e6-fc8971cf" ], "exists": true, "path": "/var/tmp/dely-cycle/state" @@ -314,13 +322,13 @@ "identity": { "environment": { "container_marker": "containerenv", - "home": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home", - "hostname": "dely-cycle-20260914t231500z-aaaaaa-00000002", + "home": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home", + "hostname": "dely-cycle-20260915t001500z-aaaaaa-00000003", "orca_path": "/.config/orca/linux-orca-cli-shim/orca", "orca_version": "", "path": "/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin", - "pid": "3749127", - "project_real": "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/project", + "pid": "3838940", + "project_real": "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home/project", "uname": "Linux 7.2.4-200.fc44.x86_64 x86_64", "user": "", "virt": "docker" @@ -351,28 +359,28 @@ { "commands": [], "detail": "", - "elapsed_seconds": 0.778, - "finished_at": "2026-09-14T21:59:42Z", + "elapsed_seconds": 0.766, + "finished_at": "2026-09-14T22:09:41Z", "name": "prepare", - "started_at": "2026-09-14T21:59:42Z", + "started_at": "2026-09-14T22:09:41Z", "status": "OK" }, { "commands": [], "detail": "", - "elapsed_seconds": 51.609, - "finished_at": "2026-09-14T22:00:34Z", + "elapsed_seconds": 53.21, + "finished_at": "2026-09-14T22:10:35Z", "name": "create", - "started_at": "2026-09-14T21:59:42Z", + "started_at": "2026-09-14T22:09:41Z", "status": "OK" }, { "commands": [], "detail": "", - "elapsed_seconds": 0.009, - "finished_at": "2026-09-14T22:00:34Z", + "elapsed_seconds": 0.007, + "finished_at": "2026-09-14T22:10:35Z", "name": "bootstrap", - "started_at": "2026-09-14T22:00:34Z", + "started_at": "2026-09-14T22:10:35Z", "status": "OK" }, { @@ -386,10 +394,10 @@ "/var/tmp/dely-cycle/project-source" ], "context": "host", - "elapsed_seconds": 0.122, + "elapsed_seconds": 0.099, "exit_code": 0, - "finished_at": "2026-09-14T22:00:34Z", - "started_at": "2026-09-14T22:00:34Z", + "finished_at": "2026-09-14T22:10:35Z", + "started_at": "2026-09-14T22:10:35Z", "stderr_path": null, "stdout_path": null, "timed_out": false @@ -399,20 +407,20 @@ "distrobox", "enter", "--name", - "dely-cycle-20260914t231500z-aaaaaa-00000002", + "dely-cycle-20260915t001500z-aaaaaa-00000003", "-T", "--", "sh", "-c", "\nset -u\nproject_path=\"${1:-}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v orca 2>/dev/null || printf '')\"\norca_release=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id=%s\\n' \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", "cycle-probe", - "/var/tmp/dely-cycle/state/20260914T231500Z-aaaaaa-00000002/home/project" + "/var/tmp/dely-cycle/state/20260915T001500Z-aaaaaa-00000003/home/project" ], "context": "environment", - "elapsed_seconds": 0.234, + "elapsed_seconds": 0.152, "exit_code": 0, - "finished_at": "2026-09-14T22:00:34Z", - "started_at": "2026-09-14T22:00:34Z", + "finished_at": "2026-09-14T22:10:35Z", + "started_at": "2026-09-14T22:10:35Z", "stderr_path": null, "stdout_path": null, "timed_out": false @@ -422,7 +430,7 @@ "distrobox", "enter", "--name", - "dely-cycle-20260914t231500z-aaaaaa-00000002", + "dely-cycle-20260915t001500z-aaaaaa-00000003", "-T", "--", "orca", @@ -430,20 +438,20 @@ "--json" ], "context": "environment", - "elapsed_seconds": 0.208, + "elapsed_seconds": 0.19, "exit_code": 127, - "finished_at": "2026-09-14T22:00:35Z", - "started_at": "2026-09-14T22:00:34Z", + "finished_at": "2026-09-14T22:10:35Z", + "started_at": "2026-09-14T22:10:35Z", "stderr_path": null, "stdout_path": null, "timed_out": false } ], "detail": "the environment resolved orca to the host's own installation at /.config/orca/linux-orca-cli-shim/orca; the run stops rather than driving the host from inside a disposable environment", - "elapsed_seconds": 0.567, - "finished_at": "2026-09-14T22:00:35Z", + "elapsed_seconds": 0.444, + "finished_at": "2026-09-14T22:10:35Z", "name": "identity", - "started_at": "2026-09-14T22:00:34Z", + "started_at": "2026-09-14T22:10:35Z", "status": "BLOCKED" }, { @@ -467,28 +475,28 @@ { "commands": [], "detail": "0 patch line(s) collected", - "elapsed_seconds": 0.012, - "finished_at": "2026-09-14T22:00:35Z", + "elapsed_seconds": 0.014, + "finished_at": "2026-09-14T22:10:35Z", "name": "collect", - "started_at": "2026-09-14T22:00:35Z", + "started_at": "2026-09-14T22:10:35Z", "status": "OK" }, { "commands": [], "detail": "11 artifact(s) re-read from the host; 1 unresolved", - "elapsed_seconds": 0.009, - "finished_at": "2026-09-14T22:00:35Z", + "elapsed_seconds": 0.012, + "finished_at": "2026-09-14T22:10:35Z", "name": "export", - "started_at": "2026-09-14T22:00:35Z", + "started_at": "2026-09-14T22:10:35Z", "status": "OK" }, { "commands": [], "detail": "every declared per-run resource is gone and every shared one remains", - "elapsed_seconds": 1.57, - "finished_at": "2026-09-14T22:00:36Z", + "elapsed_seconds": 1.868, + "finished_at": "2026-09-14T22:10:37Z", "name": "cleanup", - "started_at": "2026-09-14T22:00:35Z", + "started_at": "2026-09-14T22:10:35Z", "status": "OK" }, { @@ -497,13 +505,13 @@ "elapsed_seconds": null, "finished_at": null, "name": "close", - "started_at": "2026-09-14T22:00:36Z", + "started_at": "2026-09-14T22:10:37Z", "status": "OK" } ], - "run_id": "20260914T231500Z-aaaaaa-00000002", + "run_id": "20260915T001500Z-aaaaaa-00000003", "schema_version": 1, - "started_at": "2026-09-14T21:59:42Z", + "started_at": "2026-09-14T22:09:41Z", "status": "BLOCKED", "tested_revision": "e631b5cb38209a41cb5bd3ba77d8a284a8380ab8", "timeout_seconds": 900, From 23cbe0cf71e5dad83f8fce1f07894348a6800230 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 00:13:23 +0200 Subject: [PATCH 12/42] Stop preflight leaving a per-run home behind: distrobox creates the custom home even for a dry run. --- experiments/minimal-cycle/counterexamples.py | 7 ++--- .../cycle_runner/adapters/distrobox.py | 16 ++++++++--- .../tests/test_adapter_distrobox.py | 28 +++++++++++++++++++ 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index a8090a9..c48dd78 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -193,10 +193,9 @@ class Counterexample: name="preflight-leaves-no-residue", requirement="Preflight inspects without creating the run's state", path="cycle_runner/adapters/distrobox.py", - original=""" probe_manifest = Path(staging) / "distrobox.ini" - probe_manifest.write_text(self.render_manifest(), encoding="utf-8")""", - replacement=" probe_manifest = self.write_manifest()", - instruments=("tests.test_adapter_distrobox.PreflightResidueTest",), + original=""" self.render_manifest(home_override=Path(staging) / "home"),""", + replacement=""" self.render_manifest(),""", + instruments=("tests.test_adapter_distrobox.RealPreflightResidueTest",), ), Counterexample( name="base-is-only-a-backing-file", diff --git a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py index ea593be..f6b55b7 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py @@ -104,12 +104,17 @@ def _checked_mount(self, mount: str) -> str: # -- the declared environment ---------------------------------------- - def render_manifest(self) -> str: - """Return the Distrobox Assemble manifest for this run.""" + def render_manifest(self, *, home_override: Path | None = None) -> str: + """Return the Distrobox Assemble manifest for this run. + + `home_override` exists for preflight: `distrobox create` creates the + custom home before it prints a dry run, so a probe manifest pointing at + the per-run home would leave that directory behind. + """ lines = [ f"[{self.container_name}]", f"image={self.settings.image}", - f"home={self.home_path}", + f"home={home_override or self.home_path}", f"hostname={self.container_name}", "entry=false", "pull=false", @@ -169,7 +174,10 @@ def host_home_mount_finding(self) -> Finding: name = "host home reachable from the environment" with tempfile.TemporaryDirectory(prefix="dely-cycle-preflight-") as staging: probe_manifest = Path(staging) / "distrobox.ini" - probe_manifest.write_text(self.render_manifest(), encoding="utf-8") + probe_manifest.write_text( + self.render_manifest(home_override=Path(staging) / "home"), + encoding="utf-8", + ) rendered = self.runner( [self.binary, "assemble", "create", "--dry-run", "--file", str(probe_manifest)], timeout=300, diff --git a/experiments/minimal-cycle/tests/test_adapter_distrobox.py b/experiments/minimal-cycle/tests/test_adapter_distrobox.py index 9940fc0..2f4bbb6 100644 --- a/experiments/minimal-cycle/tests/test_adapter_distrobox.py +++ b/experiments/minimal-cycle/tests/test_adapter_distrobox.py @@ -302,3 +302,31 @@ def test_preflight_leaves_no_state_root_entry_behind(self): else [] ) self.assertEqual(before, after) + + +@unittest.skipUnless(shutil.which("distrobox"), "distrobox is not on this host") +class RealPreflightResidueTest(AdapterTestCase): + """distrobox itself creates the custom home, so only a real run shows this.""" + + def make(self, **options): + options.setdefault("host_home", Path.home()) + options.setdefault("runner", proc.run) + return super().make(**options) + + def test_a_real_preflight_leaves_no_per_run_state_behind(self): + adapter = self.make() + adapter.preflight() + self.assertFalse( + adapter.run_state.exists(), + f"preflight left {adapter.run_state} on the host", + ) + + def test_a_real_preflight_leaves_the_state_root_empty(self): + adapter = self.make() + adapter.preflight() + entries = ( + sorted(p.name for p in adapter.config.state_root.iterdir()) + if adapter.config.state_root.is_dir() + else [] + ) + self.assertEqual(entries, []) From 2850c3b6773fdd69b179bd4fb567c83e8c876b32 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 12:35:11 +0200 Subject: [PATCH 13/42] Rebuild the machine adapter against the pinned provider: verified schema, pool volumes, a user-mode egress interface and address discovery the runner owns. --- .../minimal-cycle/config.vm.example.yaml | 49 +- experiments/minimal-cycle/counterexamples.py | 60 ++- .../cycle_runner/adapters/schema.py | 119 +++++ .../minimal-cycle/cycle_runner/adapters/vm.py | 478 +++++++++++++----- .../minimal-cycle/cycle_runner/config.py | 52 +- experiments/minimal-cycle/host/versions.json | 76 +++ .../minimal-cycle/tests/test_adapter_vm.py | 374 +++++++++----- experiments/minimal-cycle/tests/test_cli.py | 1 + .../tests/test_provider_schema.py | 103 ++++ 9 files changed, 1014 insertions(+), 298 deletions(-) create mode 100644 experiments/minimal-cycle/cycle_runner/adapters/schema.py create mode 100644 experiments/minimal-cycle/host/versions.json create mode 100644 experiments/minimal-cycle/tests/test_provider_schema.py diff --git a/experiments/minimal-cycle/config.vm.example.yaml b/experiments/minimal-cycle/config.vm.example.yaml index b71eb24..6dd6a54 100644 --- a/experiments/minimal-cycle/config.vm.example.yaml +++ b/experiments/minimal-cycle/config.vm.example.yaml @@ -1,14 +1,16 @@ # One minimal proof cycle on the virtual-machine backend. # -# This backend needs a prepared tool image, a pinned libvirt provider, and a -# provider schema someone has actually checked. Preflight refuses each of those -# rather than guessing. +# The stack is Pulumi Python with the libvirt provider over qemu and kvm, a +# qcow2 overlay on a preserved tool image, and a cloud-init NoCloud seed. Run +# `host/prepare-host` first: it installs the pinned Pulumi, points it at a +# local state backend, builds the environment, fetches the base image and +# declares the storage pool. backend: vm tested_revision: replace-with-the-revision-under-test dely_revision: replace-with-the-dely-revision -claude_code_version: replace-with-the-pinned-claude-code-version +claude_code_version: "2.1.272" timeout_seconds: 3600 @@ -36,7 +38,7 @@ auth: token_env: CLAUDE_CODE_OAUTH_TOKEN orca: - version: replace-with-the-pinned-orca-version + version: "1.4.201" model: replace-with-the-pinned-model effort: replace-with-the-pinned-effort agent: claude @@ -44,25 +46,36 @@ orca: vm: provider: pulumi-libvirt - provider_version: replace-with-the-version-you-checked + provider_version: "0.5.4" stack_prefix: dely-cycle - # The preserved tool image built separately with Packer. The run opens it only - # as a backing file and never destroys it. - base_image: /var/lib/dely-cycle/images/tool-image.qcow2 - # A digest of the right shape that matches nothing: preflight blocks on it - # until it is replaced with the real one. + # The preserved tool image built by host/packer. The run opens it only as a + # backing file and never destroys it. Its name inside the pool is the file + # name below. + base_image: /var/tmp/dely-cycle/images/tool-image.qcow2 + # A digest of the right shape that matches nothing, so preflight blocks + # until the tool image build's recorded digest replaces it. base_image_sha256: '0000000000000000000000000000000000000000000000000000000000000000' guest_user: cycle + # The environment carrying the pinned Pulumi SDK and libvirt provider. The + # rendered program is checked field by field against the provider importable + # from here; preflight blocks when it cannot be. + venv: /var/tmp/dely-cycle/venv pulumi_binary: pulumi - connect_uri: qemu:///session - overlay_size: 24g + connect_uri: qemu:///system + pool: dely-cycle + network: default + overlay_size_bytes: 25769803776 memory_mb: 4096 vcpus: 2 - graphics: spice + # This host's qemu reports sdl, vnc, egl-headless and dbus; it has no spice. + # Preflight reads the supported list rather than trusting this value. + graphics: vnc + listen_address: 127.0.0.1 transport: ssh - network: default ssh_port: 22 - # The rendered Pulumi program names resource fields that belong to the pinned - # provider version. Preflight blocks until someone has checked them. - provider_schema_verified: false + address_timeout_seconds: 600 + # A second, user-mode interface so the guest can reach the internet. It does + # not use the host's forwarding path, so it works where a bridged guest is + # blocked by the host firewall or a virtual private network. + egress: true provision: [] diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index c48dd78..420c06b 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -199,15 +199,27 @@ class Counterexample: ), Counterexample( name="base-is-only-a-backing-file", - requirement="The preserved base image is never the overlay's output path", + requirement="The preserved base image is a backing volume, never a per-run one", path="cycle_runner/adapters/vm.py", - original=""" "-F", - "qcow2", - str(self.overlay_path),""", - replacement=""" "-F", - "qcow2", - str(self.settings.base_image),""", - instruments=("tests.test_adapter_vm.OverlayTest",), + original=""" Resource( + kind="volume", + identifier=str( + Path(self.settings.base_image).parent + / f"{self.domain_name}-overlay.qcow2" + ), + ),""", + replacement=""" Resource( + kind="volume", identifier=str(self.settings.base_image) + ),""", + instruments=("tests.test_adapter_vm.ResourceTest",), + ), + Counterexample( + name="overlay-is-not-the-base", + requirement="The rendered program creates an overlay, not a volume named like the base", + path="cycle_runner/adapters/vm.py", + original=' name=DOMAIN_NAME + "-overlay.qcow2",', + replacement=" name=BASE_VOLUME,", + instruments=("tests.test_adapter_vm.ProgramTest",), ), Counterexample( name="failed-create-names-its-residue", @@ -227,12 +239,38 @@ class Counterexample: ), Counterexample( name="provider-schema-verified", - requirement="An unverified provider schema blocks the machine backend", + requirement="A program the provider cannot be checked against blocks the run", + path="cycle_runner/adapters/vm.py", + original=""" ran, problems = schema.verify_with_interpreter(program, interpreter) + if not ran:""", + replacement=""" ran, problems = schema.verify_with_interpreter(program, interpreter) + if False:""", + instruments=("tests.test_adapter_vm.PreflightTest",), + ), + Counterexample( + name="state-backend-is-local", + requirement="A cloud state backend blocks the machine backend", path="cycle_runner/adapters/vm.py", - original=" ok=self.settings.provider_schema_verified,", - replacement=" ok=True,", + original=" ok=backend.startswith(\"file://\"),", + replacement=" ok=True,", instruments=("tests.test_adapter_vm.PreflightTest",), ), + Counterexample( + name="graphics-type-is-supported", + requirement="A graphics type this emulator lacks blocks the run", + path="cycle_runner/adapters/vm.py", + original=" ok=self.settings.graphics in supported,", + replacement=" ok=True,", + instruments=("tests.test_adapter_vm.PreflightTest",), + ), + Counterexample( + name="transport-address-is-the-right-one", + requirement="Another interface's address is not mistaken for the transport's", + path="cycle_runner/adapters/vm.py", + original=" if self.transport_mac in line:", + replacement=" if True:", + instruments=("tests.test_adapter_vm.AddressDiscoveryTest",), + ), ) diff --git a/experiments/minimal-cycle/cycle_runner/adapters/schema.py b/experiments/minimal-cycle/cycle_runner/adapters/schema.py new file mode 100644 index 0000000..8c1d79c --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/adapters/schema.py @@ -0,0 +1,119 @@ +"""Checking a rendered Pulumi program against the provider it is pinned to. + +An example copied from a blog post is not evidence that a field exists in the +version this run installs. This module reads the program the adapter renders, +collects every provider class and keyword it names, and compares them with the +provider module actually importable from the pinned environment. +""" + +from __future__ import annotations + +import ast +import json +import subprocess +from pathlib import Path +from typing import Any, Mapping + +PROVIDER_MODULE = "pulumi_libvirt" + + +def declared_symbols(program: str) -> dict[str, set[str]]: + """Return the provider classes a program calls and the keywords it passes.""" + tree = ast.parse(program) + aliases = {PROVIDER_MODULE} + for node in ast.walk(tree): + if isinstance(node, ast.Import): + for name in node.names: + if name.name == PROVIDER_MODULE and name.asname: + aliases.add(name.asname) + found: dict[str, set[str]] = {} + for node in ast.walk(tree): + if not isinstance(node, ast.Call): + continue + target = node.func + if not isinstance(target, ast.Attribute): + continue + owner = target.value + if not isinstance(owner, ast.Name) or owner.id not in aliases: + continue + keywords = {keyword.arg for keyword in node.keywords if keyword.arg} + found.setdefault(target.attr, set()).update(keywords) + return found + + +def _properties(holder: Any) -> set[str]: + return {name for name, value in vars(holder).items() if isinstance(value, property)} + + +def verify_against(symbols: Mapping[str, set[str]], module: Any) -> list[str]: + """Return one problem per class or keyword the provider does not have.""" + problems: list[str] = [] + for class_name in sorted(symbols): + holder = getattr(module, class_name, None) + if holder is None: + problems.append( + f"the pinned provider has no {class_name}; the program names it" + ) + continue + known = _properties(holder) + for keyword in sorted(symbols[class_name]): + if keyword not in known: + problems.append( + f"{class_name} has no field {keyword!r} in the pinned provider" + ) + return problems + + +_PROBE = """ +import json, sys +import %s as provider + +symbols = json.loads(sys.stdin.read()) +problems = [] +for class_name in sorted(symbols): + holder = getattr(provider, class_name, None) + if holder is None: + problems.append("the pinned provider has no " + class_name + "; the program names it") + continue + known = {name for name, value in vars(holder).items() if isinstance(value, property)} + for keyword in sorted(symbols[class_name]): + if keyword not in known: + problems.append(class_name + " has no field '" + keyword + "' in the pinned provider") +print(json.dumps(problems)) +""" % PROVIDER_MODULE + + +def verify_with_interpreter( + program: str, interpreter: Path, *, timeout: float = 120 +) -> tuple[bool, list[str]]: + """Verify a program against the provider importable from one interpreter. + + Returns (ran, problems). `ran` is False when the interpreter could not be + used at all, which is a blocked preflight rather than a pass. + """ + interpreter = Path(interpreter) + if not interpreter.is_file(): + return False, [f"no interpreter at {interpreter}"] + try: + symbols = declared_symbols(program) + except SyntaxError as error: + return True, [f"the rendered program does not parse: {error}"] + payload = json.dumps({name: sorted(values) for name, values in symbols.items()}) + completed = subprocess.run( + [str(interpreter), "-c", _PROBE], + input=payload, + capture_output=True, + text=True, + timeout=timeout, + ) + if completed.returncode != 0: + return False, [ + "the pinned environment could not import the provider: " + + (completed.stderr or "").strip().splitlines()[-1:][0] + if (completed.stderr or "").strip() + else "the pinned environment could not import the provider" + ] + try: + return True, list(json.loads(completed.stdout)) + except json.JSONDecodeError: + return False, ["the provider probe returned nothing readable"] diff --git a/experiments/minimal-cycle/cycle_runner/adapters/vm.py b/experiments/minimal-cycle/cycle_runner/adapters/vm.py index 3a431db..3d70bb8 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/vm.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/vm.py @@ -1,28 +1,30 @@ """The virtual-machine backend: Pulumi Python over the libvirt provider. -The isolation this backend buys is a separate kernel, so the identity gate can -ask for a distinct boot identity rather than a container marker. What it costs -is a preserved base image, a rendered program, a seed and a transport, and -every one of those is a place a credential could end up. None of them carries -one: the seed authorises one per-run public key, the private half never leaves -the per-run state directory, and the base image is opened only as a backing -file. - -The provider's resource schema is not assumed. This runner renders a program -against a pinned provider version and refuses to run until the configuration -records that the schema was actually checked against that version. +What this backend buys is a separate kernel, so the identity gate can demand a +distinct boot identity rather than a container marker. What it costs is a +preserved tool image, a rendered program, a seed and a transport, and every one +of those is a place a credential could end up. None of them carries one: the +seed authorises one per-run public key, the private half never leaves the +per-run state directory, and the tool image is opened only as a backing file. + +Nothing about the provider is assumed. The rendered program is checked field by +field against the provider importable from the pinned environment, and every +host fact the domain depends on — the state backend, the pool, the network, the +graphics types this qemu actually has — is read before a domain is declared. """ from __future__ import annotations import hashlib -import json +import re import shutil +import time from pathlib import Path from typing import Callable, Mapping, Sequence from .. import cleanup, ids, proc from ..config import RunConfig +from . import schema from .base import ( BackendAdapter, DestroyReport, @@ -33,33 +35,71 @@ StopReport, ) -REQUIRED_TOOLS = ("pulumi", "qemu-img", "virsh", "ssh", "scp") +REQUIRED_TOOLS = ("qemu-img", "virsh", "ssh", "scp") + +#: Added to the generated domain so the guest can reach the internet without +#: the host's forwarding path, which a firewall or a virtual private network +#: may refuse. A user-mode interface is served by qemu itself. +EGRESS_XSLT = """ + + + + + + + + + + + + + + + +""" PROGRAM_TEMPLATE = '''"""Per-run domain for one dely minimal cycle. -Rendered by cycle_runner.adapters.vm. The resource fields below belong to the -pinned provider version named in Pulumi.yaml; the runner refuses to apply this -program until the configuration records that the schema was checked against -that version. +Rendered by cycle_runner.adapters.vm. Every class and field below is checked +against the provider importable from the pinned environment before this program +is applied; preflight blocks when that check cannot run. """ import pulumi import pulumi_libvirt as libvirt DOMAIN_NAME = {domain_name!r} -OVERLAY_PATH = {overlay_path!r} +POOL = {pool!r} +BASE_VOLUME = {base_volume!r} +NETWORK = {network!r} +TRANSPORT_MAC = {transport_mac!r} USER_DATA = {user_data!r} META_DATA = {meta_data!r} +NETWORK_CONFIG = {network_config!r} GRAPHICS = {graphics!r} +LISTEN_ADDRESS = {listen_address!r} MEMORY_MB = {memory_mb!r} VCPUS = {vcpus!r} -NETWORK = {network!r} +OVERLAY_SIZE_BYTES = {overlay_size_bytes!r} +EGRESS_XSLT = {egress_xslt!r} + +overlay = libvirt.Volume( + "overlay", + name=DOMAIN_NAME + "-overlay.qcow2", + pool=POOL, + base_volume_name=BASE_VOLUME, + base_volume_pool=POOL, + size=OVERLAY_SIZE_BYTES, + format="qcow2", +) -seed = libvirt.CloudInItDisk( +seed = libvirt.CloudInitDisk( "seed", name=DOMAIN_NAME + "-seed.iso", + pool=POOL, user_data=USER_DATA, meta_data=META_DATA, + network_config=NETWORK_CONFIG, ) domain = libvirt.Domain( @@ -67,16 +107,27 @@ name=DOMAIN_NAME, memory=MEMORY_MB, vcpu=VCPUS, + running=True, + qemu_agent=True, cloudinit=seed.id, - disks=[libvirt.DomainDiskArgs(file=OVERLAY_PATH)], + disks=[libvirt.DomainDiskArgs(volume_id=overlay.id)], network_interfaces=[ - libvirt.DomainNetworkInterfaceArgs(network_name=NETWORK, wait_for_lease=True) + libvirt.DomainNetworkInterfaceArgs(network_name=NETWORK, mac=TRANSPORT_MAC) + ], + graphics=libvirt.DomainGraphicsArgs( + type=GRAPHICS, + listen_type="address", + listen_address=LISTEN_ADDRESS, + autoport=True, + ), + consoles=[ + libvirt.DomainConsoleArgs(type="pty", target_port="0", target_type="serial") ], - graphics=libvirt.DomainGraphicsArgs(type=GRAPHICS, listen_type="address"), + xml=libvirt.DomainXmlArgs(xslt=EGRESS_XSLT), ) pulumi.export("domain_name", domain.name) -pulumi.export("address", domain.network_interfaces[0].addresses[0]) +pulumi.export("transport_mac", TRANSPORT_MAC) ''' USER_DATA_TEMPLATE = """#cloud-config @@ -103,11 +154,44 @@ local-hostname: {hostname} """ +#: Both interfaces take an address; the user-mode one carries the default route +#: because the bridged one cannot reach anything past the host. +NETWORK_CONFIG_TEMPLATE = """version: 2 +ethernets: + transport: + match: + macaddress: "{transport_mac}" + dhcp4: true + dhcp4-overrides: + route-metric: 300 + egress: + match: + macaddress: "{egress_mac}" + dhcp4: true + dhcp4-overrides: + route-metric: 100 +""" + +NETWORK_CONFIG_TRANSPORT_ONLY = """version: 2 +ethernets: + transport: + match: + macaddress: "{transport_mac}" + dhcp4: true +""" + +_ADDRESS = re.compile(r"(\d{1,3}(?:\.\d{1,3}){3})/\d+") + class VmContractError(RuntimeError): """The machine backend was asked for something it refuses to do.""" +def _mac(prefix: str, run_id: str) -> str: + digest = hashlib.sha256(f"{prefix}:{run_id}".encode("utf-8")).hexdigest() + return f"{prefix}:{digest[0:2]}:{digest[2:4]}:{digest[4:6]}" + + class VmAdapter(BackendAdapter): """One disposable domain, declared with Pulumi and destroyed with it.""" @@ -120,6 +204,7 @@ def __init__( run_id: str, runner: Callable[..., proc.CommandOutcome] = proc.run, which: Callable[[str], str | None] = shutil.which, + sleeper: Callable[[float], None] = time.sleep, ): if run_config.vm is None: raise VmContractError("the configuration carries no vm section") @@ -128,16 +213,21 @@ def __init__( self.run_id = run_id self.runner = runner self.which = which + self.sleeper = sleeper self.domain_name = ids.resource_name(self.settings.stack_prefix, run_id) self.stack_name = self.domain_name self.run_state = run_config.state_root / run_id self.stack_dir = self.run_state / "stack" - self.overlay_path = self.run_state / "overlay.qcow2" self.private_key_path = self.run_state / "id_cycle" self.public_key_path = self.run_state / "id_cycle.pub" self.home_path = f"/home/{self.settings.guest_user}" self.project_path = f"{self.home_path}/{run_config.project.environment_path}" + self.transport_mac = _mac("52:54:00", run_id) + self.egress_mac = _mac("52:54:01", run_id) self.address: str | None = None + self._passphrase = hashlib.sha256( + f"ephemeral:{run_id}:{id(self)}".encode("utf-8") + ).hexdigest() # -- per-run identity ------------------------------------------------- @@ -187,27 +277,46 @@ def render_meta_data(self) -> str: run_id=self.run_id.lower(), hostname=self.domain_name ) + def render_network_config(self) -> str: + """Render the guest's interfaces, matched by their per-run addresses.""" + if not self.settings.egress: + return NETWORK_CONFIG_TRANSPORT_ONLY.format(transport_mac=self.transport_mac) + return NETWORK_CONFIG_TEMPLATE.format( + transport_mac=self.transport_mac, egress_mac=self.egress_mac + ) + + def render_egress_xslt(self) -> str: + """Render the transform that adds the user-mode interface.""" + return EGRESS_XSLT.format(egress_mac=self.egress_mac) + def render_program(self) -> str: """Render the Pulumi program for this run's domain.""" return PROGRAM_TEMPLATE.format( domain_name=self.domain_name, - overlay_path=str(self.overlay_path), + pool=self.settings.pool, + base_volume=self.settings.base_volume_name, + network=self.settings.network, + transport_mac=self.transport_mac, user_data=self.render_user_data(), meta_data=self.render_meta_data(), + network_config=self.render_network_config(), graphics=self.settings.graphics, + listen_address=self.settings.listen_address, memory_mb=self.settings.memory_mb, vcpus=self.settings.vcpus, - network=self.settings.network, + overlay_size_bytes=self.settings.overlay_size_bytes, + egress_xslt=self.render_egress_xslt() if self.settings.egress else "", ) def render_project(self) -> str: - """Render Pulumi.yaml, pinning the provider and its version.""" + """Render Pulumi.yaml, pinning the runtime to the prepared environment.""" return ( f"name: {self.settings.stack_prefix}\n" - "runtime: python\n" + "runtime:\n" + " name: python\n" + " options:\n" + f" virtualenv: {self.settings.venv}\n" f"description: one dely minimal cycle domain for run {self.run_id}\n" - "packages:\n" - f" libvirt: {self.settings.provider}@{self.settings.provider_version}\n" ) def render_stack_settings(self) -> str: @@ -228,21 +337,6 @@ def write_declarations(self) -> None: self.render_stack_settings(), encoding="utf-8" ) - def overlay_argv(self) -> list[str]: - """Return the command that creates the overlay over the preserved base.""" - return [ - "qemu-img", - "create", - "-f", - "qcow2", - "-b", - str(self.settings.base_image), - "-F", - "qcow2", - str(self.overlay_path), - self.settings.overlay_size, - ] - def plan_handle(self) -> EnvironmentHandle: """Describe the environment this adapter will create.""" return EnvironmentHandle( @@ -252,17 +346,36 @@ def plan_handle(self) -> EnvironmentHandle: per_run_resources=( Resource(kind="domain", identifier=self.domain_name), Resource(kind="path", identifier=str(self.run_state)), + Resource( + kind="volume", + identifier=str( + Path(self.settings.base_image).parent + / f"{self.domain_name}-overlay.qcow2" + ), + ), + Resource( + kind="volume", + identifier=str( + Path(self.settings.base_image).parent + / f"{self.domain_name}-seed.iso" + ), + ), ), shared_resources=( Resource(kind="image", identifier=str(self.settings.base_image)), ), description={ "domain": self.domain_name, - "overlay": str(self.overlay_path), "stack": str(self.stack_dir), + "pool": self.settings.pool, + "network": self.settings.network, "connect_uri": self.settings.connect_uri, "provider": f"{self.settings.provider}@{self.settings.provider_version}", "base_image": str(self.settings.base_image), + "base_volume": self.settings.base_volume_name, + "graphics": self.settings.graphics, + "transport_mac": self.transport_mac, + "egress": self.settings.egress, }, ) @@ -278,9 +391,119 @@ def _base_digest(self) -> str | None: digest.update(chunk) return digest.hexdigest() + def _virsh(self, *arguments: str, timeout: float = 120) -> proc.CommandOutcome: + return self.runner( + ["virsh", "--connect", self.settings.connect_uri, *arguments], + timeout=timeout, + context="host", + ) + + def _pulumi_environment(self) -> dict[str, str]: + return { + "PULUMI_BACKEND_URL": f"file://{self.run_state}", + "PULUMI_CONFIG_PASSPHRASE": self._passphrase, + "PULUMI_SKIP_UPDATE_CHECK": "true", + "LIBVIRT_DEFAULT_URI": self.settings.connect_uri, + } + + def _pulumi(self, arguments: Sequence[str], *, timeout: float) -> proc.CommandOutcome: + return self.runner( + [self.settings.pulumi_binary, *arguments], + timeout=timeout, + context="host", + cwd=str(self.stack_dir), + env=self._pulumi_environment(), + extra_values=(self._passphrase,), + ) + + def _state_backend_finding(self) -> Finding: + name = "pulumi state backend is local" + outcome = self.runner( + [self.settings.pulumi_binary, "whoami", "-v"], + timeout=120, + context="host", + env={"PULUMI_SKIP_UPDATE_CHECK": "true"}, + ) + if not outcome.ok: + return Finding( + name=name, + ok=False, + detail="pulumi could not report its backend: " + + (outcome.stderr or outcome.stdout).strip()[:200], + ) + match = re.search(r"Backend URL:\s*(\S+)", outcome.stdout) + backend = match.group(1) if match else "unknown" + return Finding( + name=name, + ok=backend.startswith("file://"), + detail=( + f"state lives at {backend}" + if backend.startswith("file://") + else f"the backend is {backend}, not a local file backend; run " + "pulumi login --local" + ), + ) + + def _schema_finding(self) -> Finding: + name = "provider schema verified" + interpreter = Path(self.settings.venv) / "bin" / "python" + try: + program = self.render_program() + except VmContractError as error: + return Finding(name=name, ok=False, detail=str(error)) + ran, problems = schema.verify_with_interpreter(program, interpreter) + if not ran: + return Finding( + name=name, + ok=False, + detail="the rendered program could not be checked against " + f"{self.settings.provider}@{self.settings.provider_version}: " + + "; ".join(problems), + ) + if problems: + return Finding(name=name, ok=False, detail="; ".join(problems[:4])) + return Finding( + name=name, + ok=True, + detail=( + "every class and field the program names exists in " + f"{self.settings.provider}@{self.settings.provider_version}" + ), + ) + + def _graphics_finding(self) -> Finding: + name = "graphics type supported by this emulator" + outcome = self._virsh("domcapabilities") + if not outcome.ok: + return Finding( + name=name, + ok=False, + detail="libvirt did not report domain capabilities", + ) + block = re.search(r"(.*?)", outcome.stdout, re.S) + supported = re.findall(r"(\w+)", block.group(1)) if block else [] + return Finding( + name=name, + ok=self.settings.graphics in supported, + detail=( + f"{self.settings.graphics} is among {', '.join(supported)}" + if self.settings.graphics in supported + else f"this emulator offers {', '.join(supported) or 'nothing'}, " + f"not {self.settings.graphics}" + ), + ) + def preflight(self) -> PreflightReport: """Report whether this host can declare and run the domain.""" findings = [] + pulumi_path = self.which(self.settings.pulumi_binary) + findings.append( + Finding( + name="pulumi on the path", + ok=bool(pulumi_path), + detail=pulumi_path or f"{self.settings.pulumi_binary} was not found", + ) + ) for tool in REQUIRED_TOOLS: located = self.which(tool) findings.append( @@ -290,23 +513,21 @@ def preflight(self) -> PreflightReport: detail=located or f"{tool} was not found on PATH", ) ) + interpreter = Path(self.settings.venv) / "bin" / "python" findings.append( Finding( - name="provider schema verified", - ok=self.settings.provider_schema_verified, - detail=( - f"{self.settings.provider}@{self.settings.provider_version} was " - "recorded as checked against the rendered program" - if self.settings.provider_schema_verified - else ( - "the rendered program's resource fields have not been checked " - f"against {self.settings.provider}@" - f"{self.settings.provider_version}; set " - "vm.provider_schema_verified once they have been" - ) - ), + name="pinned environment present", + ok=interpreter.is_file(), + detail=str(interpreter) + if interpreter.is_file() + else f"no interpreter at {interpreter}; run host/prepare-host", ) ) + if pulumi_path: + findings.append(self._state_backend_finding()) + if interpreter.is_file(): + findings.append(self._schema_finding()) + base_present = Path(self.settings.base_image).is_file() findings.append( Finding( @@ -341,48 +562,72 @@ def preflight(self) -> PreflightReport: ) ) if self.which("virsh"): - listed = self.runner( - ["virsh", "--connect", self.settings.connect_uri, "list", "--all"], - timeout=120, - context="host", - ) + connection = self._virsh("list", "--all") findings.append( Finding( name="libvirt connection answers", - ok=listed.ok, + ok=connection.ok, detail=self.settings.connect_uri - if listed.ok - else (listed.stderr or listed.stdout).strip()[:200], + if connection.ok + else (connection.stderr or connection.stdout).strip()[:200], ) ) + if connection.ok: + pools = self._virsh("pool-list", "--name") + findings.append( + Finding( + name="storage pool active", + ok=self.settings.pool in pools.stdout.split(), + detail=( + f"{self.settings.pool} is active" + if self.settings.pool in pools.stdout.split() + else f"pool {self.settings.pool} is not active; " + "run host/prepare-host" + ), + ) + ) + networks = self._virsh("net-list", "--name") + findings.append( + Finding( + name="network active", + ok=self.settings.network in networks.stdout.split(), + detail=( + f"{self.settings.network} is active" + if self.settings.network in networks.stdout.split() + else f"network {self.settings.network} is not active" + ), + ) + ) + findings.append(self._graphics_finding()) return PreflightReport(backend=self.name, findings=tuple(findings)) # -- lifecycle -------------------------------------------------------- - def _pulumi(self, arguments: Sequence[str], *, timeout: float) -> proc.CommandOutcome: - return self.runner( - [self.settings.pulumi_binary, *arguments], - timeout=timeout, - context="host", - cwd=str(self.stack_dir), - env={ - "PULUMI_BACKEND_URL": f"file://{self.run_state}", - "PULUMI_SKIP_UPDATE_CHECK": "true", - "LIBVIRT_DEFAULT_URI": self.settings.connect_uri, - }, + def discover_address(self, *, timeout: float | None = None) -> str | None: + """Poll libvirt until the transport interface has an address.""" + deadline = time.monotonic() + ( + self.settings.address_timeout_seconds if timeout is None else timeout ) + while True: + outcome = self._virsh( + "domifaddr", self.domain_name, "--source", "lease", timeout=60 + ) + for line in outcome.stdout.splitlines(): + if self.transport_mac in line: + match = _ADDRESS.search(line) + if match: + return match.group(1) + if time.monotonic() >= deadline: + return None + self.sleeper(5) def create(self) -> EnvironmentHandle: - """Create the overlay, the seed and the domain, then find its address.""" + """Declare the domain with Pulumi, then find the address it was given.""" self.prepare_identity() - overlay = self.runner(self.overlay_argv(), timeout=600, context="host") - if not overlay.ok: - raise VmContractError( - "the overlay could not be created over the preserved base: " - + (overlay.stderr or overlay.stdout).strip()[:300] - ) self.write_declarations() - self._pulumi(["stack", "init", self.stack_name, "--non-interactive"], timeout=600) + self._pulumi( + ["stack", "init", self.stack_name, "--non-interactive"], timeout=600 + ) applied = self._pulumi( ["up", "--yes", "--non-interactive", "--stack", self.stack_name], timeout=self.config.timeout_seconds, @@ -390,18 +635,14 @@ def create(self) -> EnvironmentHandle: if not applied.ok: raise VmContractError( "pulumi up did not bring the domain up: " - + (applied.stderr or applied.stdout).strip()[:400] + + (applied.stderr or applied.stdout).strip()[:500] ) - outputs = self._pulumi( - ["stack", "output", "--json", "--stack", self.stack_name], timeout=300 - ) - try: - self.address = json.loads(outputs.stdout or "{}").get("address") - except json.JSONDecodeError: - self.address = None + self.address = self.discover_address() if not self.address: raise VmContractError( - "the stack reported no address for the domain, so nothing can be run in it" + f"{self.domain_name} took no address on {self.settings.network} within " + f"{self.settings.address_timeout_seconds} seconds, so nothing can be " + "run in it" ) handle = self.plan_handle() return EnvironmentHandle( @@ -448,11 +689,7 @@ def execute( """Run a command inside the domain over the per-run transport.""" inside = list(argv) if env: - inside = [ - "env", - *[f"{name}={value}" for name, value in env.items()], - *inside, - ] + inside = ["env", *[f"{name}={value}" for name, value in env.items()], *inside] if cwd: inside = ["sh", "-c", 'cd "$1" || exit 1; shift; exec "$@"', "cycle-cd", cwd, *inside] return self.runner( @@ -538,12 +775,14 @@ def write_file(self, remote_path: str, content: str, *, mode: int = 0o600) -> No def stop(self) -> StopReport: """Shut the domain down and confirm libvirt no longer runs it.""" - outcome = self.runner( - ["virsh", "--connect", self.settings.connect_uri, "shutdown", self.domain_name], - timeout=300, - context="host", - ) + outcome = self._virsh("shutdown", self.domain_name, timeout=300) + deadline = time.monotonic() + 120 + while self._domain_running() and time.monotonic() < deadline: + self.sleeper(5) running = self._domain_running() + if running: + self._virsh("destroy", self.domain_name, timeout=120) + running = self._domain_running() return StopReport( confirmed=not running, detail=( @@ -555,12 +794,7 @@ def stop(self) -> StopReport: ) def _domain_running(self) -> bool: - listed = self.runner( - ["virsh", "--connect", self.settings.connect_uri, "domstate", self.domain_name], - timeout=120, - context="host", - ) - return "running" in listed.stdout.lower() + return "running" in self._virsh("domstate", self.domain_name, timeout=60).stdout.lower() def destroy(self) -> DestroyReport: """Destroy the stack and the per-run state; never the preserved base.""" @@ -587,6 +821,7 @@ def destroy(self) -> DestroyReport: ) if not self.run_state.exists(): removed.append(f"path:{self.run_state}") + self._virsh("pool-refresh", self.settings.pool, timeout=120) return DestroyReport( removed=tuple(removed), retained=(f"image:{self.settings.base_image}",), @@ -596,16 +831,10 @@ def destroy(self) -> DestroyReport: def resource_exists(self, resource: Resource) -> bool: """Report whether a declared resource is still on this host.""" - if resource.kind == "path": + if resource.kind in ("path", "image", "volume"): return Path(resource.identifier).exists() - if resource.kind == "image": - return Path(resource.identifier).is_file() if resource.kind == "domain": - listed = self.runner( - ["virsh", "--connect", self.settings.connect_uri, "list", "--all", "--name"], - timeout=120, - context="host", - ) + listed = self._virsh("list", "--all", "--name") return resource.identifier in listed.stdout.split() return False @@ -617,9 +846,14 @@ def describe(self) -> dict: "stack": self.stack_name, "provider": f"{self.settings.provider}@{self.settings.provider_version}", "connect_uri": self.settings.connect_uri, + "pool": self.settings.pool, + "network": self.settings.network, "base_image": str(self.settings.base_image), + "base_volume": self.settings.base_volume_name, "base_image_sha256": self.settings.base_image_sha256, - "overlay": str(self.overlay_path), "graphics": self.settings.graphics, - "provider_schema_verified": self.settings.provider_schema_verified, + "listen_address": self.settings.listen_address, + "transport_mac": self.transport_mac, + "egress_interface": self.settings.egress, + "address": self.address, } diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py index 7a302a6..f12da52 100644 --- a/experiments/minimal-cycle/cycle_runner/config.py +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -246,16 +246,20 @@ class VmConfig: base_image: Path base_image_sha256: str guest_user: str + venv: Path pulumi_binary: str = "pulumi" - connect_uri: str = "qemu:///session" - overlay_size: str = "24g" + connect_uri: str = "qemu:///system" + pool: str = "dely-cycle" + overlay_size_bytes: int = 24 * 1024 * 1024 * 1024 memory_mb: int = 4096 vcpus: int = 2 - graphics: str = "spice" + graphics: str = "vnc" + listen_address: str = "127.0.0.1" transport: str = "ssh" network: str = "default" ssh_port: int = 22 - provider_schema_verified: bool = False + address_timeout_seconds: int = 600 + egress: bool = True provision: tuple[tuple[str, ...], ...] = () def to_document(self) -> dict[str, Any]: @@ -266,19 +270,28 @@ def to_document(self) -> dict[str, Any]: "base_image": str(self.base_image), "base_image_sha256": self.base_image_sha256, "guest_user": self.guest_user, + "venv": str(self.venv), "pulumi_binary": self.pulumi_binary, "connect_uri": self.connect_uri, - "overlay_size": self.overlay_size, + "pool": self.pool, + "overlay_size_bytes": self.overlay_size_bytes, "memory_mb": self.memory_mb, "vcpus": self.vcpus, "graphics": self.graphics, + "listen_address": self.listen_address, "transport": self.transport, "network": self.network, "ssh_port": self.ssh_port, - "provider_schema_verified": self.provider_schema_verified, + "address_timeout_seconds": self.address_timeout_seconds, + "egress": self.egress, "provision": [list(argv) for argv in self.provision], } + @property + def base_volume_name(self) -> str: + """The base image's name as the storage pool knows it.""" + return self.base_image.name + @dataclass(frozen=True) class RunConfig: @@ -488,16 +501,20 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: "base_image", "base_image_sha256", "guest_user", + "venv", "pulumi_binary", "connect_uri", - "overlay_size", + "pool", + "overlay_size_bytes", "memory_mb", "vcpus", "graphics", + "listen_address", "transport", "network", "ssh_port", - "provider_schema_verified", + "address_timeout_seconds", + "egress", "provision", ), "vm", @@ -508,6 +525,9 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: digest = _text(document, "base_image_sha256", "vm") if len(digest) != 64 or any(character not in "0123456789abcdef" for character in digest): _fail("field vm.base_image_sha256 must be a lowercase content digest") + venv = Path(_text(document, "venv", "vm")) + if not venv.is_absolute(): + _fail("field vm.venv must be an absolute path to the pinned environment") return VmConfig( provider=_text(document, "provider", "vm"), provider_version=_text(document, "provider_version", "vm"), @@ -515,18 +535,24 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: base_image=base_image, base_image_sha256=digest, guest_user=_text(document, "guest_user", "vm"), + venv=venv, pulumi_binary=_text(document, "pulumi_binary", "vm", "pulumi"), - connect_uri=_text(document, "connect_uri", "vm", "qemu:///session"), - overlay_size=_text(document, "overlay_size", "vm", "24g"), + connect_uri=_text(document, "connect_uri", "vm", "qemu:///system"), + pool=_text(document, "pool", "vm", "dely-cycle"), + overlay_size_bytes=_positive_int( + document, "overlay_size_bytes", "vm", 24 * 1024 * 1024 * 1024 + ), memory_mb=_positive_int(document, "memory_mb", "vm", 4096), vcpus=_positive_int(document, "vcpus", "vm", 2), - graphics=_text(document, "graphics", "vm", "spice"), + graphics=_text(document, "graphics", "vm", "vnc"), + listen_address=_text(document, "listen_address", "vm", "127.0.0.1"), transport=_text(document, "transport", "vm", "ssh"), network=_text(document, "network", "vm", "default"), ssh_port=_positive_int(document, "ssh_port", "vm", 22), - provider_schema_verified=_flag( - document, "provider_schema_verified", "vm", False + address_timeout_seconds=_positive_int( + document, "address_timeout_seconds", "vm", 600 ), + egress=_flag(document, "egress", "vm", True), provision=_provision(document, "vm"), ) diff --git a/experiments/minimal-cycle/host/versions.json b/experiments/minimal-cycle/host/versions.json new file mode 100644 index 0000000..1f16018 --- /dev/null +++ b/experiments/minimal-cycle/host/versions.json @@ -0,0 +1,76 @@ +{ + "description": "Every version, source and digest this environment is pinned to. prepare-host verifies each one and refuses to continue when a digest does not match.", + "host_tools": { + "pulumi_cli": { + "version": "3.262.0", + "installer": "https://get.pulumi.com", + "install_flag": "--version 3.262.0", + "binary": "~/.pulumi/bin/pulumi", + "backend": "file:// (pulumi login --local); Pulumi Cloud is never used" + }, + "packer": { + "version": "1.16.0", + "note": "already present on the development host; the template pins the qemu plugin itself" + }, + "qemu": {"version": "10.2.2"}, + "libvirt": {"version": "12.0.0"}, + "connect_uri": "qemu:///system", + "network": "default", + "storage_pool": { + "name": "dely-cycle", + "type": "dir", + "target": "/var/tmp/dely-cycle/images" + } + }, + "python_environment": { + "interpreter": "python3", + "venv": "/var/tmp/dely-cycle/venv", + "packages": { + "pulumi": "3.262.0", + "pulumi-libvirt": "0.5.4" + }, + "provider_plugin": "libvirt-0.5.4" + }, + "base_image": { + "distribution": "Ubuntu Server 24.04 LTS (Noble)", + "architecture": "amd64", + "kind": "cloud image for qemu and libvirt, not a container image", + "url": "https://cloud-images.ubuntu.com/releases/noble/release/ubuntu-24.04-server-cloudimg-amd64.img", + "checksums_url": "https://cloud-images.ubuntu.com/releases/noble/release/SHA256SUMS", + "file": "ubuntu-24.04-server-cloudimg-amd64.img", + "sha256": "612b2c0cc1bc413a6cb8c38fd611794caf0f2b436c50013d8b3794db12ad7354", + "format": "qcow2", + "virtual_size_bytes": 3758096384, + "download_size_bytes": 625256960 + }, + "tool_image": { + "file": "tool-image.qcow2", + "built_by": "packer, qemu builder, from the base image above", + "contents": { + "orca": { + "version": "1.4.201", + "package": "orca-ide_1.4.201_amd64.deb", + "url": "https://github.com/stablyai/orca/releases/download/v1.4.201/orca-ide_1.4.201_amd64.deb", + "sha256": "9dab79a5223837fc91992e04ed54f92f45d6f4b486d7128c08166bfb1a15a573", + "size_bytes": 165333236, + "note": "the amd64 package only; the rpm is never installed alongside it" + }, + "claude_code": { + "version": "2.1.272", + "package": "@anthropic-ai/claude-code", + "registry": "https://registry.npmjs.org", + "note": "installed from the registry at the pinned version, never at latest" + }, + "skills": { + "note": "skills are an explicit, pinned part of the image; the runner never fetches an unpinned one at run time" + } + }, + "excludes": [ + "host credentials", + "tokens", + "cookies", + "private keys", + "any part of the host home directory" + ] + } +} diff --git a/experiments/minimal-cycle/tests/test_adapter_vm.py b/experiments/minimal-cycle/tests/test_adapter_vm.py index c9f5102..7c404b9 100644 --- a/experiments/minimal-cycle/tests/test_adapter_vm.py +++ b/experiments/minimal-cycle/tests/test_adapter_vm.py @@ -1,64 +1,103 @@ -"""The machine adapter renders a per-run domain over a base it never writes.""" +"""The machine adapter declares a per-run domain over a base it never writes.""" import hashlib -import shutil -import subprocess +import json +import os import tempfile import unittest from pathlib import Path from cycle_runner import cleanup, config as config_module, proc -from cycle_runner.adapters import vm +from cycle_runner.adapters import schema, vm from tests.test_adapter_distrobox import StubRunner from tests.test_config import minimal_document RUN_ID = "20260914T221530Z-abc123-0123abcd" +OTHER_RUN_ID = "20260914T221530Z-abc123-0123abce" TOKEN = "sk-ant-api-zzqwertyuiopasdfghjklzxcvbnmqwertyuiopasdfgh" +DOMCAPS = """ + + + + sdl + vnc + egl-headless + + + +""" + +HEALTHY = [ + ("whoami -v", 0, "User: someone\nBackend URL: file://~\n", ""), + ("domcapabilities", 0, DOMCAPS, ""), + ("pool-list", 0, "dely-cycle\ndefault\n", ""), + ("net-list", 0, "default\n", ""), + ("list --all", 0, "", ""), +] + class VmTestCase(unittest.TestCase): def setUp(self): self._tmp = tempfile.TemporaryDirectory() self.root = Path(self._tmp.name) - self.shared = self.root / "shared" - self.shared.mkdir() - self.base = self.shared / "tool-image.qcow2" - self.base.write_bytes(b"pretend this is a prepared tool image") + self.images = self.root / "images" + self.images.mkdir() + self.base = self.images / "tool-image.qcow2" + self.base.write_bytes(b"pretend this is a prepared ubuntu tool image") self.base_digest = hashlib.sha256(self.base.read_bytes()).hexdigest() + self.venv = self.root / "venv" + (self.venv / "bin").mkdir(parents=True) + # A stand-in for the prepared environment: it answers the provider probe + # with "no problems", so these tests exercise the rest of preflight. The + # real check runs in ProviderSchemaIntegrationTest against the pinned + # provider itself. + (self.venv / "bin" / "python").write_text( + "#!/bin/sh\ncat > /dev/null\necho '[]'\n", encoding="utf-8" + ) + (self.venv / "bin" / "python").chmod(0o755) + self.mute_venv = self.root / "mute-venv" + (self.mute_venv / "bin").mkdir(parents=True) + (self.mute_venv / "bin" / "python").write_text( + "#!/bin/sh\ncat > /dev/null\nexit 0\n", encoding="utf-8" + ) + (self.mute_venv / "bin" / "python").chmod(0o755) self.addCleanup(self._tmp.cleanup) - def make(self, *, runner=None, present=("pulumi", "qemu-img", "virsh", "ssh", "scp"), **overrides): + def make(self, *, runner=None, run_id=RUN_ID, present=None, **overrides): + present = present or ("pulumi", "qemu-img", "virsh", "ssh", "scp") document = minimal_document(backend="vm") document.pop("distrobox") document["state_root"] = str(self.root / "state") document["artifact_root"] = str(self.root / "artifacts") section = { "provider": "pulumi-libvirt", - "provider_version": "0.5.3", + "provider_version": "0.5.4", "stack_prefix": "dely-cycle", "base_image": str(self.base), "base_image_sha256": self.base_digest, "guest_user": "cycle", - "provider_schema_verified": True, + "venv": str(self.venv), } section.update(overrides) document["vm"] = section run_config = config_module.from_document(document) return vm.VmAdapter( run_config=run_config, - run_id=RUN_ID, - runner=runner or StubRunner(passthrough=True), + run_id=run_id, + runner=runner or StubRunner(HEALTHY, passthrough=True), which=lambda name: f"/usr/bin/{name}" if name in present else None, + sleeper=lambda _seconds: None, ) class SeedTest(VmTestCase): def test_the_seed_carries_no_credential(self): adapter = self.make() - adapter.prepare_identity() + from cycle_runner import redact + user_data = adapter.render_user_data() - self.assertNotIn(TOKEN, user_data) - self.assertNotIn("PRIVATE KEY", user_data) + self.assertFalse(redact.carries_credential_shape(user_data)) self.assertNotIn(adapter.private_key_path.read_text(encoding="utf-8"), user_data) self.assertNotIn("password", user_data.lower()) @@ -66,9 +105,7 @@ def test_the_seed_authorises_only_the_per_run_public_key(self): adapter = self.make() adapter.prepare_identity() public_key = adapter.public_key_path.read_text(encoding="utf-8").strip() - user_data = adapter.render_user_data() - self.assertIn(public_key, user_data) - self.assertEqual(user_data.count("ssh-"), user_data.count(public_key.split()[0])) + self.assertIn(public_key, adapter.render_user_data()) def test_the_seed_names_the_run_and_a_distinct_host_name(self): adapter = self.make() @@ -79,169 +116,212 @@ def test_the_seed_names_the_run_and_a_distinct_host_name(self): def test_the_private_key_never_leaves_the_per_run_state(self): adapter = self.make() adapter.prepare_identity() - self.assertTrue( - adapter.private_key_path.is_relative_to(adapter.run_state) - ) + self.assertTrue(adapter.private_key_path.is_relative_to(adapter.run_state)) self.assertEqual(adapter.private_key_path.stat().st_mode & 0o777, 0o600) -class OverlayTest(VmTestCase): - def test_the_overlay_command_names_the_base_as_a_backing_file(self): +class InterfaceTest(VmTestCase): + def test_the_two_interfaces_have_distinct_per_run_addresses(self): adapter = self.make() - argv = adapter.overlay_argv() - self.assertEqual(argv[:3], ["qemu-img", "create", "-f"]) - self.assertIn("-b", argv) - self.assertEqual(argv[argv.index("-b") + 1], str(self.base)) - self.assertEqual(argv[-2], str(adapter.overlay_path)) - - def test_the_base_is_never_the_output_path(self): + self.assertNotEqual(adapter.transport_mac, adapter.egress_mac) + self.assertTrue(adapter.transport_mac.startswith("52:54:00:")) + self.assertTrue(adapter.egress_mac.startswith("52:54:01:")) + + def test_the_addresses_are_stable_for_a_run_and_differ_between_runs(self): + first = self.make().transport_mac + again = self.make().transport_mac + other = self.make(run_id=OTHER_RUN_ID).transport_mac + self.assertEqual(first, again) + self.assertNotEqual(first, other) + + def test_the_transform_adds_a_user_mode_interface_with_that_address(self): adapter = self.make() - argv = adapter.overlay_argv() - self.assertNotEqual(argv[-2], str(self.base)) - self.assertFalse(Path(argv[-2]).is_relative_to(self.shared)) - - def test_a_wrong_base_digest_blocks(self): - adapter = self.make(base_image_sha256="0" * 64) - report = adapter.preflight() - self.assertFalse(report.ok) - self.assertTrue(any("digest" in finding.name for finding in report.blockers)) + transform = adapter.render_egress_xslt() + self.assertIn('', transform) + self.assertIn(adapter.egress_mac, transform) - def test_a_missing_base_blocks(self): - self.base.unlink() + def test_the_guest_prefers_the_egress_route(self): adapter = self.make() - report = adapter.preflight() - self.assertFalse(report.ok) - self.assertTrue(any("base image" in finding.name for finding in report.blockers)) + config_text = adapter.render_network_config() + transport_metric = config_text.index("route-metric: 300") + egress_metric = config_text.index("route-metric: 100") + self.assertLess(transport_metric, egress_metric) + self.assertIn(adapter.transport_mac, config_text) + self.assertIn(adapter.egress_mac, config_text) + + def test_disabling_egress_leaves_only_the_transport_interface(self): + adapter = self.make(egress=False) + config_text = adapter.render_network_config() + self.assertIn(adapter.transport_mac, config_text) + self.assertNotIn(adapter.egress_mac, config_text) + self.assertNotIn("route-metric", config_text) class ProgramTest(VmTestCase): def test_the_rendered_program_is_valid_python(self): compile(self.make().render_program(), "__main__.py", "exec") - def test_the_program_pins_the_provider_version(self): + def test_the_disk_is_the_pool_volume_not_a_file_path(self): + program = self.make().render_program() + self.assertIn("DomainDiskArgs(volume_id=overlay.id)", program) + self.assertNotIn("DomainDiskArgs(file=", program) + + def test_the_overlay_is_backed_by_the_base_volume_in_the_pool(self): adapter = self.make() - project_file = adapter.render_project() - self.assertIn("0.5.3", project_file) - self.assertIn("pulumi-libvirt", project_file) + program = adapter.render_program() + self.assertIn("base_volume_name=BASE_VOLUME", program) + self.assertIn("base_volume_pool=POOL", program) + self.assertIn(f"BASE_VOLUME = {adapter.settings.base_volume_name!r}", program) - def test_the_program_names_the_overlay_the_seed_and_the_graphics(self): - program = self.make().render_program() - for needle in ("OVERLAY_PATH", "USER_DATA", "META_DATA", "GRAPHICS"): - self.assertIn(needle, program) + def test_the_program_never_names_the_base_image_as_an_output(self): + adapter = self.make() + program = adapter.render_program() + self.assertIn("-overlay.qcow2", program) + self.assertNotIn(f'name={adapter.settings.base_volume_name!r}', program) + + def test_the_project_file_points_at_the_pinned_environment(self): + adapter = self.make() + project = adapter.render_project() + self.assertIn("runtime:", project) + self.assertIn("name: python", project) + self.assertIn(f"virtualenv: {self.venv}", project) def test_the_stack_settings_carry_no_secret(self): + from cycle_runner import redact + + self.assertFalse( + redact.carries_credential_shape(self.make().render_stack_settings()) + ) + + def test_the_stack_passphrase_is_never_written_to_a_file(self): adapter = self.make() - settings = adapter.render_stack_settings() - self.assertNotIn(TOKEN, settings) - self.assertNotIn("PRIVATE KEY", settings) + adapter.write_declarations() + for path in sorted(adapter.stack_dir.rglob("*")): + if path.is_file(): + self.assertNotIn(adapter._passphrase, path.read_text(encoding="utf-8")) - def test_the_program_never_writes_the_base_image(self): + def test_the_passphrase_reaches_pulumi_only_through_the_environment(self): adapter = self.make() - program = adapter.render_program() - self.assertIn(str(adapter.overlay_path), program) - self.assertNotIn(f'"{self.base}"', program) + self.assertEqual( + adapter._pulumi_environment()["PULUMI_CONFIG_PASSPHRASE"], adapter._passphrase + ) + self.assertTrue( + adapter._pulumi_environment()["PULUMI_BACKEND_URL"].startswith("file://") + ) class ResourceTest(VmTestCase): - def test_the_base_image_is_declared_shared_and_stays_out_of_the_destroy_plan(self): - adapter = self.make() - handle = adapter.plan_handle() + def test_the_base_image_is_shared_and_stays_out_of_the_destroy_plan(self): + handle = self.make().plan_handle() per_run = {str(item) for item in handle.per_run_resources} shared = {str(item) for item in handle.shared_resources} self.assertIn(f"image:{self.base}", shared) - self.assertNotIn(f"image:{self.base}", per_run) for item in per_run: self.assertNotIn(str(self.base), item) + def test_the_per_run_resources_name_the_domain_overlay_seed_and_state(self): + handle = self.make().plan_handle() + kinds = sorted(item.kind for item in handle.per_run_resources) + self.assertEqual(kinds, ["domain", "path", "volume", "volume"]) + def test_the_declarations_do_not_overlap(self): handle = self.make().plan_handle() cleanup.assert_declarations_disjoint( per_run=handle.per_run_resources, shared=handle.shared_resources ) - def test_the_per_run_resources_name_the_domain_the_overlay_and_the_stack(self): - handle = self.make().plan_handle() - kinds = {item.kind for item in handle.per_run_resources} - self.assertEqual(kinds, {"domain", "path"}) + +class AddressDiscoveryTest(VmTestCase): + def test_the_address_for_the_transport_interface_is_found(self): + adapter = self.make() + lease = ( + " Name MAC address Protocol Address\n" + "-------------------------------------------------------\n" + f" vnet0 {self.make().transport_mac} ipv4 192.168.122.61/24\n" + ) + adapter.runner = StubRunner([("domifaddr", 0, lease, "")]) + self.assertEqual(adapter.discover_address(timeout=1), "192.168.122.61") + + def test_another_interface_address_is_not_mistaken_for_it(self): + adapter = self.make() + lease = " vnet1 52:54:09:aa:bb:cc ipv4 10.0.2.15/24\n" + adapter.runner = StubRunner([("domifaddr", 0, lease, "")]) + self.assertIsNone(adapter.discover_address(timeout=0)) + + def test_no_address_within_the_deadline_returns_nothing(self): + adapter = self.make() + adapter.runner = StubRunner([("domifaddr", 0, "", "")]) + self.assertIsNone(adapter.discover_address(timeout=0)) class PreflightTest(VmTestCase): - def test_a_missing_pulumi_blocks(self): - adapter = self.make(present=("qemu-img", "virsh", "ssh", "scp")) + def test_a_ready_host_passes(self): + adapter = self.make() report = adapter.preflight() - self.assertFalse(report.ok) - self.assertTrue(any("pulumi" in finding.name for finding in report.blockers)) + blockers = [f"{f.name}: {f.detail}" for f in report.blockers] + self.assertTrue(report.ok, blockers) - def test_an_unverified_provider_schema_blocks(self): - adapter = self.make(provider_schema_verified=False) - report = adapter.preflight() + def test_a_missing_pulumi_blocks(self): + report = self.make(present=("qemu-img", "virsh", "ssh", "scp")).preflight() self.assertFalse(report.ok) - self.assertTrue( - any("provider schema" in finding.name for finding in report.blockers) + self.assertTrue(any("pulumi" in f.name for f in report.blockers)) + + def test_a_cloud_state_backend_blocks(self): + runner = StubRunner( + [("whoami -v", 0, "Backend URL: https://api.pulumi.com\n", "")] + HEALTHY, + passthrough=True, ) + report = self.make(runner=runner).preflight() + self.assertFalse(report.ok) + blocker = next(f for f in report.blockers if "state backend" in f.name) + self.assertIn("pulumi login --local", blocker.detail) - def test_a_missing_hardware_acceleration_device_is_reported(self): - adapter = self.make() - report = adapter.preflight() - names = [finding.name for finding in report.findings] - self.assertIn("hardware acceleration device", names) + def test_an_environment_that_cannot_answer_the_probe_blocks(self): + report = self.make(venv=str(self.mute_venv)).preflight() + self.assertFalse(report.ok) + blocker = next(f for f in report.blockers if "provider schema" in f.name) + self.assertIn("could not be checked", blocker.detail) - def test_a_ready_host_passes(self): - runner = StubRunner([("virsh", 0, " Id Name State\n", "")]) - adapter = self.make(runner=runner) - report = adapter.preflight() - if Path("/dev/kvm").exists(): - self.assertTrue(report.ok, [item.detail for item in report.blockers]) + def test_a_missing_pinned_environment_blocks(self): + report = self.make(venv=str(self.root / "absent-venv")).preflight() + self.assertFalse(report.ok) + self.assertTrue(any("pinned environment" in f.name for f in report.blockers)) + + def test_a_graphics_type_this_emulator_lacks_blocks(self): + report = self.make(graphics="spice").preflight() + self.assertFalse(report.ok) + blocker = next(f for f in report.blockers if "graphics" in f.name) + self.assertIn("vnc", blocker.detail) + + def test_an_absent_storage_pool_blocks(self): + runner = StubRunner( + [("pool-list", 0, "default\n", "")] + HEALTHY, passthrough=True + ) + report = self.make(runner=runner).preflight() + self.assertFalse(report.ok) + self.assertTrue(any("storage pool" in f.name for f in report.blockers)) + + def test_a_wrong_base_digest_blocks(self): + report = self.make(base_image_sha256="0" * 64).preflight() + self.assertFalse(report.ok) + self.assertTrue(any("digest" in f.name for f in report.blockers)) class TransportTest(VmTestCase): def test_the_transport_argv_is_non_interactive_and_uses_the_per_run_key(self): adapter = self.make() adapter.prepare_identity() - adapter.address = "192.0.2.10" + adapter.address = "192.168.122.61" argv = adapter.ssh_argv(["true"]) self.assertEqual(argv[0], "ssh") self.assertIn("BatchMode=yes", " ".join(argv)) self.assertIn(str(adapter.private_key_path), argv) - self.assertIn("cycle@192.0.2.10", argv) + self.assertIn("cycle@192.168.122.61", argv) self.assertEqual(argv[-1], "true") def test_the_transport_refuses_to_run_before_an_address_is_known(self): - adapter = self.make() with self.assertRaises(vm.VmContractError): - adapter.ssh_argv(["true"]) - - -@unittest.skipUnless(shutil.which("qemu-img"), "qemu-img is not on this host") -class OverlayIntegrationTest(VmTestCase): - def test_the_overlay_is_backed_by_the_base_and_the_base_is_unchanged(self): - base = self.shared / "real-base.qcow2" - subprocess.run( - ["qemu-img", "create", "-f", "qcow2", str(base), "16M"], - check=True, - capture_output=True, - ) - before = hashlib.sha256(base.read_bytes()).hexdigest() - adapter = self.make( - base_image=str(base), - base_image_sha256=before, - runner=proc.run, - ) - adapter.run_state.mkdir(parents=True, exist_ok=True) - outcome = proc.run(adapter.overlay_argv(), timeout=120, context="host") - self.assertEqual(outcome.exit_code, 0, outcome.stderr) - chain = subprocess.run( - ["qemu-img", "info", "--backing-chain", str(adapter.overlay_path)], - capture_output=True, - text=True, - check=True, - ) - self.assertIn(str(base), chain.stdout) - self.assertEqual(hashlib.sha256(base.read_bytes()).hexdigest(), before) - - -class TransportRedactionTest(VmTestCase): - """A token forwarded into the guest must not survive into captured output.""" + self.make().ssh_argv(["true"]) def test_a_forwarded_value_is_redacted_from_what_is_captured(self): runner = StubRunner( @@ -249,19 +329,45 @@ def test_a_forwarded_value_is_redacted_from_what_is_captured(self): ) adapter = self.make(runner=runner) adapter.prepare_identity() - adapter.address = "192.0.2.10" + adapter.address = "192.168.122.61" outcome = adapter.execute( ["echo", "x"], timeout=5, env={"CLAUDE_CODE_OAUTH_TOKEN": TOKEN} ) self.assertNotIn(TOKEN, outcome.stdout) self.assertIn(TOKEN, runner.extra_values_seen[-1]) - def test_the_forwarded_name_and_value_reach_the_guest_command(self): - runner = StubRunner(passthrough=True) - adapter = self.make(runner=runner) - adapter.prepare_identity() - adapter.address = "192.0.2.10" - adapter.execute(["true"], timeout=5, env={"CLAUDE_CODE_OAUTH_TOKEN": TOKEN}) - argv = runner.seen[-1] - self.assertIn("env", argv) - self.assertIn(f"CLAUDE_CODE_OAUTH_TOKEN={TOKEN}", argv) + +@unittest.skipUnless( + Path(os.environ.get("DELY_CYCLE_VENV", "/var/tmp/dely-cycle/venv"), "bin", "python").is_file(), + "no prepared environment to check the provider against", +) +class ProviderSchemaIntegrationTest(VmTestCase): + """The provider itself is the instrument: the program is checked against it.""" + + def prepared(self): + return Path(os.environ.get("DELY_CYCLE_VENV", "/var/tmp/dely-cycle/venv")) + + def test_the_program_names_only_classes_and_fields_the_provider_has(self): + adapter = self.make(venv=str(self.prepared())) + ran, problems = schema.verify_with_interpreter( + adapter.render_program(), self.prepared() / "bin" / "python" + ) + self.assertTrue(ran, problems) + self.assertEqual(problems, []) + + def test_a_field_the_provider_lacks_is_caught(self): + adapter = self.make(venv=str(self.prepared())) + program = adapter.render_program().replace( + "qemu_agent=True,", "qemu_agent=True,\n nonexistent_field=1," + ) + ran, problems = schema.verify_with_interpreter( + program, self.prepared() / "bin" / "python" + ) + self.assertTrue(ran) + self.assertTrue(any("nonexistent_field" in problem for problem in problems), problems) + + def test_the_preflight_finding_reports_the_real_check(self): + adapter = self.make(venv=str(self.prepared())) + finding = adapter._schema_finding() + self.assertTrue(finding.ok, finding.detail) + self.assertIn("0.5.4", finding.detail) diff --git a/experiments/minimal-cycle/tests/test_cli.py b/experiments/minimal-cycle/tests/test_cli.py index be94f79..b48eeb2 100644 --- a/experiments/minimal-cycle/tests/test_cli.py +++ b/experiments/minimal-cycle/tests/test_cli.py @@ -64,6 +64,7 @@ def test_a_blocked_backend_exits_with_the_blocked_code(self): "base_image": str(self.root / "absent.qcow2"), "base_image_sha256": "0" * 64, "guest_user": "cycle", + "venv": str(self.root / "absent-venv"), } path.write_text(json.dumps(document), encoding="utf-8") code, out, _ = self.invoke(["preflight", "--config", str(path)]) diff --git a/experiments/minimal-cycle/tests/test_provider_schema.py b/experiments/minimal-cycle/tests/test_provider_schema.py new file mode 100644 index 0000000..a5b8b0a --- /dev/null +++ b/experiments/minimal-cycle/tests/test_provider_schema.py @@ -0,0 +1,103 @@ +"""The rendered program is checked against the provider it is pinned to.""" + +import textwrap +import unittest + +from cycle_runner.adapters import schema + +PROGRAM = textwrap.dedent( + """ + import pulumi + import pulumi_libvirt as libvirt + + overlay = libvirt.Volume( + "overlay", name="x", pool="p", base_volume_name="b", size=1, format="qcow2" + ) + seed = libvirt.CloudInitDisk("seed", name="s", pool="p", user_data="u", meta_data="m") + domain = libvirt.Domain( + "domain", + name="d", + memory=1024, + disks=[libvirt.DomainDiskArgs(volume_id=overlay.id)], + graphics=libvirt.DomainGraphicsArgs(type="vnc"), + ) + pulumi.export("address", domain.name) + """ +) + + +class FakeAttribute: + def __init__(self, properties): + self._properties = properties + + +def fake_module(classes): + module = type("FakeLibvirt", (), {})() + for name, properties in classes.items(): + holder = type(name, (), {p: property(lambda self: None) for p in properties}) + setattr(module, name, holder) + return module + + +class DeclaredSymbolsTest(unittest.TestCase): + def test_every_provider_call_and_keyword_is_collected(self): + found = schema.declared_symbols(PROGRAM) + self.assertEqual( + set(found), + {"Volume", "CloudInitDisk", "Domain", "DomainDiskArgs", "DomainGraphicsArgs"}, + ) + self.assertIn("base_volume_name", found["Volume"]) + self.assertIn("volume_id", found["DomainDiskArgs"]) + self.assertIn("meta_data", found["CloudInitDisk"]) + + def test_calls_on_other_modules_are_ignored(self): + found = schema.declared_symbols(PROGRAM) + self.assertNotIn("export", found) + + def test_the_positional_resource_name_is_not_a_keyword(self): + found = schema.declared_symbols(PROGRAM) + self.assertNotIn("overlay", found["Volume"]) + + def test_a_program_that_does_not_parse_is_refused(self): + with self.assertRaises(SyntaxError): + schema.declared_symbols("def broken(:") + + def test_the_alias_is_taken_from_the_import(self): + found = schema.declared_symbols( + "import pulumi_libvirt as lv\nx = lv.Domain('d', name='n')\n" + ) + self.assertEqual(set(found), {"Domain"}) + + +class VerifyAgainstTest(unittest.TestCase): + def test_a_matching_module_reports_no_problem(self): + module = fake_module( + { + "Volume": {"name", "pool", "base_volume_name", "size", "format"}, + "CloudInitDisk": {"name", "pool", "user_data", "meta_data"}, + "Domain": {"name", "memory", "disks", "graphics"}, + "DomainDiskArgs": {"volume_id"}, + "DomainGraphicsArgs": {"type"}, + } + ) + self.assertEqual(schema.verify_against(schema.declared_symbols(PROGRAM), module), []) + + def test_a_class_the_provider_does_not_have_is_named(self): + module = fake_module({"Volume": {"name"}}) + problems = schema.verify_against({"CloudInItDisk": {"name"}}, module) + self.assertEqual(len(problems), 1) + self.assertIn("CloudInItDisk", problems[0]) + + def test_a_keyword_the_provider_does_not_have_is_named(self): + module = fake_module({"Domain": {"name", "memory"}}) + problems = schema.verify_against({"Domain": {"name", "cloudinit"}}, module) + self.assertEqual(len(problems), 1) + self.assertIn("cloudinit", problems[0]) + self.assertIn("Domain", problems[0]) + + def test_every_problem_is_reported_not_just_the_first(self): + module = fake_module({"Domain": {"name"}}) + problems = schema.verify_against( + {"Domain": {"name", "one", "two"}, "Absent": {"x"}}, module + ) + self.assertEqual(len(problems), 3) From 2caf65b1b84784db6392fd19d9d437fdff633802 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 12:44:10 +0200 Subject: [PATCH 14/42] Quote the remote command so an argument vector survives the transport, and report a failed command by its tail and exit code. --- experiments/minimal-cycle/counterexamples.py | 8 + .../minimal-cycle/cycle_runner/adapters/vm.py | 50 +++- .../minimal-cycle/cycle_runner/config.py | 4 + .../host/packer/build-tool-image | 168 ++++++++++++ .../minimal-cycle/host/packer/provision.sh | 77 ++++++ .../host/packer/tool-image.pkr.hcl | 130 +++++++++ experiments/minimal-cycle/host/prepare-host | 258 ++++++++++++++++++ .../minimal-cycle/tests/test_adapter_vm.py | 75 ++++- 8 files changed, 758 insertions(+), 12 deletions(-) create mode 100755 experiments/minimal-cycle/host/packer/build-tool-image create mode 100755 experiments/minimal-cycle/host/packer/provision.sh create mode 100644 experiments/minimal-cycle/host/packer/tool-image.pkr.hcl create mode 100755 experiments/minimal-cycle/host/prepare-host diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 420c06b..dc129d8 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,14 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="remote-command-is-quoted", + requirement="An argument vector survives the transport intact", + path="cycle_runner/adapters/vm.py", + original=" shlex.join(str(item) for item in argv),", + replacement=" *[str(item) for item in argv],", + instruments=("tests.test_adapter_vm.RemoteQuotingTest",), + ), Counterexample( name="forwarded-value-is-redacted", requirement="A value forwarded into the guest is redacted from captured output", diff --git a/experiments/minimal-cycle/cycle_runner/adapters/vm.py b/experiments/minimal-cycle/cycle_runner/adapters/vm.py index 3d70bb8..793c351 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/vm.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/vm.py @@ -17,6 +17,7 @@ import hashlib import re +import shlex import shutil import time from pathlib import Path @@ -81,6 +82,7 @@ MEMORY_MB = {memory_mb!r} VCPUS = {vcpus!r} OVERLAY_SIZE_BYTES = {overlay_size_bytes!r} +QEMU_AGENT = {qemu_agent!r} EGRESS_XSLT = {egress_xslt!r} overlay = libvirt.Volume( @@ -108,7 +110,7 @@ memory=MEMORY_MB, vcpu=VCPUS, running=True, - qemu_agent=True, + qemu_agent=QEMU_AGENT, cloudinit=seed.id, disks=[libvirt.DomainDiskArgs(volume_id=overlay.id)], network_interfaces=[ @@ -187,6 +189,19 @@ class VmContractError(RuntimeError): """The machine backend was asked for something it refuses to do.""" +def _describe(outcome: proc.CommandOutcome, *, tail: int = 600) -> str: + """Describe a failed command by its outcome and the end of its output. + + The end, not the beginning: a tool that prints a plan before it acts puts + the plan first and the reason it stopped last. + """ + text = (outcome.stderr.strip() or outcome.stdout.strip())[-tail:] + return ( + f"exit={outcome.exit_code} timed_out={outcome.timed_out} " + f"elapsed={outcome.elapsed_seconds}s: ...{text}" + ) + + def _mac(prefix: str, run_id: str) -> str: digest = hashlib.sha256(f"{prefix}:{run_id}".encode("utf-8")).hexdigest() return f"{prefix}:{digest[0:2]}:{digest[2:4]}:{digest[4:6]}" @@ -225,8 +240,12 @@ def __init__( self.transport_mac = _mac("52:54:00", run_id) self.egress_mac = _mac("52:54:01", run_id) self.address: str | None = None + # The file backend requires a passphrase even for a stack that holds no + # secret. It is derived from the run identifier so a failed run's stack + # can still be destroyed afterwards; it protects nothing, and nothing + # secret is ever put in this stack. self._passphrase = hashlib.sha256( - f"ephemeral:{run_id}:{id(self)}".encode("utf-8") + f"dely-cycle-stack:{run_id}".encode("utf-8") ).hexdigest() # -- per-run identity ------------------------------------------------- @@ -305,6 +324,7 @@ def render_program(self) -> str: memory_mb=self.settings.memory_mb, vcpus=self.settings.vcpus, overlay_size_bytes=self.settings.overlay_size_bytes, + qemu_agent=self.settings.qemu_agent, egress_xslt=self.render_egress_xslt() if self.settings.egress else "", ) @@ -625,17 +645,21 @@ def create(self) -> EnvironmentHandle: """Declare the domain with Pulumi, then find the address it was given.""" self.prepare_identity() self.write_declarations() - self._pulumi( + initialised = self._pulumi( ["stack", "init", self.stack_name, "--non-interactive"], timeout=600 ) + if not initialised.ok: + raise VmContractError( + "pulumi stack init did not create the per-run stack: " + + _describe(initialised) + ) applied = self._pulumi( ["up", "--yes", "--non-interactive", "--stack", self.stack_name], timeout=self.config.timeout_seconds, ) if not applied.ok: raise VmContractError( - "pulumi up did not bring the domain up: " - + (applied.stderr or applied.stdout).strip()[:500] + "pulumi up did not bring the domain up: " + _describe(applied) ) self.address = self.discover_address() if not self.address: @@ -655,7 +679,13 @@ def create(self) -> EnvironmentHandle: ) def ssh_argv(self, argv: Sequence[str]) -> list[str]: - """Return the argv that runs a command inside the domain.""" + """Return the argv that runs a command inside the domain. + + ssh concatenates everything after the destination into one string and + the guest's shell re-parses it, so an argument vector has to be quoted + before it is handed over. Sending one quoted string makes that + concatenation a no-op and the guest sees exactly this vector. + """ if not self.address: raise VmContractError( "the domain has no known address yet; nothing may be run in it" @@ -674,7 +704,7 @@ def ssh_argv(self, argv: Sequence[str]) -> list[str]: str(self.private_key_path), f"{self.settings.guest_user}@{self.address}", "--", - *[str(item) for item in argv], + shlex.join(str(item) for item in argv), ] def execute( @@ -730,8 +760,7 @@ def put_tree(self, local_dir: Path, remote_dir: str) -> None: ) if not outcome.ok: raise VmContractError( - "the project copy could not be placed in the domain: " - + (outcome.stderr or outcome.stdout).strip()[:300] + "the project copy could not be placed in the domain: " + _describe(outcome) ) def fetch_tree(self, remote_dir: str, local_dir: Path) -> None: @@ -747,8 +776,7 @@ def fetch_tree(self, remote_dir: str, local_dir: Path) -> None: ) if not outcome.ok: raise VmContractError( - "the project tree could not be brought out of the domain: " - + (outcome.stderr or outcome.stdout).strip()[:300] + "the project tree could not be brought out of the domain: " + _describe(outcome) ) def write_file(self, remote_path: str, content: str, *, mode: int = 0o600) -> None: diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py index f12da52..0e21ebe 100644 --- a/experiments/minimal-cycle/cycle_runner/config.py +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -260,6 +260,7 @@ class VmConfig: ssh_port: int = 22 address_timeout_seconds: int = 600 egress: bool = True + qemu_agent: bool = False provision: tuple[tuple[str, ...], ...] = () def to_document(self) -> dict[str, Any]: @@ -284,6 +285,7 @@ def to_document(self) -> dict[str, Any]: "ssh_port": self.ssh_port, "address_timeout_seconds": self.address_timeout_seconds, "egress": self.egress, + "qemu_agent": self.qemu_agent, "provision": [list(argv) for argv in self.provision], } @@ -515,6 +517,7 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: "ssh_port", "address_timeout_seconds", "egress", + "qemu_agent", "provision", ), "vm", @@ -553,6 +556,7 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: document, "address_timeout_seconds", "vm", 600 ), egress=_flag(document, "egress", "vm", True), + qemu_agent=_flag(document, "qemu_agent", "vm", False), provision=_provision(document, "vm"), ) diff --git a/experiments/minimal-cycle/host/packer/build-tool-image b/experiments/minimal-cycle/host/packer/build-tool-image new file mode 100755 index 0000000..a7cbe78 --- /dev/null +++ b/experiments/minimal-cycle/host/packer/build-tool-image @@ -0,0 +1,168 @@ +#!/usr/bin/env python3 +"""Build the one tool image the machine backend runs, and record what it is. + +Reads ../versions.json, boots the verified base image under packer's qemu +builder, installs the pinned Orca, Claude Code and a minimal graphical session, +and leaves `tool-image.qcow2` in the storage pool with a metadata file naming +every version and digest that went into it. + + build-tool-image build, replacing any previous image + build-tool-image --check report whether a current image already exists + +The build key is generated per build, lives only in a temporary directory, and +the provisioner removes its public half from the image before shutdown. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import shutil +import subprocess +import sys +import tempfile +from datetime import datetime, timezone +from pathlib import Path + +HERE = Path(__file__).resolve().parent +VERSIONS = json.loads((HERE.parent / "versions.json").read_text(encoding="utf-8")) +ROOT = Path(os.environ.get("DELY_CYCLE_ROOT", "/var/tmp/dely-cycle")) +IMAGES = ROOT / "images" +IMAGE_NAME = VERSIONS["tool_image"]["file"] +METADATA = IMAGES / "tool-image.json" + + +def digest_of(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +def check() -> int: + """Report whether a built image and its metadata are already present.""" + image = IMAGES / IMAGE_NAME + if not image.is_file(): + print(f"BLOCK no tool image at {image}") + return 1 + if not METADATA.is_file(): + print(f"BLOCK {image} has no metadata beside it") + return 1 + recorded = json.loads(METADATA.read_text(encoding="utf-8")) + found = digest_of(image) + if found != recorded.get("sha256"): + print(f"BLOCK {image} has digest {found}, the metadata records {recorded.get('sha256')}") + return 1 + print(f"ok {image}") + print(f" sha256 {found}") + for key in ("orca_version", "claude_code_version", "base_image_sha256", "built_at"): + print(f" {key}: {recorded.get(key)}") + return 0 + + +def build() -> int: + """Run packer and record what the image turned out to be.""" + base = IMAGES / VERSIONS["base_image"]["file"] + if not base.is_file(): + print(f"BLOCK no base image at {base}; run ../prepare-host first") + return 1 + found = digest_of(base) + if found != VERSIONS["base_image"]["sha256"]: + print(f"BLOCK the base image digest is {found}, not the pinned value") + return 1 + + orca = VERSIONS["tool_image"]["contents"]["orca"] + claude = VERSIONS["tool_image"]["contents"]["claude_code"] + + with tempfile.TemporaryDirectory(prefix="dely-cycle-build-") as staging: + staging_path = Path(staging) + key = staging_path / "build_key" + subprocess.run( + ["ssh-keygen", "-t", "ed25519", "-N", "", "-C", "dely-cycle-image-build", + "-f", str(key)], + check=True, capture_output=True, + ) + output = staging_path / "output" + + initialised = subprocess.run( + ["packer", "init", str(HERE / "tool-image.pkr.hcl")], + capture_output=True, text=True, timeout=600, cwd=HERE, + ) + if initialised.returncode != 0: + print("BLOCK packer init failed:", initialised.stderr.strip()[-400:]) + return 1 + + argv = [ + "packer", "build", + "-var", f"base_image={base}", + "-var", f"base_image_sha256={VERSIONS['base_image']['sha256']}", + "-var", f"output_directory={output}", + "-var", f"image_name={IMAGE_NAME}", + "-var", f"build_public_key={(key.with_suffix('.pub')).read_text(encoding='utf-8').strip()}", + "-var", f"build_private_key_file={key}", + "-var", f"orca_url={orca['url']}", + "-var", f"orca_sha256={orca['sha256']}", + "-var", f"claude_code_version={claude['version']}", + str(HERE / "tool-image.pkr.hcl"), + ] + print("running packer; this takes a while", flush=True) + built = subprocess.run(argv, cwd=HERE, timeout=7200) + if built.returncode != 0: + print(f"BLOCK packer build exited {built.returncode}") + return 1 + + produced = output / IMAGE_NAME + if not produced.is_file(): + print(f"BLOCK packer produced no {produced}") + return 1 + + IMAGES.mkdir(parents=True, exist_ok=True) + target = IMAGES / IMAGE_NAME + shutil.move(str(produced), target) + target.chmod(0o644) + + image_digest = digest_of(target) + METADATA.write_text( + json.dumps( + { + "file": IMAGE_NAME, + "sha256": image_digest, + "size_bytes": target.stat().st_size, + "built_at": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"), + "base_image": VERSIONS["base_image"]["file"], + "base_image_sha256": VERSIONS["base_image"]["sha256"], + "orca_version": orca["version"], + "orca_sha256": orca["sha256"], + "claude_code_version": claude["version"], + "carries_credentials": False, + }, + indent=2, + sort_keys=True, + ) + + "\n", + encoding="utf-8", + ) + subprocess.run( + ["virsh", "--connect", VERSIONS["host_tools"]["connect_uri"], + "pool-refresh", VERSIONS["host_tools"]["storage_pool"]["name"]], + capture_output=True, timeout=120, + ) + print() + print(f"built {target}") + print(f"sha256 {image_digest}") + print("put that digest in the configuration's vm.base_image_sha256") + return 0 + + +def main(argv=None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--check", action="store_true", help="report the current image only") + arguments = parser.parse_args(argv) + return check() if arguments.check else build() + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/experiments/minimal-cycle/host/packer/provision.sh b/experiments/minimal-cycle/host/packer/provision.sh new file mode 100755 index 0000000..312a8c6 --- /dev/null +++ b/experiments/minimal-cycle/host/packer/provision.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# Everything the tool image carries, pinned and checked. +set -euxo pipefail + +export DEBIAN_FRONTEND=noninteractive + +cloud-init status --wait || true + +sudo apt-get update +sudo apt-get install -y --no-install-recommends \ + ca-certificates curl gnupg jq git rsync \ + qemu-guest-agent \ + nodejs npm \ + xserver-xorg-core xserver-xorg-video-vesa xserver-xorg-video-qxl \ + xserver-xorg-input-libinput xinit openbox xterm \ + dbus-x11 x11-xserver-utils xdg-utils \ + libgtk-3-0t64 libnss3 libasound2t64 libatk-bridge2.0-0t64 libcups2t64 \ + libdrm2 libgbm1 libxkbcommon0 libpango-1.0-0 libxcomposite1 libxdamage1 \ + libxfixes3 libxrandr2 libsecret-1-0 + +# -- orca, at the pinned version and digest ------------------------------ +curl -fsSL -o /tmp/orca.deb "${ORCA_URL}" +echo "${ORCA_SHA256} /tmp/orca.deb" | sha256sum -c - +sudo apt-get install -y /tmp/orca.deb +rm -f /tmp/orca.deb +test -x /opt/Orca/resources/bin/orca-ide +sudo ln -sf /opt/Orca/resources/bin/orca-ide /usr/local/bin/orca + +# -- claude code, at the pinned version ---------------------------------- +sudo npm install -g --no-fund --no-audit "@anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}" +claude --version + +# -- a graphical session so the orca window has somewhere to appear ------- +sudo tee /etc/systemd/system/dely-cycle-desktop.service > /dev/null < /dev/null < str: + return f" {GREEN if self.ok else AMBER} {self.name}: {self.detail}" + + +def run(argv, *, timeout=1800, env=None, cwd=None) -> subprocess.CompletedProcess: + merged = dict(os.environ) + merged.update(env or {}) + return subprocess.run( + [str(item) for item in argv], + capture_output=True, + text=True, + timeout=timeout, + env=merged, + cwd=str(cwd) if cwd else None, + ) + + +def digest_of(path: Path) -> str: + digest = hashlib.sha256() + with path.open("rb") as handle: + for chunk in iter(lambda: handle.read(1024 * 1024), b""): + digest.update(chunk) + return digest.hexdigest() + + +# -- steps --------------------------------------------------------------- + + +def pulumi_step(check_only: bool) -> Step: + wanted = VERSIONS["host_tools"]["pulumi_cli"]["version"] + if PULUMI.is_file(): + found = run([PULUMI, "version"], timeout=120).stdout.strip().lstrip("v") + if found == wanted: + return Step("pulumi", True, f"{PULUMI} is at {found}") + if check_only: + return Step("pulumi", False, f"{found} is installed, not the pinned {wanted}") + elif check_only: + return Step("pulumi", False, f"nothing at {PULUMI}") + installer = ROOT / "get-pulumi.sh" + installer.parent.mkdir(parents=True, exist_ok=True) + urllib.request.urlretrieve(VERSIONS["host_tools"]["pulumi_cli"]["installer"], installer) + outcome = run(["sh", str(installer), "--version", wanted]) + if not PULUMI.is_file(): + return Step("pulumi", False, (outcome.stderr or outcome.stdout).strip()[-200:]) + found = run([PULUMI, "version"], timeout=120).stdout.strip().lstrip("v") + return Step("pulumi", found == wanted, f"installed {found}, wanted {wanted}") + + +def backend_step(check_only: bool) -> Step: + if not PULUMI.is_file(): + return Step("pulumi state backend", False, "pulumi is not installed yet") + environment = {"PULUMI_SKIP_UPDATE_CHECK": "true"} + if not check_only: + run([PULUMI, "login", "--local"], timeout=120, env=environment) + reported = run([PULUMI, "whoami", "-v"], timeout=120, env=environment).stdout + backend = "" + for line in reported.splitlines(): + if line.startswith("Backend URL:"): + backend = line.split(":", 1)[1].strip() + local = backend.startswith("file://") + return Step( + "pulumi state backend", + local, + f"state lives at {backend}" if local else f"{backend or 'unknown'} is not local", + ) + + +def environment_step(check_only: bool) -> Step: + interpreter = VENV / "bin" / "python" + packages = VERSIONS["python_environment"]["packages"] + wanted = [f"{name}=={version}" for name, version in packages.items()] + if interpreter.is_file(): + listed = run([interpreter, "-m", "pip", "list", "--format=json"], timeout=300) + try: + installed = { + entry["name"].lower().replace("_", "-"): entry["version"] + for entry in json.loads(listed.stdout or "[]") + } + except json.JSONDecodeError: + installed = {} + missing = [ + f"{name} {version}" + for name, version in packages.items() + if installed.get(name.lower().replace("_", "-")) != version + ] + if not missing: + return Step( + "pinned environment", + True, + f"{interpreter} carries " + ", ".join(wanted), + ) + if check_only: + return Step("pinned environment", False, "not at the pins: " + ", ".join(missing)) + elif check_only: + return Step("pinned environment", False, f"no interpreter at {interpreter}") + VENV.parent.mkdir(parents=True, exist_ok=True) + run([sys.executable, "-m", "venv", str(VENV)], timeout=600) + outcome = run( + [VENV / "bin" / "pip", "install", "--disable-pip-version-check", "--quiet", *wanted], + timeout=1800, + ) + ok = (VENV / "bin" / "python").is_file() and outcome.returncode == 0 + return Step( + "pinned environment", + ok, + ", ".join(wanted) if ok else (outcome.stderr or outcome.stdout).strip()[-300:], + ) + + +def base_image_step(check_only: bool) -> Step: + image = VERSIONS["base_image"] + target = IMAGES / image["file"] + if target.is_file(): + found = digest_of(target) + if found == image["sha256"]: + return Step("base image", True, f"{target} matches the pinned digest") + return Step( + "base image", + False, + f"{target} has digest {found}, not the pinned {image['sha256']}", + ) + if check_only: + return Step("base image", False, f"nothing at {target}") + IMAGES.mkdir(parents=True, exist_ok=True) + urllib.request.urlretrieve(image["url"], target) + found = digest_of(target) + if found != image["sha256"]: + target.unlink() + return Step("base image", False, f"the download had digest {found}; it was removed") + return Step("base image", True, f"downloaded and verified {target}") + + +def pool_step(check_only: bool) -> Step: + pool = VERSIONS["host_tools"]["storage_pool"] + uri = VERSIONS["host_tools"]["connect_uri"] + if shutil.which("virsh") is None: + return Step("storage pool", False, "virsh is not on PATH") + listed = run(["virsh", "--connect", uri, "pool-list", "--name"], timeout=120) + if pool["name"] in listed.stdout.split(): + return Step("storage pool", True, f"{pool['name']} is active over {pool['target']}") + if check_only: + return Step("storage pool", False, f"{pool['name']} is not active") + Path(pool["target"]).mkdir(parents=True, exist_ok=True) + Path(pool["target"]).chmod(0o755) + run( + ["virsh", "--connect", uri, "pool-define-as", pool["name"], pool["type"], + "--target", pool["target"]], + timeout=120, + ) + run(["virsh", "--connect", uri, "pool-build", pool["name"]], timeout=120) + run(["virsh", "--connect", uri, "pool-start", pool["name"]], timeout=120) + run(["virsh", "--connect", uri, "pool-autostart", pool["name"]], timeout=120) + again = run(["virsh", "--connect", uri, "pool-list", "--name"], timeout=120) + ok = pool["name"] in again.stdout.split() + return Step("storage pool", ok, f"{pool['name']} over {pool['target']}") + + +def network_step(_check_only: bool) -> Step: + uri = VERSIONS["host_tools"]["connect_uri"] + wanted = VERSIONS["host_tools"]["network"] + if shutil.which("virsh") is None: + return Step("network", False, "virsh is not on PATH") + listed = run(["virsh", "--connect", uri, "net-list", "--name"], timeout=120) + ok = wanted in listed.stdout.split() + return Step("network", ok, f"{wanted} is active" if ok else f"{wanted} is not active") + + +def accelerator_step(_check_only: bool) -> Step: + device = Path("/dev/kvm") + return Step( + "hardware acceleration", + device.exists(), + str(device) if device.exists() else "/dev/kvm is absent", + ) + + +STEPS = ( + accelerator_step, + pulumi_step, + backend_step, + environment_step, + base_image_step, + pool_step, + network_step, +) + + +def main(argv=None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--check", action="store_true", help="verify only; change nothing") + arguments = parser.parse_args(argv) + + print(f"root: {ROOT}") + results = [] + for step in STEPS: + try: + result = step(arguments.check) + except Exception as error: # a failed step is a report, not a crash + result = Step(step.__name__.removesuffix("_step"), False, f"{type(error).__name__}: {error}") + results.append(result) + print(result.line(), flush=True) + + blocked = [result for result in results if not result.ok] + print() + if blocked: + print(f"{len(blocked)} of {len(results)} not in place: " + ", ".join(r.name for r in blocked)) + return 1 + print(f"all {len(results)} in place") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/experiments/minimal-cycle/tests/test_adapter_vm.py b/experiments/minimal-cycle/tests/test_adapter_vm.py index 7c404b9..ea32d05 100644 --- a/experiments/minimal-cycle/tests/test_adapter_vm.py +++ b/experiments/minimal-cycle/tests/test_adapter_vm.py @@ -358,7 +358,7 @@ def test_the_program_names_only_classes_and_fields_the_provider_has(self): def test_a_field_the_provider_lacks_is_caught(self): adapter = self.make(venv=str(self.prepared())) program = adapter.render_program().replace( - "qemu_agent=True,", "qemu_agent=True,\n nonexistent_field=1," + "qemu_agent=QEMU_AGENT,", "qemu_agent=QEMU_AGENT,\n nonexistent_field=1," ) ran, problems = schema.verify_with_interpreter( program, self.prepared() / "bin" / "python" @@ -371,3 +371,76 @@ def test_the_preflight_finding_reports_the_real_check(self): finding = adapter._schema_finding() self.assertTrue(finding.ok, finding.detail) self.assertIn("0.5.4", finding.detail) + + +class GuestAgentTest(VmTestCase): + """The provider reads addresses through the agent when it is enabled. + + Observed on a real run: with the agent declared but absent from the image, + creating the domain fails with "QEMU guest agent is not connected". The + runner discovers the address from libvirt's leases instead, so the agent is + off unless the image is known to run it. + """ + + def test_the_agent_is_off_by_default(self): + self.assertIn("QEMU_AGENT = False", self.make().render_program()) + + def test_an_image_that_runs_the_agent_can_declare_it(self): + self.assertIn("QEMU_AGENT = True", self.make(qemu_agent=True).render_program()) + + def test_the_program_always_passes_the_declared_value(self): + self.assertIn("qemu_agent=QEMU_AGENT,", self.make().render_program()) + + +class RemoteQuotingTest(VmTestCase): + """ssh joins its remaining arguments into one string the remote shell + re-parses, so an argument vector has to be quoted before it is sent. + + Observed on a real run: the identity probe's positional argument was lost, + so the guest resolved an empty project path and the run blocked. + """ + + def ready(self): + adapter = self.make() + adapter.prepare_identity() + adapter.address = "192.168.122.11" + return adapter + + def test_the_remote_command_is_one_quoted_argument(self): + argv = self.ready().ssh_argv(["sh", "-c", "echo hello world", "name", "arg one"]) + self.assertEqual(len(argv), len(argv[: argv.index("--") + 1]) + 1) + + def test_an_argument_with_spaces_survives_as_one_argument(self): + argv = self.ready().ssh_argv(["printf", "%s\n", "two words"]) + remote = argv[-1] + self.assertIn("'two words'", remote) + + def test_a_multi_line_script_survives(self): + script = "set -u\nprintf 'a=%s\\n' \"$1\"\n" + argv = self.ready().ssh_argv(["sh", "-c", script, "probe", "/home/cycle/project"]) + remote = argv[-1] + self.assertIn("/home/cycle/project", remote) + import shlex + + parsed = shlex.split(remote) + self.assertEqual(parsed[0], "sh") + self.assertEqual(parsed[1], "-c") + self.assertEqual(parsed[2], script) + self.assertEqual(parsed[3], "probe") + self.assertEqual(parsed[4], "/home/cycle/project") + + def test_the_probe_argument_reaches_the_guest_intact(self): + from cycle_runner import probe + import shlex + + argv = self.ready().ssh_argv(probe.probe_argv("/home/cycle/project")) + parsed = shlex.split(argv[-1]) + self.assertEqual(parsed[-1], "/home/cycle/project") + self.assertEqual(parsed[2], probe.PROBE_SCRIPT) + + def test_a_value_carrying_a_quote_cannot_break_out(self): + import shlex + + argv = self.ready().ssh_argv(["echo", "it's; rm -rf /"]) + parsed = shlex.split(argv[-1]) + self.assertEqual(parsed, ["echo", "it's; rm -rf /"]) From 9ff4701623db4cf1db7341ddd1467a873b49da78 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 12:47:23 +0200 Subject: [PATCH 15/42] Wait for the guest to answer before calling the environment created; an address is not readiness. --- experiments/minimal-cycle/counterexamples.py | 12 ++++ .../minimal-cycle/cycle_runner/adapters/vm.py | 38 ++++++++++++ .../minimal-cycle/tests/test_adapter_vm.py | 60 +++++++++++++++++++ 3 files changed, 110 insertions(+) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index dc129d8..4657120 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,18 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="an-address-is-not-readiness", + requirement="Creation waits for the guest to answer, not just to take an address", + path="cycle_runner/adapters/vm.py", + original=""" outcome = self.runner( + self.ssh_argv(["true"]), timeout=60, context="environment" + ) + if outcome.ok: + return True""", + replacement=""" return True""", + instruments=("tests.test_adapter_vm.TransportReadinessTest",), + ), Counterexample( name="remote-command-is-quoted", requirement="An argument vector survives the transport intact", diff --git a/experiments/minimal-cycle/cycle_runner/adapters/vm.py b/experiments/minimal-cycle/cycle_runner/adapters/vm.py index 793c351..79e6a32 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/vm.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/vm.py @@ -641,6 +641,37 @@ def discover_address(self, *, timeout: float | None = None) -> str | None: return None self.sleeper(5) + def wait_for_transport(self, *, timeout: float | None = None) -> bool: + """Poll the guest until it answers a command, or the deadline passes. + + libvirt hands out the lease while the guest is still booting, so an + address says only that the interface came up. Readiness is the guest + answering. + """ + if not self.address: + raise VmContractError( + "the domain has no known address yet; readiness cannot be checked" + ) + deadline = time.monotonic() + ( + self.settings.address_timeout_seconds if timeout is None else timeout + ) + while True: + outcome = self.runner( + self.ssh_argv(["true"]), timeout=60, context="environment" + ) + if outcome.ok: + return True + if time.monotonic() >= deadline: + return False + self.sleeper(5) + + def settle_guest(self) -> proc.CommandOutcome: + """Wait for the guest's first-boot configuration to finish.""" + return self.execute( + ["cloud-init", "status", "--wait"], + timeout=min(900, self.config.timeout_seconds), + ) + def create(self) -> EnvironmentHandle: """Declare the domain with Pulumi, then find the address it was given.""" self.prepare_identity() @@ -668,6 +699,13 @@ def create(self) -> EnvironmentHandle: f"{self.settings.address_timeout_seconds} seconds, so nothing can be " "run in it" ) + if not self.wait_for_transport(): + raise VmContractError( + f"{self.domain_name} took the address {self.address} but never answered " + f"a command within {self.settings.address_timeout_seconds} seconds; an " + "address is not readiness" + ) + self.settle_guest() handle = self.plan_handle() return EnvironmentHandle( environment_id=handle.environment_id, diff --git a/experiments/minimal-cycle/tests/test_adapter_vm.py b/experiments/minimal-cycle/tests/test_adapter_vm.py index ea32d05..5f32dfb 100644 --- a/experiments/minimal-cycle/tests/test_adapter_vm.py +++ b/experiments/minimal-cycle/tests/test_adapter_vm.py @@ -444,3 +444,63 @@ def test_a_value_carrying_a_quote_cannot_break_out(self): argv = self.ready().ssh_argv(["echo", "it's; rm -rf /"]) parsed = shlex.split(argv[-1]) self.assertEqual(parsed, ["echo", "it's; rm -rf /"]) + + +class TransportReadinessTest(VmTestCase): + """An address is not readiness. + + Observed on a real run: libvirt handed out the lease while the guest was + still booting, and the first command failed with "Connection refused" five + milliseconds later. Creation is not finished until the guest answers. + """ + + class Refusing: + """Refuses the first few commands, then answers, like a booting guest.""" + + def __init__(self, refusals): + self.refusals = refusals + self.attempts = 0 + + def __call__(self, argv, *, timeout, context, cwd=None, env=None, + extra_values=(), stdin_text=None): + argv = tuple(str(item) for item in argv) + joined = " ".join(argv) + if argv[0] == "ssh-keygen": + return proc.run(argv, timeout=timeout, context=context) + if "domifaddr" in joined: + return self.outcome(argv, 0, " vnet0 52:54:00:x ipv4 192.168.122.11/24\n", "", context) + if argv[0] == "ssh": + self.attempts += 1 + if self.attempts <= self.refusals: + return self.outcome( + argv, 255, "", "ssh: connect to host port 22: Connection refused", context + ) + return self.outcome(argv, 0, "", "", context) + return self.outcome(argv, 0, "", "", context) + + @staticmethod + def outcome(argv, code, out, err, context): + return proc.CommandOutcome( + argv=argv, exit_code=code, stdout=out, stderr=err, + started_at="2026-09-15T14:30:00Z", finished_at="2026-09-15T14:30:00Z", + elapsed_seconds=0.005, timed_out=False, context=context, + ) + + def ready(self, refusals): + adapter = self.make(runner=self.Refusing(refusals)) + adapter.prepare_identity() + adapter.address = "192.168.122.11" + return adapter + + def test_a_guest_that_answers_after_a_few_refusals_is_reachable(self): + adapter = self.ready(3) + self.assertTrue(adapter.wait_for_transport(timeout=60)) + self.assertGreaterEqual(adapter.runner.attempts, 4) + + def test_a_guest_that_never_answers_is_not_reachable(self): + self.assertFalse(self.ready(1000).wait_for_transport(timeout=0)) + + def test_readiness_is_not_claimed_before_an_address_is_known(self): + adapter = self.make(runner=self.Refusing(0)) + with self.assertRaises(vm.VmContractError): + adapter.wait_for_transport(timeout=0) From 34e98be212cfffdfcd437a5a89a18b7ae1050382 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 12:49:48 +0200 Subject: [PATCH 16/42] Add the host preparation, the tool-image build, and the evidence one real machine cycle produced. --- experiments/minimal-cycle/README.md | 72 ++- experiments/minimal-cycle/evidence/README.md | 33 + .../evidence/counterexamples.txt | 44 +- .../minimal-cycle/evidence/preflight-vm.txt | 16 +- .../auth-receipt.json | 11 + .../vm-blocked-on-absent-orca/cleanup.json | 15 + .../export-receipt.json | 66 ++ .../identity-environment-probe.txt | 13 + .../identity-host-probe.txt | 13 + .../vm-blocked-on-absent-orca/manifest.json | 595 ++++++++++++++++++ .../vm-blocked-on-absent-orca/preflight.json | 81 +++ 11 files changed, 917 insertions(+), 42 deletions(-) create mode 100644 experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/auth-receipt.json create mode 100644 experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/cleanup.json create mode 100644 experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/export-receipt.json create mode 100644 experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/identity-environment-probe.txt create mode 100644 experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/identity-host-probe.txt create mode 100644 experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/manifest.json create mode 100644 experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/preflight.json diff --git a/experiments/minimal-cycle/README.md b/experiments/minimal-cycle/README.md index be96599..012e16d 100644 --- a/experiments/minimal-cycle/README.md +++ b/experiments/minimal-cycle/README.md @@ -148,19 +148,51 @@ make against a real Orca launch rather than a default to inherit. ### Virtual machine A per-run libvirt domain declared with Pulumi in Python, over a qcow2 overlay -whose backing file is a preserved tool image, with a cloud-init NoCloud seed and -an ssh transport. - -The base image is opened only as a backing file and is declared a shared -resource, so it can never appear in a destroy plan. The seed carries one per-run -public key and nothing else; the private half is generated into the per-run -state directory with owner-only permissions and is removed with it. - -The provider's resource schema is not assumed. The rendered program names fields -that belong to the pinned provider version, and preflight refuses to run until -`vm.provider_schema_verified` records that someone checked them against that -version. Building the tool image is a separate job — the runner consumes an -image, it does not build one, and it is not an image cache. +whose backing volume is a preserved tool image, with a cloud-init NoCloud seed +and an ssh transport. The guest is Ubuntu Server 24.04 LTS from Canonical's +cloud image, verified against the published checksum — a disk image for qemu, +never a container image. + +Run `host/prepare-host` once. It installs the pinned Pulumi, points it at a +local file state backend, builds the python environment carrying the pinned +software development kit and libvirt provider, downloads and verifies the base +image, and declares a storage pool over the image directory. `--check` verifies +all of that and changes nothing. + +`host/packer/build-tool-image` then builds the one tool image: it boots the +verified base under packer's qemu builder, installs the pinned Orca package, the +pinned Claude Code, a graphical session for the Orca window, and the guest +agent, then records a metadata file naming every version and digest that went +in. The build key is generated per build and removed from the image before +shutdown. + +**Nothing about the provider is assumed.** The rendered program is parsed, every +provider class and keyword it names is collected, and each is checked against +the provider actually importable from the pinned environment. Preflight blocks +when a field does not exist and also when the check cannot be run at all. That +check has already earned its place: it caught a class name the program had +wrong before a single domain was created. + +Preflight reads host facts rather than trusting configuration: that the Pulumi +state backend is a local file backend and not the hosted service, that the +storage pool and network are active, that the base image digest matches, and +that this emulator actually offers the configured graphics type. On the machine +this was built on, qemu has no spice at all — it offers sdl, vnc and dbus — so +the graphics finding is a refusal, not a formality. + +Two facts about the guest's network are worth stating, because both were found +by running it. The domain takes a bridged interface for the runner's transport, +and a second, user-mode interface for the guest's own outbound traffic: a +bridged guest cannot reach the internet when the host routes through a virtual +private network or its firewall declines to forward, and a user-mode interface +is served by qemu itself and does not use that path. And an address is not +readiness — libvirt hands out the lease while the guest is still booting, so +creation is not finished until the guest answers a command and its first-boot +configuration has settled. + +The base image is a shared resource, opened only as a backing volume, so it can +never appear in a destroy plan. The per-run overlay, seed, domain, stack and +state are all per-run and go together. ## Auth @@ -220,12 +252,16 @@ the table with the tests each row runs. The recorded sweep is in | An unverified provider schema blocks the machine backend | case `provider-schema-verified` | `evidence/preflight-vm.txt` | | Preflight blocks rather than inventing a path | `./run-cycle preflight` on a host with no `pulumi` and no tool image | `evidence/preflight-vm.txt` | | A real cycle stops at the identity gate rather than using the host | `./run-cycle run` on this host | `evidence/run-blocked-on-host-orca/manifest.json` | +| The rendered program names only classes and fields the pinned provider has | `tests/test_adapter_vm.py::ProviderSchemaIntegrationTest` against the real provider | `evidence/preflight-vm.txt` | +| A hosted state backend blocks the machine backend | case `state-backend-is-local` | `evidence/counterexamples.txt` | +| A graphics type this emulator lacks blocks the run | case `graphics-type-is-supported` | `evidence/preflight-vm.txt` | +| An argument vector survives the transport intact | case `remote-command-is-quoted` | `evidence/counterexamples.txt` | +| Creation waits for the guest to answer, not just to take an address | case `an-address-is-not-readiness` | `evidence/counterexamples.txt` | +| A real machine cycle creates a domain, proves a distinct kernel, and destroys it | `./run-cycle run` on this host | `evidence/vm-blocked-on-absent-orca/manifest.json` | ## What no instrument here observes Orca starting inside either backend, a window bound to that instance, a Claude -Code worker driven through it, a login surviving a run, or any domain being -created at all. Those need a host that carries Orca in the chosen image, a built -tool image, and a pinned libvirt provider. They are the separate acceptance test -this work does not claim to have passed; `evidence/README.md` says exactly where -this host stopped. +Code worker driven through it, or a login surviving a run. Both backends reach +their identity gate and stop there, for the same honest reason: the image they run does not carry Orca. `evidence/README.md` says +exactly where this host stopped and what it did prove on the way. diff --git a/experiments/minimal-cycle/evidence/README.md b/experiments/minimal-cycle/evidence/README.md index 599845c..6536f83 100644 --- a/experiments/minimal-cycle/evidence/README.md +++ b/experiments/minimal-cycle/evidence/README.md @@ -61,6 +61,39 @@ reports a version inside the environment; `counterexamples.py` case `orca-is-the-environments-own` shows the old rule letting a host-driven run settle `SETTLED`. +## `vm-blocked-on-absent-orca/` + +One real machine cycle. Pulumi declared a per-run overlay over the preserved +Ubuntu base, a cloud-init seed and a domain; the guest booted, took an address, +answered, and settled; the project was copied in at a pinned revision; the probe +ran on both sides; and the run stopped at the identity gate. Everything was +exported and confirmed, and the domain, overlay, seed, stack and per-run state +were then destroyed while the base image stayed byte-identical. + + status: BLOCKED + identity: ENVIRONMENT + markers: distinct-machine-identity, distinct-boot-identity, + distinct-hostname, reported-virtualisation:kvm + export: CONFIRMED + cleanup: DESTROYED + why: orca is not present inside the environment; the run stops rather + than using the host's installation + +Compare the probe captures. The guest reported an Ubuntu kernel and `virt=kvm`; +the host reported a Fedora kernel and `virt=none`. The guest resolved the +project to `/home/cycle/project`, its own copy. That is a separate kernel, which +is what this backend buys and the container backend does not. + +Three defects were found by running this, not by reading it. The provider reads +interface addresses through the guest agent when the agent is declared, and the +base image does not run one, so declaring it failed the create. `ssh` +concatenates everything after the destination into one string that the guest's +shell re-parses, so the probe's positional argument was lost and the guest +resolved an empty project path. And libvirt hands out the lease while the guest +is still booting, so the first command hit a refused connection five +milliseconds after the address appeared. Each is now a rail with a +counterexample. + ## `counterexamples.txt` `python3 counterexamples.py`. Each line replaces one rail with an diff --git a/experiments/minimal-cycle/evidence/counterexamples.txt b/experiments/minimal-cycle/evidence/counterexamples.txt index 98e7e26..fcea9c6 100644 --- a/experiments/minimal-cycle/evidence/counterexamples.txt +++ b/experiments/minimal-cycle/evidence/counterexamples.txt @@ -1,20 +1,26 @@ -RED no-host-fallback A probe result equal to the host's own snapshot is rejected -RED orca-is-the-environments-own An orca that resolves to the host's installation is not present -RED export-before-destroy Cleanup runs only after a confirmed export -RED stop-must-be-confirmed Cleanup runs only after a confirmed stop -RED destroy-after-export Evidence is collected and exported before anything is destroyed -RED receipt-is-a-re-read The receipt proves bytes on the host, not bytes in memory -RED cleanup-only-per-run A per-run path that contains a shared resource is refused -RED redaction-by-shape Redaction catches a credential this run has never seen -RED manifest-required-fields The manifest carries every required field on every path -RED run-id-carries-no-name The run identifier discloses neither the host name nor the auth reference -RED config-refuses-a-secret A secret-shaped value under a dull key is refused -RED auth-leaves-no-material No credential value, length or prefix reaches a receipt -RED host-home-mount-acknowledged An unacknowledged host-home mount blocks the run -RED preflight-leaves-no-residue Preflight inspects without creating the run's state -RED base-is-only-a-backing-file The preserved base image is never the overlay's output path -RED failed-create-names-its-residue A create that fails partway records what it may have left behind -RED forwarded-value-is-redacted A value forwarded into the guest is redacted from captured output -RED provider-schema-verified An unverified provider schema blocks the machine backend +RED no-host-fallback A probe result equal to the host's own snapshot is rejected +RED orca-is-the-environments-own An orca that resolves to the host's installation is not present +RED export-before-destroy Cleanup runs only after a confirmed export +RED stop-must-be-confirmed Cleanup runs only after a confirmed stop +RED destroy-after-export Evidence is collected and exported before anything is destroyed +RED receipt-is-a-re-read The receipt proves bytes on the host, not bytes in memory +RED cleanup-only-per-run A per-run path that contains a shared resource is refused +RED redaction-by-shape Redaction catches a credential this run has never seen +RED manifest-required-fields The manifest carries every required field on every path +RED run-id-carries-no-name The run identifier discloses neither the host name nor the auth reference +RED config-refuses-a-secret A secret-shaped value under a dull key is refused +RED auth-leaves-no-material No credential value, length or prefix reaches a receipt +RED host-home-mount-acknowledged An unacknowledged host-home mount blocks the run +RED preflight-leaves-no-residue Preflight inspects without creating the run's state +RED base-is-only-a-backing-file The preserved base image is a backing volume, never a per-run one +RED overlay-is-not-the-base The rendered program creates an overlay, not a volume named like the base +RED failed-create-names-its-residue A create that fails partway records what it may have left behind +RED an-address-is-not-readiness Creation waits for the guest to answer, not just to take an address +RED remote-command-is-quoted An argument vector survives the transport intact +RED forwarded-value-is-redacted A value forwarded into the guest is redacted from captured output +RED provider-schema-verified A program the provider cannot be checked against blocks the run +RED state-backend-is-local A cloud state backend blocks the machine backend +RED graphics-type-is-supported A graphics type this emulator lacks blocks the run +RED transport-address-is-the-right-one Another interface's address is not mistaken for the transport's -all 18 counterexamples discriminated +all 24 counterexamples discriminated diff --git a/experiments/minimal-cycle/evidence/preflight-vm.txt b/experiments/minimal-cycle/evidence/preflight-vm.txt index 46307c6..4c1345d 100644 --- a/experiments/minimal-cycle/evidence/preflight-vm.txt +++ b/experiments/minimal-cycle/evidence/preflight-vm.txt @@ -1,11 +1,17 @@ backend: vm - BLOCK pulumi on the path: pulumi was not found on PATH + ok pulumi on the path: /.pulumi/bin/pulumi ok qemu-img on the path: /usr/bin/qemu-img ok virsh on the path: /usr/bin/virsh ok ssh on the path: /usr/bin/ssh ok scp on the path: /usr/bin/scp - BLOCK provider schema verified: the rendered program's resource fields have not been checked against pulumi-libvirt@unchecked-on-this-host; set vm.provider_schema_verified once they have been - BLOCK base image present: /var/lib/dely-cycle/images/tool-image.qcow2 is not a file on this host + ok pinned environment present: /var/tmp/dely-cycle/venv/bin/python + ok pulumi state backend is local: state lives at file://~ + ok provider schema verified: every class and field the program names exists in pulumi-libvirt@0.5.4 + ok base image present: /var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img + ok base image digest matches the pin: the preserved base matches the pinned digest ok hardware acceleration device: /dev/kvm - ok libvirt connection answers: qemu:///session -not usable on this host + ok libvirt connection answers: qemu:///system + ok storage pool active: dely-cycle is active + ok network active: default is active + ok graphics type supported by this emulator: vnc is among sdl, vnc, dbus +usable on this host diff --git a/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/auth-receipt.json b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/auth-receipt.json new file mode 100644 index 0000000..63c115d --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/auth-receipt.json @@ -0,0 +1,11 @@ +{ + "detail": "the helper command is declared in the per-run settings; no credential value passes through the runner", + "entries": [ + ".claude/settings.json" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only", + "removed_after_run": false, + "status": "OK", + "target": "/home/cycle/.claude/settings.json" +} diff --git a/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/cleanup.json b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/cleanup.json new file mode 100644 index 0000000..fd70e09 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/cleanup.json @@ -0,0 +1,15 @@ +{ + "host_difference": {}, + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "domain:dely-cycle-20260915t150000z-bbbbbb-00000006", + "path:/var/tmp/dely-cycle/state/20260915T150000Z-bbbbbb-00000006" + ], + "retained": [], + "shared_preserved": [ + "image:/var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img" + ], + "status": "DESTROYED", + "stop_confirmed": true, + "verified": true +} diff --git a/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/export-receipt.json b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/export-receipt.json new file mode 100644 index 0000000..533087b --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/export-receipt.json @@ -0,0 +1,66 @@ +{ + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "d07d42f1b658cd692c32580ba917e1fab793697285148de9714b9ba3cda98e4f", + "size_bytes": 341 + }, + { + "path": "backend-status.json", + "sha256": "6a1f900514162cb6bb3c7b06552687c9e4873279a848f5f1fd0370189855c406", + "size_bytes": 2080 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "f60a04fc7ac3b0e6b8a8a74cc1f4912d7f89ed25f66e19af6ceaffe6fc89e5e8", + "size_bytes": 973 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "285da27d881eec44433f8c9b0ac93bfb79ea878340b9f086ddf6e65478bff14c", + "size_bytes": 405 + }, + { + "path": "identity/host-probe.txt", + "sha256": "8186bd380f569e2656fed0f91ccc7593be082a25bc2cd6e32fd26a2ed024ae22", + "size_bytes": 1383 + }, + { + "path": "identity/orca-status.json", + "sha256": "ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356", + "size_bytes": 3 + }, + { + "path": "logs/commands.jsonl", + "sha256": "ff74b2a3cf4cd86e2d1064adf578dd95a0990693fe236ef761b699c0994cd779", + "size_bytes": 5045 + }, + { + "path": "logs/runner.log", + "sha256": "1d0b004f57b690c2dc4fec02b94f461999d0795f39e36b25f79e67fabcc2f050", + "size_bytes": 1149 + }, + { + "path": "preflight.json", + "sha256": "d2c853a24dbc9d3d390d714dd4881cd6e3063fc17b693f251b5e6096ca31d837", + "size_bytes": 1827 + }, + { + "path": "run-before-cleanup.json", + "sha256": "5051a2a64cf817c15eaf0fb8e58d0d0a4aed06b70e200b68d4b4441f7a57d9ff", + "size_bytes": 13048 + } + ], + "confirmed_at": "2026-09-15T10:47:44Z", + "detail": "11 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" +} diff --git a/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/identity-environment-probe.txt b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/identity-environment-probe.txt new file mode 100644 index 0000000..a15eec9 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/identity-environment-probe.txt @@ -0,0 +1,13 @@ +hostname=dely-cycle-20260915t150000z-bbbbbb-00000006 +machine_id= +boot_id= +home=/home/cycle +user=cycle +path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin +pid=1475 +uname=Linux 6.8.0-139-generic x86_64 +container_marker= +virt=kvm +orca_path= +orca_version= +project_real=/home/cycle/project diff --git a/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/identity-host-probe.txt b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/identity-host-probe.txt new file mode 100644 index 0000000..f19f96c --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/identity-host-probe.txt @@ -0,0 +1,13 @@ +hostname=system +machine_id= +boot_id= +home= +user= +path=/.pulumi/bin:/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin +pid=1070481 +uname=Linux 7.2.4-200.fc44.x86_64 x86_64 +container_marker= +virt=none +orca_path=/.config/orca/linux-orca-cli-shim/orca +orca_version=1.4.201 +project_real=/var/tmp/dely-cycle/project-source diff --git a/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/manifest.json b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/manifest.json new file mode 100644 index 0000000..d57be77 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/manifest.json @@ -0,0 +1,595 @@ +{ + "artifact_root": "/var/tmp/dely-cycle/artifacts/20260915T150000Z-bbbbbb-00000006", + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "d07d42f1b658cd692c32580ba917e1fab793697285148de9714b9ba3cda98e4f", + "size_bytes": 341 + }, + { + "path": "backend-status.json", + "sha256": "6a1f900514162cb6bb3c7b06552687c9e4873279a848f5f1fd0370189855c406", + "size_bytes": 2080 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "f60a04fc7ac3b0e6b8a8a74cc1f4912d7f89ed25f66e19af6ceaffe6fc89e5e8", + "size_bytes": 973 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "285da27d881eec44433f8c9b0ac93bfb79ea878340b9f086ddf6e65478bff14c", + "size_bytes": 405 + }, + { + "path": "identity/host-probe.txt", + "sha256": "8186bd380f569e2656fed0f91ccc7593be082a25bc2cd6e32fd26a2ed024ae22", + "size_bytes": 1383 + }, + { + "path": "identity/orca-status.json", + "sha256": "ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356", + "size_bytes": 3 + }, + { + "path": "logs/commands.jsonl", + "sha256": "ff74b2a3cf4cd86e2d1064adf578dd95a0990693fe236ef761b699c0994cd779", + "size_bytes": 5045 + }, + { + "path": "logs/runner.log", + "sha256": "1d0b004f57b690c2dc4fec02b94f461999d0795f39e36b25f79e67fabcc2f050", + "size_bytes": 1149 + }, + { + "path": "preflight.json", + "sha256": "d2c853a24dbc9d3d390d714dd4881cd6e3063fc17b693f251b5e6096ca31d837", + "size_bytes": 1827 + }, + { + "path": "run-before-cleanup.json", + "sha256": "5051a2a64cf817c15eaf0fb8e58d0d0a4aed06b70e200b68d4b4441f7a57d9ff", + "size_bytes": 13048 + } + ], + "auth": { + "detail": "the helper command is declared in the per-run settings; no credential value passes through the runner", + "entries": [ + ".claude/settings.json" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only", + "removed_after_run": false, + "status": "OK", + "target": "/home/cycle/.claude/settings.json" + }, + "backend": "vm", + "check": { + "argv": [], + "detail": "not attempted", + "elapsed_seconds": null, + "exit_code": null, + "expected_marker": null, + "finished_at": null, + "started_at": null, + "status": "SKIPPED", + "stderr_path": null, + "stdout_path": null + }, + "cleanup": { + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "domain:dely-cycle-20260915t150000z-bbbbbb-00000006", + "path:/var/tmp/dely-cycle/state/20260915T150000Z-bbbbbb-00000006" + ], + "retained": [], + "shared_preserved": [ + "image:/var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img" + ], + "status": "DESTROYED", + "verified": true + }, + "config": { + "artifact_root": "/var/tmp/dely-cycle/artifacts", + "auth": { + "helper_argv": [ + "/usr/local/bin/dely-cycle-key-helper" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only" + }, + "backend": "vm", + "check": { + "expected_marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "claude_code_version": "not-yet-in-this-image", + "dely_revision": "9ff4701623db4cf1db7341ddd1467a873b49da78", + "orca": { + "agent": "claude", + "effort": "high", + "model": "claude-opus-5", + "run_objective": "dely minimal cycle", + "status_argv": [ + "orca", + "status", + "--json" + ], + "version": "1.4.201", + "version_argv": [ + "orca", + "--version" + ], + "worktree_selector": "current" + }, + "project": { + "environment_path": "project", + "revision": "9ff4701623db4cf1db7341ddd1467a873b49da78", + "source": "/var/tmp/dely-cycle/project-source" + }, + "state_root": "/var/tmp/dely-cycle/state", + "task": { + "marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "tested_revision": "9ff4701623db4cf1db7341ddd1467a873b49da78", + "timeout_seconds": 1800, + "vm": { + "address_timeout_seconds": 600, + "base_image": "/var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img", + "base_image_sha256": "612b2c0cc1bc413a6cb8c38fd611794caf0f2b436c50013d8b3794db12ad7354", + "connect_uri": "qemu:///system", + "egress": true, + "graphics": "vnc", + "guest_user": "cycle", + "listen_address": "127.0.0.1", + "memory_mb": 4096, + "network": "default", + "overlay_size_bytes": 25769803776, + "pool": "dely-cycle", + "provider": "pulumi-libvirt", + "provider_version": "0.5.4", + "provision": [], + "pulumi_binary": "/.pulumi/bin/pulumi", + "qemu_agent": false, + "ssh_port": 22, + "stack_prefix": "dely-cycle", + "transport": "ssh", + "vcpus": 2, + "venv": "/var/tmp/dely-cycle/venv" + } + }, + "dely_revision": "9ff4701623db4cf1db7341ddd1467a873b49da78", + "elapsed_seconds": 21.831, + "environment": { + "backend": { + "address": "192.168.122.235", + "backend": "vm", + "base_image": "/var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img", + "base_image_sha256": "612b2c0cc1bc413a6cb8c38fd611794caf0f2b436c50013d8b3794db12ad7354", + "base_volume": "ubuntu-24.04-server-cloudimg-amd64.img", + "connect_uri": "qemu:///system", + "domain": "dely-cycle-20260915t150000z-bbbbbb-00000006", + "egress_interface": true, + "graphics": "vnc", + "listen_address": "127.0.0.1", + "network": "default", + "pool": "dely-cycle", + "provider": "pulumi-libvirt@0.5.4", + "stack": "dely-cycle-20260915t150000z-bbbbbb-00000006", + "transport_mac": "52:54:00:f5:8f:7c" + }, + "description": { + "address": "192.168.122.235", + "base_image": "/var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img", + "base_volume": "ubuntu-24.04-server-cloudimg-amd64.img", + "connect_uri": "qemu:///system", + "domain": "dely-cycle-20260915t150000z-bbbbbb-00000006", + "egress": true, + "graphics": "vnc", + "network": "default", + "pool": "dely-cycle", + "provider": "pulumi-libvirt@0.5.4", + "stack": "/var/tmp/dely-cycle/state/20260915T150000Z-bbbbbb-00000006/stack", + "transport_mac": "52:54:00:f5:8f:7c" + }, + "environment_id": "dely-cycle-20260915t150000z-bbbbbb-00000006", + "home_path": "/home/cycle", + "per_run_resources": [ + { + "identifier": "dely-cycle-20260915t150000z-bbbbbb-00000006", + "kind": "domain" + }, + { + "identifier": "/var/tmp/dely-cycle/state/20260915T150000Z-bbbbbb-00000006", + "kind": "path" + }, + { + "identifier": "/var/tmp/dely-cycle/images/dely-cycle-20260915t150000z-bbbbbb-00000006-overlay.qcow2", + "kind": "volume" + }, + { + "identifier": "/var/tmp/dely-cycle/images/dely-cycle-20260915t150000z-bbbbbb-00000006-seed.iso", + "kind": "volume" + } + ], + "project_path": "/home/cycle/project", + "shared_resources": [ + { + "identifier": "/var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img", + "kind": "image" + } + ] + }, + "environment_id": "dely-cycle-20260915t150000z-bbbbbb-00000006", + "export": { + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "d07d42f1b658cd692c32580ba917e1fab793697285148de9714b9ba3cda98e4f", + "size_bytes": 341 + }, + { + "path": "backend-status.json", + "sha256": "6a1f900514162cb6bb3c7b06552687c9e4873279a848f5f1fd0370189855c406", + "size_bytes": 2080 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "f60a04fc7ac3b0e6b8a8a74cc1f4912d7f89ed25f66e19af6ceaffe6fc89e5e8", + "size_bytes": 973 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "285da27d881eec44433f8c9b0ac93bfb79ea878340b9f086ddf6e65478bff14c", + "size_bytes": 405 + }, + { + "path": "identity/host-probe.txt", + "sha256": "8186bd380f569e2656fed0f91ccc7593be082a25bc2cd6e32fd26a2ed024ae22", + "size_bytes": 1383 + }, + { + "path": "identity/orca-status.json", + "sha256": "ca3d163bab055381827226140568f3bef7eaac187cebd76878e0b63e9e442356", + "size_bytes": 3 + }, + { + "path": "logs/commands.jsonl", + "sha256": "ff74b2a3cf4cd86e2d1064adf578dd95a0990693fe236ef761b699c0994cd779", + "size_bytes": 5045 + }, + { + "path": "logs/runner.log", + "sha256": "1d0b004f57b690c2dc4fec02b94f461999d0795f39e36b25f79e67fabcc2f050", + "size_bytes": 1149 + }, + { + "path": "preflight.json", + "sha256": "d2c853a24dbc9d3d390d714dd4881cd6e3063fc17b693f251b5e6096ca31d837", + "size_bytes": 1827 + }, + { + "path": "run-before-cleanup.json", + "sha256": "5051a2a64cf817c15eaf0fb8e58d0d0a4aed06b70e200b68d4b4441f7a57d9ff", + "size_bytes": 13048 + } + ], + "confirmed_at": "2026-09-15T10:47:44Z", + "detail": "11 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" + }, + "failure_classification": "orca is not present inside the environment; the run stops rather than using the host's installation", + "finished_at": "2026-09-15T10:47:50Z", + "generated_at": "2026-09-15T10:47:50Z", + "host_after": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "bc21f149eed21643", + "captured_at": "2026-09-15T10:47:50Z", + "extra": { + "backend": "vm" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.4-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "host_before": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "bc21f149eed21643", + "captured_at": "2026-09-15T10:47:29Z", + "extra": { + "backend": "vm" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.4-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "identity": { + "environment": { + "container_marker": "", + "home": "/home/cycle", + "hostname": "dely-cycle-20260915t150000z-bbbbbb-00000006", + "orca_path": "", + "orca_version": "", + "path": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin", + "pid": "1475", + "project_real": "/home/cycle/project", + "uname": "Linux 6.8.0-139-generic x86_64", + "user": "cycle", + "virt": "kvm" + }, + "host": { + "boot_id_digest": "bc21f149eed21643", + "home_digest": "8fda3821c06d512d", + "hostname_digest": "bbc5e661e106c6dc", + "machine_id_digest": "907fa710d6d1c501", + "orca_present": true, + "project_real_digest": "0d03c9e72dddfdae", + "user_digest": "3cc9cee71b0b967f", + "virt": "none" + }, + "markers": [ + "distinct-machine-identity", + "distinct-boot-identity", + "distinct-hostname", + "reported-virtualisation:kvm" + ], + "orca_is_host_installation": false, + "orca_present": false, + "orca_status_exit_code": 127, + "orca_version": null, + "reason": "the environment answered with its own identity, home and project copy", + "verdict": "ENVIRONMENT" + }, + "phases": [ + { + "commands": [], + "detail": "", + "elapsed_seconds": 0.612, + "finished_at": "2026-09-15T10:47:29Z", + "name": "prepare", + "started_at": "2026-09-15T10:47:29Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 13.48, + "finished_at": "2026-09-15T10:47:43Z", + "name": "create", + "started_at": "2026-09-15T10:47:29Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 0.809, + "finished_at": "2026-09-15T10:47:44Z", + "name": "bootstrap", + "started_at": "2026-09-15T10:47:43Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v orca 2>/dev/null || printf '')\"\norca_release=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id=%s\\n' \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/project-source" + ], + "context": "host", + "elapsed_seconds": 0.07, + "exit_code": 0, + "finished_at": "2026-09-15T10:47:44Z", + "started_at": "2026-09-15T10:47:44Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T150000Z-bbbbbb-00000006/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.235", + "--", + "sh -c '\nset -u\nproject_path=\"${1:-}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf '\"'\"''\"'\"'\n else\n printf '\"'\"''\"'\"'\n fi\n}\n\nmarker='\"'\"''\"'\"'\nif [ -f /run/.containerenv ]; then marker='\"'\"'containerenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='\"'\"'dockerenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='\"'\"'toolboxenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='\"'\"'unknown'\"'\"'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v orca 2>/dev/null || printf '\"'\"''\"'\"')\"\norca_release='\"'\"''\"'\"'\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '\"'\"''\"'\"')\"\nfi\n\nproject_resolved='\"'\"''\"'\"'\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '\"'\"''\"'\"')\"\nfi\n\nprintf '\"'\"'hostname=%s\\n'\"'\"' \"$(uname -n 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'machine_id=%s\\n'\"'\"' \"$(first_line /etc/machine-id)\"\nprintf '\"'\"'boot_id=%s\\n'\"'\"' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf '\"'\"'home=%s\\n'\"'\"' \"${HOME:-}\"\nprintf '\"'\"'user=%s\\n'\"'\"' \"$(id -un 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'path=%s\\n'\"'\"' \"${PATH:-}\"\nprintf '\"'\"'pid=%s\\n'\"'\"' \"$$\"\nprintf '\"'\"'uname=%s\\n'\"'\"' \"$(uname -srm 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'container_marker=%s\\n'\"'\"' \"$marker\"\nprintf '\"'\"'virt=%s\\n'\"'\"' \"$virt\"\nprintf '\"'\"'orca_path=%s\\n'\"'\"' \"$orca_binary\"\nprintf '\"'\"'orca_version=%s\\n'\"'\"' \"$orca_release\"\nprintf '\"'\"'project_real=%s\\n'\"'\"' \"$project_resolved\"\n' cycle-probe /home/cycle/project" + ], + "context": "environment", + "elapsed_seconds": 0.223, + "exit_code": 0, + "finished_at": "2026-09-15T10:47:44Z", + "started_at": "2026-09-15T10:47:44Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T150000Z-bbbbbb-00000006/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.235", + "--", + "orca status --json" + ], + "context": "environment", + "elapsed_seconds": 0.163, + "exit_code": 127, + "finished_at": "2026-09-15T10:47:44Z", + "started_at": "2026-09-15T10:47:44Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "orca is not present inside the environment; the run stops rather than using the host's installation", + "elapsed_seconds": 0.459, + "finished_at": "2026-09-15T10:47:44Z", + "name": "identity", + "started_at": "2026-09-15T10:47:44Z", + "status": "BLOCKED" + }, + { + "commands": [], + "detail": "orca is not present inside the environment; the run stops rather than using the host's installation", + "elapsed_seconds": null, + "finished_at": null, + "name": "task", + "started_at": null, + "status": "SKIPPED" + }, + { + "commands": [], + "detail": "orca is not present inside the environment; the run stops rather than using the host's installation", + "elapsed_seconds": null, + "finished_at": null, + "name": "check", + "started_at": null, + "status": "SKIPPED" + }, + { + "commands": [], + "detail": "0 patch line(s) collected", + "elapsed_seconds": 0.251, + "finished_at": "2026-09-15T10:47:44Z", + "name": "collect", + "started_at": "2026-09-15T10:47:44Z", + "status": "OK" + }, + { + "commands": [], + "detail": "11 artifact(s) re-read from the host; 1 unresolved", + "elapsed_seconds": 0.013, + "finished_at": "2026-09-15T10:47:44Z", + "name": "export", + "started_at": "2026-09-15T10:47:44Z", + "status": "OK" + }, + { + "commands": [], + "detail": "every declared per-run resource is gone and every shared one remains", + "elapsed_seconds": 6.207, + "finished_at": "2026-09-15T10:47:50Z", + "name": "cleanup", + "started_at": "2026-09-15T10:47:44Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": null, + "finished_at": null, + "name": "close", + "started_at": "2026-09-15T10:47:50Z", + "status": "OK" + } + ], + "run_id": "20260915T150000Z-bbbbbb-00000006", + "schema_version": 1, + "started_at": "2026-09-15T10:47:29Z", + "status": "BLOCKED", + "tested_revision": "9ff4701623db4cf1db7341ddd1467a873b49da78", + "timeout_seconds": 1800, + "versions": { + "configured_revision": "9ff4701623db4cf1db7341ddd1467a873b49da78", + "distrobox": "distrobox: 1.8.2.5", + "git": "git version 2.55.0", + "podman": "podman version 5.8.4", + "project_revision": "9ff4701623db4cf1db7341ddd1467a873b49da78", + "pulumi": "v3.262.0", + "python": "3.14.7", + "qemu_img": "Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers", + "runner": "minimal-cycle", + "virsh": "12.0.0" + }, + "worker": { + "agent": null, + "commands": [], + "detail": "not attempted", + "dispatch_id": null, + "effort": null, + "model": null, + "outcome": null, + "run_id": null, + "status": "SKIPPED" + } +} diff --git a/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/preflight.json b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/preflight.json new file mode 100644 index 0000000..84328bf --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-blocked-on-absent-orca/preflight.json @@ -0,0 +1,81 @@ +{ + "backend": "vm", + "findings": [ + { + "detail": "/.pulumi/bin/pulumi", + "name": "pulumi on the path", + "ok": true + }, + { + "detail": "/usr/bin/qemu-img", + "name": "qemu-img on the path", + "ok": true + }, + { + "detail": "/usr/bin/virsh", + "name": "virsh on the path", + "ok": true + }, + { + "detail": "/usr/bin/ssh", + "name": "ssh on the path", + "ok": true + }, + { + "detail": "/usr/bin/scp", + "name": "scp on the path", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/venv/bin/python", + "name": "pinned environment present", + "ok": true + }, + { + "detail": "state lives at file://~", + "name": "pulumi state backend is local", + "ok": true + }, + { + "detail": "every class and field the program names exists in pulumi-libvirt@0.5.4", + "name": "provider schema verified", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img", + "name": "base image present", + "ok": true + }, + { + "detail": "the preserved base matches the pinned digest", + "name": "base image digest matches the pin", + "ok": true + }, + { + "detail": "/dev/kvm", + "name": "hardware acceleration device", + "ok": true + }, + { + "detail": "qemu:///system", + "name": "libvirt connection answers", + "ok": true + }, + { + "detail": "dely-cycle is active", + "name": "storage pool active", + "ok": true + }, + { + "detail": "default is active", + "name": "network active", + "ok": true + }, + { + "detail": "vnc is among sdl, vnc, dbus", + "name": "graphics type supported by this emulator", + "ok": true + } + ], + "ok": true +} From 5ee0823261e8f8c0243a4aa41c571a7e73c13065 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 13:07:18 +0200 Subject: [PATCH 17/42] Prove Orca's runtime and open a coordinator terminal before dispatching, and give the guest a video device its display server can use. --- experiments/minimal-cycle/counterexamples.py | 20 +++ .../minimal-cycle/cycle_runner/adapters/vm.py | 5 + .../minimal-cycle/cycle_runner/config.py | 8 + .../minimal-cycle/cycle_runner/lifecycle.py | 36 +++- .../minimal-cycle/cycle_runner/orca.py | 154 ++++++++++++++++++ .../minimal-cycle/cycle_runner/result.py | 2 + .../minimal-cycle/cycle_runner/worker.py | 32 +++- .../evidence/orca-in-vm/README.md | 58 +++++++ .../evidence/orca-in-vm/image-manifest.txt | 5 + .../evidence/orca-in-vm/orca-status.json | 112 +++++++++++++ .../evidence/orca-in-vm/terminals.json | 57 +++++++ .../evidence/orca-in-vm/worker-list.json | 99 +++++++++++ .../minimal-cycle/host/packer/provision.sh | 27 ++- experiments/minimal-cycle/tests/fakes.py | 33 +++- .../minimal-cycle/tests/test_adapter_vm.py | 19 +++ .../minimal-cycle/tests/test_lifecycle.py | 36 ++++ experiments/minimal-cycle/tests/test_orca.py | 151 +++++++++++++++++ .../minimal-cycle/tests/test_worker.py | 36 ++++ 18 files changed, 875 insertions(+), 15 deletions(-) create mode 100644 experiments/minimal-cycle/cycle_runner/orca.py create mode 100644 experiments/minimal-cycle/evidence/orca-in-vm/README.md create mode 100644 experiments/minimal-cycle/evidence/orca-in-vm/image-manifest.txt create mode 100644 experiments/minimal-cycle/evidence/orca-in-vm/orca-status.json create mode 100644 experiments/minimal-cycle/evidence/orca-in-vm/terminals.json create mode 100644 experiments/minimal-cycle/evidence/orca-in-vm/worker-list.json create mode 100644 experiments/minimal-cycle/tests/test_orca.py diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 4657120..6b63030 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,26 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="a-present-orca-is-not-a-ready-one", + requirement="A dispatch waits for Orca's runtime, not just for its binary", + path="cycle_runner/lifecycle.py", + original=""" if not runtime.ready:""", + replacement=""" if False:""", + instruments=("tests.test_lifecycle.OrcaSessionTest",), + ), + Counterexample( + name="dispatch-needs-a-sender-terminal", + requirement="Every orchestration command names the terminal it is sent from", + path="cycle_runner/worker.py", + original=""" if coordinator_handle: + for command in (created, start, wait): + command.extend(["--from", coordinator_handle])""", + replacement=""" if False: + for command in (created, start, wait): + command.extend(["--from", coordinator_handle])""", + instruments=("tests.test_worker.CoordinatorTerminalTest",), + ), Counterexample( name="an-address-is-not-readiness", requirement="Creation waits for the guest to answer, not just to take an address", diff --git a/experiments/minimal-cycle/cycle_runner/adapters/vm.py b/experiments/minimal-cycle/cycle_runner/adapters/vm.py index 79e6a32..c365318 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/vm.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/vm.py @@ -78,6 +78,7 @@ META_DATA = {meta_data!r} NETWORK_CONFIG = {network_config!r} GRAPHICS = {graphics!r} +VIDEO = {video!r} LISTEN_ADDRESS = {listen_address!r} MEMORY_MB = {memory_mb!r} VCPUS = {vcpus!r} @@ -125,6 +126,7 @@ consoles=[ libvirt.DomainConsoleArgs(type="pty", target_port="0", target_type="serial") ], + video=libvirt.DomainVideoArgs(type=VIDEO), xml=libvirt.DomainXmlArgs(xslt=EGRESS_XSLT), ) @@ -320,6 +322,7 @@ def render_program(self) -> str: meta_data=self.render_meta_data(), network_config=self.render_network_config(), graphics=self.settings.graphics, + video=self.settings.video, listen_address=self.settings.listen_address, memory_mb=self.settings.memory_mb, vcpus=self.settings.vcpus, @@ -394,6 +397,7 @@ def plan_handle(self) -> EnvironmentHandle: "base_image": str(self.settings.base_image), "base_volume": self.settings.base_volume_name, "graphics": self.settings.graphics, + "video": self.settings.video, "transport_mac": self.transport_mac, "egress": self.settings.egress, }, @@ -918,6 +922,7 @@ def describe(self) -> dict: "base_volume": self.settings.base_volume_name, "base_image_sha256": self.settings.base_image_sha256, "graphics": self.settings.graphics, + "video": self.settings.video, "listen_address": self.settings.listen_address, "transport_mac": self.transport_mac, "egress_interface": self.settings.egress, diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py index 0e21ebe..58173c1 100644 --- a/experiments/minimal-cycle/cycle_runner/config.py +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -206,6 +206,7 @@ class OrcaConfig: version_argv: tuple[str, ...] = ("orca", "--version") run_objective: str = "dely minimal cycle" worktree_selector: str = "current" + ready_timeout_seconds: int = 300 def to_document(self) -> dict[str, Any]: return { @@ -217,6 +218,7 @@ def to_document(self) -> dict[str, Any]: "version_argv": list(self.version_argv), "run_objective": self.run_objective, "worktree_selector": self.worktree_selector, + "ready_timeout_seconds": self.ready_timeout_seconds, } @@ -254,6 +256,7 @@ class VmConfig: memory_mb: int = 4096 vcpus: int = 2 graphics: str = "vnc" + video: str = "virtio" listen_address: str = "127.0.0.1" transport: str = "ssh" network: str = "default" @@ -279,6 +282,7 @@ def to_document(self) -> dict[str, Any]: "memory_mb": self.memory_mb, "vcpus": self.vcpus, "graphics": self.graphics, + "video": self.video, "listen_address": self.listen_address, "transport": self.transport, "network": self.network, @@ -436,6 +440,7 @@ def _orca(document: Mapping[str, Any]) -> OrcaConfig: "version_argv", "run_objective", "worktree_selector", + "ready_timeout_seconds", ), "orca", ) @@ -448,6 +453,7 @@ def _orca(document: Mapping[str, Any]) -> OrcaConfig: version_argv=_argv(document, "version_argv", "orca", ("orca", "--version")), run_objective=_text(document, "run_objective", "orca", "dely minimal cycle"), worktree_selector=_text(document, "worktree_selector", "orca", "current"), + ready_timeout_seconds=_positive_int(document, "ready_timeout_seconds", "orca", 300), ) @@ -511,6 +517,7 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: "memory_mb", "vcpus", "graphics", + "video", "listen_address", "transport", "network", @@ -548,6 +555,7 @@ def _vm(document: Mapping[str, Any]) -> VmConfig: memory_mb=_positive_int(document, "memory_mb", "vm", 4096), vcpus=_positive_int(document, "vcpus", "vm", 2), graphics=_text(document, "graphics", "vm", "vnc"), + video=_text(document, "video", "vm", "virtio"), listen_address=_text(document, "listen_address", "vm", "127.0.0.1"), transport=_text(document, "transport", "vm", "ssh"), network=_text(document, "network", "vm", "default"), diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index fda5592..e7eb1bf 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -17,7 +17,7 @@ from pathlib import Path from typing import Any, Mapping -from . import auth, cleanup, hostinfo, manifest, probe, proc, project, redact, worker +from . import auth, cleanup, hostinfo, manifest, orca, probe, proc, project, redact, worker from .adapters.base import BackendAdapter, EnvironmentHandle from .config import RunConfig from .export import Exporter @@ -165,6 +165,7 @@ def __init__( self.task_ran = False self.stop_confirmed = True self.env_overlay: dict[str, str] = {} + self.coordinator_handle: str | None = None self.host_before: dict[str, Any] = {} # -- plumbing --------------------------------------------------------- @@ -340,6 +341,38 @@ def _identity(self) -> None: record.status = PhaseStatus.BLOCKED record.detail = reason self.blocked_reason = self.blocked_reason or reason + return + # A command that resolves is not a runtime that can take a dispatch. + runtime = orca.wait_for_runtime( + self.adapter, + self.config.orca.status_argv, + timeout=self.config.orca.ready_timeout_seconds, + ) + identity.orca_runtime = runtime.to_document() + self.exporter.write_json("identity/orca-runtime.json", runtime.to_document()) + if not runtime.ready: + record.status = PhaseStatus.BLOCKED + record.detail = ( + "orca is installed in the environment but its runtime never became " + f"ready there: {runtime.detail}" + ) + self.blocked_reason = self.blocked_reason or record.detail + return + try: + self.coordinator_handle = orca.open_coordinator_terminal( + self.adapter, + self.handle.project_path, + timeout=min(300, self.config.timeout_seconds), + ) + except orca.OrcaSessionError as error: + record.status = PhaseStatus.BLOCKED + record.detail = str(error) + self.blocked_reason = self.blocked_reason or record.detail + return + record.detail = ( + f"orca runtime {runtime.runtime_id} is ready and the coordinator " + f"terminal is {self.coordinator_handle}" + ) def _task(self) -> None: assert self.handle is not None @@ -351,6 +384,7 @@ def _task(self) -> None: handle=self.handle, timeout_seconds=self.config.timeout_seconds, env_overlay=self.env_overlay, + coordinator_handle=self.coordinator_handle, ) self.result.worker = worker_record record.status = worker_record.status diff --git a/experiments/minimal-cycle/cycle_runner/orca.py b/experiments/minimal-cycle/cycle_runner/orca.py new file mode 100644 index 0000000..94db195 --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/orca.py @@ -0,0 +1,154 @@ +"""Orca's own session inside the environment. + +Orca's command line is a client: it reaches a runtime the desktop application +owns, and every orchestration command is sent *from* a terminal that runtime +knows. So before a worker can be dispatched, two things have to be true inside +the environment, and both are observed rather than assumed: the runtime reports +itself ready, and a coordinator terminal exists in the project copy. +""" + +from __future__ import annotations + +import json +import time +from dataclasses import dataclass, field +from typing import Any, Callable, Protocol, Sequence + + +class Environment(Protocol): + """The part of a backend adapter this module needs.""" + + def execute(self, argv: Sequence[str], *, timeout: float, cwd: str | None = ..., + env: Any = ..., extra_values: Sequence[str] = ...) -> Any: ... + + +class OrcaSessionError(RuntimeError): + """Orca could not be brought to a state that can accept a dispatch.""" + + +@dataclass +class RuntimeReport: + """What Orca said about itself inside the environment.""" + + ready: bool = False + app_running: bool = False + app_pid: int | None = None + desktop_window: str | None = None + runtime_state: str | None = None + runtime_id: str | None = None + app_version: str | None = None + capabilities: list[str] = field(default_factory=list) + detail: str = "" + + def to_document(self) -> dict[str, Any]: + return { + "ready": self.ready, + "app_running": self.app_running, + "app_pid": self.app_pid, + "desktop_window": self.desktop_window, + "runtime_state": self.runtime_state, + "runtime_id": self.runtime_id, + "app_version": self.app_version, + "capabilities": list(self.capabilities), + "detail": self.detail, + } + + +def _document(text: str) -> dict[str, Any] | None: + for candidate in (text, *text.splitlines()): + stripped = candidate.strip() + if not stripped.startswith("{"): + continue + try: + parsed = json.loads(stripped) + except json.JSONDecodeError: + continue + if isinstance(parsed, dict): + return parsed + return None + + +def read_status(environment: Environment, argv: Sequence[str], *, timeout: float) -> RuntimeReport: + """Ask Orca inside the environment what state it is in.""" + outcome = environment.execute(list(argv), timeout=timeout) + if not outcome.ok: + return RuntimeReport( + detail="orca status did not answer: " + + (outcome.stderr or outcome.stdout).strip()[:200] + ) + document = _document(outcome.stdout) + if document is None: + return RuntimeReport(detail="orca status could not be read as a document") + result = document.get("result") or {} + app = result.get("app") or {} + runtime = result.get("runtime") or {} + report = RuntimeReport( + app_running=bool(app.get("running")), + app_pid=app.get("pid"), + desktop_window=app.get("desktopWindowStatus"), + runtime_state=runtime.get("state"), + runtime_id=runtime.get("runtimeId"), + app_version=runtime.get("appVersion"), + capabilities=list(runtime.get("capabilities") or []), + ) + report.ready = bool( + report.app_running and report.runtime_state == "ready" and runtime.get("reachable") + ) + report.detail = ( + f"the runtime is {report.runtime_state} and the application is " + f"{'running' if report.app_running else 'not running'}" + ) + return report + + +def wait_for_runtime( + environment: Environment, + argv: Sequence[str], + *, + timeout: float, + sleeper: Callable[[float], None] = time.sleep, + interval: float = 5.0, +) -> RuntimeReport: + """Poll until Orca reports a ready runtime, or the deadline passes.""" + deadline = time.monotonic() + timeout + while True: + report = read_status(environment, argv, timeout=min(120, max(timeout, 30))) + if report.ready: + return report + if time.monotonic() >= deadline: + return report + sleeper(interval) + + +def open_coordinator_terminal( + environment: Environment, project_path: str, *, timeout: float +) -> str: + """Register the project copy with Orca and open the terminal to send from.""" + environment.execute( + ["orca", "repo", "add", "--path", project_path, "--json"], timeout=timeout + ) + created = environment.execute( + [ + "orca", + "terminal", + "create", + "--worktree", + f"path:{project_path}", + "--json", + ], + timeout=timeout, + ) + if not created.ok: + raise OrcaSessionError( + "orca terminal create did not open a coordinator terminal: " + + (created.stderr or created.stdout).strip()[:300] + ) + document = _document(created.stdout) or {} + terminal = ((document.get("result") or {}).get("terminal")) or {} + handle = terminal.get("handle") + if not handle: + raise OrcaSessionError( + "orca terminal create returned no terminal handle, so nothing can be " + "dispatched from it" + ) + return handle diff --git a/experiments/minimal-cycle/cycle_runner/result.py b/experiments/minimal-cycle/cycle_runner/result.py index c8bf2ae..5d746dc 100644 --- a/experiments/minimal-cycle/cycle_runner/result.py +++ b/experiments/minimal-cycle/cycle_runner/result.py @@ -90,6 +90,7 @@ class IdentityRecord: orca_is_host_installation: bool = False orca_version: str | None = None orca_status_exit_code: int | None = None + orca_runtime: dict[str, Any] = field(default_factory=dict) def to_document(self) -> dict[str, Any]: return { @@ -102,6 +103,7 @@ def to_document(self) -> dict[str, Any]: "orca_is_host_installation": self.orca_is_host_installation, "orca_version": self.orca_version, "orca_status_exit_code": self.orca_status_exit_code, + "orca_runtime": dict(self.orca_runtime), } diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index 1a892e6..1365a39 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -55,7 +55,12 @@ def build_prompt(run_config: RunConfig, handle: EnvironmentHandle) -> str: def build_plan( - *, run_config: RunConfig, handle: EnvironmentHandle, timeout_seconds: int, orca_run_id: str | None + *, + run_config: RunConfig, + handle: EnvironmentHandle, + timeout_seconds: int, + orca_run_id: str | None, + coordinator_handle: str | None = None, ) -> LaunchPlan: """Compose the argv for run-create, worker-start and the completion wait.""" prompt_path = str(Path(handle.home_path) / PROMPT_NAME) @@ -95,17 +100,23 @@ def build_plan( if orca_run_id: start.extend(["--run", orca_run_id]) wait.extend(["--run", orca_run_id]) + created = [ + "orca", + "orchestration", + "run-create", + "--objective", + run_config.orca.run_objective, + "--json", + ] + # Every orchestration command is sent from a terminal the runtime knows; + # without one Orca refuses with no_active_sender_terminal. + if coordinator_handle: + for command in (created, start, wait): + command.extend(["--from", coordinator_handle]) return LaunchPlan( prompt_path=prompt_path, spec=spec, - run_create_argv=( - "orca", - "orchestration", - "run-create", - "--objective", - run_config.orca.run_objective, - "--json", - ), + run_create_argv=tuple(created), worker_start_argv=tuple(start), wait_argv=tuple(wait), ) @@ -156,6 +167,7 @@ def launch( handle: EnvironmentHandle, timeout_seconds: int, env_overlay: Mapping[str, str] | None = None, + coordinator_handle: str | None = None, ) -> WorkerRecord: """Run exactly one worker and report how it settled.""" record = WorkerRecord( @@ -171,6 +183,7 @@ def launch( handle=handle, timeout_seconds=timeout_seconds, orca_run_id=None, + coordinator_handle=coordinator_handle, ) created = adapter.execute( @@ -195,6 +208,7 @@ def launch( handle=handle, timeout_seconds=timeout_seconds, orca_run_id=record.run_id, + coordinator_handle=coordinator_handle, ) adapter.write_file(plan.prompt_path, build_prompt(run_config, handle), mode=0o644) diff --git a/experiments/minimal-cycle/evidence/orca-in-vm/README.md b/experiments/minimal-cycle/evidence/orca-in-vm/README.md new file mode 100644 index 0000000..8e6296d --- /dev/null +++ b/experiments/minimal-cycle/evidence/orca-in-vm/README.md @@ -0,0 +1,58 @@ +# Orca running inside the virtual machine + +Captured from a guest booted from `tool-image.qcow2`, with the display server +running on a virtio video device and openbox as the window manager. + +## `orca-status.json` + +`orca status --json`, run inside the guest over the per-run transport: + + app.running true + app.pid 1786 + app.desktopWindowStatus available + runtime.state ready + runtime.reachable true + runtime.connectionState connected + runtime.runtimeId 9386612e-bf79-44a0-9232-173c653dcba8 + runtime.appVersion 1.4.201 + runtime.capabilities include orchestration.contract.v1 + +This is a runtime the guest owns. The host's own Orca has a different runtime +identifier, and the guest reached this state with no host process involved. + +## The window + +A framebuffer capture taken from the host with +`virsh --connect qemu:///system screenshot ` showed the Orca window +rendered inside the guest: the first-run wizard, titled "Pick your default +agent", listing what it had detected **on that guest** — `claude` and +`orca-ide claude-teams`. That is the application's own window on the guest's +display, observed from outside the guest, independently of any layout or focus +claim made inside it. + +The capture is a binary file and is deliberately not committed: this +repository's disclosure gate greps every tracked file, and arbitrary binary +content trips it. Reproduce it with the command above against a live domain. + +## `terminals.json` and `worker-list.json` + +After `orca repo add` and `orca terminal create`, a coordinator terminal +existed, and a dispatch created a **second** terminal for the worker: + + term_50c4a5b6-5060-4cab-ba85-609b60a69f45 the coordinator + term_c862c124-5808-407a-9ef3-8a3f53eff7c7 the worker's own terminal + +`worker-list.json` ties them together: one dispatch (`ctx_098ec148783c`), one +task (`task_0e636c920592`), one run (`run_a52fcdd366b4`), and a worktree +resource owned by that dispatch. Control and the worker are separate sessions +by process and dispatch identity, not by where a panel happens to be drawn. + +The worker settled as `start_unknown` / `turn_start_unobserved`: the image +carries no credential, so Claude Code had nothing to authenticate with and its +turn never began. Orca reported that honestly rather than claiming a result. + +## What this does not show + +A worker that actually ran a turn, a task completed, or a check passed. Those +need a credential reaching the guest, which is the one thing this image is built +never to carry. diff --git a/experiments/minimal-cycle/evidence/orca-in-vm/image-manifest.txt b/experiments/minimal-cycle/evidence/orca-in-vm/image-manifest.txt new file mode 100644 index 0000000..7d751d4 --- /dev/null +++ b/experiments/minimal-cycle/evidence/orca-in-vm/image-manifest.txt @@ -0,0 +1,5 @@ +orca_version=1.4.201 +claude_code_version=2.1.272 +node_version=v18.19.1 +ubuntu=Ubuntu 24.04.5 LTS +built_at=2026-09-15T10:49:55Z diff --git a/experiments/minimal-cycle/evidence/orca-in-vm/orca-status.json b/experiments/minimal-cycle/evidence/orca-in-vm/orca-status.json new file mode 100644 index 0000000..8f62cd3 --- /dev/null +++ b/experiments/minimal-cycle/evidence/orca-in-vm/orca-status.json @@ -0,0 +1,112 @@ +{ + "id": "local-status", + "ok": true, + "result": { + "target": { + "kind": "local" + }, + "app": { + "running": true, + "pid": 1786, + "desktopWindowStatus": "available" + }, + "runtime": { + "state": "ready", + "reachable": true, + "connectionState": "connected", + "runtimeId": "9386612e-bf79-44a0-9232-173c653dcba8", + "appVersion": "1.4.201", + "remoteUpdateSupport": { + "installMode": "interactive", + "automatic": false, + "reason": "manual-service-update-required" + }, + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ] + }, + "graph": { + "state": "ready" + } + }, + "_meta": { + "runtimeId": "9386612e-bf79-44a0-9232-173c653dcba8" + } +} diff --git a/experiments/minimal-cycle/evidence/orca-in-vm/terminals.json b/experiments/minimal-cycle/evidence/orca-in-vm/terminals.json new file mode 100644 index 0000000..f988e09 --- /dev/null +++ b/experiments/minimal-cycle/evidence/orca-in-vm/terminals.json @@ -0,0 +1,57 @@ +{ + "id": "f834829b-d386-4f5e-9cea-7b57e5164887", + "ok": true, + "result": { + "terminals": [ + { + "handle": "term_50c4a5b6-5060-4cab-ba85-609b60a69f45", + "ptyId": "e8ac363c-41b0-4447-bd9a-f850ed3c837f::/home/cycle/project@@ef0e740a", + "incarnationId": "ab2cdf23-9a54-48ea-b918-53a0546e9bb6", + "orphaned": false, + "worktreeId": "e8ac363c-41b0-4447-bd9a-f850ed3c837f::/home/cycle/project", + "worktreePath": "/home/cycle/project", + "branch": "refs/heads/master", + "tabId": "279dcdae-cb06-4c20-ad82-ef7d28df9dcb", + "leafId": "5ddefe26-a667-4717-aa9b-c1eea4a727e1", + "title": "cycle@dely-cycle-spike2: ~/project", + "connected": true, + "writable": true, + "lastOutputAt": 1789469911288, + "preview": "cycle@dely-cycle-spike2:~/project$", + "executionHostId": "local" + }, + { + "handle": "term_c862c124-5808-407a-9ef3-8a3f53eff7c7", + "ptyId": "e8ac363c-41b0-4447-bd9a-f850ed3c837f::/home/cycle/project@@670ed7a6", + "incarnationId": "d6143ee6-e69d-4cc3-85c4-776e34d0cfe0", + "orphaned": false, + "worktreeId": "e8ac363c-41b0-4447-bd9a-f850ed3c837f::/home/cycle/project", + "worktreePath": "/home/cycle/project", + "branch": "refs/heads/master", + "tabId": "0b359691-1fda-43b5-970d-8c5f110a0e9e", + "leafId": "9a737bc6-6efb-4673-a9b8-a8e9aa51357b", + "title": "cycle@dely-cycle-spike2: ~/project", + "connected": true, + "writable": true, + "lastOutputAt": 1789469937962, + "preview": "cycle@dely-cycle-spike2:~/project$ claude '--dangerously-skip-permissions' '--model' 'claude-opus-5' '--effort' 'high'", + "executionHostId": "local", + "agentIdentity": "claude" + } + ], + "hostScope": { + "hostIds": [ + "local" + ], + "omittedHostIds": [] + }, + "topologyRevisions": { + "e8ac363c-41b0-4447-bd9a-f850ed3c837f::/home/cycle/project": 2 + }, + "totalCount": 2, + "truncated": false + }, + "_meta": { + "runtimeId": "9386612e-bf79-44a0-9232-173c653dcba8" + } +} diff --git a/experiments/minimal-cycle/evidence/orca-in-vm/worker-list.json b/experiments/minimal-cycle/evidence/orca-in-vm/worker-list.json new file mode 100644 index 0000000..9600e20 --- /dev/null +++ b/experiments/minimal-cycle/evidence/orca-in-vm/worker-list.json @@ -0,0 +1,99 @@ +{ + "id": "5651081b-a1d9-4071-b535-9be8a85068a8", + "ok": true, + "result": { + "workers": [ + { + "dispatchId": "ctx_098ec148783c", + "taskId": "task_0e636c920592", + "runId": "run_a52fcdd366b4", + "workerState": "start_unknown", + "dispatchStatus": "pending", + "agentTerminalHandle": "term_c862c124-5808-407a-9ef3-8a3f53eff7c7", + "terminalState": "active", + "resource": { + "id": "wtr_d189325f038a", + "ownershipState": "owned", + "releaseState": "not_requested", + "retainedReason": null, + "terminalHandle": "term_c862c124-5808-407a-9ef3-8a3f53eff7c7", + "worktreeId": "e8ac363c-41b0-4447-bd9a-f850ed3c837f::/home/cycle/project", + "endpointId": "9386612e-bf79-44a0-9232-173c653dcba8", + "endpointIncarnation": "e8ac363c-41b0-4447-bd9a-f850ed3c837f::/home/cycle/project@@670ed7a6:d6143ee6-e69d-4cc3-85c4-776e34d0cfe0", + "originDispatchId": "ctx_098ec148783c", + "ownerDispatchId": "ctx_098ec148783c", + "releaseRequestedAt": null, + "releaseCompletedAt": null, + "releaseError": null, + "recoveryAttemptCount": 0, + "lastRecoveryAt": null, + "archive": { + "source": null, + "status": null + } + }, + "projection": { + "id": "ctx_098ec148783c", + "dispatchId": "ctx_098ec148783c", + "taskId": "task_0e636c920592", + "runId": "run_a52fcdd366b4", + "role": "worker", + "parent": null, + "provider": null, + "host": { + "kind": "local", + "id": "local" + }, + "workspace": { + "id": "e8ac363c-41b0-4447-bd9a-f850ed3c837f::/home/cycle/project", + "kind": "folder_or_worktree" + }, + "stage": { + "worker": "start_unknown", + "dispatch": "pending", + "detail": "turn_start_unobserved", + "activity": "unknown" + }, + "outcome": "in_progress", + "liveness": { + "verdict": "unverifiable", + "reason": "missing_status" + }, + "evidence": { + "durable": true, + "liveStatus": "unavailable", + "lastObservedAt": null + }, + "resource": { + "state": "owned" + }, + "nextAction": { + "kind": "none", + "argv": [] + }, + "attention": { + "categories": [ + "unverifiable" + ], + "requiresAction": true + } + } + } + ], + "counts": { + "active": 1 + }, + "page": { + "limit": 100, + "total": 1, + "hasMore": false, + "nextCursor": null + }, + "scope": { + "source": "all" + } + }, + "_meta": { + "runtimeId": "9386612e-bf79-44a0-9232-173c653dcba8" + } +} diff --git a/experiments/minimal-cycle/host/packer/provision.sh b/experiments/minimal-cycle/host/packer/provision.sh index 312a8c6..dd236cb 100755 --- a/experiments/minimal-cycle/host/packer/provision.sh +++ b/experiments/minimal-cycle/host/packer/provision.sh @@ -12,7 +12,7 @@ sudo apt-get install -y --no-install-recommends \ qemu-guest-agent \ nodejs npm \ xserver-xorg-core xserver-xorg-video-vesa xserver-xorg-video-qxl \ - xserver-xorg-input-libinput xinit openbox xterm \ + xserver-xorg-input-libinput xinit openbox xterm xdotool \ dbus-x11 x11-xserver-utils xdg-utils \ libgtk-3-0t64 libnss3 libasound2t64 libatk-bridge2.0-0t64 libcups2t64 \ libdrm2 libgbm1 libxkbcommon0 libpango-1.0-0 libxcomposite1 libxdamage1 \ @@ -35,21 +35,38 @@ sudo tee /etc/systemd/system/dely-cycle-desktop.service > /dev/null < /dev/null < Date: Tue, 15 Sep 2026 14:15:29 +0200 Subject: [PATCH 18/42] Stop the test doubles running anything they do not name: a fall-through to the real process runner launched the Orca application on the developer's own desktop. --- docs/decisions.md | 71 +++++++++++++++- experiments/minimal-cycle/counterexamples.py | 12 +++ .../minimal-cycle/cycle_runner/config.py | 8 ++ .../minimal-cycle/cycle_runner/lifecycle.py | 8 +- .../minimal-cycle/cycle_runner/orca.py | 26 ++++++ .../host/packer/build-tool-image | 15 ++++ experiments/minimal-cycle/tests/fakes.py | 39 +++++++++ .../tests/test_adapter_distrobox.py | 8 +- .../tests/test_doubles_are_contained.py | 83 +++++++++++++++++++ .../minimal-cycle/tests/test_lifecycle.py | 21 +++++ experiments/minimal-cycle/tests/test_orca.py | 39 +++++++++ 11 files changed, 327 insertions(+), 3 deletions(-) create mode 100644 experiments/minimal-cycle/tests/test_doubles_are_contained.py diff --git a/docs/decisions.md b/docs/decisions.md index 2e96b8d..c64632d 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -3,12 +3,81 @@ What has been settled, what is still open, and what was rejected and why. Rationale is kept because the reasons are the reusable part. -Last updated 2026-09-14. +Last updated 2026-09-15. --- ## Settled +### 2026-09-15 — A test double that can run a command runs it on the developer's machine + +#### Context + +The cycle runner's fake backend adapter treated its "environment" as a +directory on the host, and answered a small set of commands from a table. +Anything it did not recognise fell through to the real process runner, which +meant it ran on the machine running the tests. That was a deliberate choice: +it made the fake honest about the commands it did handle, because the check +script and the auth teardown really executed and really observed their files. + +Then the runner gained a step that starts the Orca desktop application inside +the environment. The fake did not recognise it, so it fell through. Every +lifecycle test that reached the identity phase — more than a dozen — launched a +real Orca on the developer's own desktop. The result was dozens of application +windows and a reboot to clear them. + +Nothing about that was a bug in the runner. The runner did exactly what it was +asked. The fault was entirely in the double: its fall-through was a general +permission to execute, held open for the convenience of a few specific +commands. + +#### Decision + +A test double never runs a command it does not name. The fake adapter carries +an allowlist of program names and, for `sh -c`, an allowlist of the runner's +own script names; anything else is answered with a non-zero outcome and a +reason, never executed. Removals are refused outside the fake's own root. The +stub runner's passthrough is narrowed to key generation, which is the one thing +it genuinely needs a real process for. + +`tests/test_doubles_are_contained.py` holds these as tests: starting the +application through the fake must be refused, an arbitrary program must be +refused, an unnamed shell script must be refused, and the runner's own scripts +must still run. + +#### Alternatives considered + +Removing execution from the fake entirely and scripting every answer. It would +be safe and it would make the check and the teardown prove nothing: those tests +are worth having precisely because a real file is really inspected. + +Running the suite in a container or a virtual machine. It would contain the +blast radius and it would also hide the defect, which is that a double should +not be able to do this in the first place. It is worth doing later as defence +in depth; it is not the fix. + +#### Consequences + +The allowlist has to grow deliberately when the runner learns a new command the +fake should really execute, and a reviewer sees that growth. That is the point. + +This generalises past this runner. Any test double standing in for a remote or +isolated environment, on a machine where the same commands would also work +locally, has this shape: the fall-through case is the dangerous one, and it is +usually written for convenience by someone who is thinking about the commands +they listed rather than the ones they did not. + +#### Non-goals + +Sandboxing the suite. Preventing the runner itself from starting applications +in a real environment — that is its job. + +#### Deferred + +Running the suite inside a disposable environment as defence in depth, +triggered by the suite needing to execute anything broader than the current +allowlist. + ### 2026-09-14 — One shared cycle runner, two backend adapters, export before cleanup #### Context diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 6b63030..0a20675 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,18 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="the-application-is-started", + requirement="The runner starts the application rather than hoping something did", + path="cycle_runner/lifecycle.py", + original=""" started = self.execute( + orca.start_argv(self.config.orca.app_argv, self.config.orca.display), + timeout=min(120, self.config.timeout_seconds), + ) + record.commands.append(started.to_record())""", + replacement=""" pass""", + instruments=("tests.test_lifecycle.OrcaSessionTest",), + ), Counterexample( name="a-present-orca-is-not-a-ready-one", requirement="A dispatch waits for Orca's runtime, not just for its binary", diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py index 58173c1..599ed0b 100644 --- a/experiments/minimal-cycle/cycle_runner/config.py +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -207,6 +207,8 @@ class OrcaConfig: run_objective: str = "dely minimal cycle" worktree_selector: str = "current" ready_timeout_seconds: int = 300 + app_argv: tuple[str, ...] = ("/opt/Orca/orca-ide",) + display: str = ":0" def to_document(self) -> dict[str, Any]: return { @@ -219,6 +221,8 @@ def to_document(self) -> dict[str, Any]: "run_objective": self.run_objective, "worktree_selector": self.worktree_selector, "ready_timeout_seconds": self.ready_timeout_seconds, + "app_argv": list(self.app_argv), + "display": self.display, } @@ -441,6 +445,8 @@ def _orca(document: Mapping[str, Any]) -> OrcaConfig: "run_objective", "worktree_selector", "ready_timeout_seconds", + "app_argv", + "display", ), "orca", ) @@ -454,6 +460,8 @@ def _orca(document: Mapping[str, Any]) -> OrcaConfig: run_objective=_text(document, "run_objective", "orca", "dely minimal cycle"), worktree_selector=_text(document, "worktree_selector", "orca", "current"), ready_timeout_seconds=_positive_int(document, "ready_timeout_seconds", "orca", 300), + app_argv=_argv(document, "app_argv", "orca", ("/opt/Orca/orca-ide",)), + display=_text(document, "display", "orca", ":0"), ) diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index e7eb1bf..6edb6e4 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -342,7 +342,13 @@ def _identity(self) -> None: record.detail = reason self.blocked_reason = self.blocked_reason or reason return - # A command that resolves is not a runtime that can take a dispatch. + # A command that resolves is not a runtime that can take a dispatch, + # and nothing else starts the application, so the runner does. + started = self.execute( + orca.start_argv(self.config.orca.app_argv, self.config.orca.display), + timeout=min(120, self.config.timeout_seconds), + ) + record.commands.append(started.to_record()) runtime = orca.wait_for_runtime( self.adapter, self.config.orca.status_argv, diff --git a/experiments/minimal-cycle/cycle_runner/orca.py b/experiments/minimal-cycle/cycle_runner/orca.py index 94db195..27a75cf 100644 --- a/experiments/minimal-cycle/cycle_runner/orca.py +++ b/experiments/minimal-cycle/cycle_runner/orca.py @@ -120,6 +120,32 @@ def wait_for_runtime( sleeper(interval) +#: Launched detached with its own output kept, because a window manager's +#: autostart was observed to leave only a crash directory and a stale lock. +START_SCRIPT = ( + 'rm -f "$HOME/.config/orca/SingletonLock" "$HOME/.config/orca/SingletonCookie"; ' + 'export DISPLAY="$1"; export XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"; ' + 'shift; nohup "$@" > "$HOME/orca-app.log" 2>&1 & ' + 'printf "started %s\n" "$!"' +) + + +def start_argv(app_argv: Sequence[str], display: str) -> list[str]: + """Return the command that starts the application inside the environment.""" + return ["sh", "-c", START_SCRIPT, "orca-start", display, *[str(a) for a in app_argv]] + + +def start_application( + environment: Environment, + app_argv: Sequence[str], + *, + display: str, + timeout: float, +) -> Any: + """Start the Orca application inside the environment and return the outcome.""" + return environment.execute(start_argv(app_argv, display), timeout=timeout) + + def open_coordinator_terminal( environment: Environment, project_path: str, *, timeout: float ) -> str: diff --git a/experiments/minimal-cycle/host/packer/build-tool-image b/experiments/minimal-cycle/host/packer/build-tool-image index a7cbe78..0db8d35 100755 --- a/experiments/minimal-cycle/host/packer/build-tool-image +++ b/experiments/minimal-cycle/host/packer/build-tool-image @@ -121,6 +121,21 @@ def build() -> int: IMAGES.mkdir(parents=True, exist_ok=True) target = IMAGES / IMAGE_NAME + # A previous image that a domain has used belongs to the emulator, not + # to this user, so it is removed through libvirt rather than by hand. + if target.exists(): + subprocess.run( + ["virsh", "--connect", VERSIONS["host_tools"]["connect_uri"], + "vol-delete", "--pool", VERSIONS["host_tools"]["storage_pool"]["name"], + IMAGE_NAME], + capture_output=True, timeout=300, + ) + if target.exists(): + try: + target.unlink() + except PermissionError: + print(f"BLOCK {target} could not be removed; delete it and build again") + return 1 shutil.move(str(produced), target) target.chmod(0o644) diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index 4b69201..aa90135 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -39,6 +39,11 @@ } +#: The only programs a fake environment may really run on this machine, and +#: the only `sh -c` scripts among them. `orca-start` is deliberately absent. +RUNNABLE_PROGRAMS = frozenset({"sh", "rm", "mkdir", "cat", "test", "true", "false", "printf"}) +RUNNABLE_SHELL_SCRIPTS = frozenset({"cycle-check", "auth-teardown", "cycle-cd"}) + ORCA_STATUS_REPLY = ( '{"ok": true, "result": {"app": {"running": true, "pid": 1786, ' '"desktopWindowStatus": "available"}, "runtime": {"state": "ready", ' @@ -225,6 +230,9 @@ def execute( self.calls.append("check") else: self.calls.append(f"execute:{argv[0]}") + refusal = self._refuse(argv) + if refusal is not None: + return refusal return proc.run( argv, timeout=timeout, @@ -234,6 +242,37 @@ def execute( extra_values=extra_values, ) + def _refuse(self, argv) -> proc.CommandOutcome | None: + """Answer, rather than run, anything outside the narrow allowlist. + + This fake's "environment" is a directory on the developer's own + machine, so a command it does not recognise runs *here*. That is not + hypothetical: when the runner gained a step that starts the Orca + desktop application, every lifecycle test launched a real one on the + host. A test double that can start an application will start it. + """ + program = Path(argv[0]).name + if program not in RUNNABLE_PROGRAMS: + return self._outcome( + argv, 127, "", f"the fake environment does not run {program}" + ) + if program == "sh": + name = argv[3] if len(argv) > 3 else "" + if name not in RUNNABLE_SHELL_SCRIPTS: + return self._outcome( + argv, 127, "", f"the fake environment does not run the script {name!r}" + ) + if program == "rm": + targets = [item for item in argv[1:] if not item.startswith("-")] + outside = [ + item for item in targets if not Path(item).is_relative_to(self.root) + ] + if outside: + return self._outcome( + argv, 1, "", f"refusing to remove outside the fake root: {outside}" + ) + return None + def _outcome(self, argv, exit_code, stdout, stderr, timed_out=False): return proc.CommandOutcome( argv=tuple(argv), diff --git a/experiments/minimal-cycle/tests/test_adapter_distrobox.py b/experiments/minimal-cycle/tests/test_adapter_distrobox.py index 2f4bbb6..4a59996 100644 --- a/experiments/minimal-cycle/tests/test_adapter_distrobox.py +++ b/experiments/minimal-cycle/tests/test_adapter_distrobox.py @@ -13,6 +13,12 @@ RUN_ID = "20260914T221530Z-abc123-0123abcd" +#: Passthrough exists so a test can generate a real key pair, and for nothing +#: else. Anything not named here is answered from the table, never run: a stub +#: that runs whatever it is handed runs it on the developer's own machine. +PASSTHROUGH_PROGRAMS = frozenset({"ssh-keygen"}) + + class StubRunner: """Answers commands from a table so the adapter can be tested without podman.""" @@ -30,7 +36,7 @@ def __call__(self, argv, *, timeout, context, cwd=None, env=None, extra_values=( for needle, code, out, err in self.table: if needle in joined: return self._outcome(argv, code, out, err, context, extra_values) - if self.passthrough: + if self.passthrough and Path(argv[0]).name in PASSTHROUGH_PROGRAMS: return proc.run( argv, timeout=timeout, diff --git a/experiments/minimal-cycle/tests/test_doubles_are_contained.py b/experiments/minimal-cycle/tests/test_doubles_are_contained.py new file mode 100644 index 0000000..d03e912 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_doubles_are_contained.py @@ -0,0 +1,83 @@ +"""The test doubles must not run anything on the machine running the tests. + +A fake environment here is a directory on a developer's own machine, so a +command the fake does not recognise runs *there*. When the runner gained a step +that starts the Orca desktop application, every lifecycle test launched a real +one on the host desktop — dozens of windows, and a reboot to clear them. These +tests exist so that cannot happen again. +""" + +import tempfile +import unittest +from pathlib import Path + +from tests.fakes import RUNNABLE_PROGRAMS, RUNNABLE_SHELL_SCRIPTS, FakeAdapter +from tests.test_adapter_distrobox import PASSTHROUGH_PROGRAMS, StubRunner + + +class FakeAdapterContainmentTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.adapter = FakeAdapter(Path(self._tmp.name) / "run") + self.adapter.create() + self.addCleanup(self._tmp.cleanup) + + def test_starting_the_orca_application_is_refused(self): + from cycle_runner import orca + + outcome = self.adapter.execute( + orca.start_argv(("/opt/Orca/orca-ide",), ":0"), timeout=5 + ) + self.assertNotEqual(outcome.exit_code, 0) + self.assertIn("does not run", outcome.stderr) + + def test_an_arbitrary_program_is_refused(self): + outcome = self.adapter.execute(["/usr/bin/firefox"], timeout=5) + self.assertEqual(outcome.exit_code, 127) + self.assertIn("firefox", outcome.stderr) + + def test_an_unnamed_shell_script_is_refused(self): + outcome = self.adapter.execute( + ["sh", "-c", "touch /tmp/should-not-exist", "whatever"], timeout=5 + ) + self.assertEqual(outcome.exit_code, 127) + self.assertFalse(Path("/tmp/should-not-exist").exists()) + + def test_removing_outside_the_fake_root_is_refused(self): + outcome = self.adapter.execute(["rm", "-f", "/etc/hosts"], timeout=5) + self.assertNotEqual(outcome.exit_code, 0) + self.assertIn("refusing to remove", outcome.stderr) + self.assertTrue(Path("/etc/hosts").exists()) + + def test_the_runner_own_scripts_still_run(self): + target = self.adapter.project / "evidence.txt" + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text("dely-cycle-marker", encoding="utf-8") + from cycle_runner.lifecycle import check_argv + + outcome = self.adapter.execute( + check_argv(str(self.adapter.project), "evidence.txt", "dely-cycle-marker"), + timeout=10, + ) + self.assertEqual(outcome.exit_code, 0) + + def test_the_allowlists_stay_small_and_name_nothing_graphical(self): + self.assertLessEqual(len(RUNNABLE_PROGRAMS), 8) + self.assertNotIn("orca", RUNNABLE_PROGRAMS) + self.assertNotIn("nohup", RUNNABLE_PROGRAMS) + self.assertNotIn("orca-start", RUNNABLE_SHELL_SCRIPTS) + + +class StubRunnerContainmentTest(unittest.TestCase): + def test_passthrough_is_only_for_key_generation(self): + self.assertEqual(PASSTHROUGH_PROGRAMS, frozenset({"ssh-keygen"})) + + def test_an_unlisted_program_is_answered_not_run(self): + runner = StubRunner(passthrough=True) + outcome = runner( + ["sh", "-c", "touch /tmp/stub-should-not-exist"], + timeout=5, + context="host", + ) + self.assertEqual(outcome.exit_code, 0) + self.assertFalse(Path("/tmp/stub-should-not-exist").exists()) diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index f70889c..38073d2 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -339,3 +339,24 @@ def test_the_coordinator_terminal_is_opened_before_the_dispatch(self): adapter, _ = self.run_cycle() self.assertIn("terminal-create", adapter.calls) self.assertLess(adapter.calls.index("terminal-create"), adapter.calls.index("worker")) + + +class OrcaApplicationStartTest(CycleTestCase): + """Nothing else starts the application, so the runner does.""" + + def test_the_application_is_started_inside_the_environment(self): + adapter, _ = self.run_cycle() + joined = [" ".join(argv) for argv in getattr(adapter, "executed", [])] + self.assertTrue( + any("orca-start" in line for line in joined) + or any("execute:sh" in call for call in adapter.calls), + adapter.calls, + ) + + def test_the_start_is_recorded_as_a_command_of_the_identity_phase(self): + _, outcome = self.run_cycle() + identity = outcome.run_result.phase("identity") + self.assertTrue( + any("orca-start" in " ".join(c.argv) for c in identity.commands), + [" ".join(c.argv)[:60] for c in identity.commands], + ) diff --git a/experiments/minimal-cycle/tests/test_orca.py b/experiments/minimal-cycle/tests/test_orca.py index 5ec1e1d..6511893 100644 --- a/experiments/minimal-cycle/tests/test_orca.py +++ b/experiments/minimal-cycle/tests/test_orca.py @@ -149,3 +149,42 @@ def test_a_terminal_that_is_not_created_is_reported(self): with self.assertRaises(orca.OrcaSessionError) as caught: orca.open_coordinator_terminal(environment, "/home/cycle/project", timeout=5) self.assertIn("terminal", str(caught.exception).lower()) + + +class StartApplicationTest(unittest.TestCase): + """The command line is a client; something has to start the application. + + Observed in a real guest: launched from the window manager's autostart the + application left only a crash handler directory and a singleton lock, and + the runtime never appeared. Launched as a detached command it reached + `ready`. So the runner starts it, and that start is a recorded command. + """ + + def test_the_application_is_started_detached_with_a_display(self): + environment = Environment([]) + orca.start_application( + environment, ("/opt/Orca/orca-ide",), display=":0", timeout=30 + ) + argv = environment.seen[-1] + joined = " ".join(argv) + self.assertIn("/opt/Orca/orca-ide", joined) + self.assertIn("nohup", joined) + # The display is an argument the script exports, never inlined text. + self.assertIn('export DISPLAY="$1"', joined) + self.assertIn(":0", argv) + self.assertEqual(argv[-1], "/opt/Orca/orca-ide") + + def test_a_stale_singleton_lock_is_cleared_first(self): + environment = Environment([]) + orca.start_application( + environment, ("/opt/Orca/orca-ide",), display=":0", timeout=30 + ) + joined = " ".join(" ".join(argv) for argv in environment.seen) + self.assertIn("SingletonLock", joined) + + def test_the_start_reports_what_it_ran(self): + environment = Environment([]) + outcome = orca.start_application( + environment, ("/opt/Orca/orca-ide",), display=":0", timeout=30 + ) + self.assertEqual(outcome.context, "environment") From dd2e37395be5594243d0289c82054b65d8fbca69 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 16:19:34 +0200 Subject: [PATCH 19/42] Start the application only when the runtime is down, wait for the display first, and stop the image racing the runner for it. --- experiments/minimal-cycle/counterexamples.py | 14 ++++++++------ .../minimal-cycle/cycle_runner/lifecycle.py | 18 ++++++++++++------ .../minimal-cycle/cycle_runner/orca.py | 14 ++++++++++++-- .../minimal-cycle/host/packer/provision.sh | 11 +++++------ experiments/minimal-cycle/tests/fakes.py | 12 +++++++++++- .../tests/test_doubles_are_contained.py | 19 ++++++++++++++++--- .../minimal-cycle/tests/test_lifecycle.py | 18 ++++++++++++------ 7 files changed, 76 insertions(+), 30 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 0a20675..6fa363f 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -233,12 +233,14 @@ class Counterexample: name="the-application-is-started", requirement="The runner starts the application rather than hoping something did", path="cycle_runner/lifecycle.py", - original=""" started = self.execute( - orca.start_argv(self.config.orca.app_argv, self.config.orca.display), - timeout=min(120, self.config.timeout_seconds), - ) - record.commands.append(started.to_record())""", - replacement=""" pass""", + original=""" if not runtime.ready: + started = self.execute( + orca.start_argv(self.config.orca.app_argv, self.config.orca.display), + timeout=min(180, self.config.timeout_seconds), + ) + record.commands.append(started.to_record())""", + replacement=""" if False: + pass""", instruments=("tests.test_lifecycle.OrcaSessionTest",), ), Counterexample( diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index 6edb6e4..5a7799b 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -342,13 +342,19 @@ def _identity(self) -> None: record.detail = reason self.blocked_reason = self.blocked_reason or reason return - # A command that resolves is not a runtime that can take a dispatch, - # and nothing else starts the application, so the runner does. - started = self.execute( - orca.start_argv(self.config.orca.app_argv, self.config.orca.display), - timeout=min(120, self.config.timeout_seconds), + # A command that resolves is not a runtime that can take a dispatch. + # Nothing else starts the application, so the runner does — but only + # when it is not already there, because a second one takes the + # singleton lock from the first and neither finishes starting. + runtime = orca.read_status( + self.adapter, self.config.orca.status_argv, timeout=120 ) - record.commands.append(started.to_record()) + if not runtime.ready: + started = self.execute( + orca.start_argv(self.config.orca.app_argv, self.config.orca.display), + timeout=min(180, self.config.timeout_seconds), + ) + record.commands.append(started.to_record()) runtime = orca.wait_for_runtime( self.adapter, self.config.orca.status_argv, diff --git a/experiments/minimal-cycle/cycle_runner/orca.py b/experiments/minimal-cycle/cycle_runner/orca.py index 27a75cf..742373d 100644 --- a/experiments/minimal-cycle/cycle_runner/orca.py +++ b/experiments/minimal-cycle/cycle_runner/orca.py @@ -123,9 +123,19 @@ def wait_for_runtime( #: Launched detached with its own output kept, because a window manager's #: autostart was observed to leave only a crash directory and a stale lock. START_SCRIPT = ( + 'export DISPLAY="$1"; shift; ' + 'export XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"; ' + # The graphical session starts at boot; the application must not be launched + # before it is there, or it leaves a crash directory and a stale lock. + 'waited=0; ' + 'while ! xset -q >/dev/null 2>&1; do ' + ' waited=$((waited + 2)); ' + ' if [ "$waited" -ge 120 ]; then printf "no display after %ss\n" "$waited" >&2; exit 3; fi; ' + ' sleep 2; ' + 'done; ' + 'printf "display ready after %ss\n" "$waited"; ' 'rm -f "$HOME/.config/orca/SingletonLock" "$HOME/.config/orca/SingletonCookie"; ' - 'export DISPLAY="$1"; export XDG_RUNTIME_DIR="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"; ' - 'shift; nohup "$@" > "$HOME/orca-app.log" 2>&1 & ' + 'nohup "$@" > "$HOME/orca-app.log" 2>&1 & ' 'printf "started %s\n" "$!"' ) diff --git a/experiments/minimal-cycle/host/packer/provision.sh b/experiments/minimal-cycle/host/packer/provision.sh index dd236cb..c375205 100755 --- a/experiments/minimal-cycle/host/packer/provision.sh +++ b/experiments/minimal-cycle/host/packer/provision.sh @@ -60,12 +60,11 @@ RestartSec=5 WantedBy=graphical.target UNIT -# The command line is a client of the runtime the application owns, so the -# application has to be running before anything can be dispatched. -sudo -u "${GUEST_USER}" mkdir -p "/home/${GUEST_USER}/.config/openbox" -sudo -u "${GUEST_USER}" tee "/home/${GUEST_USER}/.config/openbox/autostart" > /dev/null < 3 and argv[0] == "sh" and argv[3] == "orca-start": + # Simulated, never executed: running this would start a desktop + # application on the machine running the tests. + self.calls.append("orca-start") + self.app_started = True + return self._outcome(argv, 0, "display ready after 0s\nstarted 1786\n", "") if tuple(argv[:2]) == ("orca", "status"): self.calls.append("orca-status") - up = self.orca_present and self.runtime_ready + up = self.orca_present and self.runtime_ready and self.app_started return self._outcome(argv, 0, ORCA_STATUS_REPLY if up else ORCA_STATUS_DOWN, "") if "terminal create" in joined: self.calls.append("terminal-create") diff --git a/experiments/minimal-cycle/tests/test_doubles_are_contained.py b/experiments/minimal-cycle/tests/test_doubles_are_contained.py index d03e912..35362b7 100644 --- a/experiments/minimal-cycle/tests/test_doubles_are_contained.py +++ b/experiments/minimal-cycle/tests/test_doubles_are_contained.py @@ -22,14 +22,27 @@ def setUp(self): self.adapter.create() self.addCleanup(self._tmp.cleanup) - def test_starting_the_orca_application_is_refused(self): + def test_the_application_start_is_simulated_not_executed(self): from cycle_runner import orca + log = Path.home() / "orca-app.log" + existed = log.exists() outcome = self.adapter.execute( orca.start_argv(("/opt/Orca/orca-ide",), ":0"), timeout=5 ) - self.assertNotEqual(outcome.exit_code, 0) - self.assertIn("does not run", outcome.stderr) + self.assertIn("orca-start", self.adapter.calls) + self.assertIn("started 1786", outcome.stdout) + self.assertTrue(self.adapter.app_started) + # The real script would write this beside the home directory it ran in. + self.assertEqual(log.exists(), existed) + + def test_the_allowlist_would_refuse_the_start_if_it_ever_reached_execution(self): + from cycle_runner import orca + + refusal = self.adapter._refuse(orca.start_argv(("/opt/Orca/orca-ide",), ":0")) + self.assertIsNotNone(refusal) + self.assertEqual(refusal.exit_code, 127) + self.assertIn("orca-start", refusal.stderr) def test_an_arbitrary_program_is_refused(self): outcome = self.adapter.execute(["/usr/bin/firefox"], timeout=5) diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index 38073d2..f5b30a4 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -346,12 +346,18 @@ class OrcaApplicationStartTest(CycleTestCase): def test_the_application_is_started_inside_the_environment(self): adapter, _ = self.run_cycle() - joined = [" ".join(argv) for argv in getattr(adapter, "executed", [])] - self.assertTrue( - any("orca-start" in line for line in joined) - or any("execute:sh" in call for call in adapter.calls), - adapter.calls, - ) + self.assertIn("orca-start", adapter.calls) + self.assertLess(adapter.calls.index("orca-start"), adapter.calls.index("worker")) + + def test_an_already_ready_runtime_is_not_started_again(self): + adapter, outcome = self.run_cycle(already_running=True) + self.assertNotIn("orca-start", adapter.calls) + self.assertEqual(outcome.run_result.status, status.RunStatus.SETTLED) + + def test_a_runtime_that_stays_down_after_a_start_blocks(self): + adapter, outcome = self.run_cycle(runtime_ready=False) + self.assertIn("orca-start", adapter.calls) + self.assertEqual(outcome.run_result.status, status.RunStatus.BLOCKED) def test_the_start_is_recorded_as_a_command_of_the_identity_phase(self): _, outcome = self.run_cycle() From 4e402e5b94d5fe28ef55149d76da29c0ff5f17c4 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 16:22:10 +0200 Subject: [PATCH 20/42] Name the Orca command the environment must use, and tell two installations apart by what the path resolves to rather than by the path. --- experiments/minimal-cycle/counterexamples.py | 11 ++ .../minimal-cycle/cycle_runner/config.py | 4 + .../minimal-cycle/cycle_runner/lifecycle.py | 6 +- .../minimal-cycle/cycle_runner/probe.py | 34 ++++- experiments/minimal-cycle/tests/test_probe.py | 120 ++++++++++++++++++ 5 files changed, 168 insertions(+), 7 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 6fa363f..eda8c9f 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,17 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="the-same-path-is-not-the-same-file", + requirement="An environment with its own Orca at the host's path is not the host's", + path="cycle_runner/probe.py", + original=""" if environment_print and host_print: + is_host_installation = environment_print == host_print + else: + is_host_installation = bool(environment_orca) and environment_orca == host_orca""", + replacement=""" is_host_installation = bool(environment_orca) and environment_orca == host_orca""", + instruments=("tests.test_probe.OrcaFingerprintTest",), + ), Counterexample( name="the-application-is-started", requirement="The runner starts the application rather than hoping something did", diff --git a/experiments/minimal-cycle/cycle_runner/config.py b/experiments/minimal-cycle/cycle_runner/config.py index 599ed0b..db76e83 100644 --- a/experiments/minimal-cycle/cycle_runner/config.py +++ b/experiments/minimal-cycle/cycle_runner/config.py @@ -209,6 +209,7 @@ class OrcaConfig: ready_timeout_seconds: int = 300 app_argv: tuple[str, ...] = ("/opt/Orca/orca-ide",) display: str = ":0" + command: str = "orca" def to_document(self) -> dict[str, Any]: return { @@ -223,6 +224,7 @@ def to_document(self) -> dict[str, Any]: "ready_timeout_seconds": self.ready_timeout_seconds, "app_argv": list(self.app_argv), "display": self.display, + "command": self.command, } @@ -447,6 +449,7 @@ def _orca(document: Mapping[str, Any]) -> OrcaConfig: "ready_timeout_seconds", "app_argv", "display", + "command", ), "orca", ) @@ -462,6 +465,7 @@ def _orca(document: Mapping[str, Any]) -> OrcaConfig: ready_timeout_seconds=_positive_int(document, "ready_timeout_seconds", "orca", 300), app_argv=_argv(document, "app_argv", "orca", ("/opt/Orca/orca-ide",)), display=_text(document, "display", "orca", ":0"), + command=_text(document, "command", "orca", "orca"), ) diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index 5a7799b..0cf751f 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -304,13 +304,15 @@ def _identity(self) -> None: with self.phase("identity") as record: host_probe = self.log.record( proc.run( - probe.probe_argv(str(self.config.project.source)), + probe.probe_argv( + str(self.config.project.source), self.config.orca.command + ), timeout=min(120, self.config.timeout_seconds), context="host", ) ) environment_probe = self.execute( - probe.probe_argv(self.handle.project_path), + probe.probe_argv(self.handle.project_path, self.config.orca.command), timeout=min(120, self.config.timeout_seconds), ) self.exporter.write_text("identity/host-probe.txt", host_probe.stdout) diff --git a/experiments/minimal-cycle/cycle_runner/probe.py b/experiments/minimal-cycle/cycle_runner/probe.py index a13d3b2..b3cbb3d 100644 --- a/experiments/minimal-cycle/cycle_runner/probe.py +++ b/experiments/minimal-cycle/cycle_runner/probe.py @@ -31,6 +31,7 @@ "virt", "orca_path", "orca_version", + "orca_fingerprint", "project_real", ) @@ -39,6 +40,7 @@ PROBE_SCRIPT = r""" set -u project_path="${1:-}" +orca_command="${2:-orca}" first_line() { if [ -r "$1" ]; then @@ -60,10 +62,13 @@ if [ -n "$detected" ]; then virt="$detected"; fi fi -orca_binary="$(command -v orca 2>/dev/null || printf '')" +orca_binary="$(command -v "$orca_command" 2>/dev/null || printf '')" orca_release='' +orca_fingerprint='' if [ -n "$orca_binary" ]; then orca_release="$("$orca_binary" --version 2>/dev/null | head -n 1 || printf '')" + orca_resolved="$(readlink -f "$orca_binary" 2>/dev/null || printf '%s' "$orca_binary")" + orca_fingerprint="$(stat -c '%i:%s:%Y' "$orca_resolved" 2>/dev/null || printf '')" fi project_resolved='' @@ -83,15 +88,21 @@ printf 'virt=%s\n' "$virt" printf 'orca_path=%s\n' "$orca_binary" printf 'orca_version=%s\n' "$orca_release" +printf 'orca_fingerprint=%s\n' "$orca_fingerprint" printf 'project_real=%s\n' "$project_resolved" """ _UNINFORMATIVE_VIRT = ("", "none", "unknown") -def probe_argv(project_path: str) -> list[str]: - """Return the argv that runs the probe with the project path to resolve.""" - return ["sh", "-c", PROBE_SCRIPT, "cycle-probe", project_path] +def probe_argv(project_path: str, orca_command: str = "orca") -> list[str]: + """Return the argv that runs the probe for a project path and an Orca command. + + The command is named rather than assumed: a container inherits the host's + search path, so the bare name can resolve to the host's own launcher even + when the container has an installation of its own. + """ + return ["sh", "-c", PROBE_SCRIPT, "cycle-probe", project_path, orca_command] def parse(text: str) -> dict[str, str]: @@ -122,6 +133,10 @@ def _host_view(host: Mapping[str, str]) -> dict[str, Any]: "home_digest": digest(host.get("home", "")), "user_digest": digest(host.get("user", "")), "project_real_digest": digest(host.get("project_real", "")), + # Digests, not paths: the host's own installation may sit under a home + # directory, and this block travels into artifacts a person may share. + "orca_path_digest": digest(host.get("orca_path", "")), + "orca_fingerprint_digest": digest(host.get("orca_fingerprint", "")), "virt": host.get("virt", ""), "orca_present": bool(host.get("orca_path")), } @@ -141,6 +156,7 @@ def _environment_view(environment: Mapping[str, str]) -> dict[str, Any]: "virt", "orca_path", "orca_version", + "orca_fingerprint", "project_real", ) } @@ -178,7 +194,15 @@ def verdict( environment_orca = environment.get("orca_path", "") host_orca = host.get("orca_path", "") environment_version = environment.get("orca_version", "") - is_host_installation = bool(environment_orca) and environment_orca == host_orca + # The same path is not the same file: a container that installs its own + # Orca has it where the host does. Compare what the path resolves to, and + # fall back to the path only when nothing could be resolved. + environment_print = environment.get("orca_fingerprint", "") + host_print = host.get("orca_fingerprint", "") + if environment_print and host_print: + is_host_installation = environment_print == host_print + else: + is_host_installation = bool(environment_orca) and environment_orca == host_orca record = IdentityRecord( host=_host_view(host), environment=_environment_view(environment), diff --git a/experiments/minimal-cycle/tests/test_probe.py b/experiments/minimal-cycle/tests/test_probe.py index 434dcaf..cbdd82a 100644 --- a/experiments/minimal-cycle/tests/test_probe.py +++ b/experiments/minimal-cycle/tests/test_probe.py @@ -237,3 +237,123 @@ def test_the_gate_opens_for_an_orca_the_environment_carries(self): verdict(container_snapshot(orca_path="/usr/bin/orca", orca_version="1.4.201")) ) self.assertTrue(allowed) + + +class OrcaFingerprintTest(unittest.TestCase): + """The same path is not the same file. + + A container that installs its own Orca has it at `/opt/Orca/orca-ide`, and + so does the host. Comparing paths would call that the host's installation. + Comparing what the path resolves to does not. + """ + + def test_the_same_path_with_a_different_file_is_the_environments_own(self): + record = verdict( + container_snapshot( + orca_path="/opt/Orca/orca-ide", + orca_version="1.4.201", + orca_fingerprint="222:9001:1700000000", + ), + host=host_snapshot( + orca_path="/opt/Orca/orca-ide", + orca_fingerprint="111:9001:1600000000", + ), + ) + self.assertFalse(record.orca_is_host_installation) + self.assertTrue(record.orca_present) + + def test_the_same_path_and_the_same_file_is_the_hosts(self): + record = verdict( + container_snapshot( + orca_path="/opt/Orca/orca-ide", + orca_version="1.4.201", + orca_fingerprint="111:9001:1600000000", + ), + host=host_snapshot( + orca_path="/opt/Orca/orca-ide", + orca_fingerprint="111:9001:1600000000", + ), + ) + self.assertTrue(record.orca_is_host_installation) + self.assertFalse(record.orca_present) + + def test_without_a_fingerprint_the_path_is_still_compared(self): + record = verdict( + container_snapshot( + orca_path=host_snapshot()["orca_path"], + orca_version="1.4.201", + orca_fingerprint="", + ) + ) + self.assertTrue(record.orca_is_host_installation) + + def test_a_different_path_is_never_the_hosts(self): + record = verdict( + container_snapshot( + orca_path="/usr/local/bin/orca", + orca_version="1.4.201", + orca_fingerprint="333:100:1700000000", + ) + ) + self.assertFalse(record.orca_is_host_installation) + self.assertTrue(record.orca_present) + + def test_the_fingerprint_is_a_named_probe_field(self): + self.assertIn("orca_fingerprint", probe.REQUIRED_FIELDS) + self.assertIn("orca_fingerprint=", probe.PROBE_SCRIPT) + + +class HostViewDisclosureTest(unittest.TestCase): + """The host block travels into shared artifacts, so it carries no paths.""" + + def test_no_absolute_path_reaches_the_host_view(self): + record = verdict( + container_snapshot(), + host=host_snapshot( + orca_path="/home/someone/.config/orca/linux-orca-cli-shim/orca", + home="/home/someone", + project_real="/home/someone/code/under-test", + ), + ) + rendered = str(record.host) + self.assertNotIn("/home/someone", rendered) + self.assertNotIn("someone", rendered) + self.assertNotIn("/", rendered.replace("orca_path_digest", "").replace( + "orca_fingerprint_digest", "")) + + def test_the_digests_still_separate_two_different_installations(self): + first = verdict(container_snapshot(), host=host_snapshot(orca_path="/a/orca")) + second = verdict(container_snapshot(), host=host_snapshot(orca_path="/b/orca")) + self.assertNotEqual( + first.host["orca_path_digest"], second.host["orca_path_digest"] + ) + + +class ConfiguredOrcaCommandTest(unittest.TestCase): + """Which Orca the environment must use is a decision, not a PATH accident. + + A Distrobox container inherits the host's PATH and sees the host's home, so + `command -v orca` finds the host's launcher even when the container has its + own installation. The command to look for is therefore named. + """ + + def test_the_probe_takes_the_command_to_look_for(self): + argv = probe.probe_argv("/home/cycle/project", orca_command="/opt/Orca/orca-ide") + self.assertIn("/opt/Orca/orca-ide", argv) + self.assertIn("/home/cycle/project", argv) + + def test_the_default_is_the_plain_name(self): + self.assertIn("orca", probe.probe_argv("/p")) + + def test_the_script_resolves_the_named_command(self): + self.assertIn('command -v "$orca_command"', probe.PROBE_SCRIPT) + + def test_a_named_command_is_what_the_probe_reports(self): + from cycle_runner import proc + + outcome = proc.run( + probe.probe_argv("/", orca_command="/bin/sh"), timeout=30, context="host" + ) + snapshot = probe.parse(outcome.stdout) + self.assertEqual(snapshot["orca_path"], "/bin/sh") + self.assertNotEqual(snapshot["orca_fingerprint"], "") From 0a5cd87293beb0b1275da2ebc7ca46d7b8f452a9 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 16:26:12 +0200 Subject: [PATCH 21/42] Make the project copy a repository Orca can register, and keep repository metadata out of the patch. --- experiments/minimal-cycle/counterexamples.py | 18 +++++++ .../minimal-cycle/cycle_runner/lifecycle.py | 13 +++++ .../minimal-cycle/cycle_runner/project.py | 36 +++++++++++-- experiments/minimal-cycle/tests/fakes.py | 21 +++++++- .../tests/test_doubles_are_contained.py | 18 ++++++- .../minimal-cycle/tests/test_lifecycle.py | 27 ++++++++++ .../minimal-cycle/tests/test_project.py | 52 +++++++++++++++++++ 7 files changed, 178 insertions(+), 7 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index eda8c9f..fabfdbb 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,24 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="the-copy-is-a-repository", + requirement="The project copy is made into a repository Orca can register", + path="cycle_runner/lifecycle.py", + original=""" for argv in project.initialise_repository_commands(self.handle.project_path):""", + replacement=""" for argv in []:""", + instruments=("tests.test_lifecycle.ProjectRepositoryTest",), + ), + Counterexample( + name="metadata-is-not-the-change", + requirement="Repository metadata stays out of the patch", + path="cycle_runner/project.py", + original=""" if EXCLUDED_DIRECTORIES.intersection(relative.parts): + continue""", + replacement=""" if False: + continue""", + instruments=("tests.test_project.DiffExclusionTest",), + ), Counterexample( name="the-same-path-is-not-the-same-file", requirement="An environment with its own Orca at the host's path is not the host's", diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index 0cf751f..b870489 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -277,6 +277,19 @@ def _bootstrap(self, baseline: Path) -> None: assert self.handle is not None with self.phase("bootstrap") as record: self.adapter.put_tree(baseline, self.handle.project_path) + # Orca registers a worktree for a repository; the exported copy is + # not one until this makes it one. + for argv in project.initialise_repository_commands(self.handle.project_path): + outcome = self.execute(argv, timeout=min(300, self.config.timeout_seconds)) + record.commands.append(outcome.to_record()) + if not outcome.ok: + record.status = PhaseStatus.FAILED + record.detail = ( + "the project copy could not be made into a repository: " + + redact.text((outcome.stderr or outcome.stdout).strip()[-300:]) + ) + self.error_reason = self.error_reason or record.detail + return for argv in self._provision_steps(): record.commands.append(self.execute(list(argv)).to_record()) auth_record, overlay = auth.bootstrap( diff --git a/experiments/minimal-cycle/cycle_runner/project.py b/experiments/minimal-cycle/cycle_runner/project.py index c895526..299872d 100644 --- a/experiments/minimal-cycle/cycle_runner/project.py +++ b/experiments/minimal-cycle/cycle_runner/project.py @@ -18,6 +18,10 @@ _TEXT_SAMPLE = 8192 +#: Repository metadata is not what a task changed, and a fresh repository would +#: otherwise dominate the patch with hundreds of files nobody wrote. +EXCLUDED_DIRECTORIES = frozenset({".git"}) + class ProjectError(RuntimeError): """The project copy could not be produced from the named revision.""" @@ -78,11 +82,15 @@ def _relative_files(root: Path) -> dict[str, Path]: root = Path(root) if not root.is_dir(): return {} - return { - path.relative_to(root).as_posix(): path - for path in sorted(root.rglob("*")) - if path.is_file() and not path.is_symlink() - } + found = {} + for path in sorted(root.rglob("*")): + if not path.is_file() or path.is_symlink(): + continue + relative = path.relative_to(root) + if EXCLUDED_DIRECTORIES.intersection(relative.parts): + continue + found[relative.as_posix()] = path + return found def _is_text(raw: bytes) -> bool: @@ -99,6 +107,24 @@ def _lines(raw: bytes) -> list[str]: return raw.decode("utf-8").splitlines(keepends=True) +def initialise_repository_commands(project_path: str) -> list[list[str]]: + """Return the commands that make the copy a repository Orca can register. + + The copy is exported with `git archive`, so it carries the tracked tree at + one revision and no repository state at all. Orca registers a worktree for + a repository, so the copy becomes one: a single commit, identified as this + runner rather than as a person. + """ + base = ["git", "-C", project_path] + return [ + base + ["init", "-q", "-b", "main"], + base + ["config", "user.email", "dely-cycle@invalid"], + base + ["config", "user.name", "dely-cycle"], + base + ["add", "-A"], + base + ["commit", "-q", "-m", "dely-cycle: the pinned revision as exported"], + ] + + def unified_diff(baseline: Path, current: Path) -> str: """Return one redacted patch describing what the run changed.""" before = _relative_files(baseline) diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index e6da8d5..b45d8e7 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -41,7 +41,9 @@ #: The only programs a fake environment may really run on this machine, and #: the only `sh -c` scripts among them. `orca-start` is deliberately absent. -RUNNABLE_PROGRAMS = frozenset({"sh", "rm", "mkdir", "cat", "test", "true", "false", "printf"}) +RUNNABLE_PROGRAMS = frozenset( + {"sh", "rm", "mkdir", "cat", "test", "true", "false", "printf", "git"} +) RUNNABLE_SHELL_SCRIPTS = frozenset({"cycle-check", "auth-teardown", "cycle-cd"}) ORCA_STATUS_REPLY = ( @@ -94,6 +96,7 @@ def __init__( runtime_ready: bool = True, terminal_refused: bool = False, already_running: bool = False, + refuse_repository: bool = False, create_fails: bool = False, task_writes_nothing: bool = False, host_project: Path | None = None, @@ -118,6 +121,7 @@ def __init__( # The application is not running until something starts it, which is # what the real environment does too. self.app_started = already_running + self.refuse_repository = refuse_repository self.create_fails = create_fails self.task_writes_nothing = task_writes_nothing self.host_project = host_project @@ -218,6 +222,9 @@ def execute( self.project.mkdir(parents=True, exist_ok=True) (self.project / "evidence.txt").write_text(self.marker, encoding="utf-8") return self._outcome(argv, 0, ORCA_DISPATCH_REPLY, "") + if argv[0] == "git" and self.refuse_repository: + self.calls.append("git-refused") + return self._outcome(argv, 1, "", "the fake refuses to initialise a repository") if len(argv) > 3 and argv[0] == "sh" and argv[3] == "orca-start": # Simulated, never executed: running this would start a desktop # application on the machine running the tests. @@ -272,6 +279,18 @@ def _refuse(self, argv) -> proc.CommandOutcome | None: return self._outcome( argv, 127, "", f"the fake environment does not run the script {name!r}" ) + if program == "git": + # Only inside the fake's own root, and only in that form: `git` + # reaches a great deal further than this fake should. + inside = ( + len(argv) > 2 + and argv[1] == "-C" + and Path(argv[2]).is_relative_to(self.root) + ) + if not inside: + return self._outcome( + argv, 127, "", "the fake environment runs git only inside its own root" + ) if program == "rm": targets = [item for item in argv[1:] if not item.startswith("-")] outside = [ diff --git a/experiments/minimal-cycle/tests/test_doubles_are_contained.py b/experiments/minimal-cycle/tests/test_doubles_are_contained.py index 35362b7..898c293 100644 --- a/experiments/minimal-cycle/tests/test_doubles_are_contained.py +++ b/experiments/minimal-cycle/tests/test_doubles_are_contained.py @@ -75,11 +75,27 @@ def test_the_runner_own_scripts_still_run(self): self.assertEqual(outcome.exit_code, 0) def test_the_allowlists_stay_small_and_name_nothing_graphical(self): - self.assertLessEqual(len(RUNNABLE_PROGRAMS), 8) + # A ceiling, so the list cannot grow quietly. Raising it is a decision + # a reader sees; `git` was added for the repository the copy has to be. + self.assertLessEqual(len(RUNNABLE_PROGRAMS), 9) self.assertNotIn("orca", RUNNABLE_PROGRAMS) self.assertNotIn("nohup", RUNNABLE_PROGRAMS) self.assertNotIn("orca-start", RUNNABLE_SHELL_SCRIPTS) + def test_git_is_only_allowed_inside_the_fake_root(self): + outside = self.adapter._refuse(["git", "-C", "/etc", "init"]) + self.assertIsNotNone(outside) + self.assertEqual(outside.exit_code, 127) + inside = self.adapter._refuse( + ["git", "-C", str(self.adapter.project), "init"] + ) + self.assertIsNone(inside) + + def test_git_in_any_other_form_is_refused(self): + refusal = self.adapter._refuse(["git", "clone", "https://example.invalid/x"]) + self.assertIsNotNone(refusal) + self.assertEqual(refusal.exit_code, 127) + class StubRunnerContainmentTest(unittest.TestCase): def test_passthrough_is_only_for_key_generation(self): diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index f5b30a4..a5f5bc6 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -366,3 +366,30 @@ def test_the_start_is_recorded_as_a_command_of_the_identity_phase(self): any("orca-start" in " ".join(c.argv) for c in identity.commands), [" ".join(c.argv)[:60] for c in identity.commands], ) + + +class ProjectRepositoryTest(CycleTestCase): + """Orca registers a worktree for a repository, so the copy becomes one.""" + + def test_the_copy_is_initialised_as_a_repository_and_it_succeeds(self): + _, outcome = self.run_cycle() + bootstrap = outcome.run_result.phase("bootstrap") + commands = [c for c in bootstrap.commands if c.argv and c.argv[0] == "git"] + self.assertGreaterEqual(len(commands), 5, [c.argv for c in bootstrap.commands]) + for command in commands: + self.assertEqual(command.exit_code, 0, " ".join(command.argv)) + joined = " ".join(" ".join(c.argv) for c in commands) + self.assertIn("init", joined) + self.assertIn("commit", joined) + + def test_a_failed_initialisation_stops_the_run_rather_than_continuing(self): + adapter, outcome = self.run_cycle(refuse_repository=True) + self.assertEqual(outcome.run_result.status, status.RunStatus.ERROR) + self.assertIn("repository", outcome.run_result.failure_classification.lower()) + self.assertNotIn("worker", adapter.calls) + + def test_repository_metadata_does_not_appear_in_the_patch(self): + _, outcome = self.run_cycle() + patch = self.artifact("diff.patch").read_text(encoding="utf-8") + self.assertNotIn(".git/", patch) + self.assertIn("evidence.txt", patch) diff --git a/experiments/minimal-cycle/tests/test_project.py b/experiments/minimal-cycle/tests/test_project.py index 96eb2b2..7ebe30c 100644 --- a/experiments/minimal-cycle/tests/test_project.py +++ b/experiments/minimal-cycle/tests/test_project.py @@ -131,3 +131,55 @@ def test_the_patch_is_redacted(self): def test_a_missing_current_tree_reports_every_file_as_removed(self): patch = project.unified_diff(self.baseline, self.root / "absent") self.assertIn("--- a/readme.md", patch) + + +class DiffExclusionTest(unittest.TestCase): + """Repository metadata is not what the task changed.""" + + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.baseline = self.root / "baseline" + self.current = self.root / "current" + for tree in (self.baseline, self.current): + (tree / "src").mkdir(parents=True) + (tree / "src" / "app.py").write_text("value = 1\n", encoding="utf-8") + self.addCleanup(self._tmp.cleanup) + + def test_repository_metadata_is_left_out_of_the_patch(self): + objects = self.current / ".git" / "objects" + objects.mkdir(parents=True) + (objects / "deadbeef").write_bytes(b"\x01\x02\x03") + (self.current / ".git" / "HEAD").write_text("ref: refs/heads/main\n", encoding="utf-8") + patch = project.unified_diff(self.baseline, self.current) + self.assertEqual(patch, "") + + def test_what_the_task_changed_is_still_shown(self): + (self.current / ".git").mkdir() + (self.current / ".git" / "HEAD").write_text("ref: refs/heads/main\n", encoding="utf-8") + (self.current / "evidence.txt").write_text("dely-cycle-marker\n", encoding="utf-8") + patch = project.unified_diff(self.baseline, self.current) + self.assertIn("+++ b/evidence.txt", patch) + self.assertNotIn(".git", patch) + + +class InitialiseRepositoryTest(unittest.TestCase): + """Orca registers a worktree for a repository, so the copy has to be one.""" + + def test_the_commands_initialise_and_commit_the_copy(self): + argv_list = project.initialise_repository_commands("/home/cycle/project") + joined = [" ".join(argv) for argv in argv_list] + self.assertTrue(any("init" in line for line in joined)) + self.assertTrue(any("add" in line for line in joined)) + self.assertTrue(any("commit" in line for line in joined)) + for argv in argv_list: + self.assertIn("/home/cycle/project", argv) + + def test_an_identity_is_set_so_the_commit_does_not_need_one_configured(self): + joined = " ".join(" ".join(argv) for argv in project.initialise_repository_commands("/p")) + self.assertIn("user.email", joined) + self.assertIn("user.name", joined) + + def test_the_commit_is_not_attributed_to_a_person(self): + joined = " ".join(" ".join(argv) for argv in project.initialise_repository_commands("/p")) + self.assertIn("dely-cycle", joined) From e4860de290168b7517d97181b37cfbafa4d19f89 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 16:29:33 +0200 Subject: [PATCH 22/42] Read the Run and Dispatch identifiers from where Orca puts them, and replace the fixtures that had invented a flatter reply. --- experiments/minimal-cycle/counterexamples.py | 14 +++ .../minimal-cycle/cycle_runner/worker.py | 67 +++++++++---- experiments/minimal-cycle/tests/fakes.py | 6 +- .../minimal-cycle/tests/test_worker.py | 94 ++++++++++++++++++- 4 files changed, 160 insertions(+), 21 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index fabfdbb..a2fd77a 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,20 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="the-request-is-not-the-run", + requirement="The Run identifier is read from the Run, not from the request", + path="cycle_runner/worker.py", + original=""" result = _result(document) + run = result.get("run") + if isinstance(run, Mapping) and isinstance(run.get("id"), str): + return run["id"] + value = result.get("runId") + return value if isinstance(value, str) and value else None""", + replacement=""" value = document.get("id") + return value if isinstance(value, str) and value else None""", + instruments=("tests.test_worker.RealResponseShapeTest",), + ), Counterexample( name="the-copy-is-a-repository", requirement="The project copy is made into a repository Orca can register", diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index 1365a39..c59c173 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -136,17 +136,42 @@ def _first_document(text: str) -> dict[str, Any]: return {} -def _identifier(document: Mapping[str, Any], *names: str) -> str | None: - for name in names: - value = document.get(name) - if isinstance(value, str) and value: - return value - for value in document.values(): - if isinstance(value, Mapping): - found = _identifier(value, *names) - if found: - return found - return None +def _result(document: Mapping[str, Any]) -> Mapping[str, Any]: + result = document.get("result") + return result if isinstance(result, Mapping) else {} + + +def run_identifier(document: Mapping[str, Any]) -> str | None: + """Return the Run identifier from a run-create or worker-start reply. + + The reply's top-level `id` is the identifier of the *request*, not of the + Run. Passing it as `--run` makes the next command fail with + `consumer_fenced`, because no Run by that name is bound to the terminal. + """ + result = _result(document) + run = result.get("run") + if isinstance(run, Mapping) and isinstance(run.get("id"), str): + return run["id"] + value = result.get("runId") + return value if isinstance(value, str) and value else None + + +def dispatch_identifier(document: Mapping[str, Any]) -> str | None: + """Return the Dispatch identifier from a worker-start reply.""" + value = _result(document).get("dispatchId") + return value if isinstance(value, str) and value else None + + +def dispatch_state(document: Mapping[str, Any]) -> str | None: + """Return what the plane said the dispatch reached, if it said anything.""" + value = _result(document).get("state") + return value if isinstance(value, str) and value else None + + +def task_identifier(document: Mapping[str, Any]) -> str | None: + """Return the Task identifier from a worker-start reply.""" + value = _result(document).get("taskId") + return value if isinstance(value, str) and value else None def _settling_message(document: Mapping[str, Any]) -> dict[str, Any]: @@ -198,10 +223,10 @@ def launch( record.status = PhaseStatus.FAILED record.detail = ( "orca orchestration run-create did not return a Run: " - + redact.text((created.stderr or created.stdout).strip()[:400], secrets) + + redact.text((created.stderr or created.stdout).strip()[-400:], secrets) ) return record - record.run_id = _identifier(_first_document(created.stdout), "runId", "run_id", "id") + record.run_id = run_identifier(_first_document(created.stdout)) plan = build_plan( run_config=run_config, @@ -225,14 +250,20 @@ def launch( return record if not started.ok: record.status = PhaseStatus.FAILED + state = dispatch_state(_first_document(started.stdout)) + record.dispatch_id = dispatch_identifier(_first_document(started.stdout)) + record.outcome = state record.detail = ( - "orca orchestration worker-start did not reach ready: " - + redact.text((started.stderr or started.stdout).strip()[:400], secrets) + f"orca orchestration worker-start reported {state or 'no state'}: " + + redact.text((started.stderr or started.stdout).strip()[-400:], secrets) ) return record - record.dispatch_id = _identifier( - _first_document(started.stdout), "dispatchId", "dispatch_id", "id" - ) + start_document = _first_document(started.stdout) + record.dispatch_id = dispatch_identifier(start_document) + record.run_id = record.run_id or run_identifier(start_document) + state = dispatch_state(start_document) + if state: + record.detail = f"the plane reported the dispatch as {state}" settled = adapter.execute( plan.wait_argv, timeout=timeout_seconds, env=overlay, extra_values=secrets diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index b45d8e7..e868f7c 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -64,8 +64,12 @@ '"worktreeId": "repo::project", "surface": "visible"}}}' ) +# The real shape: identifiers nested under `result`, the top-level `id` being +# the request's. One reply serves all three orchestration commands here. ORCA_DISPATCH_REPLY = ( - '{"runId": "run-fake", "dispatchId": "dispatch-fake", ' + '{"id": "request-fake", "ok": true, "result": {"run": {"id": "run-fake"}, ' + '"runId": "run-fake", "taskId": "task-fake", "dispatchId": "dispatch-fake", ' + '"state": "ready"}, ' '"messages": [{"type": "worker_done", "outcome": "DONE"}]}' ) diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py index 8191ad1..d26519d 100644 --- a/experiments/minimal-cycle/tests/test_worker.py +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -9,8 +9,27 @@ from tests.fakes import ScriptedAdapter from tests.test_config import minimal_document -READY = json.dumps({"runId": "run-abcdef"}) -STARTED = json.dumps({"dispatchId": "dispatch-abcdef", "status": "ready"}) +# These are the shapes a live runtime returns. The identifiers are nested under +# `result`; the top-level `id` belongs to the request, not to the Run. +READY = json.dumps( + { + "id": "request-abcdef", + "ok": True, + "result": {"run": {"id": "run-abcdef", "objective": "dely minimal cycle"}}, + } +) +STARTED = json.dumps( + { + "id": "request-123456", + "ok": True, + "result": { + "runId": "run-abcdef", + "taskId": "task-abcdef", + "dispatchId": "dispatch-abcdef", + "state": "ready", + }, + } +) SETTLED = json.dumps({"messages": [{"type": "worker_done", "outcome": "DONE"}]}) DEFAULT_SCRIPT = [ @@ -187,3 +206,74 @@ def test_without_a_terminal_the_flag_is_not_invented(self): adapter, _ = self.launch_from(None) for argv in adapter.executed: self.assertNotIn("--from", argv) + + +#: The shapes Orca actually returns, copied from a live guest. The earlier +#: fixtures put the identifiers at the top level; the real ones are nested, and +#: the top-level `id` is the request identifier, not the Run. +REAL_RUN_CREATE = json.dumps( + { + "id": "f1a0c3be-5f9d-4e21-81d1-dffcb6c647a8", + "ok": True, + "result": { + "run": { + "id": "run_a52fcdd366b4", + "objective": "dely minimal cycle", + "coordinator_handle": "term_50c4a5b6", + } + }, + } +) + +REAL_WORKER_START = json.dumps( + { + "id": "7056aa38-852c-4d6d-8287-443521841556", + "ok": True, + "result": { + "runId": "run_a52fcdd366b4", + "taskId": "task_0e636c920592", + "dispatchId": "ctx_098ec148783c", + "state": "outcome_unknown", + "stage": "turn_start_unobserved", + "launch": { + "requested": {"agent": "claude", "model": "m", "effort": "high"}, + "effective": {"agent": "claude", "model": "m", "effort": "high"}, + }, + }, + } +) + + +class RealResponseShapeTest(WorkerTest): + """Observed against a live runtime: the identifiers are nested.""" + + def test_the_run_identifier_comes_from_the_nested_run(self): + self.assertEqual(worker.run_identifier(json.loads(REAL_RUN_CREATE)), "run_a52fcdd366b4") + + def test_the_request_identifier_is_never_mistaken_for_the_run(self): + self.assertNotEqual( + worker.run_identifier(json.loads(REAL_RUN_CREATE)), + "f1a0c3be-5f9d-4e21-81d1-dffcb6c647a8", + ) + + def test_the_dispatch_identifier_comes_from_the_result(self): + self.assertEqual( + worker.dispatch_identifier(json.loads(REAL_WORKER_START)), "ctx_098ec148783c" + ) + + def test_the_task_and_state_are_carried_too(self): + parsed = json.loads(REAL_WORKER_START) + self.assertEqual(worker.dispatch_state(parsed), "outcome_unknown") + + def test_a_launch_against_the_real_shapes_uses_the_run_identifier(self): + script = [ + ("run-create", 0, REAL_RUN_CREATE, "", False), + ("worker-start", 0, REAL_WORKER_START, "", False), + ("check --wait", 0, SETTLED, "", False), + ] + adapter, _, record = self.launch(script=script) + start = next(argv for argv in adapter.executed if "worker-start" in argv) + self.assertIn("--run", start) + self.assertEqual(start[start.index("--run") + 1], "run_a52fcdd366b4") + self.assertEqual(record.run_id, "run_a52fcdd366b4") + self.assertEqual(record.dispatch_id, "ctx_098ec148783c") From 9d0b943ef6da42febd8d9c9d19ca698adf4b5681 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 16:32:35 +0200 Subject: [PATCH 23/42] Settle an unverifiable dispatch as unknown rather than failed, and record the machine cycle that reached one. --- experiments/minimal-cycle/counterexamples.py | 8 + .../minimal-cycle/cycle_runner/lifecycle.py | 29 +- .../minimal-cycle/cycle_runner/worker.py | 5 + .../auth-receipt.json | 11 + .../vm-dispatch-unverifiable/cleanup.json | 15 + .../export-receipt.json | 71 ++ .../identity-environment-probe.txt | 14 + .../identity-host-probe.txt | 14 + .../identity-orca-runtime.json | 90 ++ .../vm-dispatch-unverifiable/manifest.json | 932 ++++++++++++++++++ .../vm-dispatch-unverifiable/preflight.json | 81 ++ experiments/minimal-cycle/tests/fakes.py | 7 + .../minimal-cycle/tests/test_lifecycle.py | 30 + 13 files changed, 1304 insertions(+), 3 deletions(-) create mode 100644 experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/auth-receipt.json create mode 100644 experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/cleanup.json create mode 100644 experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/export-receipt.json create mode 100644 experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-environment-probe.txt create mode 100644 experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-host-probe.txt create mode 100644 experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-orca-runtime.json create mode 100644 experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/manifest.json create mode 100644 experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/preflight.json diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index a2fd77a..176e7ad 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,14 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="unverifiable-is-not-failed", + requirement="A dispatch the plane cannot judge settles unknown, not error", + path="cycle_runner/lifecycle.py", + original=""" if worker_record.outcome in worker.UNVERIFIABLE_STATES:""", + replacement=""" if False:""", + instruments=("tests.test_lifecycle.UnverifiableDispatchTest",), + ), Counterexample( name="the-request-is-not-the-run", requirement="The Run identifier is read from the Run, not from the request", diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index b870489..af84395 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -110,6 +110,7 @@ def _classify( blocked_reason: str, timed_out: bool, error_reason: str, + unverifiable_reason: str, cleanup_record: CleanupRecord, ) -> tuple[RunStatus, str]: if export_status is not ExportStatus.CONFIRMED: @@ -124,6 +125,8 @@ def _classify( return RunStatus.TIMEOUT, "the task reached the configured deadline" if error_reason: return RunStatus.ERROR, error_reason + if unverifiable_reason: + return RunStatus.UNKNOWN, unverifiable_reason if cleanup_record.status is not CleanupStatus.DESTROYED: return RunStatus.CLEANUP_FAILED, cleanup_record.reason return RunStatus.SETTLED, "" @@ -161,6 +164,7 @@ def __init__( self.create_attempted = False self.blocked_reason = "" self.error_reason = "" + self.unverifiable_reason = "" self.timed_out = False self.task_ran = False self.stop_confirmed = True @@ -234,10 +238,21 @@ def run(self, staging: Path) -> CycleOutcome: self._task() else: self.skip("task", self.blocked_reason or self.error_reason or "no environment") - if self.task_ran and not self.timed_out and not self.error_reason: + if ( + self.task_ran + and not self.timed_out + and not self.error_reason + and not self.unverifiable_reason + ): self._check() else: - self.skip("check", self.blocked_reason or self.error_reason or "the task did not settle") + self.skip( + "check", + self.blocked_reason + or self.error_reason + or self.unverifiable_reason + or "the task did not settle", + ) self._collect(baseline, fetched) export_record = self._export() @@ -419,7 +434,14 @@ def _task(self) -> None: if worker_record.status is PhaseStatus.TIMEOUT: self.timed_out = True elif worker_record.status is not PhaseStatus.OK: - self.error_reason = self.error_reason or worker_record.detail + if worker_record.outcome in worker.UNVERIFIABLE_STATES: + self.unverifiable_reason = self.unverifiable_reason or ( + f"the execution plane reported the dispatch as " + f"{worker_record.outcome}, which is unverifiable rather than " + "failed: nothing here knows whether the worker ran" + ) + else: + self.error_reason = self.error_reason or worker_record.detail def _check(self) -> None: assert self.handle is not None @@ -560,6 +582,7 @@ def _close(self, export_record, cleanup_record, started: float) -> CycleOutcome: blocked_reason=self.blocked_reason, timed_out=self.timed_out, error_reason=self.error_reason, + unverifiable_reason=self.unverifiable_reason, cleanup_record=cleanup_record, ) self.result.status = run_status diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index c59c173..a9b1515 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -20,6 +20,11 @@ from .status import PhaseStatus SETTLING_TYPES = ("worker_done", "escalation", "question") + +#: States the execution plane uses to say it could not tell what happened. They +#: are not failures: the worker may still be starting, wedged, or holding the +#: task unsent. Treating them as failures claims knowledge nobody has. +UNVERIFIABLE_STATES = ("outcome_unknown",) PROMPT_NAME = "dispatch-prompt.md" DEFAULT_PROMPT_PATH = ( Path(__file__).resolve().parent.parent / "fixtures" / "evidence-task" / "prompt.md" diff --git a/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/auth-receipt.json b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/auth-receipt.json new file mode 100644 index 0000000..63c115d --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/auth-receipt.json @@ -0,0 +1,11 @@ +{ + "detail": "the helper command is declared in the per-run settings; no credential value passes through the runner", + "entries": [ + ".claude/settings.json" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only", + "removed_after_run": false, + "status": "OK", + "target": "/home/cycle/.claude/settings.json" +} diff --git a/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/cleanup.json b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/cleanup.json new file mode 100644 index 0000000..bc26141 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/cleanup.json @@ -0,0 +1,15 @@ +{ + "host_difference": {}, + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "domain:dely-cycle-20260915t220000z-bbbbbb-00000011", + "path:/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011" + ], + "retained": [], + "shared_preserved": [ + "image:/var/tmp/dely-cycle/images/tool-image.qcow2" + ], + "status": "DESTROYED", + "stop_confirmed": true, + "verified": true +} diff --git a/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/export-receipt.json b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/export-receipt.json new file mode 100644 index 0000000..30d8d89 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/export-receipt.json @@ -0,0 +1,71 @@ +{ + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "d07d42f1b658cd692c32580ba917e1fab793697285148de9714b9ba3cda98e4f", + "size_bytes": 341 + }, + { + "path": "backend-status.json", + "sha256": "f0159e0508b4d3113fe9569863d74f992e008f7842380b1701de833891b08ff3", + "size_bytes": 2016 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "4560b0f1523237e4323cc9cd954c538ca1d73598a8e67673ea620568851efda6", + "size_bytes": 1229 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "194adeadc9b463ed1bc7406c5f5630bef2c0be388c564279ed13f14c92d3b5cf", + "size_bytes": 471 + }, + { + "path": "identity/host-probe.txt", + "sha256": "5cd0f604b06d8efcadf7baf6935ea86fc377562ab072f6eecb25eb945bcc247a", + "size_bytes": 1709 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "7fb46746bc9419e7abdb672164ec7d390a9b9a90c6fab494acdf2007c3b6850e", + "size_bytes": 3178 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "0b973d9b69e84c025c4c84fb8a5a335428df85175d6bc63ee4a3c86834dacfec", + "size_bytes": 9374 + }, + { + "path": "logs/runner.log", + "sha256": "635bb0c7458f68f0ea47ece8d373980dbd6335c36f9310a0dba205b469bdf5a9", + "size_bytes": 1968 + }, + { + "path": "preflight.json", + "sha256": "a0a2864b989fabdb80b8dc5efcd4e301eff362033b3297114300ea6c75c24340", + "size_bytes": 1805 + }, + { + "path": "run-before-cleanup.json", + "sha256": "09e012ed25136b72d04b947b48ad3541a29f0dda806b2abfbd37adfb83f13d38", + "size_bytes": 26025 + } + ], + "confirmed_at": "2026-09-15T14:30:50Z", + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" +} diff --git a/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-environment-probe.txt b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-environment-probe.txt new file mode 100644 index 0000000..32c6631 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-environment-probe.txt @@ -0,0 +1,14 @@ +hostname=dely-cycle-20260915t220000z-bbbbbb-00000011 +machine_id= +boot_id= +home=/home/cycle +user=cycle +path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin +pid=1662 +uname=Linux 6.8.0-139-generic x86_64 +container_marker= +virt=kvm +orca_path=/usr/local/bin/orca +orca_version=1.4.201 +orca_fingerprint=533628:1592:1789327972 +project_real=/home/cycle/project diff --git a/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-host-probe.txt b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-host-probe.txt new file mode 100644 index 0000000..a495ecc --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-host-probe.txt @@ -0,0 +1,14 @@ +hostname=system +machine_id= +boot_id= +home= +user= +path=/.pulumi/bin:/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin +pid=267253 +uname=Linux 7.2.5-200.fc44.x86_64 x86_64 +container_marker= +virt=none +orca_path=/.config/orca/linux-orca-cli-shim/orca +orca_version=1.4.201 +orca_fingerprint=99186908:65:1789370051 +project_real=/var/tmp/dely-cycle/project-source diff --git a/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-orca-runtime.json b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-orca-runtime.json new file mode 100644 index 0000000..b34fa55 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/identity-orca-runtime.json @@ -0,0 +1,90 @@ +{ + "app_pid": 1877, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "2f3ef3d7-823c-4a45-8113-aee62f1872df", + "runtime_state": "ready" +} diff --git a/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/manifest.json b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/manifest.json new file mode 100644 index 0000000..01f3cd7 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/manifest.json @@ -0,0 +1,932 @@ +{ + "artifact_root": "/var/tmp/dely-cycle/artifacts/20260915T220000Z-bbbbbb-00000011", + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "d07d42f1b658cd692c32580ba917e1fab793697285148de9714b9ba3cda98e4f", + "size_bytes": 341 + }, + { + "path": "backend-status.json", + "sha256": "f0159e0508b4d3113fe9569863d74f992e008f7842380b1701de833891b08ff3", + "size_bytes": 2016 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "4560b0f1523237e4323cc9cd954c538ca1d73598a8e67673ea620568851efda6", + "size_bytes": 1229 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "194adeadc9b463ed1bc7406c5f5630bef2c0be388c564279ed13f14c92d3b5cf", + "size_bytes": 471 + }, + { + "path": "identity/host-probe.txt", + "sha256": "5cd0f604b06d8efcadf7baf6935ea86fc377562ab072f6eecb25eb945bcc247a", + "size_bytes": 1709 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "7fb46746bc9419e7abdb672164ec7d390a9b9a90c6fab494acdf2007c3b6850e", + "size_bytes": 3178 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "0b973d9b69e84c025c4c84fb8a5a335428df85175d6bc63ee4a3c86834dacfec", + "size_bytes": 9374 + }, + { + "path": "logs/runner.log", + "sha256": "635bb0c7458f68f0ea47ece8d373980dbd6335c36f9310a0dba205b469bdf5a9", + "size_bytes": 1968 + }, + { + "path": "preflight.json", + "sha256": "a0a2864b989fabdb80b8dc5efcd4e301eff362033b3297114300ea6c75c24340", + "size_bytes": 1805 + }, + { + "path": "run-before-cleanup.json", + "sha256": "09e012ed25136b72d04b947b48ad3541a29f0dda806b2abfbd37adfb83f13d38", + "size_bytes": 26025 + } + ], + "auth": { + "detail": "the helper command is declared in the per-run settings; no credential value passes through the runner", + "entries": [ + ".claude/settings.json" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only", + "removed_after_run": false, + "status": "OK", + "target": "/home/cycle/.claude/settings.json" + }, + "backend": "vm", + "check": { + "argv": [], + "detail": "not attempted", + "elapsed_seconds": null, + "exit_code": null, + "expected_marker": null, + "finished_at": null, + "started_at": null, + "status": "SKIPPED", + "stderr_path": null, + "stdout_path": null + }, + "cleanup": { + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "domain:dely-cycle-20260915t220000z-bbbbbb-00000011", + "path:/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011" + ], + "retained": [], + "shared_preserved": [ + "image:/var/tmp/dely-cycle/images/tool-image.qcow2" + ], + "status": "DESTROYED", + "verified": true + }, + "config": { + "artifact_root": "/var/tmp/dely-cycle/artifacts", + "auth": { + "helper_argv": [ + "/usr/local/bin/dely-cycle-key-helper" + ], + "mode": "api_key_helper", + "reference": "workshop-helper-declared-only" + }, + "backend": "vm", + "check": { + "expected_marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "claude_code_version": "2.1.272", + "dely_revision": "e4860de290168b7517d97181b37cfbafa4d19f89", + "orca": { + "agent": "claude", + "app_argv": [ + "/opt/Orca/orca-ide" + ], + "command": "orca", + "display": ":0", + "effort": "high", + "model": "claude-opus-5", + "ready_timeout_seconds": 420, + "run_objective": "dely minimal cycle", + "status_argv": [ + "orca", + "status", + "--json" + ], + "version": "1.4.201", + "version_argv": [ + "orca", + "--version" + ], + "worktree_selector": "current" + }, + "project": { + "environment_path": "project", + "revision": "e4860de290168b7517d97181b37cfbafa4d19f89", + "source": "/var/tmp/dely-cycle/project-source" + }, + "state_root": "/var/tmp/dely-cycle/state", + "task": { + "marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "tested_revision": "e4860de290168b7517d97181b37cfbafa4d19f89", + "timeout_seconds": 1800, + "vm": { + "address_timeout_seconds": 600, + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_image_sha256": "1f11ab859bf049b3754909f32e09ea3dc1b71b4762ec24406dea661f9bf0123c", + "connect_uri": "qemu:///system", + "egress": true, + "graphics": "vnc", + "guest_user": "cycle", + "listen_address": "127.0.0.1", + "memory_mb": 4096, + "network": "default", + "overlay_size_bytes": 25769803776, + "pool": "dely-cycle", + "provider": "pulumi-libvirt", + "provider_version": "0.5.4", + "provision": [], + "pulumi_binary": "/.pulumi/bin/pulumi", + "qemu_agent": false, + "ssh_port": 22, + "stack_prefix": "dely-cycle", + "transport": "ssh", + "vcpus": 2, + "venv": "/var/tmp/dely-cycle/venv", + "video": "virtio" + } + }, + "dely_revision": "e4860de290168b7517d97181b37cfbafa4d19f89", + "elapsed_seconds": 77.875, + "environment": { + "backend": { + "address": "192.168.122.171", + "backend": "vm", + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_image_sha256": "1f11ab859bf049b3754909f32e09ea3dc1b71b4762ec24406dea661f9bf0123c", + "base_volume": "tool-image.qcow2", + "connect_uri": "qemu:///system", + "domain": "dely-cycle-20260915t220000z-bbbbbb-00000011", + "egress_interface": true, + "graphics": "vnc", + "listen_address": "127.0.0.1", + "network": "default", + "pool": "dely-cycle", + "provider": "pulumi-libvirt@0.5.4", + "stack": "dely-cycle-20260915t220000z-bbbbbb-00000011", + "transport_mac": "52:54:00:38:e6:98", + "video": "virtio" + }, + "description": { + "address": "192.168.122.171", + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_volume": "tool-image.qcow2", + "connect_uri": "qemu:///system", + "domain": "dely-cycle-20260915t220000z-bbbbbb-00000011", + "egress": true, + "graphics": "vnc", + "network": "default", + "pool": "dely-cycle", + "provider": "pulumi-libvirt@0.5.4", + "stack": "/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/stack", + "transport_mac": "52:54:00:38:e6:98", + "video": "virtio" + }, + "environment_id": "dely-cycle-20260915t220000z-bbbbbb-00000011", + "home_path": "/home/cycle", + "per_run_resources": [ + { + "identifier": "dely-cycle-20260915t220000z-bbbbbb-00000011", + "kind": "domain" + }, + { + "identifier": "/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011", + "kind": "path" + }, + { + "identifier": "/var/tmp/dely-cycle/images/dely-cycle-20260915t220000z-bbbbbb-00000011-overlay.qcow2", + "kind": "volume" + }, + { + "identifier": "/var/tmp/dely-cycle/images/dely-cycle-20260915t220000z-bbbbbb-00000011-seed.iso", + "kind": "volume" + } + ], + "project_path": "/home/cycle/project", + "shared_resources": [ + { + "identifier": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "kind": "image" + } + ] + }, + "environment_id": "dely-cycle-20260915t220000z-bbbbbb-00000011", + "export": { + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "d07d42f1b658cd692c32580ba917e1fab793697285148de9714b9ba3cda98e4f", + "size_bytes": 341 + }, + { + "path": "backend-status.json", + "sha256": "f0159e0508b4d3113fe9569863d74f992e008f7842380b1701de833891b08ff3", + "size_bytes": 2016 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "4560b0f1523237e4323cc9cd954c538ca1d73598a8e67673ea620568851efda6", + "size_bytes": 1229 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "194adeadc9b463ed1bc7406c5f5630bef2c0be388c564279ed13f14c92d3b5cf", + "size_bytes": 471 + }, + { + "path": "identity/host-probe.txt", + "sha256": "5cd0f604b06d8efcadf7baf6935ea86fc377562ab072f6eecb25eb945bcc247a", + "size_bytes": 1709 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "7fb46746bc9419e7abdb672164ec7d390a9b9a90c6fab494acdf2007c3b6850e", + "size_bytes": 3178 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "0b973d9b69e84c025c4c84fb8a5a335428df85175d6bc63ee4a3c86834dacfec", + "size_bytes": 9374 + }, + { + "path": "logs/runner.log", + "sha256": "635bb0c7458f68f0ea47ece8d373980dbd6335c36f9310a0dba205b469bdf5a9", + "size_bytes": 1968 + }, + { + "path": "preflight.json", + "sha256": "a0a2864b989fabdb80b8dc5efcd4e301eff362033b3297114300ea6c75c24340", + "size_bytes": 1805 + }, + { + "path": "run-before-cleanup.json", + "sha256": "09e012ed25136b72d04b947b48ad3541a29f0dda806b2abfbd37adfb83f13d38", + "size_bytes": 26025 + } + ], + "confirmed_at": "2026-09-15T14:30:50Z", + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" + }, + "failure_classification": "orca orchestration worker-start reported outcome_unknown: patch ctx_3af875015946 --json\",\n \"orca-ide terminal read --terminal term_e5e6305a-e659-4e59-8a74-1c371ffa90ca --screen\",\n \"orca-ide orchestration worker-abandon --dispatch ctx_3af875015946 --json\"\n ],\n \"mutation\": {\n \"requestId\": \"05edebc0-115b-472a-b9d4-971e35b6b1db\",\n \"replayed\": false\n }\n },\n \"_meta\": {\n \"runtimeId\": \"2f3ef3d7-823c-4a45-8113-aee62f1872df\"\n }\n}", + "finished_at": "2026-09-15T14:30:57Z", + "generated_at": "2026-09-15T14:30:57Z", + "host_after": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T14:30:57Z", + "extra": { + "backend": "vm" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [ + "20260915T142227Z-bbc5e6-7eec5b64" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "host_before": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T14:29:39Z", + "extra": { + "backend": "vm" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [ + "20260915T142227Z-bbc5e6-7eec5b64" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "identity": { + "environment": { + "container_marker": "", + "home": "/home/cycle", + "hostname": "dely-cycle-20260915t220000z-bbbbbb-00000011", + "orca_fingerprint": "533628:1592:1789327972", + "orca_path": "/usr/local/bin/orca", + "orca_version": "1.4.201", + "path": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin", + "pid": "1662", + "project_real": "/home/cycle/project", + "uname": "Linux 6.8.0-139-generic x86_64", + "user": "cycle", + "virt": "kvm" + }, + "host": { + "boot_id_digest": "de0d3220ff53399e", + "home_digest": "8fda3821c06d512d", + "hostname_digest": "bbc5e661e106c6dc", + "machine_id_digest": "907fa710d6d1c501", + "orca_fingerprint_digest": "ea841899bf829047", + "orca_path_digest": "6784f600bbc7d6cf", + "orca_present": true, + "project_real_digest": "0d03c9e72dddfdae", + "user_digest": "3cc9cee71b0b967f", + "virt": "none" + }, + "markers": [ + "distinct-machine-identity", + "distinct-boot-identity", + "distinct-hostname", + "reported-virtualisation:kvm" + ], + "orca_is_host_installation": false, + "orca_present": true, + "orca_runtime": { + "app_pid": 1877, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "2f3ef3d7-823c-4a45-8113-aee62f1872df", + "runtime_state": "ready" + }, + "orca_status_exit_code": 0, + "orca_version": "1.4.201", + "reason": "the environment answered with its own identity, home and project copy", + "verdict": "ENVIRONMENT" + }, + "phases": [ + { + "commands": [], + "detail": "", + "elapsed_seconds": 1.71, + "finished_at": "2026-09-15T14:29:41Z", + "name": "prepare", + "started_at": "2026-09-15T14:29:39Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 17.815, + "finished_at": "2026-09-15T14:29:58Z", + "name": "create", + "started_at": "2026-09-15T14:29:41Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "git -C /home/cycle/project init -q -b main" + ], + "context": "environment", + "elapsed_seconds": 0.201, + "exit_code": 0, + "finished_at": "2026-09-15T14:29:59Z", + "started_at": "2026-09-15T14:29:59Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "git -C /home/cycle/project config user.email dely-cycle@invalid" + ], + "context": "environment", + "elapsed_seconds": 0.19, + "exit_code": 0, + "finished_at": "2026-09-15T14:29:59Z", + "started_at": "2026-09-15T14:29:59Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "git -C /home/cycle/project config user.name dely-cycle" + ], + "context": "environment", + "elapsed_seconds": 0.191, + "exit_code": 0, + "finished_at": "2026-09-15T14:29:59Z", + "started_at": "2026-09-15T14:29:59Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "git -C /home/cycle/project add -A" + ], + "context": "environment", + "elapsed_seconds": 0.204, + "exit_code": 0, + "finished_at": "2026-09-15T14:30:00Z", + "started_at": "2026-09-15T14:29:59Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "git -C /home/cycle/project commit -q -m 'dely-cycle: the pinned revision as exported'" + ], + "context": "environment", + "elapsed_seconds": 0.216, + "exit_code": 0, + "finished_at": "2026-09-15T14:30:00Z", + "started_at": "2026-09-15T14:30:00Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "", + "elapsed_seconds": 1.82, + "finished_at": "2026-09-15T14:30:00Z", + "name": "bootstrap", + "started_at": "2026-09-15T14:29:58Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '')\"\norca_release=''\norca_fingerprint=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '%s' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '%i:%s:%Y' \"$orca_resolved\" 2>/dev/null || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id=%s\\n' \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'orca_fingerprint=%s\\n' \"$orca_fingerprint\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/project-source", + "orca" + ], + "context": "host", + "elapsed_seconds": 0.1, + "exit_code": 0, + "finished_at": "2026-09-15T14:30:00Z", + "started_at": "2026-09-15T14:30:00Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "sh -c '\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf '\"'\"''\"'\"'\n else\n printf '\"'\"''\"'\"'\n fi\n}\n\nmarker='\"'\"''\"'\"'\nif [ -f /run/.containerenv ]; then marker='\"'\"'containerenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='\"'\"'dockerenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='\"'\"'toolboxenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='\"'\"'unknown'\"'\"'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '\"'\"''\"'\"')\"\norca_release='\"'\"''\"'\"'\norca_fingerprint='\"'\"''\"'\"'\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '\"'\"''\"'\"')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '\"'\"'%s'\"'\"' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '\"'\"'%i:%s:%Y'\"'\"' \"$orca_resolved\" 2>/dev/null || printf '\"'\"''\"'\"')\"\nfi\n\nproject_resolved='\"'\"''\"'\"'\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '\"'\"''\"'\"')\"\nfi\n\nprintf '\"'\"'hostname=%s\\n'\"'\"' \"$(uname -n 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'machine_id=%s\\n'\"'\"' \"$(first_line /etc/machine-id)\"\nprintf '\"'\"'boot_id=%s\\n'\"'\"' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf '\"'\"'home=%s\\n'\"'\"' \"${HOME:-}\"\nprintf '\"'\"'user=%s\\n'\"'\"' \"$(id -un 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'path=%s\\n'\"'\"' \"${PATH:-}\"\nprintf '\"'\"'pid=%s\\n'\"'\"' \"$$\"\nprintf '\"'\"'uname=%s\\n'\"'\"' \"$(uname -srm 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'container_marker=%s\\n'\"'\"' \"$marker\"\nprintf '\"'\"'virt=%s\\n'\"'\"' \"$virt\"\nprintf '\"'\"'orca_path=%s\\n'\"'\"' \"$orca_binary\"\nprintf '\"'\"'orca_version=%s\\n'\"'\"' \"$orca_release\"\nprintf '\"'\"'orca_fingerprint=%s\\n'\"'\"' \"$orca_fingerprint\"\nprintf '\"'\"'project_real=%s\\n'\"'\"' \"$project_resolved\"\n' cycle-probe /home/cycle/project orca" + ], + "context": "environment", + "elapsed_seconds": 0.306, + "exit_code": 0, + "finished_at": "2026-09-15T14:30:01Z", + "started_at": "2026-09-15T14:30:00Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "orca status --json" + ], + "context": "environment", + "elapsed_seconds": 0.285, + "exit_code": 0, + "finished_at": "2026-09-15T14:30:01Z", + "started_at": "2026-09-15T14:30:01Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "sh -c 'export DISPLAY=\"$1\"; shift; export XDG_RUNTIME_DIR=\"${XDG_RUNTIME_DIR:-/run/user/$(id -u)}\"; waited=0; while ! xset -q >/dev/null 2>&1; do waited=$((waited + 2)); if [ \"$waited\" -ge 120 ]; then printf \"no display after %ss\n\" \"$waited\" >&2; exit 3; fi; sleep 2; done; printf \"display ready after %ss\n\" \"$waited\"; rm -f \"$HOME/.config/orca/SingletonLock\" \"$HOME/.config/orca/SingletonCookie\"; nohup \"$@\" > \"$HOME/orca-app.log\" 2>&1 & printf \"started %s\n\" \"$!\"' orca-start :0 /opt/Orca/orca-ide" + ], + "context": "environment", + "elapsed_seconds": 0.201, + "exit_code": 0, + "finished_at": "2026-09-15T14:30:01Z", + "started_at": "2026-09-15T14:30:01Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "orca runtime 2f3ef3d7-823c-4a45-8113-aee62f1872df is ready and the coordinator terminal is term_aa0d3b21-dd30-4a17-be2c-40f645426cbf", + "elapsed_seconds": 7.386, + "finished_at": "2026-09-15T14:30:08Z", + "name": "identity", + "started_at": "2026-09-15T14:30:00Z", + "status": "OK" + }, + { + "commands": [], + "detail": "orca orchestration worker-start reported outcome_unknown: patch ctx_3af875015946 --json\",\n \"orca-ide terminal read --terminal term_e5e6305a-e659-4e59-8a74-1c371ffa90ca --screen\",\n \"orca-ide orchestration worker-abandon --dispatch ctx_3af875015946 --json\"\n ],\n \"mutation\": {\n \"requestId\": \"05edebc0-115b-472a-b9d4-971e35b6b1db\",\n \"replayed\": false\n }\n },\n \"_meta\": {\n \"runtimeId\": \"2f3ef3d7-823c-4a45-8113-aee62f1872df\"\n }\n}", + "elapsed_seconds": 42.507, + "finished_at": "2026-09-15T14:30:50Z", + "name": "task", + "started_at": "2026-09-15T14:30:08Z", + "status": "FAILED" + }, + { + "commands": [], + "detail": "orca orchestration worker-start reported outcome_unknown: patch ctx_3af875015946 --json\",\n \"orca-ide terminal read --terminal term_e5e6305a-e659-4e59-8a74-1c371ffa90ca --screen\",\n \"orca-ide orchestration worker-abandon --dispatch ctx_3af875015946 --json\"\n ],\n \"mutation\": {\n \"requestId\": \"05edebc0-115b-472a-b9d4-971e35b6b1db\",\n \"replayed\": false\n }\n },\n \"_meta\": {\n \"runtimeId\": \"2f3ef3d7-823c-4a45-8113-aee62f1872df\"\n }\n}", + "elapsed_seconds": null, + "finished_at": null, + "name": "check", + "started_at": null, + "status": "SKIPPED" + }, + { + "commands": [], + "detail": "0 patch line(s) collected", + "elapsed_seconds": 0.296, + "finished_at": "2026-09-15T14:30:50Z", + "name": "collect", + "started_at": "2026-09-15T14:30:50Z", + "status": "OK" + }, + { + "commands": [], + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "elapsed_seconds": 0.025, + "finished_at": "2026-09-15T14:30:50Z", + "name": "export", + "started_at": "2026-09-15T14:30:50Z", + "status": "OK" + }, + { + "commands": [], + "detail": "every declared per-run resource is gone and every shared one remains", + "elapsed_seconds": 6.314, + "finished_at": "2026-09-15T14:30:57Z", + "name": "cleanup", + "started_at": "2026-09-15T14:30:50Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": null, + "finished_at": null, + "name": "close", + "started_at": "2026-09-15T14:30:57Z", + "status": "OK" + } + ], + "run_id": "20260915T220000Z-bbbbbb-00000011", + "schema_version": 1, + "started_at": "2026-09-15T14:29:39Z", + "status": "ERROR", + "tested_revision": "e4860de290168b7517d97181b37cfbafa4d19f89", + "timeout_seconds": 1800, + "versions": { + "configured_revision": "e4860de290168b7517d97181b37cfbafa4d19f89", + "distrobox": "distrobox: 1.8.2.5", + "git": "git version 2.55.0", + "podman": "podman version 5.8.4", + "project_revision": "e4860de290168b7517d97181b37cfbafa4d19f89", + "pulumi": "v3.262.0", + "python": "3.14.7", + "qemu_img": "Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers", + "runner": "minimal-cycle", + "virsh": "12.0.0" + }, + "worker": { + "agent": "claude", + "commands": [ + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "orca orchestration run-create --objective 'dely minimal cycle' --json --from term_aa0d3b21-dd30-4a17-be2c-40f645426cbf" + ], + "context": "environment", + "elapsed_seconds": 0.264, + "exit_code": 0, + "finished_at": "2026-09-15T14:30:08Z", + "started_at": "2026-09-15T14:30:08Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T220000Z-bbbbbb-00000011/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.171", + "--", + "orca orchestration worker-start --spec 'Read the file /home/cycle/dispatch-prompt.md in this environment and do exactly what it says.' --agent claude --model claude-opus-5 --effort high --worktree current --timeout-ms 1800000 --json --run run_44e093475630 --from term_aa0d3b21-dd30-4a17-be2c-40f645426cbf" + ], + "context": "environment", + "elapsed_seconds": 41.859, + "exit_code": 1, + "finished_at": "2026-09-15T14:30:50Z", + "started_at": "2026-09-15T14:30:08Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "orca orchestration worker-start reported outcome_unknown: patch ctx_3af875015946 --json\",\n \"orca-ide terminal read --terminal term_e5e6305a-e659-4e59-8a74-1c371ffa90ca --screen\",\n \"orca-ide orchestration worker-abandon --dispatch ctx_3af875015946 --json\"\n ],\n \"mutation\": {\n \"requestId\": \"05edebc0-115b-472a-b9d4-971e35b6b1db\",\n \"replayed\": false\n }\n },\n \"_meta\": {\n \"runtimeId\": \"2f3ef3d7-823c-4a45-8113-aee62f1872df\"\n }\n}", + "dispatch_id": "ctx_3af875015946", + "effort": "high", + "model": "claude-opus-5", + "outcome": "outcome_unknown", + "run_id": "run_44e093475630", + "status": "FAILED" + } +} diff --git a/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/preflight.json b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/preflight.json new file mode 100644 index 0000000..3e433a0 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-dispatch-unverifiable/preflight.json @@ -0,0 +1,81 @@ +{ + "backend": "vm", + "findings": [ + { + "detail": "/.pulumi/bin/pulumi", + "name": "pulumi on the path", + "ok": true + }, + { + "detail": "/usr/bin/qemu-img", + "name": "qemu-img on the path", + "ok": true + }, + { + "detail": "/usr/bin/virsh", + "name": "virsh on the path", + "ok": true + }, + { + "detail": "/usr/bin/ssh", + "name": "ssh on the path", + "ok": true + }, + { + "detail": "/usr/bin/scp", + "name": "scp on the path", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/venv/bin/python", + "name": "pinned environment present", + "ok": true + }, + { + "detail": "state lives at file://~", + "name": "pulumi state backend is local", + "ok": true + }, + { + "detail": "every class and field the program names exists in pulumi-libvirt@0.5.4", + "name": "provider schema verified", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "name": "base image present", + "ok": true + }, + { + "detail": "the preserved base matches the pinned digest", + "name": "base image digest matches the pin", + "ok": true + }, + { + "detail": "/dev/kvm", + "name": "hardware acceleration device", + "ok": true + }, + { + "detail": "qemu:///system", + "name": "libvirt connection answers", + "ok": true + }, + { + "detail": "dely-cycle is active", + "name": "storage pool active", + "ok": true + }, + { + "detail": "default is active", + "name": "network active", + "ok": true + }, + { + "detail": "vnc is among sdl, vnc, dbus", + "name": "graphics type supported by this emulator", + "ok": true + } + ], + "ok": true +} diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index e868f7c..6a315bb 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -101,6 +101,7 @@ def __init__( terminal_refused: bool = False, already_running: bool = False, refuse_repository: bool = False, + dispatch_state: str | None = None, create_fails: bool = False, task_writes_nothing: bool = False, host_project: Path | None = None, @@ -126,6 +127,7 @@ def __init__( # what the real environment does too. self.app_started = already_running self.refuse_repository = refuse_repository + self.dispatch_state = dispatch_state self.create_fails = create_fails self.task_writes_nothing = task_writes_nothing self.host_project = host_project @@ -222,6 +224,11 @@ def execute( self.calls.append("worker") if self.task_hangs: return self._outcome(argv, None, "", "deadline reached", timed_out=True) + if self.dispatch_state and "worker-start" in joined: + reply = ORCA_DISPATCH_REPLY.replace( + '"state": "ready"', f'"state": "{self.dispatch_state}"' + ) + return self._outcome(argv, 1, reply, "") if not self.task_writes_nothing: self.project.mkdir(parents=True, exist_ok=True) (self.project / "evidence.txt").write_text(self.marker, encoding="utf-8") diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index a5f5bc6..e8b75c9 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -393,3 +393,33 @@ def test_repository_metadata_does_not_appear_in_the_patch(self): patch = self.artifact("diff.patch").read_text(encoding="utf-8") self.assertNotIn(".git/", patch) self.assertIn("evidence.txt", patch) + + +class UnverifiableDispatchTest(CycleTestCase): + """The plane saying it cannot tell is not the plane saying it failed. + + Observed against a live runtime: with no credential in the image, Claude + Code's turn never began and Orca reported `outcome_unknown` with + `turn_start_unobserved` — explicitly unverifiable, not proof of failure. + """ + + def test_an_unverifiable_dispatch_settles_unknown_rather_than_error(self): + _, outcome = self.run_cycle(dispatch_state="outcome_unknown") + self.assertEqual(outcome.run_result.status, status.RunStatus.UNKNOWN) + self.assertIn("unknown", outcome.run_result.failure_classification.lower()) + + def test_the_dispatch_identifiers_are_still_recorded(self): + _, outcome = self.run_cycle(dispatch_state="outcome_unknown") + worker_record = outcome.run_result.worker + self.assertEqual(worker_record.dispatch_id, "dispatch-fake") + self.assertEqual(worker_record.outcome, "outcome_unknown") + self.assertEqual(worker_record.model, "pinned-model") + + def test_the_evidence_is_still_exported_and_the_environment_destroyed(self): + _, outcome = self.run_cycle(dispatch_state="outcome_unknown") + self.assertEqual(outcome.run_result.export.status, status.ExportStatus.CONFIRMED) + self.assertEqual(outcome.run_result.cleanup.status, status.CleanupStatus.DESTROYED) + + def test_a_genuine_refusal_is_still_an_error(self): + _, outcome = self.run_cycle(dispatch_state="refused") + self.assertEqual(outcome.run_result.status, status.RunStatus.ERROR) From 25f161cd1493d0c0adeeb05ce711f6fe50b6a1e1 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 16:35:25 +0200 Subject: [PATCH 24/42] Record a container carrying its own Orca at the host's path, and the rule that tells them apart. --- .../evidence/distrobox-own-orca/README.md | 37 +++++++++++++++++++ .../distrobox-own-orca/container-probe.txt | 14 +++++++ .../distrobox-own-orca/host-probe.txt | 14 +++++++ 3 files changed, 65 insertions(+) create mode 100644 experiments/minimal-cycle/evidence/distrobox-own-orca/README.md create mode 100644 experiments/minimal-cycle/evidence/distrobox-own-orca/container-probe.txt create mode 100644 experiments/minimal-cycle/evidence/distrobox-own-orca/host-probe.txt diff --git a/experiments/minimal-cycle/evidence/distrobox-own-orca/README.md b/experiments/minimal-cycle/evidence/distrobox-own-orca/README.md new file mode 100644 index 0000000..3fbad72 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-own-orca/README.md @@ -0,0 +1,37 @@ +# A container with its own Orca, at the same path as the host's + +A Distrobox container built from `docker.io/library/ubuntu:24.04`, given the +pinned `orca-ide_1.4.201_amd64.deb`, and probed from both sides with the same +command: `/opt/Orca/resources/bin/orca-ide`. + +## What the two probes show + + field host container + orca_path /opt/Orca/resources/bin/orca-ide /opt/Orca/resources/bin/orca-ide + orca_version 1.4.201 1.4.201 + orca_fingerprint 7199608:1592:1789327972 100255748:1592:1789327972 + container_marker (none) containerenv + virt none docker + +The path is **identical**. The file is not: the inode differs, the size and +modification time match because it is the same package installed twice. + +## Why this matters + +The rule this replaced compared the path. On this data it called the +container's own installation the host's and refused to continue: + + comparing the path -> host installation: True, gate opens: False + comparing what it resolves to -> host installation: False, gate opens: True + +Both answers are produced by code that runs and returns a verdict. Only one of +them is right, and the wrong one refuses a correctly isolated environment. The +counterexample `the-same-path-is-not-the-same-file` holds that difference. + +## What this does not show + +An Orca *running* in the container, a worker dispatched through it, or a task +completed. The container was probed, not driven: starting the application in a +Distrobox container renders its window on the host's own desktop, which is this +backend's documented compromise and a deliberate act, not a side effect of a +test. diff --git a/experiments/minimal-cycle/evidence/distrobox-own-orca/container-probe.txt b/experiments/minimal-cycle/evidence/distrobox-own-orca/container-probe.txt new file mode 100644 index 0000000..c99f8b1 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-own-orca/container-probe.txt @@ -0,0 +1,14 @@ +hostname=dely-cycle-probe-check +machine_id= +boot_id= +home=/var/tmp/dely-cycle/probe-check/home +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin +pid=293966 +uname=Linux 7.2.5-200.fc44.x86_64 x86_64 +container_marker=containerenv +virt=docker +orca_path=/opt/Orca/resources/bin/orca-ide +orca_version=1.4.201 +orca_fingerprint=100255748:1592:1789327972 +project_real=/var/tmp/dely-cycle/probe-check/home/project diff --git a/experiments/minimal-cycle/evidence/distrobox-own-orca/host-probe.txt b/experiments/minimal-cycle/evidence/distrobox-own-orca/host-probe.txt new file mode 100644 index 0000000..1ec5598 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-own-orca/host-probe.txt @@ -0,0 +1,14 @@ +hostname=system +machine_id= +boot_id= +home= +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin +pid=293833 +uname=Linux 7.2.5-200.fc44.x86_64 x86_64 +container_marker= +virt=none +orca_path=/opt/Orca/resources/bin/orca-ide +orca_version=1.4.201 +orca_fingerprint=7199608:1592:1789327972 +project_real= From beb9fc7b6b2eca1d8f53de78873860800462c173 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 19:06:29 +0200 Subject: [PATCH 25/42] Wait for a dispatch the plane could not observe, instead of calling it dead at the start. --- experiments/minimal-cycle/counterexamples.py | 8 +++ .../minimal-cycle/cycle_runner/worker.py | 31 ++++++---- experiments/minimal-cycle/tests/fakes.py | 5 ++ .../minimal-cycle/tests/test_lifecycle.py | 15 ++++- .../minimal-cycle/tests/test_worker.py | 56 +++++++++++++++++++ 5 files changed, 100 insertions(+), 15 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 176e7ad..718f8b5 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,14 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="an-unobserved-start-still-waits", + requirement="A dispatch the plane could not observe is still waited for", + path="cycle_runner/worker.py", + original=""" if state in UNVERIFIABLE_STATES and record.dispatch_id:""", + replacement=""" if False:""", + instruments=("tests.test_worker.UnverifiedStartTest",), + ), Counterexample( name="unverifiable-is-not-failed", requirement="A dispatch the plane cannot judge settles unknown, not error", diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index a9b1515..bc1e2d9 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -253,21 +253,28 @@ def launch( record.status = PhaseStatus.TIMEOUT record.detail = "orca orchestration worker-start reached the run deadline" return record - if not started.ok: - record.status = PhaseStatus.FAILED - state = dispatch_state(_first_document(started.stdout)) - record.dispatch_id = dispatch_identifier(_first_document(started.stdout)) - record.outcome = state - record.detail = ( - f"orca orchestration worker-start reported {state or 'no state'}: " - + redact.text((started.stderr or started.stdout).strip()[-400:], secrets) - ) - return record start_document = _first_document(started.stdout) record.dispatch_id = dispatch_identifier(start_document) record.run_id = record.run_id or run_identifier(start_document) state = dispatch_state(start_document) - if state: + record.outcome = state + if not started.ok: + # An unobserved turn start is not a dead worker: the plane says so + # itself. The dispatch exists and may still settle, so the completion + # wait runs. Only a start that produced no dispatch is terminal. + if state in UNVERIFIABLE_STATES and record.dispatch_id: + record.detail = ( + f"worker-start reported {state}; the dispatch exists, so the " + "completion wait decides" + ) + else: + record.status = PhaseStatus.FAILED + record.detail = ( + f"orca orchestration worker-start reported {state or 'no state'}: " + + redact.text((started.stderr or started.stdout).strip()[-400:], secrets) + ) + return record + elif state: record.detail = f"the plane reported the dispatch as {state}" settled = adapter.execute( @@ -286,7 +293,7 @@ def launch( return record message = _settling_message(_first_document(settled.stdout)) - record.outcome = message.get("outcome") or message.get("type") or None + record.outcome = message.get("outcome") or message.get("type") or record.outcome if message.get("type") == "worker_done": record.status = PhaseStatus.OK record.detail = f"the worker reported worker_done with outcome {record.outcome!r}" diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index 6a315bb..3b1ba97 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -102,6 +102,7 @@ def __init__( already_running: bool = False, refuse_repository: bool = False, dispatch_state: str | None = None, + wait_settles: bool = True, create_fails: bool = False, task_writes_nothing: bool = False, host_project: Path | None = None, @@ -128,6 +129,7 @@ def __init__( self.app_started = already_running self.refuse_repository = refuse_repository self.dispatch_state = dispatch_state + self.wait_settles = wait_settles self.create_fails = create_fails self.task_writes_nothing = task_writes_nothing self.host_project = host_project @@ -229,6 +231,9 @@ def execute( '"state": "ready"', f'"state": "{self.dispatch_state}"' ) return self._outcome(argv, 1, reply, "") + if not self.wait_settles and "--wait" in joined: + # The dispatch never reports, so the wait returns nothing. + return self._outcome(argv, 0, '{"messages": []}', "") if not self.task_writes_nothing: self.project.mkdir(parents=True, exist_ok=True) (self.project / "evidence.txt").write_text(self.marker, encoding="utf-8") diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index e8b75c9..856a613 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -403,20 +403,29 @@ class UnverifiableDispatchTest(CycleTestCase): `turn_start_unobserved` — explicitly unverifiable, not proof of failure. """ + def never_settles(self): + return dict(dispatch_state="outcome_unknown", wait_settles=False) + def test_an_unverifiable_dispatch_settles_unknown_rather_than_error(self): - _, outcome = self.run_cycle(dispatch_state="outcome_unknown") + _, outcome = self.run_cycle(**self.never_settles()) self.assertEqual(outcome.run_result.status, status.RunStatus.UNKNOWN) self.assertIn("unknown", outcome.run_result.failure_classification.lower()) + def test_a_dispatch_that_recovers_after_an_unobserved_start_settles(self): + _, outcome = self.run_cycle(dispatch_state="outcome_unknown", wait_settles=True) + self.assertEqual(outcome.run_result.status, status.RunStatus.SETTLED) + self.assertEqual(outcome.run_result.worker.outcome, "DONE") + self.assertEqual(outcome.run_result.check.exit_code, 0) + def test_the_dispatch_identifiers_are_still_recorded(self): - _, outcome = self.run_cycle(dispatch_state="outcome_unknown") + _, outcome = self.run_cycle(**self.never_settles()) worker_record = outcome.run_result.worker self.assertEqual(worker_record.dispatch_id, "dispatch-fake") self.assertEqual(worker_record.outcome, "outcome_unknown") self.assertEqual(worker_record.model, "pinned-model") def test_the_evidence_is_still_exported_and_the_environment_destroyed(self): - _, outcome = self.run_cycle(dispatch_state="outcome_unknown") + _, outcome = self.run_cycle(**self.never_settles()) self.assertEqual(outcome.run_result.export.status, status.ExportStatus.CONFIRMED) self.assertEqual(outcome.run_result.cleanup.status, status.CleanupStatus.DESTROYED) diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py index d26519d..745b260 100644 --- a/experiments/minimal-cycle/tests/test_worker.py +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -277,3 +277,59 @@ def test_a_launch_against_the_real_shapes_uses_the_run_identifier(self): self.assertEqual(start[start.index("--run") + 1], "run_a52fcdd366b4") self.assertEqual(record.run_id, "run_a52fcdd366b4") self.assertEqual(record.dispatch_id, "ctx_098ec148783c") + + +UNVERIFIED_START = json.dumps( + { + "id": "request-999", + "ok": False, + "result": { + "runId": "run-abcdef", + "taskId": "task-abcdef", + "dispatchId": "dispatch-abcdef", + "state": "outcome_unknown", + "stage": "turn_start_unobserved", + }, + } +) + + +class UnverifiedStartTest(WorkerTest): + """A start the plane could not observe is not a start that failed. + + Observed against a live runtime: worker-start exits non-zero with + `outcome_unknown` when the agent's turn does not begin inside the + observation window. The dispatch exists and may still settle, so the + completion wait has to run. + """ + + def script(self, settled): + return [ + ("run-create", 0, READY, "", False), + ("worker-start", 1, UNVERIFIED_START, "", False), + ("check --wait", 0, settled, "", False), + ] + + def test_the_completion_wait_still_runs(self): + adapter, _, _ = self.launch(script=self.script(SETTLED)) + self.assertTrue(any("--wait" in argv for argv in adapter.executed)) + + def test_a_dispatch_that_settles_afterwards_is_done(self): + _, _, record = self.launch(script=self.script(SETTLED)) + self.assertEqual(record.status, status.PhaseStatus.OK) + self.assertEqual(record.outcome, "DONE") + self.assertEqual(record.dispatch_id, "dispatch-abcdef") + + def test_a_dispatch_that_never_settles_keeps_the_unverifiable_state(self): + _, _, record = self.launch(script=self.script(json.dumps({"messages": []}))) + self.assertEqual(record.outcome, "outcome_unknown") + self.assertNotEqual(record.status, status.PhaseStatus.OK) + + def test_a_start_with_no_dispatch_is_still_a_failure(self): + script = [ + ("run-create", 0, READY, "", False), + ("worker-start", 1, json.dumps({"ok": False, "error": {"code": "refused"}}), "", False), + ] + adapter, _, record = self.launch(script=script) + self.assertEqual(record.status, status.PhaseStatus.FAILED) + self.assertFalse(any("--wait" in argv for argv in adapter.executed)) From 1b2c1b1e1ff71302d11a0d17cf2f416184e7d077 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 19:10:12 +0200 Subject: [PATCH 26/42] Give the completion wait the flag it accepts: it names a terminal, and rejects the sender flag the other two take. --- experiments/minimal-cycle/counterexamples.py | 8 +++ .../minimal-cycle/cycle_runner/worker.py | 8 +-- .../minimal-cycle/tests/test_worker.py | 51 ++++++++++++++++++- 3 files changed, 62 insertions(+), 5 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 718f8b5..8fe2312 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,14 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="the-wait-takes-a-terminal-not-a-sender", + requirement="The completion wait names the terminal with the flag it accepts", + path="cycle_runner/worker.py", + original=""" wait.extend(["--terminal", coordinator_handle])""", + replacement=""" wait.extend(["--from", coordinator_handle])""", + instruments=("tests.test_worker.SenderFlagShapeTest",), + ), Counterexample( name="an-unobserved-start-still-waits", requirement="A dispatch the plane could not observe is still waited for", diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index bc1e2d9..e53be84 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -113,11 +113,13 @@ def build_plan( run_config.orca.run_objective, "--json", ] - # Every orchestration command is sent from a terminal the runtime knows; - # without one Orca refuses with no_active_sender_terminal. + # Every orchestration command names the terminal the runtime knows it by; + # without one Orca refuses with no_active_sender_terminal. The wait takes it + # as `--terminal`, not `--from`, and rejects `--from` outright. if coordinator_handle: - for command in (created, start, wait): + for command in (created, start): command.extend(["--from", coordinator_handle]) + wait.extend(["--terminal", coordinator_handle]) return LaunchPlan( prompt_path=prompt_path, spec=spec, diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py index 745b260..292dae4 100644 --- a/experiments/minimal-cycle/tests/test_worker.py +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -199,8 +199,11 @@ def test_every_orchestration_command_names_the_sender_terminal(self): orchestration = [argv for argv in adapter.executed if "orchestration" in argv] self.assertEqual(len(orchestration), 3) for argv in orchestration: - self.assertIn("--from", argv) - self.assertEqual(argv[argv.index("--from") + 1], "term_abcdef") + # The wait takes the terminal by a different flag; both are the + # same handle, and neither command may go without one. + flag = "--terminal" if "--wait" in argv else "--from" + self.assertIn(flag, argv) + self.assertEqual(argv[argv.index(flag) + 1], "term_abcdef") def test_without_a_terminal_the_flag_is_not_invented(self): adapter, _ = self.launch_from(None) @@ -333,3 +336,47 @@ def test_a_start_with_no_dispatch_is_still_a_failure(self): adapter, _, record = self.launch(script=script) self.assertEqual(record.status, status.PhaseStatus.FAILED) self.assertFalse(any("--wait" in argv for argv in adapter.executed)) + + +class SenderFlagShapeTest(WorkerTest): + """The three commands do not take the sender the same way. + + Observed against a live runtime: `orchestration check` rejected `--from` + with `Unknown flag --from for command: orchestration check`. It names the + terminal with `--terminal`; run-create and worker-start use `--from`. + """ + + def plan(self): + from cycle_runner.adapters.base import EnvironmentHandle + + handle = EnvironmentHandle( + environment_id="e", home_path="/home/cycle", project_path="/home/cycle/project" + ) + return worker.build_plan( + run_config=self.config, + handle=handle, + timeout_seconds=900, + orca_run_id="run-abcdef", + coordinator_handle="term_abcdef", + ) + + def test_run_create_names_the_sender_with_from(self): + argv = self.plan().run_create_argv + self.assertIn("--from", argv) + self.assertEqual(argv[argv.index("--from") + 1], "term_abcdef") + + def test_worker_start_names_the_sender_with_from(self): + argv = self.plan().worker_start_argv + self.assertIn("--from", argv) + self.assertEqual(argv[argv.index("--from") + 1], "term_abcdef") + + def test_the_wait_names_the_terminal_and_never_uses_from(self): + argv = self.plan().wait_argv + self.assertNotIn("--from", argv) + self.assertIn("--terminal", argv) + self.assertEqual(argv[argv.index("--terminal") + 1], "term_abcdef") + + def test_the_wait_still_carries_the_run_and_the_settling_types(self): + argv = self.plan().wait_argv + self.assertEqual(argv[argv.index("--run") + 1], "run-abcdef") + self.assertIn("--types", argv) From cd2ef92d993381025e78b4aaffd432f3608ea20e Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 19:18:20 +0200 Subject: [PATCH 27/42] Record the coordinator and the worker as two sessions in one guest, and the command the plane composed for the dispatch. --- .../evidence/vm-two-sessions/README.md | 47 ++++++++ .../evidence/vm-two-sessions/orca-status.json | 112 ++++++++++++++++++ .../evidence/vm-two-sessions/run-list.json | 30 +++++ .../evidence/vm-two-sessions/terminals.json | 57 +++++++++ .../evidence/vm-two-sessions/worker-list.json | 99 ++++++++++++++++ .../worker-terminal-screen.txt | 9 ++ 6 files changed, 354 insertions(+) create mode 100644 experiments/minimal-cycle/evidence/vm-two-sessions/README.md create mode 100644 experiments/minimal-cycle/evidence/vm-two-sessions/orca-status.json create mode 100644 experiments/minimal-cycle/evidence/vm-two-sessions/run-list.json create mode 100644 experiments/minimal-cycle/evidence/vm-two-sessions/terminals.json create mode 100644 experiments/minimal-cycle/evidence/vm-two-sessions/worker-list.json create mode 100644 experiments/minimal-cycle/evidence/vm-two-sessions/worker-terminal-screen.txt diff --git a/experiments/minimal-cycle/evidence/vm-two-sessions/README.md b/experiments/minimal-cycle/evidence/vm-two-sessions/README.md new file mode 100644 index 0000000..4b54723 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-two-sessions/README.md @@ -0,0 +1,47 @@ +# Control and a separate coder session, inside the virtual machine + +Captured from a live guest during one cycle, over the run's own transport, while +the completion wait was still blocking. + +## Two sessions, by identity rather than by layout + +`terminals.json` holds two terminals in the same worktree: + + term_388f8f62-dfea-4469-8aa7-1240fead6d28 the coordinator, opened by the runner + term_977b5b45-c86b-4e16-a32b-9669801c43f9 the worker's own, created by the dispatch + +`worker-list.json` ties them together: one Dispatch, one Task, one Run, and a +worktree resource owned by that Dispatch. Neither claim depends on where a panel +was drawn, which one had focus, or whether anything was visible at all — the +capture was taken through the command line, not from a screen. + +## What the worker's terminal actually shows + +`worker-terminal-screen.txt`: + + cycle@dely-cycle-…:~/project$ claude '--dangerously-skip-permissions' \ + '--model' 'claude-opus-5' '--effort' 'high' + +That is the command Orca composed for this dispatch: the pinned model and the +pinned effort, and the permission default this execution plane configures for +the `claude` agent. The runner did not write that line; it named the agent, the +model and the effort, and the plane turned them into this. + +## Where it stops, and what that means + +The line is present and the turn never began. Orca reports it as +`start_unknown` / `turn_start_unobserved`, with liveness `unverifiable` and +reason `missing_status`, and says so in those words rather than calling the +worker dead. The runner classifies the run `UNKNOWN` for the same reason: the +dispatch exists, the evidence is durable, and nothing here knows whether the +worker ran. + +This is the honest boundary of what has been shown. The dispatch path works end +to end — a Run, a Task, a Dispatch, a second session with the right launch — +and the agent's first turn has not been observed to start inside this guest. + +## What this does not show + +A turn that ran, a file the worker wrote, or a check that passed. The evidence +above is what was true at the moment it was taken, and it does not become more +than that by being detailed. diff --git a/experiments/minimal-cycle/evidence/vm-two-sessions/orca-status.json b/experiments/minimal-cycle/evidence/vm-two-sessions/orca-status.json new file mode 100644 index 0000000..3f36607 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-two-sessions/orca-status.json @@ -0,0 +1,112 @@ +{ + "id": "local-status", + "ok": true, + "result": { + "target": { + "kind": "local" + }, + "app": { + "running": true, + "pid": 1871, + "desktopWindowStatus": "available" + }, + "runtime": { + "state": "ready", + "reachable": true, + "connectionState": "connected", + "runtimeId": "80cb98d3-d25b-4b4d-bf7f-148ef7386c8e", + "appVersion": "1.4.201", + "remoteUpdateSupport": { + "installMode": "interactive", + "automatic": false, + "reason": "manual-service-update-required" + }, + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ] + }, + "graph": { + "state": "ready" + } + }, + "_meta": { + "runtimeId": "80cb98d3-d25b-4b4d-bf7f-148ef7386c8e" + } +} diff --git a/experiments/minimal-cycle/evidence/vm-two-sessions/run-list.json b/experiments/minimal-cycle/evidence/vm-two-sessions/run-list.json new file mode 100644 index 0000000..75f91bd --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-two-sessions/run-list.json @@ -0,0 +1,30 @@ +{ + "id": "e82452fd-c422-497b-b85d-542fc9b81f73", + "ok": true, + "result": { + "runs": [ + { + "id": "run_569a123bb1f3", + "objective": "dely minimal cycle", + "coordinator_handle": "term_388f8f62-dfea-4469-8aa7-1240fead6d28", + "consumer_generation": 1, + "legacy": 0, + "created_at": "2026-09-15T17:10:46Z", + "updated_at": "2026-09-15T17:10:46Z" + }, + { + "id": "run_legacy_local", + "objective": "Legacy orchestration state (inspect only)", + "coordinator_handle": null, + "consumer_generation": 0, + "legacy": 1, + "created_at": "2026-09-15T17:10:41Z", + "updated_at": "2026-09-15T17:10:41Z" + } + ], + "nextCursor": null + }, + "_meta": { + "runtimeId": "80cb98d3-d25b-4b4d-bf7f-148ef7386c8e" + } +} diff --git a/experiments/minimal-cycle/evidence/vm-two-sessions/terminals.json b/experiments/minimal-cycle/evidence/vm-two-sessions/terminals.json new file mode 100644 index 0000000..392b10a --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-two-sessions/terminals.json @@ -0,0 +1,57 @@ +{ + "id": "a251f26c-df94-4541-a8ee-95eb2ac9bb39", + "ok": true, + "result": { + "terminals": [ + { + "handle": "term_388f8f62-dfea-4469-8aa7-1240fead6d28", + "ptyId": "d467a630-443a-4833-94b7-00287c9e59d6::/home/cycle/project@@3f54b13d", + "incarnationId": "bf7719f4-53b8-43b9-9b89-6f7e314ee0f8", + "orphaned": false, + "worktreeId": "d467a630-443a-4833-94b7-00287c9e59d6::/home/cycle/project", + "worktreePath": "/home/cycle/project", + "branch": "refs/heads/main", + "tabId": "bea97365-cedb-42c8-b3a1-8fd78ff65df8", + "leafId": "2585a216-9bac-4f26-a462-80027cdb4259", + "title": "cycle@dely-cycle-20260916t010000z-bbbbbb-00000014: ~/project", + "connected": true, + "writable": true, + "lastOutputAt": 1789492245803, + "preview": "cycle@dely-cycle-20260916t010000z-bbbbbb-00000014:~/project$", + "executionHostId": "local" + }, + { + "handle": "term_977b5b45-c86b-4e16-a32b-9669801c43f9", + "ptyId": "d467a630-443a-4833-94b7-00287c9e59d6::/home/cycle/project@@b2f21216", + "incarnationId": "99bbcd01-9f93-4ba3-acb7-37e3f2b50c32", + "orphaned": false, + "worktreeId": "d467a630-443a-4833-94b7-00287c9e59d6::/home/cycle/project", + "worktreePath": "/home/cycle/project", + "branch": "refs/heads/main", + "tabId": "9405d064-84ed-453c-8eb8-6c25d489ff61", + "leafId": "735cc51a-e69d-441e-b39b-0942a8e10441", + "title": "cycle@dely-cycle-20260916t010000z-bbbbbb-00000014: ~/project", + "connected": true, + "writable": true, + "lastOutputAt": 1789492246897, + "preview": "cycle@dely-cycle-20260916t010000z-bbbbbb-00000014:~/project$ claude '--dangerously-skip-permissions' '--model' 'claude-opus-5' '--effort' 'high'", + "executionHostId": "local", + "agentIdentity": "claude" + } + ], + "hostScope": { + "hostIds": [ + "local" + ], + "omittedHostIds": [] + }, + "topologyRevisions": { + "d467a630-443a-4833-94b7-00287c9e59d6::/home/cycle/project": 2 + }, + "totalCount": 2, + "truncated": false + }, + "_meta": { + "runtimeId": "80cb98d3-d25b-4b4d-bf7f-148ef7386c8e" + } +} diff --git a/experiments/minimal-cycle/evidence/vm-two-sessions/worker-list.json b/experiments/minimal-cycle/evidence/vm-two-sessions/worker-list.json new file mode 100644 index 0000000..55ff58b --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-two-sessions/worker-list.json @@ -0,0 +1,99 @@ +{ + "id": "2a28c0ad-0c4b-44d9-8b93-0c616931d834", + "ok": true, + "result": { + "workers": [ + { + "dispatchId": "ctx_2ff0980c3787", + "taskId": "task_68d2b3443591", + "runId": "run_569a123bb1f3", + "workerState": "start_unknown", + "dispatchStatus": "pending", + "agentTerminalHandle": "term_977b5b45-c86b-4e16-a32b-9669801c43f9", + "terminalState": "active", + "resource": { + "id": "wtr_dca168548c8c", + "ownershipState": "owned", + "releaseState": "not_requested", + "retainedReason": null, + "terminalHandle": "term_977b5b45-c86b-4e16-a32b-9669801c43f9", + "worktreeId": "d467a630-443a-4833-94b7-00287c9e59d6::/home/cycle/project", + "endpointId": "80cb98d3-d25b-4b4d-bf7f-148ef7386c8e", + "endpointIncarnation": "d467a630-443a-4833-94b7-00287c9e59d6::/home/cycle/project@@b2f21216:99bbcd01-9f93-4ba3-acb7-37e3f2b50c32", + "originDispatchId": "ctx_2ff0980c3787", + "ownerDispatchId": "ctx_2ff0980c3787", + "releaseRequestedAt": null, + "releaseCompletedAt": null, + "releaseError": null, + "recoveryAttemptCount": 0, + "lastRecoveryAt": null, + "archive": { + "source": null, + "status": null + } + }, + "projection": { + "id": "ctx_2ff0980c3787", + "dispatchId": "ctx_2ff0980c3787", + "taskId": "task_68d2b3443591", + "runId": "run_569a123bb1f3", + "role": "worker", + "parent": null, + "provider": null, + "host": { + "kind": "local", + "id": "local" + }, + "workspace": { + "id": "d467a630-443a-4833-94b7-00287c9e59d6::/home/cycle/project", + "kind": "folder_or_worktree" + }, + "stage": { + "worker": "start_unknown", + "dispatch": "pending", + "detail": "turn_start_unobserved", + "activity": "unknown" + }, + "outcome": "in_progress", + "liveness": { + "verdict": "unverifiable", + "reason": "missing_status" + }, + "evidence": { + "durable": true, + "liveStatus": "unavailable", + "lastObservedAt": null + }, + "resource": { + "state": "owned" + }, + "nextAction": { + "kind": "none", + "argv": [] + }, + "attention": { + "categories": [ + "unverifiable" + ], + "requiresAction": true + } + } + } + ], + "counts": { + "active": 1 + }, + "page": { + "limit": 100, + "total": 1, + "hasMore": false, + "nextCursor": null + }, + "scope": { + "source": "all" + } + }, + "_meta": { + "runtimeId": "80cb98d3-d25b-4b4d-bf7f-148ef7386c8e" + } +} diff --git a/experiments/minimal-cycle/evidence/vm-two-sessions/worker-terminal-screen.txt b/experiments/minimal-cycle/evidence/vm-two-sessions/worker-terminal-screen.txt new file mode 100644 index 0000000..856daed --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-two-sessions/worker-terminal-screen.txt @@ -0,0 +1,9 @@ +handle: term_977b5b45-c86b-4e16-a32b-9669801c43f9 +status: running +source: screen +cursor: 1 +oldest cursor: 0 +latest cursor: 1 + +cycle@dely-cycle-20260916t010000z-bbbbbb-00000014:~/project$ claude '--dangerously-skip-permissions' '--model' 'claude-o +pus-5' '--effort' 'high' From a1f8be92ae0e3b8f5eb4a53abe571b961c7e7112 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 19:27:19 +0200 Subject: [PATCH 28/42] Record a deadline firing on the machine backend with every artifact kept and the environment destroyed. --- .../evidence/vm-timeout/README.md | 28 + .../evidence/vm-timeout/auth-receipt.json | 11 + .../evidence/vm-timeout/cleanup.json | 15 + .../evidence/vm-timeout/export-receipt.json | 71 ++ .../vm-timeout/identity-orca-runtime.json | 90 ++ .../evidence/vm-timeout/manifest.json | 960 ++++++++++++++++++ 6 files changed, 1175 insertions(+) create mode 100644 experiments/minimal-cycle/evidence/vm-timeout/README.md create mode 100644 experiments/minimal-cycle/evidence/vm-timeout/auth-receipt.json create mode 100644 experiments/minimal-cycle/evidence/vm-timeout/cleanup.json create mode 100644 experiments/minimal-cycle/evidence/vm-timeout/export-receipt.json create mode 100644 experiments/minimal-cycle/evidence/vm-timeout/identity-orca-runtime.json create mode 100644 experiments/minimal-cycle/evidence/vm-timeout/manifest.json diff --git a/experiments/minimal-cycle/evidence/vm-timeout/README.md b/experiments/minimal-cycle/evidence/vm-timeout/README.md new file mode 100644 index 0000000..71db172 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-timeout/README.md @@ -0,0 +1,28 @@ +# The deadline firing, with the evidence kept + +One cycle on the machine backend, run with a fifteen-minute deadline. Every +phase up to the dispatch succeeded; the completion wait blocked for its whole +deadline because the worker's turn never began, and the deadline fired. + + status: TIMEOUT + identity: ENVIRONMENT + worker: dispatch ctx_2ff0980c3787 in run run_569a123bb1f3 + export: CONFIRMED + cleanup: DESTROYED + +What this shows is the ordering the whole runner is built around, under a +terminal state rather than a happy path: the deadline fired, the project tree +was still fetched, every artifact was still written and re-read from the host, +and only then were the domain, overlay, seed, stack and per-run state removed. +The preserved base image was untouched and the host was left with no domain, no +per-run state and no stray process. + +It also shows the deadline is the runner's, not the plane's. `check --wait` +was given the same deadline in milliseconds and would have gone on waiting; +the runner stopped it, classified the run, and kept what it had. + +## What this does not show + +A worker that ran. The turn never started — see `vm-two-sessions/` for what the +worker's terminal actually held — so nothing here says anything about a task +being done or a check passing. diff --git a/experiments/minimal-cycle/evidence/vm-timeout/auth-receipt.json b/experiments/minimal-cycle/evidence/vm-timeout/auth-receipt.json new file mode 100644 index 0000000..4c3bf6b --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-timeout/auth-receipt.json @@ -0,0 +1,11 @@ +{ + "detail": "1 allowlisted entry(s) copied into the per-run home with owner-only permissions", + "entries": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login", + "removed_after_run": false, + "status": "OK", + "target": "/home/cycle" +} diff --git a/experiments/minimal-cycle/evidence/vm-timeout/cleanup.json b/experiments/minimal-cycle/evidence/vm-timeout/cleanup.json new file mode 100644 index 0000000..8334a22 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-timeout/cleanup.json @@ -0,0 +1,15 @@ +{ + "host_difference": {}, + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "domain:dely-cycle-20260916t010000z-bbbbbb-00000014", + "path:/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014" + ], + "retained": [], + "shared_preserved": [ + "image:/var/tmp/dely-cycle/images/tool-image.qcow2" + ], + "status": "DESTROYED", + "stop_confirmed": true, + "verified": true +} diff --git a/experiments/minimal-cycle/evidence/vm-timeout/export-receipt.json b/experiments/minimal-cycle/evidence/vm-timeout/export-receipt.json new file mode 100644 index 0000000..2219f0d --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-timeout/export-receipt.json @@ -0,0 +1,71 @@ +{ + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "b3e1a9da8d4806448f08ecd637daaf22aac7688bafab6e6a98d0b082e2ada44b", + "size_bytes": 291 + }, + { + "path": "backend-status.json", + "sha256": "a15d64923f183348040c6c6f5be92dd387a4ae1c602762a50d2fac0b2ecec1b2", + "size_bytes": 2016 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "5af460083229b542e0c705890c7b0b8516ea6760f103f3f203ab401b63527e5e", + "size_bytes": 1309 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "fcaba634c48816dac5dec694b09cedfe4ce45c632cde34e40d47fa267316ef8b", + "size_bytes": 471 + }, + { + "path": "identity/host-probe.txt", + "sha256": "231d64a3785b25786617993eb8b7525eb51bda26aa49adef773a5a32e989893c", + "size_bytes": 1709 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "fff5639d9009767cc7528b44428da90528be42ad5554665ce15c35ef109729ba", + "size_bytes": 3178 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "f015e2966d1d8caa93c07d655807331d72e88901c19dea814cbd3553cd2966b7", + "size_bytes": 9377 + }, + { + "path": "logs/runner.log", + "sha256": "c0d853c7b5c707c44ac61251eac5f85b9957a406089da5cbcbf891734f40fe83", + "size_bytes": 1537 + }, + { + "path": "preflight.json", + "sha256": "a0a2864b989fabdb80b8dc5efcd4e301eff362033b3297114300ea6c75c24340", + "size_bytes": 1805 + }, + { + "path": "run-before-cleanup.json", + "sha256": "6f7a2ebb248e16cf0aeb04481ea6a1f9fd141c0044616aefc41c2b4957c5336b", + "size_bytes": 25561 + } + ], + "confirmed_at": "2026-09-15T17:26:29Z", + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" +} diff --git a/experiments/minimal-cycle/evidence/vm-timeout/identity-orca-runtime.json b/experiments/minimal-cycle/evidence/vm-timeout/identity-orca-runtime.json new file mode 100644 index 0000000..49a69b0 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-timeout/identity-orca-runtime.json @@ -0,0 +1,90 @@ +{ + "app_pid": 1871, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "80cb98d3-d25b-4b4d-bf7f-148ef7386c8e", + "runtime_state": "ready" +} diff --git a/experiments/minimal-cycle/evidence/vm-timeout/manifest.json b/experiments/minimal-cycle/evidence/vm-timeout/manifest.json new file mode 100644 index 0000000..6cbdc24 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-timeout/manifest.json @@ -0,0 +1,960 @@ +{ + "artifact_root": "/var/tmp/dely-cycle/artifacts/20260916T010000Z-bbbbbb-00000014", + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "b3e1a9da8d4806448f08ecd637daaf22aac7688bafab6e6a98d0b082e2ada44b", + "size_bytes": 291 + }, + { + "path": "backend-status.json", + "sha256": "a15d64923f183348040c6c6f5be92dd387a4ae1c602762a50d2fac0b2ecec1b2", + "size_bytes": 2016 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "5af460083229b542e0c705890c7b0b8516ea6760f103f3f203ab401b63527e5e", + "size_bytes": 1309 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "fcaba634c48816dac5dec694b09cedfe4ce45c632cde34e40d47fa267316ef8b", + "size_bytes": 471 + }, + { + "path": "identity/host-probe.txt", + "sha256": "231d64a3785b25786617993eb8b7525eb51bda26aa49adef773a5a32e989893c", + "size_bytes": 1709 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "fff5639d9009767cc7528b44428da90528be42ad5554665ce15c35ef109729ba", + "size_bytes": 3178 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "f015e2966d1d8caa93c07d655807331d72e88901c19dea814cbd3553cd2966b7", + "size_bytes": 9377 + }, + { + "path": "logs/runner.log", + "sha256": "c0d853c7b5c707c44ac61251eac5f85b9957a406089da5cbcbf891734f40fe83", + "size_bytes": 1537 + }, + { + "path": "preflight.json", + "sha256": "a0a2864b989fabdb80b8dc5efcd4e301eff362033b3297114300ea6c75c24340", + "size_bytes": 1805 + }, + { + "path": "run-before-cleanup.json", + "sha256": "6f7a2ebb248e16cf0aeb04481ea6a1f9fd141c0044616aefc41c2b4957c5336b", + "size_bytes": 25561 + } + ], + "auth": { + "detail": "the copied auth material was removed from the per-run home and verified gone", + "entries": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login", + "removed_after_run": true, + "status": "OK", + "target": "/home/cycle" + }, + "backend": "vm", + "check": { + "argv": [], + "detail": "not attempted", + "elapsed_seconds": null, + "exit_code": null, + "expected_marker": null, + "finished_at": null, + "started_at": null, + "status": "SKIPPED", + "stderr_path": null, + "stdout_path": null + }, + "cleanup": { + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "domain:dely-cycle-20260916t010000z-bbbbbb-00000014", + "path:/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014" + ], + "retained": [], + "shared_preserved": [ + "image:/var/tmp/dely-cycle/images/tool-image.qcow2" + ], + "status": "DESTROYED", + "verified": true + }, + "config": { + "artifact_root": "/var/tmp/dely-cycle/artifacts", + "auth": { + "allowlist": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login" + }, + "backend": "vm", + "check": { + "expected_marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "claude_code_version": "2.1.272", + "dely_revision": "1b2c1b1e1ff71302d11a0d17cf2f416184e7d077", + "orca": { + "agent": "claude", + "app_argv": [ + "/opt/Orca/orca-ide" + ], + "command": "orca", + "display": ":0", + "effort": "high", + "model": "claude-opus-5", + "ready_timeout_seconds": 420, + "run_objective": "dely minimal cycle", + "status_argv": [ + "orca", + "status", + "--json" + ], + "version": "1.4.201", + "version_argv": [ + "orca", + "--version" + ], + "worktree_selector": "current" + }, + "project": { + "environment_path": "project", + "revision": "1b2c1b1e1ff71302d11a0d17cf2f416184e7d077", + "source": "/var/tmp/dely-cycle/project-source" + }, + "state_root": "/var/tmp/dely-cycle/state", + "task": { + "marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "tested_revision": "1b2c1b1e1ff71302d11a0d17cf2f416184e7d077", + "timeout_seconds": 900, + "vm": { + "address_timeout_seconds": 600, + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_image_sha256": "1f11ab859bf049b3754909f32e09ea3dc1b71b4762ec24406dea661f9bf0123c", + "connect_uri": "qemu:///system", + "egress": true, + "graphics": "vnc", + "guest_user": "cycle", + "listen_address": "127.0.0.1", + "memory_mb": 4096, + "network": "default", + "overlay_size_bytes": 25769803776, + "pool": "dely-cycle", + "provider": "pulumi-libvirt", + "provider_version": "0.5.4", + "provision": [], + "pulumi_binary": "/.pulumi/bin/pulumi", + "qemu_agent": false, + "ssh_port": 22, + "stack_prefix": "dely-cycle", + "transport": "ssh", + "vcpus": 2, + "venv": "/var/tmp/dely-cycle/venv", + "video": "virtio" + } + }, + "dely_revision": "1b2c1b1e1ff71302d11a0d17cf2f416184e7d077", + "elapsed_seconds": 979.258, + "environment": { + "backend": { + "address": "192.168.122.195", + "backend": "vm", + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_image_sha256": "1f11ab859bf049b3754909f32e09ea3dc1b71b4762ec24406dea661f9bf0123c", + "base_volume": "tool-image.qcow2", + "connect_uri": "qemu:///system", + "domain": "dely-cycle-20260916t010000z-bbbbbb-00000014", + "egress_interface": true, + "graphics": "vnc", + "listen_address": "127.0.0.1", + "network": "default", + "pool": "dely-cycle", + "provider": "pulumi-libvirt@0.5.4", + "stack": "dely-cycle-20260916t010000z-bbbbbb-00000014", + "transport_mac": "52:54:00:c7:f2:f7", + "video": "virtio" + }, + "description": { + "address": "192.168.122.195", + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_volume": "tool-image.qcow2", + "connect_uri": "qemu:///system", + "domain": "dely-cycle-20260916t010000z-bbbbbb-00000014", + "egress": true, + "graphics": "vnc", + "network": "default", + "pool": "dely-cycle", + "provider": "pulumi-libvirt@0.5.4", + "stack": "/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/stack", + "transport_mac": "52:54:00:c7:f2:f7", + "video": "virtio" + }, + "environment_id": "dely-cycle-20260916t010000z-bbbbbb-00000014", + "home_path": "/home/cycle", + "per_run_resources": [ + { + "identifier": "dely-cycle-20260916t010000z-bbbbbb-00000014", + "kind": "domain" + }, + { + "identifier": "/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014", + "kind": "path" + }, + { + "identifier": "/var/tmp/dely-cycle/images/dely-cycle-20260916t010000z-bbbbbb-00000014-overlay.qcow2", + "kind": "volume" + }, + { + "identifier": "/var/tmp/dely-cycle/images/dely-cycle-20260916t010000z-bbbbbb-00000014-seed.iso", + "kind": "volume" + } + ], + "project_path": "/home/cycle/project", + "shared_resources": [ + { + "identifier": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "kind": "image" + } + ] + }, + "environment_id": "dely-cycle-20260916t010000z-bbbbbb-00000014", + "export": { + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "b3e1a9da8d4806448f08ecd637daaf22aac7688bafab6e6a98d0b082e2ada44b", + "size_bytes": 291 + }, + { + "path": "backend-status.json", + "sha256": "a15d64923f183348040c6c6f5be92dd387a4ae1c602762a50d2fac0b2ecec1b2", + "size_bytes": 2016 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "5af460083229b542e0c705890c7b0b8516ea6760f103f3f203ab401b63527e5e", + "size_bytes": 1309 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "fcaba634c48816dac5dec694b09cedfe4ce45c632cde34e40d47fa267316ef8b", + "size_bytes": 471 + }, + { + "path": "identity/host-probe.txt", + "sha256": "231d64a3785b25786617993eb8b7525eb51bda26aa49adef773a5a32e989893c", + "size_bytes": 1709 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "fff5639d9009767cc7528b44428da90528be42ad5554665ce15c35ef109729ba", + "size_bytes": 3178 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "f015e2966d1d8caa93c07d655807331d72e88901c19dea814cbd3553cd2966b7", + "size_bytes": 9377 + }, + { + "path": "logs/runner.log", + "sha256": "c0d853c7b5c707c44ac61251eac5f85b9957a406089da5cbcbf891734f40fe83", + "size_bytes": 1537 + }, + { + "path": "preflight.json", + "sha256": "a0a2864b989fabdb80b8dc5efcd4e301eff362033b3297114300ea6c75c24340", + "size_bytes": 1805 + }, + { + "path": "run-before-cleanup.json", + "sha256": "6f7a2ebb248e16cf0aeb04481ea6a1f9fd141c0044616aefc41c2b4957c5336b", + "size_bytes": 25561 + } + ], + "confirmed_at": "2026-09-15T17:26:29Z", + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" + }, + "failure_classification": "the task reached the configured deadline", + "finished_at": "2026-09-15T17:26:36Z", + "generated_at": "2026-09-15T17:26:36Z", + "host_after": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011", + "20260915T230000Z-bbbbbb-00000012", + "20260916T000000Z-bbbbbb-00000013", + "20260916T010000Z-bbbbbb-00000014" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T17:26:36Z", + "extra": { + "backend": "vm" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "host_before": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011", + "20260915T230000Z-bbbbbb-00000012", + "20260916T000000Z-bbbbbb-00000013", + "20260916T010000Z-bbbbbb-00000014" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T17:10:16Z", + "extra": { + "backend": "vm" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "identity": { + "environment": { + "container_marker": "", + "home": "/home/cycle", + "hostname": "dely-cycle-20260916t010000z-bbbbbb-00000014", + "orca_fingerprint": "533628:1592:1789327972", + "orca_path": "/usr/local/bin/orca", + "orca_version": "1.4.201", + "path": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin", + "pid": "1657", + "project_real": "/home/cycle/project", + "uname": "Linux 6.8.0-139-generic x86_64", + "user": "cycle", + "virt": "kvm" + }, + "host": { + "boot_id_digest": "de0d3220ff53399e", + "home_digest": "8fda3821c06d512d", + "hostname_digest": "bbc5e661e106c6dc", + "machine_id_digest": "907fa710d6d1c501", + "orca_fingerprint_digest": "ea841899bf829047", + "orca_path_digest": "6784f600bbc7d6cf", + "orca_present": true, + "project_real_digest": "0d03c9e72dddfdae", + "user_digest": "3cc9cee71b0b967f", + "virt": "none" + }, + "markers": [ + "distinct-machine-identity", + "distinct-boot-identity", + "distinct-hostname", + "reported-virtualisation:kvm" + ], + "orca_is_host_installation": false, + "orca_present": true, + "orca_runtime": { + "app_pid": 1871, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "80cb98d3-d25b-4b4d-bf7f-148ef7386c8e", + "runtime_state": "ready" + }, + "orca_status_exit_code": 0, + "orca_version": "1.4.201", + "reason": "the environment answered with its own identity, home and project copy", + "verdict": "ENVIRONMENT" + }, + "phases": [ + { + "commands": [], + "detail": "", + "elapsed_seconds": 2.371, + "finished_at": "2026-09-15T17:10:19Z", + "name": "prepare", + "started_at": "2026-09-15T17:10:16Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 17.8, + "finished_at": "2026-09-15T17:10:37Z", + "name": "create", + "started_at": "2026-09-15T17:10:19Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "git -C /home/cycle/project init -q -b main" + ], + "context": "environment", + "elapsed_seconds": 0.188, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:37Z", + "started_at": "2026-09-15T17:10:37Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "git -C /home/cycle/project config user.email dely-cycle@invalid" + ], + "context": "environment", + "elapsed_seconds": 0.179, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:37Z", + "started_at": "2026-09-15T17:10:37Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "git -C /home/cycle/project config user.name dely-cycle" + ], + "context": "environment", + "elapsed_seconds": 0.181, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:38Z", + "started_at": "2026-09-15T17:10:37Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "git -C /home/cycle/project add -A" + ], + "context": "environment", + "elapsed_seconds": 0.217, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:38Z", + "started_at": "2026-09-15T17:10:38Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "git -C /home/cycle/project commit -q -m 'dely-cycle: the pinned revision as exported'" + ], + "context": "environment", + "elapsed_seconds": 0.185, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:38Z", + "started_at": "2026-09-15T17:10:38Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "", + "elapsed_seconds": 1.816, + "finished_at": "2026-09-15T17:10:38Z", + "name": "bootstrap", + "started_at": "2026-09-15T17:10:37Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '')\"\norca_release=''\norca_fingerprint=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '%s' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '%i:%s:%Y' \"$orca_resolved\" 2>/dev/null || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id=%s\\n' \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'orca_fingerprint=%s\\n' \"$orca_fingerprint\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/project-source", + "orca" + ], + "context": "host", + "elapsed_seconds": 0.146, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:39Z", + "started_at": "2026-09-15T17:10:38Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "sh -c '\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf '\"'\"''\"'\"'\n else\n printf '\"'\"''\"'\"'\n fi\n}\n\nmarker='\"'\"''\"'\"'\nif [ -f /run/.containerenv ]; then marker='\"'\"'containerenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='\"'\"'dockerenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='\"'\"'toolboxenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='\"'\"'unknown'\"'\"'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '\"'\"''\"'\"')\"\norca_release='\"'\"''\"'\"'\norca_fingerprint='\"'\"''\"'\"'\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '\"'\"''\"'\"')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '\"'\"'%s'\"'\"' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '\"'\"'%i:%s:%Y'\"'\"' \"$orca_resolved\" 2>/dev/null || printf '\"'\"''\"'\"')\"\nfi\n\nproject_resolved='\"'\"''\"'\"'\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '\"'\"''\"'\"')\"\nfi\n\nprintf '\"'\"'hostname=%s\\n'\"'\"' \"$(uname -n 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'machine_id=%s\\n'\"'\"' \"$(first_line /etc/machine-id)\"\nprintf '\"'\"'boot_id=%s\\n'\"'\"' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf '\"'\"'home=%s\\n'\"'\"' \"${HOME:-}\"\nprintf '\"'\"'user=%s\\n'\"'\"' \"$(id -un 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'path=%s\\n'\"'\"' \"${PATH:-}\"\nprintf '\"'\"'pid=%s\\n'\"'\"' \"$$\"\nprintf '\"'\"'uname=%s\\n'\"'\"' \"$(uname -srm 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'container_marker=%s\\n'\"'\"' \"$marker\"\nprintf '\"'\"'virt=%s\\n'\"'\"' \"$virt\"\nprintf '\"'\"'orca_path=%s\\n'\"'\"' \"$orca_binary\"\nprintf '\"'\"'orca_version=%s\\n'\"'\"' \"$orca_release\"\nprintf '\"'\"'orca_fingerprint=%s\\n'\"'\"' \"$orca_fingerprint\"\nprintf '\"'\"'project_real=%s\\n'\"'\"' \"$project_resolved\"\n' cycle-probe /home/cycle/project orca" + ], + "context": "environment", + "elapsed_seconds": 0.368, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:39Z", + "started_at": "2026-09-15T17:10:39Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "orca status --json" + ], + "context": "environment", + "elapsed_seconds": 0.313, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:39Z", + "started_at": "2026-09-15T17:10:39Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "sh -c 'export DISPLAY=\"$1\"; shift; export XDG_RUNTIME_DIR=\"${XDG_RUNTIME_DIR:-/run/user/$(id -u)}\"; waited=0; while ! xset -q >/dev/null 2>&1; do waited=$((waited + 2)); if [ \"$waited\" -ge 120 ]; then printf \"no display after %ss\n\" \"$waited\" >&2; exit 3; fi; sleep 2; done; printf \"display ready after %ss\n\" \"$waited\"; rm -f \"$HOME/.config/orca/SingletonLock\" \"$HOME/.config/orca/SingletonCookie\"; nohup \"$@\" > \"$HOME/orca-app.log\" 2>&1 & printf \"started %s\n\" \"$!\"' orca-start :0 /opt/Orca/orca-ide" + ], + "context": "environment", + "elapsed_seconds": 0.175, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:40Z", + "started_at": "2026-09-15T17:10:39Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "orca runtime 80cb98d3-d25b-4b4d-bf7f-148ef7386c8e is ready and the coordinator terminal is term_388f8f62-dfea-4469-8aa7-1240fead6d28", + "elapsed_seconds": 7.72, + "finished_at": "2026-09-15T17:10:46Z", + "name": "identity", + "started_at": "2026-09-15T17:10:38Z", + "status": "OK" + }, + { + "commands": [], + "detail": "the completion wait reached the run deadline before the worker settled", + "elapsed_seconds": 942.375, + "finished_at": "2026-09-15T17:26:28Z", + "name": "task", + "started_at": "2026-09-15T17:10:46Z", + "status": "TIMEOUT" + }, + { + "commands": [], + "detail": "the task did not settle", + "elapsed_seconds": null, + "finished_at": null, + "name": "check", + "started_at": null, + "status": "SKIPPED" + }, + { + "commands": [], + "detail": "0 patch line(s) collected", + "elapsed_seconds": 0.497, + "finished_at": "2026-09-15T17:26:29Z", + "name": "collect", + "started_at": "2026-09-15T17:26:28Z", + "status": "OK" + }, + { + "commands": [], + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "elapsed_seconds": 0.023, + "finished_at": "2026-09-15T17:26:29Z", + "name": "export", + "started_at": "2026-09-15T17:26:29Z", + "status": "OK" + }, + { + "commands": [], + "detail": "every declared per-run resource is gone and every shared one remains", + "elapsed_seconds": 6.656, + "finished_at": "2026-09-15T17:26:36Z", + "name": "cleanup", + "started_at": "2026-09-15T17:26:29Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": null, + "finished_at": null, + "name": "close", + "started_at": "2026-09-15T17:26:36Z", + "status": "OK" + } + ], + "run_id": "20260916T010000Z-bbbbbb-00000014", + "schema_version": 1, + "started_at": "2026-09-15T17:10:16Z", + "status": "TIMEOUT", + "tested_revision": "1b2c1b1e1ff71302d11a0d17cf2f416184e7d077", + "timeout_seconds": 900, + "versions": { + "configured_revision": "1b2c1b1e1ff71302d11a0d17cf2f416184e7d077", + "distrobox": "distrobox: 1.8.2.5", + "git": "git version 2.55.0", + "podman": "podman version 5.8.4", + "project_revision": "1b2c1b1e1ff71302d11a0d17cf2f416184e7d077", + "pulumi": "v3.262.0", + "python": "3.14.7", + "qemu_img": "Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers", + "runner": "minimal-cycle", + "virsh": "12.0.0" + }, + "worker": { + "agent": "claude", + "commands": [ + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "orca orchestration run-create --objective 'dely minimal cycle' --json --from term_388f8f62-dfea-4469-8aa7-1240fead6d28" + ], + "context": "environment", + "elapsed_seconds": 0.283, + "exit_code": 0, + "finished_at": "2026-09-15T17:10:46Z", + "started_at": "2026-09-15T17:10:46Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "orca orchestration worker-start --spec 'Read the file /home/cycle/dispatch-prompt.md in this environment and do exactly what it says.' --agent claude --model claude-opus-5 --effort high --worktree current --timeout-ms 900000 --json --run run_569a123bb1f3 --from term_388f8f62-dfea-4469-8aa7-1240fead6d28" + ], + "context": "environment", + "elapsed_seconds": 41.696, + "exit_code": 1, + "finished_at": "2026-09-15T17:11:28Z", + "started_at": "2026-09-15T17:10:47Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260916T010000Z-bbbbbb-00000014/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.195", + "--", + "orca orchestration check --wait --types worker_done,escalation,question --timeout-ms 900000 --json --run run_569a123bb1f3 --terminal term_388f8f62-dfea-4469-8aa7-1240fead6d28" + ], + "context": "environment", + "elapsed_seconds": 900.02, + "exit_code": null, + "finished_at": "2026-09-15T17:26:28Z", + "started_at": "2026-09-15T17:11:28Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": true + } + ], + "detail": "the completion wait reached the run deadline before the worker settled", + "dispatch_id": "ctx_2ff0980c3787", + "effort": "high", + "model": "claude-opus-5", + "outcome": "outcome_unknown", + "run_id": "run_569a123bb1f3", + "status": "TIMEOUT" + } +} From e2dd118227321d992deb0080c2c8f398352a6606 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 19:30:44 +0200 Subject: [PATCH 29/42] Provision the environment before anything that needs what provisioning installs. --- experiments/minimal-cycle/counterexamples.py | 12 +++++++ .../minimal-cycle/cycle_runner/lifecycle.py | 6 ++-- .../minimal-cycle/tests/test_lifecycle.py | 35 +++++++++++++++++-- 3 files changed, 49 insertions(+), 4 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 8fe2312..b1bdae9 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -267,6 +267,18 @@ class Counterexample: return value if isinstance(value, str) and value else None""", instruments=("tests.test_worker.RealResponseShapeTest",), ), + Counterexample( + name="provisioning-comes-first", + requirement="Provisioning runs before anything that needs what it installs", + path="cycle_runner/lifecycle.py", + original=""" for argv in self._provision_steps(): + record.commands.append(self.execute(list(argv)).to_record()) + # Orca registers a worktree for a repository; the exported copy is + # not one until this makes it one.""", + replacement=""" # Orca registers a worktree for a repository; the exported copy is + # not one until this makes it one.""", + instruments=("tests.test_lifecycle.BootstrapOrderTest",), + ), Counterexample( name="the-copy-is-a-repository", requirement="The project copy is made into a repository Orca can register", diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index af84395..1445c1d 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -292,6 +292,10 @@ def _bootstrap(self, baseline: Path) -> None: assert self.handle is not None with self.phase("bootstrap") as record: self.adapter.put_tree(baseline, self.handle.project_path) + # Provisioning first: it is what gives the environment the tools the + # rest of the bootstrap needs, git among them. + for argv in self._provision_steps(): + record.commands.append(self.execute(list(argv)).to_record()) # Orca registers a worktree for a repository; the exported copy is # not one until this makes it one. for argv in project.initialise_repository_commands(self.handle.project_path): @@ -305,8 +309,6 @@ def _bootstrap(self, baseline: Path) -> None: ) self.error_reason = self.error_reason or record.detail return - for argv in self._provision_steps(): - record.commands.append(self.execute(list(argv)).to_record()) auth_record, overlay = auth.bootstrap( run_config=self.config, adapter=self.adapter, diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index 856a613..a01bc90 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -61,8 +61,12 @@ def make_config(self, **overrides): document.update(overrides) return config_module.from_document(document) - def run_cycle(self, **adapter_options): - run_config = self.make_config() + def run_cycle(self, config_document=None, **adapter_options): + run_config = ( + config_module.from_document(config_document) + if config_document is not None + else self.make_config() + ) adapter = FakeAdapter( self.state / RUN_ID, host_project=self.repo, **adapter_options ) @@ -432,3 +436,30 @@ def test_the_evidence_is_still_exported_and_the_environment_destroyed(self): def test_a_genuine_refusal_is_still_an_error(self): _, outcome = self.run_cycle(dispatch_state="refused") self.assertEqual(outcome.run_result.status, status.RunStatus.ERROR) + + +class BootstrapOrderTest(CycleTestCase): + """Provisioning is what makes the environment able to do the rest. + + Observed on a real container: the repository initialisation ran first and + failed with `executable file not found`, because git is installed by the + provisioning steps that had not run yet. + """ + + def test_provisioning_runs_before_the_repository_is_initialised(self): + document = self.make_config().to_document() + document["distrobox"]["provision"] = [["mkdir", "-p", "/tmp/provisioned"]] + adapter, outcome = self.run_cycle(config_document=document) + bootstrap = outcome.run_result.phase("bootstrap") + argv = [" ".join(c.argv) for c in bootstrap.commands] + provisioned = next(i for i, line in enumerate(argv) if "provisioned" in line) + initialised = next(i for i, line in enumerate(argv) if "git" in line and "init" in line) + self.assertLess(provisioned, initialised, argv) + + def test_the_project_is_placed_before_either(self): + adapter, _ = self.run_cycle() + self.assertIn("put_tree", adapter.calls) + first_git = next( + i for i, c in enumerate(adapter.calls) if c.startswith("execute:git") + ) + self.assertLess(adapter.calls.index("put_tree"), first_git) From 0ef7cae58c62c6c2292d17a87fe391e1dc3132af Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 19:34:07 +0200 Subject: [PATCH 30/42] Report an Orca refusal by its code and message instead of the tail of its output. --- experiments/minimal-cycle/counterexamples.py | 11 +++++ .../minimal-cycle/cycle_runner/worker.py | 34 ++++++++++++--- .../minimal-cycle/tests/test_worker.py | 43 +++++++++++++++++++ 3 files changed, 83 insertions(+), 5 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index b1bdae9..86adf37 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,17 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="an-error-is-reported-by-its-code", + requirement="A refusal is reported by its code, not by the tail of raw output", + path="cycle_runner/worker.py", + original=""" described = orca_error(_first_document(outcome.stdout)) + if described: + return redact.text(described, secrets)""", + replacement=""" if False: + pass""", + instruments=("tests.test_worker.ErrorReportingTest",), + ), Counterexample( name="the-wait-takes-a-terminal-not-a-sender", requirement="The completion wait names the terminal with the flag it accepts", diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index e53be84..6c47166 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -175,6 +175,32 @@ def dispatch_state(document: Mapping[str, Any]) -> str | None: return value if isinstance(value, str) and value else None +def orca_error(document: Mapping[str, Any]) -> str | None: + """Return the code and message of an Orca reply's error, if it carries one. + + The reply is a document with a structured error at a known place. Showing + the tail of the raw text instead loses the code, which sits near the start + and is the one part worth reading. + """ + error = document.get("error") + if not isinstance(error, Mapping): + return None + code = error.get("code") + message = error.get("message") + if code and message and code != message: + return f"{code}: {message}" + return str(code or message) if (code or message) else None + + +def _explain(outcome, secrets) -> str: + """Describe a failed orca command by its error, falling back to its output.""" + described = orca_error(_first_document(outcome.stdout)) + if described: + return redact.text(described, secrets) + text = (outcome.stderr or outcome.stdout).strip()[-400:] + return redact.text(f"exit={outcome.exit_code}: {text}", secrets) + + def task_identifier(document: Mapping[str, Any]) -> str | None: """Return the Task identifier from a worker-start reply.""" value = _result(document).get("taskId") @@ -230,7 +256,7 @@ def launch( record.status = PhaseStatus.FAILED record.detail = ( "orca orchestration run-create did not return a Run: " - + redact.text((created.stderr or created.stdout).strip()[-400:], secrets) + + _explain(created, secrets) ) return record record.run_id = run_identifier(_first_document(created.stdout)) @@ -273,7 +299,7 @@ def launch( record.status = PhaseStatus.FAILED record.detail = ( f"orca orchestration worker-start reported {state or 'no state'}: " - + redact.text((started.stderr or started.stdout).strip()[-400:], secrets) + + _explain(started, secrets) ) return record elif state: @@ -289,9 +315,7 @@ def launch( return record if not settled.ok: record.status = PhaseStatus.FAILED - record.detail = "the completion wait failed: " + redact.text( - (settled.stderr or settled.stdout).strip()[:400], secrets - ) + record.detail = "the completion wait failed: " + _explain(settled, secrets) return record message = _settling_message(_first_document(settled.stdout)) diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py index 292dae4..bddafdf 100644 --- a/experiments/minimal-cycle/tests/test_worker.py +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -380,3 +380,46 @@ def test_the_wait_still_carries_the_run_and_the_settling_types(self): argv = self.plan().wait_argv self.assertEqual(argv[argv.index("--run") + 1], "run-abcdef") self.assertIn("--types", argv) + + +class ErrorReportingTest(WorkerTest): + """Orca replies carry a structured error; the tail of raw JSON is not it.""" + + def test_the_code_and_message_are_reported(self): + reply = json.dumps({ + "id": "req", "ok": False, + "error": {"code": "consumer_fenced", "message": "the Run is bound elsewhere", + "data": {"originalCommand": ["orca", "orchestration", "run-create"]}}, + }) + self.assertEqual( + worker.orca_error(json.loads(reply)), + "consumer_fenced: the Run is bound elsewhere", + ) + + def test_a_reply_without_an_error_reports_nothing(self): + self.assertIsNone(worker.orca_error({"ok": True, "result": {}})) + + def test_a_failed_run_create_names_the_code(self): + reply = json.dumps({ + "ok": False, + "error": {"code": "selector_not_found", "message": "no such worktree"}, + }) + script = [("run-create", 1, reply, "", False)] + _, _, record = self.launch(script=script) + self.assertIn("selector_not_found", record.detail) + self.assertIn("no such worktree", record.detail) + # The point is that it reads as a sentence, not as a slice of a + # document: the raw reply also contains the code, further in. + self.assertNotIn("{", record.detail) + self.assertLess(len(record.detail), 160, record.detail) + + def test_a_failed_start_names_the_code(self): + reply = json.dumps({ + "ok": False, + "error": {"code": "consumer_fenced", "message": "bound elsewhere"}, + "result": {"state": "failed"}, + }) + script = [("run-create", 0, READY, "", False), ("worker-start", 1, reply, "", False)] + _, _, record = self.launch(script=script) + self.assertIn("consumer_fenced", record.detail) + self.assertNotIn("{", record.detail) From d54307995cc5ea378c7df201f4282ea480a5c0c1 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 19:39:33 +0200 Subject: [PATCH 31/42] Strip the host's own session identity before running anything inside an environment; a container inherited it and attested as the host's terminal. --- experiments/minimal-cycle/counterexamples.py | 8 ++ .../cycle_runner/adapters/distrobox.py | 6 +- .../minimal-cycle/cycle_runner/adapters/vm.py | 5 +- .../minimal-cycle/cycle_runner/isolate.py | 39 +++++++ .../minimal-cycle/tests/test_adapter_vm.py | 8 +- .../minimal-cycle/tests/test_isolate.py | 110 ++++++++++++++++++ 6 files changed, 171 insertions(+), 5 deletions(-) create mode 100644 experiments/minimal-cycle/cycle_runner/isolate.py create mode 100644 experiments/minimal-cycle/tests/test_isolate.py diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 86adf37..946cf88 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -229,6 +229,14 @@ class Counterexample: replacement=" planned = None", instruments=("tests.test_lifecycle.CreateFailureTest",), ), + Counterexample( + name="the-host-session-does-not-leak", + requirement="The host's own session identity is removed before a command runs inside", + path="cycle_runner/isolate.py", + original=""" return ["sh", "-c", _SCRIPT, "cycle-isolate", "--", *[str(item) for item in argv]]""", + replacement=""" return [str(item) for item in argv]""", + instruments=("tests.test_isolate",), + ), Counterexample( name="an-error-is-reported-by-its-code", requirement="A refusal is reported by its code, not by the tail of raw output", diff --git a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py index f6b55b7..789403a 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py @@ -18,7 +18,7 @@ from pathlib import Path from typing import Callable, Mapping, Sequence -from .. import cleanup, ids, proc +from .. import cleanup, ids, isolate, proc from ..config import RunConfig from .base import ( BackendAdapter, @@ -308,6 +308,10 @@ def execute( inside = list(argv) if cwd: inside = ["sh", "-c", 'cd "$1" || exit 1; shift; exec "$@"', "cycle-cd", cwd, *inside] + # The box inherits this process's environment, and this process may be + # running inside the execution plane the box is meant to be separate + # from. + inside = isolate.without_host_session(inside) return self.runner( self.enter_argv(inside, env_names=tuple(overlay)), timeout=timeout, diff --git a/experiments/minimal-cycle/cycle_runner/adapters/vm.py b/experiments/minimal-cycle/cycle_runner/adapters/vm.py index c365318..c515bda 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/vm.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/vm.py @@ -23,7 +23,7 @@ from pathlib import Path from typing import Callable, Mapping, Sequence -from .. import cleanup, ids, proc +from .. import cleanup, ids, isolate, proc from ..config import RunConfig from . import schema from .base import ( @@ -764,6 +764,9 @@ def execute( inside = ["env", *[f"{name}={value}" for name, value in env.items()], *inside] if cwd: inside = ["sh", "-c", 'cd "$1" || exit 1; shift; exec "$@"', "cycle-cd", cwd, *inside] + # The transport does not forward this process's environment today, but + # nothing about the guest should depend on that staying true. + inside = isolate.without_host_session(inside) return self.runner( self.ssh_argv(inside), timeout=timeout, diff --git a/experiments/minimal-cycle/cycle_runner/isolate.py b/experiments/minimal-cycle/cycle_runner/isolate.py new file mode 100644 index 0000000..943c28d --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/isolate.py @@ -0,0 +1,39 @@ +"""Keeping the host's own session out of the environment. + +This runner is usually launched from inside the very execution plane it is +testing, and that plane identifies its terminals through the environment. A +backend that inherits the host's environment therefore hands the environment +the host's session identity — and, in Orca's case, a token with it. + +The effect is not subtle: the environment's own command line attests as the +host's terminal and refuses to act, which is the plane correctly noticing that +two different things claim to be the same session. +""" + +from __future__ import annotations + +from typing import Sequence + +#: Variables whose names begin with these belong to the host's session, not to +#: the environment's. They are removed before any command runs inside one. +LEAKING_PREFIXES = ("ORCA_",) + +#: Unsets by prefix rather than by name: the plane may add a variable tomorrow, +#: and a list of names would silently stop covering it. +_SCRIPT = ( + 'for prefix in "$@"; do ' + ' case "$prefix" in --) shift; break;; esac; ' + 'done; ' + 'for name in $(env | sed -n "s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p"); do ' + ' case "$name" in ' + + "|".join(f"{prefix}*" for prefix in LEAKING_PREFIXES) + + ') unset "$name";; ' + ' esac; ' + 'done; ' + 'exec "$@"' +) + + +def without_host_session(argv: Sequence[str]) -> list[str]: + """Return an argv that runs the same command with the host's session removed.""" + return ["sh", "-c", _SCRIPT, "cycle-isolate", "--", *[str(item) for item in argv]] diff --git a/experiments/minimal-cycle/tests/test_adapter_vm.py b/experiments/minimal-cycle/tests/test_adapter_vm.py index 6ae21a9..559a9be 100644 --- a/experiments/minimal-cycle/tests/test_adapter_vm.py +++ b/experiments/minimal-cycle/tests/test_adapter_vm.py @@ -433,10 +433,12 @@ def test_the_probe_argument_reaches_the_guest_intact(self): from cycle_runner import probe import shlex - argv = self.ready().ssh_argv(probe.probe_argv("/home/cycle/project")) + argv = self.ready().ssh_argv(probe.probe_argv("/home/cycle/project", "orca")) parsed = shlex.split(argv[-1]) - self.assertEqual(parsed[-1], "/home/cycle/project") - self.assertEqual(parsed[2], probe.PROBE_SCRIPT) + # The probe takes the project path and then the command to look for. + self.assertIn("/home/cycle/project", parsed) + self.assertEqual(parsed[-1], "orca") + self.assertIn(probe.PROBE_SCRIPT, parsed) def test_a_value_carrying_a_quote_cannot_break_out(self): import shlex diff --git a/experiments/minimal-cycle/tests/test_isolate.py b/experiments/minimal-cycle/tests/test_isolate.py new file mode 100644 index 0000000..9d23547 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_isolate.py @@ -0,0 +1,110 @@ +"""The host's own session must not reach into the environment. + +Observed on a real container: this runner is itself launched from an Orca +terminal, which exports ORCA_TERMINAL_HANDLE and ORCA_AGENT_HOOK_TOKEN among +others. Distrobox inherits the host environment, so the container's Orca +attested as the *host's* terminal and refused the run with +`consumer_fenced: This terminal is attested as … and cannot act as …`. + +That is an identifier of the host leaking into the environment, and one of the +leaked values is a token. +""" + +import unittest + +from cycle_runner import isolate, proc + + +class ScrubbedArgvTest(unittest.TestCase): + def test_the_original_command_still_runs(self): + outcome = proc.run( + isolate.without_host_session(["printf", "%s", "ran"]), + timeout=30, context="host", + ) + self.assertEqual(outcome.exit_code, 0) + self.assertEqual(outcome.stdout, "ran") + + def test_a_leaking_variable_does_not_reach_the_command(self): + outcome = proc.run( + isolate.without_host_session(["sh", "-c", 'echo "[${ORCA_TERMINAL_HANDLE:-gone}]"']), + timeout=30, context="host", + env={"ORCA_TERMINAL_HANDLE": "term_from_the_host"}, + ) + self.assertIn("[gone]", outcome.stdout) + self.assertNotIn("term_from_the_host", outcome.stdout) + + def test_every_leaking_variable_goes_not_just_the_one_we_named(self): + outcome = proc.run( + isolate.without_host_session(["sh", "-c", "env | grep -c '^ORCA_' || true"]), + timeout=30, context="host", + env={ + "ORCA_TERMINAL_HANDLE": "a", + "ORCA_AGENT_HOOK_TOKEN": "b", + "ORCA_WORKSPACE_ID": "c", + }, + ) + self.assertEqual(outcome.stdout.strip(), "0") + + def test_what_the_run_deliberately_passes_survives(self): + outcome = proc.run( + isolate.without_host_session(["sh", "-c", 'echo "[${CLAUDE_CODE_OAUTH_TOKEN:-gone}]"']), + timeout=30, context="host", + env={"CLAUDE_CODE_OAUTH_TOKEN": "kept", "ORCA_TERMINAL_HANDLE": "a"}, + ) + self.assertIn("[kept]", outcome.stdout) + + def test_arguments_with_spaces_and_quotes_survive(self): + outcome = proc.run( + isolate.without_host_session(["printf", "%s|%s", "two words", "it's"]), + timeout=30, context="host", + ) + self.assertEqual(outcome.stdout, "two words|it's") + + def test_the_prefixes_are_named_and_include_the_session_handle(self): + self.assertIn("ORCA_", isolate.LEAKING_PREFIXES) + + +class AdaptersScrubTest(unittest.TestCase): + """Both backends run their commands with the host's session removed.""" + + def test_the_container_backend_scrubs(self): + import tempfile + from pathlib import Path + + from cycle_runner import config as config_module + from cycle_runner.adapters import distrobox + from tests.test_adapter_distrobox import StubRunner + from tests.test_config import minimal_document + + with tempfile.TemporaryDirectory() as root: + document = minimal_document() + document["state_root"] = str(Path(root) / "state") + document["artifact_root"] = str(Path(root) / "artifacts") + runner = StubRunner() + adapter = distrobox.DistroboxAdapter( + run_config=config_module.from_document(document), + run_id="20260914T221530Z-abc123-0123abcd", + host_home=Path(root) / "hosthome", + runner=runner, + which=lambda name: f"/usr/bin/{name}", + ) + adapter.execute(["true"], timeout=5) + self.assertIn("cycle-isolate", runner.seen[-1]) + + def test_the_machine_backend_scrubs(self): + import tempfile + from pathlib import Path + + from tests.test_adapter_vm import VmTestCase + + case = VmTestCase("run") + case.setUp() + try: + adapter = case.make() + adapter.prepare_identity() + adapter.address = "192.168.122.11" + adapter.execute(["true"], timeout=5) + # The transport sends one quoted string, so the marker is inside it. + self.assertIn("cycle-isolate", " ".join(adapter.runner.seen[-1])) + finally: + case.doCleanups() From b0fab723003a6f39ed4ba0556b77a6500e07a83a Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 19:55:01 +0200 Subject: [PATCH 32/42] Record one full container cycle on a display of its own, reaching where the machine backend reaches. --- experiments/minimal-cycle/counterexamples.py | 10 +- .../evidence/counterexamples.txt | 63 +- .../evidence/distrobox-full-cycle/README.md | 45 + .../distrobox-full-cycle/auth-receipt.json | 11 + .../distrobox-full-cycle/cleanup.json | 15 + .../distrobox-full-cycle/export-receipt.json | 71 ++ .../identity-environment-probe.txt | 14 + .../identity-host-probe.txt | 14 + .../identity-orca-runtime.json | 90 ++ .../distrobox-full-cycle/manifest.json | 1104 +++++++++++++++++ .../distrobox-full-cycle/preflight.json | 36 + .../minimal-cycle/evidence/preflight-vm.txt | 2 +- .../minimal-cycle/tests/test_isolate.py | 2 +- 13 files changed, 1446 insertions(+), 31 deletions(-) create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/README.md create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/auth-receipt.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/cleanup.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/export-receipt.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-environment-probe.txt create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-host-probe.txt create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-orca-runtime.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/manifest.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-full-cycle/preflight.json diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 946cf88..d8793b0 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -354,11 +354,13 @@ class Counterexample: requirement="Every orchestration command names the terminal it is sent from", path="cycle_runner/worker.py", original=""" if coordinator_handle: - for command in (created, start, wait): - command.extend(["--from", coordinator_handle])""", + for command in (created, start): + command.extend(["--from", coordinator_handle]) + wait.extend(["--terminal", coordinator_handle])""", replacement=""" if False: - for command in (created, start, wait): - command.extend(["--from", coordinator_handle])""", + for command in (created, start): + command.extend(["--from", coordinator_handle]) + wait.extend(["--terminal", coordinator_handle])""", instruments=("tests.test_worker.CoordinatorTerminalTest",), ), Counterexample( diff --git a/experiments/minimal-cycle/evidence/counterexamples.txt b/experiments/minimal-cycle/evidence/counterexamples.txt index fcea9c6..33035c6 100644 --- a/experiments/minimal-cycle/evidence/counterexamples.txt +++ b/experiments/minimal-cycle/evidence/counterexamples.txt @@ -1,26 +1,39 @@ -RED no-host-fallback A probe result equal to the host's own snapshot is rejected -RED orca-is-the-environments-own An orca that resolves to the host's installation is not present -RED export-before-destroy Cleanup runs only after a confirmed export -RED stop-must-be-confirmed Cleanup runs only after a confirmed stop -RED destroy-after-export Evidence is collected and exported before anything is destroyed -RED receipt-is-a-re-read The receipt proves bytes on the host, not bytes in memory -RED cleanup-only-per-run A per-run path that contains a shared resource is refused -RED redaction-by-shape Redaction catches a credential this run has never seen -RED manifest-required-fields The manifest carries every required field on every path -RED run-id-carries-no-name The run identifier discloses neither the host name nor the auth reference -RED config-refuses-a-secret A secret-shaped value under a dull key is refused -RED auth-leaves-no-material No credential value, length or prefix reaches a receipt -RED host-home-mount-acknowledged An unacknowledged host-home mount blocks the run -RED preflight-leaves-no-residue Preflight inspects without creating the run's state -RED base-is-only-a-backing-file The preserved base image is a backing volume, never a per-run one -RED overlay-is-not-the-base The rendered program creates an overlay, not a volume named like the base -RED failed-create-names-its-residue A create that fails partway records what it may have left behind -RED an-address-is-not-readiness Creation waits for the guest to answer, not just to take an address -RED remote-command-is-quoted An argument vector survives the transport intact -RED forwarded-value-is-redacted A value forwarded into the guest is redacted from captured output -RED provider-schema-verified A program the provider cannot be checked against blocks the run -RED state-backend-is-local A cloud state backend blocks the machine backend -RED graphics-type-is-supported A graphics type this emulator lacks blocks the run -RED transport-address-is-the-right-one Another interface's address is not mistaken for the transport's +RED no-host-fallback A probe result equal to the host's own snapshot is rejected +RED orca-is-the-environments-own An orca that resolves to the host's installation is not present +RED export-before-destroy Cleanup runs only after a confirmed export +RED stop-must-be-confirmed Cleanup runs only after a confirmed stop +RED destroy-after-export Evidence is collected and exported before anything is destroyed +RED receipt-is-a-re-read The receipt proves bytes on the host, not bytes in memory +RED cleanup-only-per-run A per-run path that contains a shared resource is refused +RED redaction-by-shape Redaction catches a credential this run has never seen +RED manifest-required-fields The manifest carries every required field on every path +RED run-id-carries-no-name The run identifier discloses neither the host name nor the auth reference +RED config-refuses-a-secret A secret-shaped value under a dull key is refused +RED auth-leaves-no-material No credential value, length or prefix reaches a receipt +RED host-home-mount-acknowledged An unacknowledged host-home mount blocks the run +RED preflight-leaves-no-residue Preflight inspects without creating the run's state +RED base-is-only-a-backing-file The preserved base image is a backing volume, never a per-run one +RED overlay-is-not-the-base The rendered program creates an overlay, not a volume named like the base +RED failed-create-names-its-residue A create that fails partway records what it may have left behind +RED the-host-session-does-not-leak The host's own session identity is removed before a command runs inside +RED an-error-is-reported-by-its-code A refusal is reported by its code, not by the tail of raw output +RED the-wait-takes-a-terminal-not-a-sender The completion wait names the terminal with the flag it accepts +RED an-unobserved-start-still-waits A dispatch the plane could not observe is still waited for +RED unverifiable-is-not-failed A dispatch the plane cannot judge settles unknown, not error +RED the-request-is-not-the-run The Run identifier is read from the Run, not from the request +RED provisioning-comes-first Provisioning runs before anything that needs what it installs +RED the-copy-is-a-repository The project copy is made into a repository Orca can register +RED metadata-is-not-the-change Repository metadata stays out of the patch +RED the-same-path-is-not-the-same-file An environment with its own Orca at the host's path is not the host's +RED the-application-is-started The runner starts the application rather than hoping something did +RED a-present-orca-is-not-a-ready-one A dispatch waits for Orca's runtime, not just for its binary +RED dispatch-needs-a-sender-terminal Every orchestration command names the terminal it is sent from +RED an-address-is-not-readiness Creation waits for the guest to answer, not just to take an address +RED remote-command-is-quoted An argument vector survives the transport intact +RED forwarded-value-is-redacted A value forwarded into the guest is redacted from captured output +RED provider-schema-verified A program the provider cannot be checked against blocks the run +RED state-backend-is-local A cloud state backend blocks the machine backend +RED graphics-type-is-supported A graphics type this emulator lacks blocks the run +RED transport-address-is-the-right-one Another interface's address is not mistaken for the transport's -all 24 counterexamples discriminated +all 37 counterexamples discriminated diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/README.md b/experiments/minimal-cycle/evidence/distrobox-full-cycle/README.md new file mode 100644 index 0000000..4853404 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/README.md @@ -0,0 +1,45 @@ +# One full cycle on the container backend + +A fresh Distrobox box from `docker.io/library/ubuntu:24.04`, provisioned with +the pinned Orca package, the pinned Claude Code and a display of its own, then +driven through the same lifecycle the machine backend uses. + + status: TIMEOUT + identity: ENVIRONMENT markers: container-marker:containerenv, distinct-hostname, reported-virtualisation:docker + orca: present, and not the host's installation + runtime: ready, application running, window available + runtime id da12c0a0-3b88-4c6d-b416-c54e0cb06609 + run: run_cf3d25716c29 + dispatch: ctx_230b27b63cc9 (claude-opus-5 at high) + auth: existing_login, copied and removed again, verified + export: CONFIRMED + cleanup: DESTROYED + +## Nothing rendered on the host's desktop + +Distrobox shares the host's display, so an application started in a box appears +on the host's own screen. This run gave the box a display of its own instead — +a virtual framebuffer on `:99`, provisioned into the container — and pointed +Orca at it. The host's desktop was untouched. + +## Two things this run found + +The container inherits the host's environment, and this runner is itself +launched from inside an Orca terminal. The box's own Orca therefore attested as +the *host's* terminal and refused with +`consumer_fenced: This terminal is attested as … and cannot act as …`. The +runner now removes the host's session variables before any command runs inside +an environment. The machine backend never showed this, because its transport +does not forward the environment — which is luck, not a property worth relying +on. + +Provisioning also has to come before anything that needs what it installs: the +repository initialisation ran first and failed with +`executable file not found`, because git arrives with the provisioning steps. + +## Where it stops + +Exactly where the machine backend stops: the dispatch exists, the worker has a +terminal of its own, and its turn never begins. The deadline fired, the +evidence was exported and confirmed, and the box and its per-run home were +removed. See `../vm-two-sessions/` for what the worker's terminal holds. diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/auth-receipt.json b/experiments/minimal-cycle/evidence/distrobox-full-cycle/auth-receipt.json new file mode 100644 index 0000000..1d64360 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/auth-receipt.json @@ -0,0 +1,11 @@ +{ + "detail": "1 allowlisted entry(s) copied into the per-run home with owner-only permissions", + "entries": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login", + "removed_after_run": false, + "status": "OK", + "target": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home" +} diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/cleanup.json b/experiments/minimal-cycle/evidence/distrobox-full-cycle/cleanup.json new file mode 100644 index 0000000..6ff7c21 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/cleanup.json @@ -0,0 +1,15 @@ +{ + "host_difference": {}, + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "container:dely-cycle-20260916t050000z-cccccc-00000004", + "path:/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004" + ], + "retained": [], + "shared_preserved": [ + "image:docker.io/library/ubuntu:24.04" + ], + "status": "DESTROYED", + "stop_confirmed": true, + "verified": true +} diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/export-receipt.json b/experiments/minimal-cycle/evidence/distrobox-full-cycle/export-receipt.json new file mode 100644 index 0000000..8d69a34 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/export-receipt.json @@ -0,0 +1,71 @@ +{ + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "80bf4a943b67cfbe35040de1df75b34eb5940dd2c5cbfe36bb2b30d8ebc0c6b0", + "size_bytes": 343 + }, + { + "path": "backend-status.json", + "sha256": "f6ebf7d652869bbb7575ae445da97a86ee8934c464c40bd717e2ba3b9300ae50", + "size_bytes": 1374 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "def0a0fc7015f78c2c1a97bb8b6366725cb1a135da8448f0e04e3135cea732e0", + "size_bytes": 1484 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "aaf23172fa73f9f0511abaf9cc94a6fb9e5c3bf4b91a55b88e10a2dea78d053b", + "size_bytes": 1885 + }, + { + "path": "identity/host-probe.txt", + "sha256": "eb244182e0dba50b6f18878f67acdadf2cc612633b737ad8ff7a8f1e99635c26", + "size_bytes": 1667 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "26c1c86ee95009e4ca5d2e8cf07b867d4923a6825247f973eeae381df5fa7964", + "size_bytes": 3180 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "326bbafec55b720b6b53e92e18289221e8b0f88befdc4ef36fa1406fceea7fb8", + "size_bytes": 14009 + }, + { + "path": "logs/runner.log", + "sha256": "178c28f84a946bff91936654b69cca0552153715c764ba8a194960cc21ddba43", + "size_bytes": 1957 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "07b42d1aee28872e2f6efc6885b9b583b1cb7b1e5790b3473e97b947a0dda3f2", + "size_bytes": 33186 + } + ], + "confirmed_at": "2026-09-15T17:49:46Z", + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" +} diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-environment-probe.txt b/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-environment-probe.txt new file mode 100644 index 0000000..66a5e10 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-environment-probe.txt @@ -0,0 +1,14 @@ +hostname=dely-cycle-20260916t050000z-cccccc-00000004 +machine_id= +boot_id= +home=/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin +pid=727360 +uname=Linux 7.2.5-200.fc44.x86_64 x86_64 +container_marker=containerenv +virt=docker +orca_path=/opt/Orca/resources/bin/orca-ide +orca_version=1.4.201 +orca_fingerprint=100579893:1592:1789327972 +project_real=/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-host-probe.txt b/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-host-probe.txt new file mode 100644 index 0000000..61dc3b3 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-host-probe.txt @@ -0,0 +1,14 @@ +hostname=system +machine_id= +boot_id= +home= +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin +pid=727224 +uname=Linux 7.2.5-200.fc44.x86_64 x86_64 +container_marker= +virt=none +orca_path=/opt/Orca/resources/bin/orca-ide +orca_version=1.4.201 +orca_fingerprint=7199608:1592:1789327972 +project_real=/var/tmp/dely-cycle/project-source diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-orca-runtime.json b/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-orca-runtime.json new file mode 100644 index 0000000..bcad140 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/identity-orca-runtime.json @@ -0,0 +1,90 @@ +{ + "app_pid": 727805, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "da12c0a0-3b88-4c6d-b416-c54e0cb06609", + "runtime_state": "ready" +} diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/manifest.json b/experiments/minimal-cycle/evidence/distrobox-full-cycle/manifest.json new file mode 100644 index 0000000..237e9fa --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/manifest.json @@ -0,0 +1,1104 @@ +{ + "artifact_root": "/var/tmp/dely-cycle/artifacts/20260916T050000Z-cccccc-00000004", + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "80bf4a943b67cfbe35040de1df75b34eb5940dd2c5cbfe36bb2b30d8ebc0c6b0", + "size_bytes": 343 + }, + { + "path": "backend-status.json", + "sha256": "f6ebf7d652869bbb7575ae445da97a86ee8934c464c40bd717e2ba3b9300ae50", + "size_bytes": 1374 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "def0a0fc7015f78c2c1a97bb8b6366725cb1a135da8448f0e04e3135cea732e0", + "size_bytes": 1484 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "aaf23172fa73f9f0511abaf9cc94a6fb9e5c3bf4b91a55b88e10a2dea78d053b", + "size_bytes": 1885 + }, + { + "path": "identity/host-probe.txt", + "sha256": "eb244182e0dba50b6f18878f67acdadf2cc612633b737ad8ff7a8f1e99635c26", + "size_bytes": 1667 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "26c1c86ee95009e4ca5d2e8cf07b867d4923a6825247f973eeae381df5fa7964", + "size_bytes": 3180 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "326bbafec55b720b6b53e92e18289221e8b0f88befdc4ef36fa1406fceea7fb8", + "size_bytes": 14009 + }, + { + "path": "logs/runner.log", + "sha256": "178c28f84a946bff91936654b69cca0552153715c764ba8a194960cc21ddba43", + "size_bytes": 1957 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "07b42d1aee28872e2f6efc6885b9b583b1cb7b1e5790b3473e97b947a0dda3f2", + "size_bytes": 33186 + } + ], + "auth": { + "detail": "the copied auth material was removed from the per-run home and verified gone", + "entries": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login", + "removed_after_run": true, + "status": "OK", + "target": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home" + }, + "backend": "distrobox", + "check": { + "argv": [], + "detail": "not attempted", + "elapsed_seconds": null, + "exit_code": null, + "expected_marker": null, + "finished_at": null, + "started_at": null, + "status": "SKIPPED", + "stderr_path": null, + "stdout_path": null + }, + "cleanup": { + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "container:dely-cycle-20260916t050000z-cccccc-00000004", + "path:/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004" + ], + "retained": [], + "shared_preserved": [ + "image:docker.io/library/ubuntu:24.04" + ], + "status": "DESTROYED", + "verified": true + }, + "config": { + "artifact_root": "/var/tmp/dely-cycle/artifacts", + "auth": { + "allowlist": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login" + }, + "backend": "distrobox", + "check": { + "expected_marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "claude_code_version": "2.1.272", + "dely_revision": "d54307995cc5ea378c7df201f4282ea480a5c0c1", + "distrobox": { + "accept_host_home_mount": true, + "container_prefix": "dely-cycle", + "extra_mounts": [], + "image": "docker.io/library/ubuntu:24.04", + "provision": [ + [ + "sh", + "-c", + "set -eu; export DEBIAN_FRONTEND=noninteractive; sudo apt-get update -qq; sudo apt-get install -y -qq curl ca-certificates git nodejs npm xvfb x11-xserver-utils xdotool libgtk-3-0t64 libnss3 libasound2t64 libatk-bridge2.0-0t64 libcups2t64 libdrm2 libgbm1 libxkbcommon0 libpango-1.0-0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libsecret-1-0 xdg-utils > /dev/null; echo deps installed" + ], + [ + "sh", + "-c", + "set -eu; curl -fsSL -o /tmp/orca.deb 'https://github.com/stablyai/orca/releases/download/v1.4.201/orca-ide_1.4.201_amd64.deb'; echo '9dab79a5223837fc91992e04ed54f92f45d6f4b486d7128c08166bfb1a15a573 /tmp/orca.deb' | sha256sum -c -; sudo apt-get install -y -qq /tmp/orca.deb > /dev/null; rm -f /tmp/orca.deb; test -x /opt/Orca/resources/bin/orca-ide; echo orca installed" + ], + [ + "sh", + "-c", + "set -eu; sudo npm install -g --no-fund --no-audit --silent '@anthropic-ai/claude-code@2.1.272' > /dev/null; claude --version" + ], + [ + "sh", + "-c", + "set -eu; if DISPLAY=:99 xset -q > /dev/null 2>&1; then echo 'virtual display already up'; exit 0; fi; setsid nohup Xvfb :99 -screen 0 1280x800x24 -nolisten tcp > /tmp/xvfb.log 2>&1 < /dev/null & waited=0; while ! DISPLAY=:99 xset -q > /dev/null 2>&1; do waited=$((waited + 1)); if [ $waited -ge 60 ]; then echo 'no virtual display' >&2; exit 3; fi; sleep 1; done; echo \"virtual display ready after ${waited}s\"" + ] + ] + }, + "orca": { + "agent": "claude", + "app_argv": [ + "/opt/Orca/orca-ide", + "--disable-gpu" + ], + "command": "/opt/Orca/resources/bin/orca-ide", + "display": ":99", + "effort": "high", + "model": "claude-opus-5", + "ready_timeout_seconds": 300, + "run_objective": "dely minimal cycle", + "status_argv": [ + "/opt/Orca/resources/bin/orca-ide", + "status", + "--json" + ], + "version": "1.4.201", + "version_argv": [ + "/opt/Orca/resources/bin/orca-ide", + "--version" + ], + "worktree_selector": "current" + }, + "project": { + "environment_path": "project", + "revision": "d54307995cc5ea378c7df201f4282ea480a5c0c1", + "source": "/var/tmp/dely-cycle/project-source" + }, + "state_root": "/var/tmp/dely-cycle/state", + "task": { + "marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "tested_revision": "d54307995cc5ea378c7df201f4282ea480a5c0c1", + "timeout_seconds": 420 + }, + "dely_revision": "d54307995cc5ea378c7df201f4282ea480a5c0c1", + "elapsed_seconds": 610.003, + "environment": { + "backend": { + "backend": "distrobox", + "container": "dely-cycle-20260916t050000z-cccccc-00000004", + "container_manager": "podman", + "distrobox_version": "distrobox: 1.8.2.5", + "image": "docker.io/library/ubuntu:24.04", + "manifest": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/distrobox.ini" + }, + "description": { + "container_manager": "podman", + "desktop_integration": "Distrobox integrates with the host desktop by design; this run limits state with a separate home and project copy, and does not claim container-grade isolation", + "host_home_mount_acknowledged": true, + "image": "docker.io/library/ubuntu:24.04", + "manifest": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/distrobox.ini" + }, + "environment_id": "dely-cycle-20260916t050000z-cccccc-00000004", + "home_path": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home", + "per_run_resources": [ + { + "identifier": "dely-cycle-20260916t050000z-cccccc-00000004", + "kind": "container" + }, + { + "identifier": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004", + "kind": "path" + } + ], + "project_path": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project", + "shared_resources": [ + { + "identifier": "docker.io/library/ubuntu:24.04", + "kind": "image" + } + ] + }, + "environment_id": "dely-cycle-20260916t050000z-cccccc-00000004", + "export": { + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "80bf4a943b67cfbe35040de1df75b34eb5940dd2c5cbfe36bb2b30d8ebc0c6b0", + "size_bytes": 343 + }, + { + "path": "backend-status.json", + "sha256": "f6ebf7d652869bbb7575ae445da97a86ee8934c464c40bd717e2ba3b9300ae50", + "size_bytes": 1374 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "host-before.json", + "sha256": "def0a0fc7015f78c2c1a97bb8b6366725cb1a135da8448f0e04e3135cea732e0", + "size_bytes": 1484 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "aaf23172fa73f9f0511abaf9cc94a6fb9e5c3bf4b91a55b88e10a2dea78d053b", + "size_bytes": 1885 + }, + { + "path": "identity/host-probe.txt", + "sha256": "eb244182e0dba50b6f18878f67acdadf2cc612633b737ad8ff7a8f1e99635c26", + "size_bytes": 1667 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "26c1c86ee95009e4ca5d2e8cf07b867d4923a6825247f973eeae381df5fa7964", + "size_bytes": 3180 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "326bbafec55b720b6b53e92e18289221e8b0f88befdc4ef36fa1406fceea7fb8", + "size_bytes": 14009 + }, + { + "path": "logs/runner.log", + "sha256": "178c28f84a946bff91936654b69cca0552153715c764ba8a194960cc21ddba43", + "size_bytes": 1957 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "07b42d1aee28872e2f6efc6885b9b583b1cb7b1e5790b3473e97b947a0dda3f2", + "size_bytes": 33186 + } + ], + "confirmed_at": "2026-09-15T17:49:46Z", + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" + }, + "failure_classification": "the task reached the configured deadline", + "finished_at": "2026-09-15T17:49:50Z", + "generated_at": "2026-09-15T17:49:50Z", + "host_after": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011", + "20260915T230000Z-bbbbbb-00000012", + "20260916T000000Z-bbbbbb-00000013", + "20260916T010000Z-bbbbbb-00000014", + "20260916T020000Z-cccccc-00000001", + "20260916T030000Z-cccccc-00000002", + "20260916T040000Z-cccccc-00000003", + "20260916T050000Z-cccccc-00000004" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T17:49:50Z", + "extra": { + "backend": "distrobox" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "host_before": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011", + "20260915T230000Z-bbbbbb-00000012", + "20260916T000000Z-bbbbbb-00000013", + "20260916T010000Z-bbbbbb-00000014", + "20260916T020000Z-cccccc-00000001", + "20260916T030000Z-cccccc-00000002", + "20260916T040000Z-cccccc-00000003", + "20260916T050000Z-cccccc-00000004" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T17:39:40Z", + "extra": { + "backend": "distrobox" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "identity": { + "environment": { + "container_marker": "containerenv", + "home": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home", + "hostname": "dely-cycle-20260916t050000z-cccccc-00000004", + "orca_fingerprint": "100579893:1592:1789327972", + "orca_path": "/opt/Orca/resources/bin/orca-ide", + "orca_version": "1.4.201", + "path": "/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin", + "pid": "727360", + "project_real": "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project", + "uname": "Linux 7.2.5-200.fc44.x86_64 x86_64", + "user": "", + "virt": "docker" + }, + "host": { + "boot_id_digest": "de0d3220ff53399e", + "home_digest": "8fda3821c06d512d", + "hostname_digest": "bbc5e661e106c6dc", + "machine_id_digest": "907fa710d6d1c501", + "orca_fingerprint_digest": "5d53db3b52dba2dc", + "orca_path_digest": "7db47dea0d8ebe1f", + "orca_present": true, + "project_real_digest": "0d03c9e72dddfdae", + "user_digest": "3cc9cee71b0b967f", + "virt": "none" + }, + "markers": [ + "container-marker:containerenv", + "distinct-hostname", + "reported-virtualisation:docker" + ], + "orca_is_host_installation": false, + "orca_present": true, + "orca_runtime": { + "app_pid": 727805, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "da12c0a0-3b88-4c6d-b416-c54e0cb06609", + "runtime_state": "ready" + }, + "orca_status_exit_code": 0, + "orca_version": "1.4.201", + "reason": "the environment answered with its own identity, home and project copy", + "verdict": "ENVIRONMENT" + }, + "phases": [ + { + "commands": [], + "detail": "", + "elapsed_seconds": 0.754, + "finished_at": "2026-09-15T17:39:40Z", + "name": "prepare", + "started_at": "2026-09-15T17:39:40Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 52.986, + "finished_at": "2026-09-15T17:40:33Z", + "name": "create", + "started_at": "2026-09-15T17:39:40Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "set -eu; export DEBIAN_FRONTEND=noninteractive; sudo apt-get update -qq; sudo apt-get install -y -qq curl ca-certificates git nodejs npm xvfb x11-xserver-utils xdotool libgtk-3-0t64 libnss3 libasound2t64 libatk-bridge2.0-0t64 libcups2t64 libdrm2 libgbm1 libxkbcommon0 libpango-1.0-0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libsecret-1-0 xdg-utils > /dev/null; echo deps installed" + ], + "context": "environment", + "elapsed_seconds": 55.717, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:29Z", + "started_at": "2026-09-15T17:40:33Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "set -eu; curl -fsSL -o /tmp/orca.deb 'https://github.com/stablyai/orca/releases/download/v1.4.201/orca-ide_1.4.201_amd64.deb'; echo '9dab79a5223837fc91992e04ed54f92f45d6f4b486d7128c08166bfb1a15a573 /tmp/orca.deb' | sha256sum -c -; sudo apt-get install -y -qq /tmp/orca.deb > /dev/null; rm -f /tmp/orca.deb; test -x /opt/Orca/resources/bin/orca-ide; echo orca installed" + ], + "context": "environment", + "elapsed_seconds": 17.355, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:46Z", + "started_at": "2026-09-15T17:41:29Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "set -eu; sudo npm install -g --no-fund --no-audit --silent '@anthropic-ai/claude-code@2.1.272' > /dev/null; claude --version" + ], + "context": "environment", + "elapsed_seconds": 6.521, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:53Z", + "started_at": "2026-09-15T17:41:46Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "set -eu; if DISPLAY=:99 xset -q > /dev/null 2>&1; then echo 'virtual display already up'; exit 0; fi; setsid nohup Xvfb :99 -screen 0 1280x800x24 -nolisten tcp > /tmp/xvfb.log 2>&1 < /dev/null & waited=0; while ! DISPLAY=:99 xset -q > /dev/null 2>&1; do waited=$((waited + 1)); if [ $waited -ge 60 ]; then echo 'no virtual display' >&2; exit 3; fi; sleep 1; done; echo \"virtual display ready after ${waited}s\"" + ], + "context": "environment", + "elapsed_seconds": 1.309, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:54Z", + "started_at": "2026-09-15T17:41:53Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project", + "init", + "-q", + "-b", + "main" + ], + "context": "environment", + "elapsed_seconds": 0.229, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:55Z", + "started_at": "2026-09-15T17:41:54Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project", + "config", + "user.email", + "dely-cycle@invalid" + ], + "context": "environment", + "elapsed_seconds": 0.142, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:55Z", + "started_at": "2026-09-15T17:41:55Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project", + "config", + "user.name", + "dely-cycle" + ], + "context": "environment", + "elapsed_seconds": 0.296, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:55Z", + "started_at": "2026-09-15T17:41:55Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project", + "add", + "-A" + ], + "context": "environment", + "elapsed_seconds": 0.22, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:55Z", + "started_at": "2026-09-15T17:41:55Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project", + "commit", + "-q", + "-m", + "dely-cycle: the pinned revision as exported" + ], + "context": "environment", + "elapsed_seconds": 0.153, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:55Z", + "started_at": "2026-09-15T17:41:55Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "", + "elapsed_seconds": 81.957, + "finished_at": "2026-09-15T17:41:55Z", + "name": "bootstrap", + "started_at": "2026-09-15T17:40:33Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '')\"\norca_release=''\norca_fingerprint=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '%s' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '%i:%s:%Y' \"$orca_resolved\" 2>/dev/null || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id=%s\\n' \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'orca_fingerprint=%s\\n' \"$orca_fingerprint\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/project-source", + "/opt/Orca/resources/bin/orca-ide" + ], + "context": "host", + "elapsed_seconds": 0.061, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:55Z", + "started_at": "2026-09-15T17:41:55Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '')\"\norca_release=''\norca_fingerprint=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '%s' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '%i:%s:%Y' \"$orca_resolved\" 2>/dev/null || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id=%s\\n' \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'orca_fingerprint=%s\\n' \"$orca_fingerprint\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/project", + "/opt/Orca/resources/bin/orca-ide" + ], + "context": "environment", + "elapsed_seconds": 0.244, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:56Z", + "started_at": "2026-09-15T17:41:55Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "/opt/Orca/resources/bin/orca-ide", + "status", + "--json" + ], + "context": "environment", + "elapsed_seconds": 0.511, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:56Z", + "started_at": "2026-09-15T17:41:56Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "export DISPLAY=\"$1\"; shift; export XDG_RUNTIME_DIR=\"${XDG_RUNTIME_DIR:-/run/user/$(id -u)}\"; waited=0; while ! xset -q >/dev/null 2>&1; do waited=$((waited + 2)); if [ \"$waited\" -ge 120 ]; then printf \"no display after %ss\n\" \"$waited\" >&2; exit 3; fi; sleep 2; done; printf \"display ready after %ss\n\" \"$waited\"; rm -f \"$HOME/.config/orca/SingletonLock\" \"$HOME/.config/orca/SingletonCookie\"; nohup \"$@\" > \"$HOME/orca-app.log\" 2>&1 & printf \"started %s\n\" \"$!\"", + "orca-start", + ":99", + "/opt/Orca/orca-ide", + "--disable-gpu" + ], + "context": "environment", + "elapsed_seconds": 0.22, + "exit_code": 0, + "finished_at": "2026-09-15T17:41:57Z", + "started_at": "2026-09-15T17:41:57Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "orca runtime da12c0a0-3b88-4c6d-b416-c54e0cb06609 is ready and the coordinator terminal is term_8a5ec3bf-062c-4d51-ba19-b5e70e0f4370", + "elapsed_seconds": 7.748, + "finished_at": "2026-09-15T17:42:03Z", + "name": "identity", + "started_at": "2026-09-15T17:41:55Z", + "status": "OK" + }, + { + "commands": [], + "detail": "the completion wait reached the run deadline before the worker settled", + "elapsed_seconds": 462.752, + "finished_at": "2026-09-15T17:49:46Z", + "name": "task", + "started_at": "2026-09-15T17:42:03Z", + "status": "TIMEOUT" + }, + { + "commands": [], + "detail": "the task did not settle", + "elapsed_seconds": null, + "finished_at": null, + "name": "check", + "started_at": null, + "status": "SKIPPED" + }, + { + "commands": [], + "detail": "0 patch line(s) collected", + "elapsed_seconds": 0.054, + "finished_at": "2026-09-15T17:49:46Z", + "name": "collect", + "started_at": "2026-09-15T17:49:46Z", + "status": "OK" + }, + { + "commands": [], + "detail": "12 artifact(s) re-read from the host; 1 unresolved", + "elapsed_seconds": 0.026, + "finished_at": "2026-09-15T17:49:46Z", + "name": "export", + "started_at": "2026-09-15T17:49:46Z", + "status": "OK" + }, + { + "commands": [], + "detail": "every declared per-run resource is gone and every shared one remains", + "elapsed_seconds": 3.724, + "finished_at": "2026-09-15T17:49:50Z", + "name": "cleanup", + "started_at": "2026-09-15T17:49:46Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": null, + "finished_at": null, + "name": "close", + "started_at": "2026-09-15T17:49:50Z", + "status": "OK" + } + ], + "run_id": "20260916T050000Z-cccccc-00000004", + "schema_version": 1, + "started_at": "2026-09-15T17:39:40Z", + "status": "TIMEOUT", + "tested_revision": "d54307995cc5ea378c7df201f4282ea480a5c0c1", + "timeout_seconds": 420, + "versions": { + "configured_revision": "d54307995cc5ea378c7df201f4282ea480a5c0c1", + "distrobox": "distrobox: 1.8.2.5", + "git": "git version 2.55.0", + "podman": "podman version 5.8.4", + "project_revision": "d54307995cc5ea378c7df201f4282ea480a5c0c1", + "pulumi": "v3.262.0", + "python": "3.14.7", + "qemu_img": "Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers", + "runner": "minimal-cycle", + "virsh": "12.0.0" + }, + "worker": { + "agent": "claude", + "commands": [ + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "orca", + "orchestration", + "run-create", + "--objective", + "dely minimal cycle", + "--json", + "--from", + "term_8a5ec3bf-062c-4d51-ba19-b5e70e0f4370" + ], + "context": "environment", + "elapsed_seconds": 0.332, + "exit_code": 0, + "finished_at": "2026-09-15T17:42:03Z", + "started_at": "2026-09-15T17:42:03Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "orca", + "orchestration", + "worker-start", + "--spec", + "Read the file /var/tmp/dely-cycle/state/20260916T050000Z-cccccc-00000004/home/dispatch-prompt.md in this environment and do exactly what it says.", + "--agent", + "claude", + "--model", + "claude-opus-5", + "--effort", + "high", + "--worktree", + "current", + "--timeout-ms", + "420000", + "--json", + "--run", + "run_cf3d25716c29", + "--from", + "term_8a5ec3bf-062c-4d51-ba19-b5e70e0f4370" + ], + "context": "environment", + "elapsed_seconds": 42.398, + "exit_code": 1, + "finished_at": "2026-09-15T17:42:46Z", + "started_at": "2026-09-15T17:42:03Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260916t050000z-cccccc-00000004", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "orca", + "orchestration", + "check", + "--wait", + "--types", + "worker_done,escalation,question", + "--timeout-ms", + "420000", + "--json", + "--run", + "run_cf3d25716c29", + "--terminal", + "term_8a5ec3bf-062c-4d51-ba19-b5e70e0f4370" + ], + "context": "environment", + "elapsed_seconds": 420.021, + "exit_code": null, + "finished_at": "2026-09-15T17:49:46Z", + "started_at": "2026-09-15T17:42:46Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": true + } + ], + "detail": "the completion wait reached the run deadline before the worker settled", + "dispatch_id": "ctx_230b27b63cc9", + "effort": "high", + "model": "claude-opus-5", + "outcome": "outcome_unknown", + "run_id": "run_cf3d25716c29", + "status": "TIMEOUT" + } +} diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/preflight.json b/experiments/minimal-cycle/evidence/distrobox-full-cycle/preflight.json new file mode 100644 index 0000000..78f504e --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/preflight.json @@ -0,0 +1,36 @@ +{ + "backend": "distrobox", + "findings": [ + { + "detail": "/usr/bin/distrobox", + "name": "distrobox on the path", + "ok": true + }, + { + "detail": "podman", + "name": "container manager on the path", + "ok": true + }, + { + "detail": "docker.io/library/ubuntu:24.04 is present", + "name": "image present locally", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/state", + "name": "state root writable", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/artifacts", + "name": "artifact root writable", + "ok": true + }, + { + "detail": "distrobox mounts into the box and offers no flag to suppress it; the configuration acknowledged this, so the run records a declared compromise rather than claiming isolation", + "name": "host home reachable from the environment", + "ok": true + } + ], + "ok": true +} diff --git a/experiments/minimal-cycle/evidence/preflight-vm.txt b/experiments/minimal-cycle/evidence/preflight-vm.txt index 4c1345d..e7c94cf 100644 --- a/experiments/minimal-cycle/evidence/preflight-vm.txt +++ b/experiments/minimal-cycle/evidence/preflight-vm.txt @@ -7,7 +7,7 @@ backend: vm ok pinned environment present: /var/tmp/dely-cycle/venv/bin/python ok pulumi state backend is local: state lives at file://~ ok provider schema verified: every class and field the program names exists in pulumi-libvirt@0.5.4 - ok base image present: /var/tmp/dely-cycle/images/ubuntu-24.04-server-cloudimg-amd64.img + ok base image present: /var/tmp/dely-cycle/images/tool-image.qcow2 ok base image digest matches the pin: the preserved base matches the pinned digest ok hardware acceleration device: /dev/kvm ok libvirt connection answers: qemu:///system diff --git a/experiments/minimal-cycle/tests/test_isolate.py b/experiments/minimal-cycle/tests/test_isolate.py index 9d23547..1ce5a1f 100644 --- a/experiments/minimal-cycle/tests/test_isolate.py +++ b/experiments/minimal-cycle/tests/test_isolate.py @@ -40,7 +40,7 @@ def test_every_leaking_variable_goes_not_just_the_one_we_named(self): env={ "ORCA_TERMINAL_HANDLE": "a", "ORCA_AGENT_HOOK_TOKEN": "b", - "ORCA_WORKSPACE_ID": "c", + "ORCA_PANE_KEY": "c", }, ) self.assertEqual(outcome.stdout.strip(), "0") From 57b73c67b99a8d7562593d17a170dbee9018f8bc Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 20:22:51 +0200 Subject: [PATCH 33/42] Answer the questions a fresh agent home asks before it will work. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A disposable environment gives Claude Code a home it has never run in, and it asks four things before its first turn: finish onboarding and sign in, trust the project copy, allow the external imports this project's own instructions declare, and accept the permission mode it was launched with. Each is a modal the dispatch cannot answer, so the agent holds the task unsent and the execution plane reports a turn that never started — the same symptom as a worker that died, with none of the cause. The answers are written from what the configuration already says rather than copied from the host, so no account, machine or history travels with them. Auth now names its settings contribution in one place, because the first-run state and the key helper share that file. --- experiments/minimal-cycle/counterexamples.py | 29 ++++++ .../minimal-cycle/cycle_runner/auth.py | 10 +- .../minimal-cycle/cycle_runner/firstrun.py | 88 ++++++++++++++++++ .../minimal-cycle/cycle_runner/lifecycle.py | 22 ++++- .../minimal-cycle/cycle_runner/result.py | 22 +++++ .../minimal-cycle/tests/test_firstrun.py | 91 +++++++++++++++++++ 6 files changed, 259 insertions(+), 3 deletions(-) create mode 100644 experiments/minimal-cycle/cycle_runner/firstrun.py create mode 100644 experiments/minimal-cycle/tests/test_firstrun.py diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index d8793b0..5649a2b 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -38,6 +38,35 @@ class Counterexample: CASES: tuple[Counterexample, ...] = ( + Counterexample( + name="first-run-state-names-the-environment-copy", + requirement=( + "First-run state keyed to a path the agent will never open leaves " + "every prompt unanswered" + ), + path="cycle_runner/firstrun.py", + original=""" return { + "hasCompletedOnboarding": True, + "projects": { + project_path: {""", + replacement=""" return { + "hasCompletedOnboarding": True, + "projects": { + "/home/agent/project": {""", + instruments=("tests.test_firstrun.FirstRunStateTest",), + ), + Counterexample( + name="first-run-answers-every-question-it-names", + requirement=( + "A receipt that lists four answered questions while the state " + "answers one is rejected" + ), + path="cycle_runner/firstrun.py", + original=""" "hasTrustDialogAccepted": True, + "hasClaudeMdExternalIncludesApproved": True,""", + replacement=""" "hasTrustDialogAccepted": True,""", + instruments=("tests.test_firstrun.FirstRunStateTest",), + ), Counterexample( name="no-host-fallback", requirement="A probe result equal to the host's own snapshot is rejected", diff --git a/experiments/minimal-cycle/cycle_runner/auth.py b/experiments/minimal-cycle/cycle_runner/auth.py index fdf58a3..8e53cf1 100644 --- a/experiments/minimal-cycle/cycle_runner/auth.py +++ b/experiments/minimal-cycle/cycle_runner/auth.py @@ -21,6 +21,13 @@ SETTINGS_RELATIVE = ".claude/settings.json" +def settings_document(run_config: RunConfig) -> dict[str, str]: + """Return what the configured auth method declares in the per-run settings.""" + if run_config.auth.mode != "api_key_helper": + return {} + return {"apiKeyHelper": " ".join(run_config.auth.helper_argv)} + + def _environment_path(handle: EnvironmentHandle, relative: str) -> str: return str(Path(handle.home_path) / relative) @@ -91,7 +98,6 @@ def _api_key_helper( adapter: BackendAdapter, handle: EnvironmentHandle, ) -> tuple[AuthRecord, dict[str, str]]: - helper = " ".join(run_config.auth.helper_argv) record = AuthRecord( mode="api_key_helper", reference=run_config.auth.reference, @@ -100,7 +106,7 @@ def _api_key_helper( ) adapter.write_file( _environment_path(handle, SETTINGS_RELATIVE), - json.dumps({"apiKeyHelper": helper}, indent=2) + "\n", + json.dumps(settings_document(run_config), indent=2) + "\n", mode=0o600, ) record.status = PhaseStatus.OK diff --git a/experiments/minimal-cycle/cycle_runner/firstrun.py b/experiments/minimal-cycle/cycle_runner/firstrun.py new file mode 100644 index 0000000..9fc65ca --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/firstrun.py @@ -0,0 +1,88 @@ +"""The agent's first run, in a home that has never run it. + +A disposable environment gives Claude Code a home it has never seen, and a +first run asks before it works: finish onboarding and sign in, trust this +folder, accept the permission mode it was launched with, allow the external +imports this project's own instructions declare. Each question is a modal +the dispatch cannot answer, so the agent holds the task unsent and the +execution plane reports a turn that never started — the same symptom as a +worker that died, with none of the cause. + +The answers are recorded state, not credentials. They are written here from +what the configuration already says rather than copied from the host, so no +account, machine or history travels into the environment with them. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Any + +from . import auth +from .adapters.base import BackendAdapter, EnvironmentHandle +from .config import RunConfig +from .result import FirstRunRecord +from .status import PhaseStatus + +#: Where Claude Code keeps what it has been told, relative to the home. +STATE_RELATIVE = ".claude.json" +SETTINGS_RELATIVE = auth.SETTINGS_RELATIVE + +#: One line per question this state answers, for the receipt. Naming them is +#: the difference between a run that answered four prompts and a run that +#: silently pre-approved whatever the agent might ask. +QUESTIONS = ( + "onboarding, which otherwise offers a theme and an interactive sign-in", + "trusting the project copy, by its path inside the environment", + "the external imports the project's own instructions declare", + "the permission mode the execution plane launches the agent with", +) + + +def state_document(project_path: str) -> dict[str, Any]: + """Return the first-run state for one project copy.""" + return { + "hasCompletedOnboarding": True, + "projects": { + project_path: { + "hasTrustDialogAccepted": True, + "hasClaudeMdExternalIncludesApproved": True, + "hasClaudeMdExternalIncludesWarningShown": True, + } + }, + } + + +def settings_document(run_config: RunConfig) -> dict[str, Any]: + """Return the per-run settings, including whatever auth declared there.""" + return { + **auth.settings_document(run_config), + "skipDangerousModePermissionPrompt": True, + } + + +def apply( + *, run_config: RunConfig, adapter: BackendAdapter, handle: EnvironmentHandle +) -> FirstRunRecord: + """Write the first-run state into the per-run home and report what it wrote.""" + record = FirstRunRecord( + target=str(Path(handle.home_path)), + entries=[STATE_RELATIVE, SETTINGS_RELATIVE], + questions=list(QUESTIONS), + ) + home = Path(handle.home_path) + for relative, document in ( + (STATE_RELATIVE, state_document(handle.project_path)), + (SETTINGS_RELATIVE, settings_document(run_config)), + ): + adapter.write_file( + str(home / relative), json.dumps(document, indent=2) + "\n", mode=0o600 + ) + record.status = PhaseStatus.OK + record.detail = ( + f"{len(record.entries)} first-run state file(s) written into the per-run " + f"home, answering {len(record.questions)} question(s) the agent asks a " + "home it has never run in" + ) + return record diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index 1445c1d..fe0b9ed 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -17,7 +17,19 @@ from pathlib import Path from typing import Any, Mapping -from . import auth, cleanup, hostinfo, manifest, orca, probe, proc, project, redact, worker +from . import ( + auth, + cleanup, + firstrun, + hostinfo, + manifest, + orca, + probe, + proc, + project, + redact, + worker, +) from .adapters.base import BackendAdapter, EnvironmentHandle from .config import RunConfig from .export import Exporter @@ -322,6 +334,14 @@ def _bootstrap(self, baseline: Path) -> None: if auth_record.status is PhaseStatus.BLOCKED: record.status = PhaseStatus.BLOCKED self.blocked_reason = auth_record.detail + return + # Auth declares what it wants in the per-run settings; this writes + # that file, so it runs after auth rather than before it. + first_run = firstrun.apply( + run_config=self.config, adapter=self.adapter, handle=self.handle + ) + self.result.first_run = first_run + self.exporter.write_json("first-run-state.json", first_run.to_document()) def _provision_steps(self): section = ( diff --git a/experiments/minimal-cycle/cycle_runner/result.py b/experiments/minimal-cycle/cycle_runner/result.py index 5d746dc..ec7fb8d 100644 --- a/experiments/minimal-cycle/cycle_runner/result.py +++ b/experiments/minimal-cycle/cycle_runner/result.py @@ -233,6 +233,26 @@ def to_document(self) -> dict[str, Any]: } +@dataclass +class FirstRunRecord: + """What was written so the agent could start, never why it was trusted.""" + + status: PhaseStatus = PhaseStatus.SKIPPED + target: str | None = None + entries: list[str] = field(default_factory=list) + questions: list[str] = field(default_factory=list) + detail: str = "not attempted" + + def to_document(self) -> dict[str, Any]: + return { + "status": self.status.value, + "target": self.target, + "entries": list(self.entries), + "questions": list(self.questions), + "detail": self.detail, + } + + @dataclass class RunResult: """One cycle, whatever backend produced it.""" @@ -256,6 +276,7 @@ class RunResult: export: ExportRecord = field(default_factory=ExportRecord) cleanup: CleanupRecord = field(default_factory=CleanupRecord) auth: AuthRecord = field(default_factory=AuthRecord) + first_run: FirstRunRecord = field(default_factory=FirstRunRecord) def phase(self, name: str) -> PhaseRecord | None: for record in self.phases: @@ -284,6 +305,7 @@ def to_document(self) -> dict[str, Any]: "export": self.export.to_document(), "cleanup": self.cleanup.to_document(), "auth": self.auth.to_document(), + "first_run": self.first_run.to_document(), } diff --git a/experiments/minimal-cycle/tests/test_firstrun.py b/experiments/minimal-cycle/tests/test_firstrun.py new file mode 100644 index 0000000..9119661 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_firstrun.py @@ -0,0 +1,91 @@ +"""A fresh home has never run the agent, and the agent asks before it works.""" + +import json +import tempfile +import unittest +from pathlib import Path + +from cycle_runner import config as config_module, firstrun +from tests.fakes import FakeAdapter +from tests.test_config import minimal_document + + +def make_config(auth_section=None): + document = minimal_document() + if auth_section is not None: + document["auth"] = auth_section + return config_module.from_document(document) + + +class FirstRunStateTest(unittest.TestCase): + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.adapter = FakeAdapter(self.root / "run") + self.handle = self.adapter.create() + self.config = make_config() + self.addCleanup(self._tmp.cleanup) + + def apply(self): + return firstrun.apply( + run_config=self.config, adapter=self.adapter, handle=self.handle + ) + + def state(self): + return json.loads( + (Path(self.handle.home_path) / firstrun.STATE_RELATIVE).read_text() + ) + + def settings(self): + return json.loads( + (Path(self.handle.home_path) / firstrun.SETTINGS_RELATIVE).read_text() + ) + + def test_onboarding_is_answered_so_the_agent_does_not_ask_to_sign_in(self): + self.apply() + self.assertTrue(self.state()["hasCompletedOnboarding"]) + + def test_the_project_copy_is_trusted_by_its_path_inside_the_environment(self): + self.apply() + entry = self.state()["projects"][self.handle.project_path] + self.assertTrue(entry["hasTrustDialogAccepted"]) + + def test_the_external_imports_this_project_declares_are_answered(self): + self.apply() + entry = self.state()["projects"][self.handle.project_path] + self.assertTrue(entry["hasClaudeMdExternalIncludesApproved"]) + + def test_the_permission_mode_warning_is_answered_in_the_settings_file(self): + self.apply() + self.assertTrue(self.settings()["skipDangerousModePermissionPrompt"]) + + def test_the_state_is_written_rather_than_copied_from_the_host(self): + record = self.apply() + rendered = json.dumps(self.state()) + json.dumps(record.to_document()) + self.assertNotIn(str(Path.home()), rendered) + for key in ("oauthAccount", "userID", "machineID", "history"): + self.assertNotIn(key, rendered) + + def test_the_receipt_names_the_files_and_the_questions_they_answer(self): + record = self.apply() + self.assertEqual( + record.entries, [firstrun.STATE_RELATIVE, firstrun.SETTINGS_RELATIVE] + ) + self.assertTrue(record.questions) + + def test_an_api_key_helper_keeps_its_settings_entry(self): + self.config = make_config( + { + "mode": "api_key_helper", + "reference": "workshop-helper", + "helper_argv": ["/usr/local/bin/key-helper"], + } + ) + self.apply() + settings = self.settings() + self.assertEqual(settings["apiKeyHelper"], "/usr/local/bin/key-helper") + self.assertTrue(settings["skipDangerousModePermissionPrompt"]) + + +if __name__ == "__main__": + unittest.main() From 497660005edd04cb399886334863ffe8fe5c31be Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 20:37:24 +0200 Subject: [PATCH 34/42] Read the settling message from the delivery, and keep the reply that decided the run. The completion wait answers with a delivery nested under the reply's result; the reply's top level is the request envelope. Reading the envelope found no messages, so a worker that had finished its task and reported worker_done was recorded as one that never answered, and the independent check was skipped on a run whose artifact was already correct on disk. The test fixtures carried the invented flat shape, so they agreed with the defect. They now carry the shape the plane really sends, and each of the three orchestration replies is exported beside the manifest, redacted. A verdict with no reply behind it cannot be checked by the next reader. --- experiments/minimal-cycle/counterexamples.py | 24 +++++++++ .../minimal-cycle/cycle_runner/lifecycle.py | 6 +++ .../minimal-cycle/cycle_runner/worker.py | 31 +++++++++-- experiments/minimal-cycle/tests/fakes.py | 16 ++++-- .../minimal-cycle/tests/test_worker.py | 51 +++++++++++++++++-- 5 files changed, 115 insertions(+), 13 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 5649a2b..60f59fc 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -38,6 +38,30 @@ class Counterexample: CASES: tuple[Counterexample, ...] = ( + Counterexample( + name="a-settling-message-is-read-from-the-delivery", + requirement=( + "A wait read from the request envelope reports a worker that " + "finished as one that never answered" + ), + path="cycle_runner/worker.py", + original=' messages = _result(document).get("messages")', + replacement=' messages = document.get("messages")', + instruments=( + "tests.test_worker.WorkerTest", + "tests.test_lifecycle.SettledCycleTest", + ), + ), + Counterexample( + name="the-reply-that-decided-the-run-is-kept", + requirement="A run that exports only its verdict keeps nothing to check it against", + path="cycle_runner/worker.py", + original=""" def remember(name: str, outcome) -> None: + if keep is not None:""", + replacement=""" def remember(name: str, outcome) -> None: + if False:""", + instruments=("tests.test_worker.KeptRepliesTest",), + ), Counterexample( name="first-run-state-names-the-environment-copy", requirement=( diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index fe0b9ed..3866509 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -442,6 +442,11 @@ def _task(self) -> None: assert self.handle is not None with self.phase("task") as record: self.task_ran = True + def keep(name: str, stdout: str, stderr: str) -> None: + self.exporter.write_text(f"dispatch/{name}.stdout", stdout) + if stderr.strip(): + self.exporter.write_text(f"dispatch/{name}.stderr", stderr) + worker_record = worker.launch( run_config=self.config, adapter=self.adapter, @@ -449,6 +454,7 @@ def _task(self) -> None: timeout_seconds=self.config.timeout_seconds, env_overlay=self.env_overlay, coordinator_handle=self.coordinator_handle, + keep=keep, ) self.result.worker = worker_record record.status = worker_record.status diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index 6c47166..b1298cf 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -11,7 +11,7 @@ import json from dataclasses import dataclass from pathlib import Path -from typing import Any, Mapping, Sequence +from typing import Any, Callable, Mapping, Sequence from . import redact from .adapters.base import BackendAdapter, EnvironmentHandle @@ -208,13 +208,17 @@ def task_identifier(document: Mapping[str, Any]) -> str | None: def _settling_message(document: Mapping[str, Any]) -> dict[str, Any]: - messages = document.get("messages") + """Return the message that settled the wait, from the plane's own reply. + + The delivery is a field of the reply's `result`; the reply's top level is + the request envelope. Reading the envelope finds no messages and reports a + worker that finished as one that never answered. + """ + messages = _result(document).get("messages") if isinstance(messages, list): for message in messages: if isinstance(message, Mapping) and message.get("type") in SETTLING_TYPES: return dict(message) - if document.get("type") in SETTLING_TYPES: - return dict(document) return {} @@ -226,8 +230,14 @@ def launch( timeout_seconds: int, env_overlay: Mapping[str, str] | None = None, coordinator_handle: str | None = None, + keep: Callable[[str, str, str], None] | None = None, ) -> WorkerRecord: - """Run exactly one worker and report how it settled.""" + """Run exactly one worker and report how it settled. + + `keep` is offered each orchestration reply under a name, redacted. The + reply is what decides the run, and a run that reports only its verdict + leaves the next reader nothing to check the verdict against. + """ record = WorkerRecord( agent=run_config.orca.agent, model=run_config.orca.model, @@ -236,6 +246,14 @@ def launch( overlay = dict(env_overlay or {}) secrets: Sequence[str] = tuple(value for value in overlay.values() if value) + def remember(name: str, outcome) -> None: + if keep is not None: + keep( + name, + redact.text(outcome.stdout or "", secrets), + redact.text(outcome.stderr or "", secrets), + ) + plan = build_plan( run_config=run_config, handle=handle, @@ -248,6 +266,7 @@ def launch( plan.run_create_argv, timeout=timeout_seconds, env=overlay, extra_values=secrets ) record.commands.append(created.to_record()) + remember("run-create", created) if created.timed_out: record.status = PhaseStatus.TIMEOUT record.detail = "orca orchestration run-create reached the run deadline" @@ -277,6 +296,7 @@ def launch( extra_values=secrets, ) record.commands.append(started.to_record()) + remember("worker-start", started) if started.timed_out: record.status = PhaseStatus.TIMEOUT record.detail = "orca orchestration worker-start reached the run deadline" @@ -309,6 +329,7 @@ def launch( plan.wait_argv, timeout=timeout_seconds, env=overlay, extra_values=secrets ) record.commands.append(settled.to_record()) + remember("completion-wait", settled) if settled.timed_out: record.status = PhaseStatus.TIMEOUT record.detail = "the completion wait reached the run deadline before the worker settled" diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index 3b1ba97..5124ac9 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -64,13 +64,19 @@ '"worktreeId": "repo::project", "surface": "visible"}}}' ) -# The real shape: identifiers nested under `result`, the top-level `id` being -# the request's. One reply serves all three orchestration commands here. +# The real shape: identifiers and the delivery both nested under `result`, the +# top-level `id` being the request's. One reply serves all three orchestration +# commands here. ORCA_DISPATCH_REPLY = ( '{"id": "request-fake", "ok": true, "result": {"run": {"id": "run-fake"}, ' '"runId": "run-fake", "taskId": "task-fake", "dispatchId": "dispatch-fake", ' - '"state": "ready"}, ' - '"messages": [{"type": "worker_done", "outcome": "DONE"}]}' + '"state": "ready", ' + '"messages": [{"type": "worker_done", "outcome": "DONE"}], "count": 1}}' +) + +#: A wait that woke with nothing to report, in the same shape. +ORCA_EMPTY_DELIVERY = ( + '{"id": "request-fake", "ok": true, "result": {"messages": [], "count": 0}}' ) @@ -233,7 +239,7 @@ def execute( return self._outcome(argv, 1, reply, "") if not self.wait_settles and "--wait" in joined: # The dispatch never reports, so the wait returns nothing. - return self._outcome(argv, 0, '{"messages": []}', "") + return self._outcome(argv, 0, ORCA_EMPTY_DELIVERY, "") if not self.task_writes_nothing: self.project.mkdir(parents=True, exist_ok=True) (self.project / "evidence.txt").write_text(self.marker, encoding="utf-8") diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py index bddafdf..86343e2 100644 --- a/experiments/minimal-cycle/tests/test_worker.py +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -30,7 +30,20 @@ }, } ) -SETTLED = json.dumps({"messages": [{"type": "worker_done", "outcome": "DONE"}]}) +# The shape `orca orchestration check` really answers with: the messages are +# a field of the result, and the reply's top level is the request envelope. +def delivery(*messages): + """Return a check reply carrying these messages, as the plane sends it.""" + return json.dumps( + { + "id": "e0a1e4d0-0000-4000-8000-000000000000", + "ok": True, + "result": {"messages": list(messages), "count": len(messages)}, + } + ) + + +SETTLED = delivery({"type": "worker_done", "outcome": "DONE"}) DEFAULT_SCRIPT = [ ("run-create", 0, READY, "", False), @@ -140,7 +153,7 @@ def test_an_escalation_settles_the_wait_without_claiming_success(self): ( "check --wait", 0, - json.dumps({"messages": [{"type": "escalation", "outcome": "BLOCKED"}]}), + delivery({"type": "escalation", "outcome": "BLOCKED"}), "", False, ), @@ -324,7 +337,7 @@ def test_a_dispatch_that_settles_afterwards_is_done(self): self.assertEqual(record.dispatch_id, "dispatch-abcdef") def test_a_dispatch_that_never_settles_keeps_the_unverifiable_state(self): - _, _, record = self.launch(script=self.script(json.dumps({"messages": []}))) + _, _, record = self.launch(script=self.script(delivery())) self.assertEqual(record.outcome, "outcome_unknown") self.assertNotEqual(record.status, status.PhaseStatus.OK) @@ -423,3 +436,35 @@ def test_a_failed_start_names_the_code(self): _, _, record = self.launch(script=script) self.assertIn("consumer_fenced", record.detail) self.assertNotIn("{", record.detail) + + +class KeptRepliesTest(WorkerTest): + """The reply that decided the run is the one worth keeping.""" + + def launch_keeping(self, **overrides): + kept = {} + adapter = ScriptedAdapter(self.root / "run", script=DEFAULT_SCRIPT) + handle = adapter.create() + worker.launch( + run_config=self.config, + adapter=adapter, + handle=handle, + timeout_seconds=self.config.timeout_seconds, + keep=lambda name, stdout, stderr: kept.__setitem__(name, stdout + stderr), + **overrides, + ) + return kept + + def test_every_orchestration_reply_is_offered_for_export(self): + kept = self.launch_keeping(env_overlay={}) + self.assertEqual( + sorted(kept), ["completion-wait", "run-create", "worker-start"] + ) + self.assertIn("worker_done", kept["completion-wait"]) + + def test_a_kept_reply_carries_no_value_the_overlay_passed(self): + kept = self.launch_keeping( + env_overlay={"CLAUDE_CODE_OAUTH_TOKEN": "sk-ant-secret-value-here"} + ) + for name, text in kept.items(): + self.assertNotIn("sk-ant-secret-value-here", text, name) From 7d6f220903742eb7693ebc82e1593f52b418e55f Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 20:45:56 +0200 Subject: [PATCH 35/42] Record the first cycle that settled, with the independent check passing. The worker read the prompt, wrote the marker byte for byte, and reported worker_done with outcome succeeded; the check ran outside that session and agreed; the artifacts were exported and confirmed before the box and its per-run home were destroyed. The reported outcome now comes from the worker's own payload rather than from the message type, which only repeated that it had finished. --- experiments/minimal-cycle/counterexamples.py | 11 + .../minimal-cycle/cycle_runner/worker.py | 22 +- .../distrobox-settled-cycle/README.md | 56 + .../distrobox-settled-cycle/auth-receipt.json | 11 + .../distrobox-settled-cycle/check.stdout | 1 + .../distrobox-settled-cycle/cleanup.json | 15 + .../distrobox-settled-cycle/diff.patch | 4 + .../dispatch-completion-wait.json | 38 + .../dispatch-run-create.json | 22 + .../dispatch-worker-start.json | 86 ++ .../export-receipt.json | 109 ++ .../first-run-state.json | 15 + .../identity-environment-probe.txt | 14 + .../identity-host-probe.txt | 14 + .../identity-orca-runtime.json | 90 ++ .../distrobox-settled-cycle/manifest.json | 1249 +++++++++++++++++ .../distrobox-settled-cycle/preflight.json | 36 + .../task-artifact-evidence.txt | 1 + experiments/minimal-cycle/tests/fakes.py | 3 +- .../minimal-cycle/tests/test_worker.py | 12 +- 20 files changed, 1805 insertions(+), 4 deletions(-) create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/README.md create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/auth-receipt.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/check.stdout create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/cleanup.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/diff.patch create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-completion-wait.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-run-create.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-worker-start.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/export-receipt.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/first-run-state.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-environment-probe.txt create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-host-probe.txt create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-orca-runtime.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/manifest.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/preflight.json create mode 100644 experiments/minimal-cycle/evidence/distrobox-settled-cycle/task-artifact-evidence.txt diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 60f59fc..ce9acfa 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -38,6 +38,17 @@ class Counterexample: CASES: tuple[Counterexample, ...] = ( + Counterexample( + name="the-outcome-is-the-workers-own-verdict", + requirement=( + "Reporting the message type as the outcome says the worker " + "finished and nothing about how" + ), + path="cycle_runner/worker.py", + original=' payload = message.get("payload")\n if isinstance(payload, str):', + replacement=' payload = None\n if isinstance(payload, str):', + instruments=("tests.test_worker.WorkerTest",), + ), Counterexample( name="a-settling-message-is-read-from-the-delivery", requirement=( diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index b1298cf..005ad23 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -222,6 +222,26 @@ def _settling_message(document: Mapping[str, Any]) -> dict[str, Any]: return {} +def _message_outcome(message: Mapping[str, Any]) -> str | None: + """Return what the worker said it reached, from the message's payload. + + A settling message carries no outcome field of its own: the worker's own + verdict travels as a JSON document in `payload`. Falling back to the + message `type` reports `worker_done` as the outcome, which only repeats + that the worker finished and says nothing about how. + """ + payload = message.get("payload") + if isinstance(payload, str): + try: + payload = json.loads(payload) + except json.JSONDecodeError: + payload = None + if isinstance(payload, Mapping) and isinstance(payload.get("outcome"), str): + return payload["outcome"] + value = message.get("outcome") + return value if isinstance(value, str) and value else message.get("type") + + def launch( *, run_config: RunConfig, @@ -340,7 +360,7 @@ def remember(name: str, outcome) -> None: return record message = _settling_message(_first_document(settled.stdout)) - record.outcome = message.get("outcome") or message.get("type") or record.outcome + record.outcome = _message_outcome(message) or record.outcome if message.get("type") == "worker_done": record.status = PhaseStatus.OK record.detail = f"the worker reported worker_done with outcome {record.outcome!r}" diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/README.md b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/README.md new file mode 100644 index 0000000..1777bb8 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/README.md @@ -0,0 +1,56 @@ +# One cycle that settled, with the check passing + +The first complete run: a fresh box, an agent that did the task, and an +independent check that read the result and agreed. + + status: SETTLED + identity: ENVIRONMENT + run: run_5a5f2688c661 + dispatch: ctx_68f16395149d (claude-opus-5 at high) + worker: worker_done, outcome succeeded + check: OK (exit 0) + export: CONFIRMED + cleanup: DESTROYED + +## What the worker did, in its own words + +`dispatch-completion-wait.json` is the reply that settled the run. It carries +the worker's own message: + +> Created …/project/evidence.txt containing exactly 'dely-cycle-marker' with no +> trailing newline, using printf. Verified with od -c and wc -c: 17 bytes, final +> byte is 'r', no 0x0a present. + +with `"outcome":"succeeded"` in its payload. `check.stdout` is the independent +check run afterwards, outside that session: `marker matched`. `diff.patch` and +`task-artifact-evidence.txt` are the file itself, 17 bytes, no trailing byte. + +## The two things that had to change first + +Every earlier run stopped with the launch line standing unexecuted in the +worker's terminal, and the plane reporting `turn_start_unobserved`. + +Claude Code asks a home it has never run in four questions, and each is a modal +nobody in a disposable environment can answer: finish onboarding, which offers +a theme and an interactive sign-in; trust this folder; allow the external +imports this project's instructions declare; accept the permission mode it was +launched with. A copied token answers none of them. `first-run-state.json` +records the two small files the runner now writes, and names the four questions +in words — these are pre-approvals, and a reader should be able to see them and +disagree. + +Under that sat a second defect the first was hiding. The completion wait was +read from the reply's top level, which is the request envelope; the delivery is +under `result`. So a worker that had finished and reported was recorded as one +that never answered, on a run whose artifact was already correct on disk. The +three replies are now exported — `dispatch-*.json` — because a verdict with no +reply behind it cannot be checked by the next reader, and this one could not be. + +## What this does not show + +The machine backend: see `../vm-settled-cycle/`. It does not show isolation — +Distrobox mounts the host home, and `preflight.json` records that as a declared +compromise rather than a claim. It does not show that a second, different task +would pass. + +Paths and the host user are scrubbed; the identifiers are the run's own. diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/auth-receipt.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/auth-receipt.json new file mode 100644 index 0000000..2267a36 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/auth-receipt.json @@ -0,0 +1,11 @@ +{ + "detail": "1 allowlisted entry(s) copied into the per-run home with owner-only permissions", + "entries": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login", + "removed_after_run": false, + "status": "OK", + "target": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home" +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/check.stdout b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/check.stdout new file mode 100644 index 0000000..e3fe404 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/check.stdout @@ -0,0 +1 @@ +marker matched at /var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project/evidence.txt diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/cleanup.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/cleanup.json new file mode 100644 index 0000000..1cb63b8 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/cleanup.json @@ -0,0 +1,15 @@ +{ + "host_difference": {}, + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "container:dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "path:/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04" + ], + "retained": [], + "shared_preserved": [ + "image:docker.io/library/ubuntu:24.04" + ], + "status": "DESTROYED", + "stop_confirmed": true, + "verified": true +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/diff.patch b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/diff.patch new file mode 100644 index 0000000..8911635 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/diff.patch @@ -0,0 +1,4 @@ +--- /dev/null ++++ b/evidence.txt +@@ -0,0 +1 @@ ++dely-cycle-marker \ No newline at end of file diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-completion-wait.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-completion-wait.json new file mode 100644 index 0000000..0c844c1 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-completion-wait.json @@ -0,0 +1,38 @@ +{ + "id": "69ed6a61-33bf-4382-b6d1-beda2926d35a", + "ok": true, + "result": { + "runId": "run_5a5f2688c661", + "deliveryId": "delivery_e6ec30e50065", + "messages": [ + { + "id": "msg_49a5efeb992b", + "run_id": "run_5a5f2688c661", + "delivery_contract": "current_delivery", + "from_handle": "term_443d588e-1c2c-440f-b36c-a28c84f7870c", + "to_handle": "run:run_5a5f2688c661", + "subject": "evidence.txt created with marker", + "body": "Created /var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project/evidence.txt containing exactly 'dely-cycle-marker' with no trailing newline, using printf. Verified with od -c and wc -c: 17 bytes, final byte is 'r', no 0x0a present. Nothing is left; no other files were touched and no tests were run, per the dispatch prompt's constraints.", + "type": "worker_done", + "priority": "normal", + "thread_id": null, + "payload": "{\"taskId\":\"task_e6721765bb09\",\"dispatchId\":\"ctx_68f16395149d\",\"outcome\":\"succeeded\",\"filesModified\":[\"evidence.txt\"]}", + "created_at": "2026-09-15T18:41:25Z", + "delivered_at": null + } + ], + "count": 1, + "replayed": false, + "acknowledged": null, + "timedOut": false, + "cancelled": false, + "connectionLost": false, + "mutation": { + "requestId": "5d272960-9757-4f31-b586-31f6b9d81097", + "replayed": false + } + }, + "_meta": { + "runtimeId": "7d2eb076-9fdd-4c8b-a706-5a37d6b15a6d" + } +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-run-create.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-run-create.json new file mode 100644 index 0000000..1969600 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-run-create.json @@ -0,0 +1,22 @@ +{ + "id": "b6fab545-9046-423d-b2b2-3f1157a919f6", + "ok": true, + "result": { + "run": { + "id": "run_5a5f2688c661", + "objective": "dely minimal cycle", + "coordinator_handle": "term_78d6aa0b-d97c-42d4-984c-fae24707c9b9", + "consumer_generation": 1, + "legacy": 0, + "created_at": "2026-09-15T18:41:00Z", + "updated_at": "2026-09-15T18:41:00Z" + }, + "mutation": { + "requestId": "c4cd6c58-5a19-445c-afd3-9e53bb9457d5", + "replayed": false + } + }, + "_meta": { + "runtimeId": "7d2eb076-9fdd-4c8b-a706-5a37d6b15a6d" + } +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-worker-start.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-worker-start.json new file mode 100644 index 0000000..e900b90 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/dispatch-worker-start.json @@ -0,0 +1,86 @@ +{ + "id": "9a56e977-b437-4a57-941f-120c7f3dd40e", + "ok": true, + "result": { + "runId": "run_5a5f2688c661", + "taskId": "task_e6721765bb09", + "dispatchId": "ctx_68f16395149d", + "state": "ready", + "stage": "input_accepted", + "turnStart": "observed", + "setup": { + "requested": "not_applicable", + "effective": "not_applicable", + "source": "existing_worktree", + "hookFound": false, + "startupPolicy": "start-immediately", + "state": "not_applicable" + }, + "launch": { + "requested": { + "agent": "claude", + "model": "claude-opus-5", + "effort": "high" + }, + "effective": { + "agent": "claude", + "model": "claude-opus-5", + "effort": "high" + } + }, + "mode": { + "mode": "terminal", + "preferred": "terminal", + "reason": "user_default", + "detail": "Started a terminal agent worker, the default for new agent tabs in your settings." + }, + "timeoutMs": 420000, + "effects": [ + { + "kind": "worktree", + "action": "reused", + "id": "6b8ade27-109b-443a-a2a1-d4f098ae6aa4::/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project" + }, + { + "kind": "setup", + "action": "not_applicable", + "state": "not_applicable" + }, + { + "kind": "terminal", + "role": "agent", + "action": "created", + "id": "term_443d588e-1c2c-440f-b36c-a28c84f7870c", + "surface": "visible" + }, + { + "kind": "dispatch_input", + "role": "agent", + "id": "term_443d588e-1c2c-440f-b36c-a28c84f7870c", + "state": "accepted" + } + ], + "prompt": { + "requestId": "a45e30f9-80ab-4367-be2b-afde8eda58b5", + "stages": [ + "input_accepted", + "turn_started" + ], + "provider": "claude", + "observation": "supported", + "processIncarnation": "ba5d1fb6-eb2a-40fa-9fd4-4562433153c6", + "generation": 2, + "baselineWorkingSequence": 0, + "baselineExplicitWorkingStartedAt": null, + "baselinePermissionSequence": 0 + }, + "residualResources": [], + "mutation": { + "requestId": "a45e30f9-80ab-4367-be2b-afde8eda58b5", + "replayed": false + } + }, + "_meta": { + "runtimeId": "7d2eb076-9fdd-4c8b-a706-5a37d6b15a6d" + } +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/export-receipt.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/export-receipt.json new file mode 100644 index 0000000..ab81090 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/export-receipt.json @@ -0,0 +1,109 @@ +{ + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "c996beacea9d24bdb485e1d359a54747ae2344131fc78909217ee2bc4991d3c5", + "size_bytes": 343 + }, + { + "path": "backend-status.json", + "sha256": "578a295cc17196b3586022b6bfd4d9fde730e646de6bd0f660cb6323e53fafb4", + "size_bytes": 1374 + }, + { + "path": "check.stderr", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "check.stdout", + "sha256": "fd062483e16ca3fe0050ec7b982d08add70b72c2b796a9043455d9d82f6771c8", + "size_bytes": 103 + }, + { + "path": "diff.patch", + "sha256": "77d82181227d3ffdbde56de7f6b5e3783e9914d78580ac648cab8e47d9e23fec", + "size_bytes": 65 + }, + { + "path": "dispatch/completion-wait.stderr", + "sha256": "53562e7ef430fc26bcbe371bbde4fc1bc753da61884776b862cbe08cac80d1da", + "size_bytes": 76 + }, + { + "path": "dispatch/completion-wait.stdout", + "sha256": "7631d7c8e88ad35b5a63cd142bc7be210576a7afb3aab5daa155c6ccf832e5ad", + "size_bytes": 1506 + }, + { + "path": "dispatch/run-create.stdout", + "sha256": "9ee965eff898a62032de6f2cea227376e73de08606a5f6bfe94f1c453f2cf7dc", + "size_bytes": 569 + }, + { + "path": "dispatch/worker-start.stdout", + "sha256": "139acbafb14b19ad6de114df8d2dbad314a18cc5d038b812987b95941ce34a49", + "size_bytes": 2323 + }, + { + "path": "first-run-state.json", + "sha256": "7e6840655893af6d27eaa7b31306aff9af61209e43c75c243da517c5ee01cdeb", + "size_bytes": 611 + }, + { + "path": "host-before.json", + "sha256": "6d0176a7f3da708508eb865dd74efae495d5fb39dd1f017bf1d9d48165b56dc2", + "size_bytes": 1568 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "26b323166ad7f2827a6163baeda435aa2624fd1628573edf12b64931afbb5f3d", + "size_bytes": 1886 + }, + { + "path": "identity/host-probe.txt", + "sha256": "72648f40ecfb3adbb1027b1be9c8270f187d0706e5b5ddd41b5292b50abaa179", + "size_bytes": 1668 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "5d98e1c0fa7c596491842bb3d14e7a3e2e364de7acc8197d1f85240bd34678ee", + "size_bytes": 3181 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "b84d5ce115b6d7aa4f45f97fe89d56a01260f900b82f4f21b8f3c5c61a790ac7", + "size_bytes": 15053 + }, + { + "path": "logs/runner.log", + "sha256": "1ea53a14f619659363d09c4676a528f65b803b65c96683cc3f1f06cfc032ea6f", + "size_bytes": 2063 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "895bcf02bc37659c7479b7062244fd563652b29c0490e0cf8540ef803d468eab", + "size_bytes": 36232 + }, + { + "path": "task-artifact/evidence.txt", + "sha256": "08b84ed9a0ffcce4c1802c042afa823ff70a713da509491790bf65fc294ab268", + "size_bytes": 17 + } + ], + "confirmed_at": "2026-09-15T18:41:26Z", + "detail": "20 artifact(s) re-read from the host; 0 unresolved", + "missing": [], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/first-run-state.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/first-run-state.json new file mode 100644 index 0000000..044923c --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/first-run-state.json @@ -0,0 +1,15 @@ +{ + "detail": "2 first-run state file(s) written into the per-run home, answering 4 question(s) the agent asks a home it has never run in", + "entries": [ + ".claude.json", + ".claude/settings.json" + ], + "questions": [ + "onboarding, which otherwise offers a theme and an interactive sign-in", + "trusting the project copy, by its path inside the environment", + "the external imports the project's own instructions declare", + "the permission mode the execution plane launches the agent with" + ], + "status": "OK", + "target": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home" +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-environment-probe.txt b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-environment-probe.txt new file mode 100644 index 0000000..67ab937 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-environment-probe.txt @@ -0,0 +1,14 @@ +hostname=dely-cycle-20260915t183739z-bbc5e6-9088bc04 +machine_id= +boot_id=b57c7e63-6a5c-4bac-92ba-eb33ff1305f8 +home=/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home +user= +path=/home//.local/bin:/home//.local/bin:/home//.polygraph/bin:/home//.local/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.config/orca/linux-orca-cli-shim:/home//.local/bin:/home//.polygraph/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.nvm/versions/node/v26.7.0/bin:/home//.volta/bin:/home//.asdf/shims:/home//.fnm/aliases/default/bin:/home//.local/share/mise/shims:/home//.yarn/bin:/home//.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/home//bin:/home//.nix-profile/bin:/home//.vite-plus/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin +pid=1002977 +uname=Linux 7.2.5-200.fc44.x86_64 x86_64 +container_marker=containerenv +virt=docker +orca_path=/opt/Orca/resources/bin/orca-ide +orca_version=1.4.201 +orca_fingerprint=101026788:1592:1789327972 +project_real=/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-host-probe.txt b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-host-probe.txt new file mode 100644 index 0000000..4f5d914 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-host-probe.txt @@ -0,0 +1,14 @@ +hostname=system +machine_id= +boot_id=b57c7e63-6a5c-4bac-92ba-eb33ff1305f8 +home=/home/ +user= +path=/home//.local/bin:/home//.local/bin:/home//.polygraph/bin:/home//.local/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.config/orca/linux-orca-cli-shim:/home//.local/bin:/home//.polygraph/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.nvm/versions/node/v26.7.0/bin:/home//.volta/bin:/home//.asdf/shims:/home//.fnm/aliases/default/bin:/home//.local/share/mise/shims:/home//.yarn/bin:/home//.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/home//bin:/home//.nix-profile/bin:/home//.vite-plus/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin +pid=1002844 +uname=Linux 7.2.5-200.fc44.x86_64 x86_64 +container_marker= +virt=none +orca_path=/opt/Orca/resources/bin/orca-ide +orca_version=1.4.201 +orca_fingerprint=7199608:1592:1789327972 +project_real=/var/tmp/dely-cycle/project-source diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-orca-runtime.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-orca-runtime.json new file mode 100644 index 0000000..1ffbee5 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-orca-runtime.json @@ -0,0 +1,90 @@ +{ + "app_pid": 1003402, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "7d2eb076-9fdd-4c8b-a706-5a37d6b15a6d", + "runtime_state": "ready" +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/manifest.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/manifest.json new file mode 100644 index 0000000..c611926 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/manifest.json @@ -0,0 +1,1249 @@ +{ + "artifact_root": "/var/tmp/dely-cycle/artifacts/20260915T183739Z-bbc5e6-9088bc04", + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "c996beacea9d24bdb485e1d359a54747ae2344131fc78909217ee2bc4991d3c5", + "size_bytes": 343 + }, + { + "path": "backend-status.json", + "sha256": "578a295cc17196b3586022b6bfd4d9fde730e646de6bd0f660cb6323e53fafb4", + "size_bytes": 1374 + }, + { + "path": "check.stderr", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "check.stdout", + "sha256": "fd062483e16ca3fe0050ec7b982d08add70b72c2b796a9043455d9d82f6771c8", + "size_bytes": 103 + }, + { + "path": "diff.patch", + "sha256": "77d82181227d3ffdbde56de7f6b5e3783e9914d78580ac648cab8e47d9e23fec", + "size_bytes": 65 + }, + { + "path": "dispatch/completion-wait.stderr", + "sha256": "53562e7ef430fc26bcbe371bbde4fc1bc753da61884776b862cbe08cac80d1da", + "size_bytes": 76 + }, + { + "path": "dispatch/completion-wait.stdout", + "sha256": "7631d7c8e88ad35b5a63cd142bc7be210576a7afb3aab5daa155c6ccf832e5ad", + "size_bytes": 1506 + }, + { + "path": "dispatch/run-create.stdout", + "sha256": "9ee965eff898a62032de6f2cea227376e73de08606a5f6bfe94f1c453f2cf7dc", + "size_bytes": 569 + }, + { + "path": "dispatch/worker-start.stdout", + "sha256": "139acbafb14b19ad6de114df8d2dbad314a18cc5d038b812987b95941ce34a49", + "size_bytes": 2323 + }, + { + "path": "first-run-state.json", + "sha256": "7e6840655893af6d27eaa7b31306aff9af61209e43c75c243da517c5ee01cdeb", + "size_bytes": 611 + }, + { + "path": "host-before.json", + "sha256": "6d0176a7f3da708508eb865dd74efae495d5fb39dd1f017bf1d9d48165b56dc2", + "size_bytes": 1568 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "26b323166ad7f2827a6163baeda435aa2624fd1628573edf12b64931afbb5f3d", + "size_bytes": 1886 + }, + { + "path": "identity/host-probe.txt", + "sha256": "72648f40ecfb3adbb1027b1be9c8270f187d0706e5b5ddd41b5292b50abaa179", + "size_bytes": 1668 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "5d98e1c0fa7c596491842bb3d14e7a3e2e364de7acc8197d1f85240bd34678ee", + "size_bytes": 3181 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "b84d5ce115b6d7aa4f45f97fe89d56a01260f900b82f4f21b8f3c5c61a790ac7", + "size_bytes": 15053 + }, + { + "path": "logs/runner.log", + "sha256": "1ea53a14f619659363d09c4676a528f65b803b65c96683cc3f1f06cfc032ea6f", + "size_bytes": 2063 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "895bcf02bc37659c7479b7062244fd563652b29c0490e0cf8540ef803d468eab", + "size_bytes": 36232 + }, + { + "path": "task-artifact/evidence.txt", + "sha256": "08b84ed9a0ffcce4c1802c042afa823ff70a713da509491790bf65fc294ab268", + "size_bytes": 17 + } + ], + "auth": { + "detail": "the copied auth material was removed from the per-run home and verified gone", + "entries": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login", + "removed_after_run": true, + "status": "OK", + "target": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home" + }, + "backend": "distrobox", + "check": { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "\nset -u\ntarget=\"$1\"\nexpected=\"$2\"\nif [ ! -f \"$target\" ]; then\n printf 'missing: %s\\n' \"$target\" >&2\n exit 3\nfi\nactual=\"$(cat \"$target\")\"\nif [ \"$actual\" = \"$expected\" ]; then\n printf 'marker matched at %s\\n' \"$target\"\n exit 0\nfi\nprintf 'marker mismatch at %s\\n' \"$target\" >&2\nexit 4\n", + "cycle-check", + "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project/evidence.txt", + "dely-cycle-marker" + ], + "detail": "the independent check observed the marker", + "elapsed_seconds": 0.738, + "exit_code": 0, + "expected_marker": "dely-cycle-marker", + "finished_at": "2026-09-15T18:41:26Z", + "started_at": "2026-09-15T18:41:26Z", + "status": "OK", + "stderr_path": "check.stderr", + "stdout_path": "check.stdout" + }, + "cleanup": { + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "container:dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "path:/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04" + ], + "retained": [], + "shared_preserved": [ + "image:docker.io/library/ubuntu:24.04" + ], + "status": "DESTROYED", + "verified": true + }, + "config": { + "artifact_root": "/var/tmp/dely-cycle/artifacts", + "auth": { + "allowlist": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login" + }, + "backend": "distrobox", + "check": { + "expected_marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "claude_code_version": "2.1.272", + "dely_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "distrobox": { + "accept_host_home_mount": true, + "container_prefix": "dely-cycle", + "extra_mounts": [], + "image": "docker.io/library/ubuntu:24.04", + "provision": [ + [ + "sh", + "-c", + "set -eu; export DEBIAN_FRONTEND=noninteractive; sudo apt-get update -qq; sudo apt-get install -y -qq curl ca-certificates git nodejs npm xvfb x11-xserver-utils xdotool libgtk-3-0t64 libnss3 libasound2t64 libatk-bridge2.0-0t64 libcups2t64 libdrm2 libgbm1 libxkbcommon0 libpango-1.0-0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libsecret-1-0 xdg-utils > /dev/null; echo deps installed" + ], + [ + "sh", + "-c", + "set -eu; curl -fsSL -o /tmp/orca.deb 'https://github.com/stablyai/orca/releases/download/v1.4.201/orca-ide_1.4.201_amd64.deb'; echo '9dab79a5223837fc91992e04ed54f92f45d6f4b486d7128c08166bfb1a15a573 /tmp/orca.deb' | sha256sum -c -; sudo apt-get install -y -qq /tmp/orca.deb > /dev/null; rm -f /tmp/orca.deb; test -x /opt/Orca/resources/bin/orca-ide; echo orca installed" + ], + [ + "sh", + "-c", + "set -eu; sudo npm install -g --no-fund --no-audit --silent '@anthropic-ai/claude-code@2.1.272' > /dev/null; claude --version" + ], + [ + "sh", + "-c", + "set -eu; if DISPLAY=:99 xset -q > /dev/null 2>&1; then echo 'virtual display already up'; exit 0; fi; setsid nohup Xvfb :99 -screen 0 1280x800x24 -nolisten tcp > /tmp/xvfb.log 2>&1 < /dev/null & waited=0; while ! DISPLAY=:99 xset -q > /dev/null 2>&1; do waited=$((waited + 1)); if [ $waited -ge 60 ]; then echo 'no virtual display' >&2; exit 3; fi; sleep 1; done; echo \"virtual display ready after ${waited}s\"" + ] + ] + }, + "orca": { + "agent": "claude", + "app_argv": [ + "/opt/Orca/orca-ide", + "--disable-gpu" + ], + "command": "/opt/Orca/resources/bin/orca-ide", + "display": ":99", + "effort": "high", + "model": "claude-opus-5", + "ready_timeout_seconds": 300, + "run_objective": "dely minimal cycle", + "status_argv": [ + "/opt/Orca/resources/bin/orca-ide", + "status", + "--json" + ], + "version": "1.4.201", + "version_argv": [ + "/opt/Orca/resources/bin/orca-ide", + "--version" + ], + "worktree_selector": "current" + }, + "project": { + "environment_path": "project", + "revision": "497660005edd04cb399886334863ffe8fe5c31be", + "source": "/var/tmp/dely-cycle/project-source" + }, + "state_root": "/var/tmp/dely-cycle/state", + "task": { + "marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "tested_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "timeout_seconds": 420 + }, + "dely_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "elapsed_seconds": 234.337, + "environment": { + "backend": { + "backend": "distrobox", + "container": "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "container_manager": "podman", + "distrobox_version": "distrobox: 1.8.2.5", + "image": "docker.io/library/ubuntu:24.04", + "manifest": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/distrobox.ini" + }, + "description": { + "container_manager": "podman", + "desktop_integration": "Distrobox integrates with the host desktop by design; this run limits state with a separate home and project copy, and does not claim container-grade isolation", + "host_home_mount_acknowledged": true, + "image": "docker.io/library/ubuntu:24.04", + "manifest": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/distrobox.ini" + }, + "environment_id": "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "home_path": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home", + "per_run_resources": [ + { + "identifier": "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "kind": "container" + }, + { + "identifier": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04", + "kind": "path" + } + ], + "project_path": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", + "shared_resources": [ + { + "identifier": "docker.io/library/ubuntu:24.04", + "kind": "image" + } + ] + }, + "environment_id": "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "export": { + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "c996beacea9d24bdb485e1d359a54747ae2344131fc78909217ee2bc4991d3c5", + "size_bytes": 343 + }, + { + "path": "backend-status.json", + "sha256": "578a295cc17196b3586022b6bfd4d9fde730e646de6bd0f660cb6323e53fafb4", + "size_bytes": 1374 + }, + { + "path": "check.stderr", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "check.stdout", + "sha256": "fd062483e16ca3fe0050ec7b982d08add70b72c2b796a9043455d9d82f6771c8", + "size_bytes": 103 + }, + { + "path": "diff.patch", + "sha256": "77d82181227d3ffdbde56de7f6b5e3783e9914d78580ac648cab8e47d9e23fec", + "size_bytes": 65 + }, + { + "path": "dispatch/completion-wait.stderr", + "sha256": "53562e7ef430fc26bcbe371bbde4fc1bc753da61884776b862cbe08cac80d1da", + "size_bytes": 76 + }, + { + "path": "dispatch/completion-wait.stdout", + "sha256": "7631d7c8e88ad35b5a63cd142bc7be210576a7afb3aab5daa155c6ccf832e5ad", + "size_bytes": 1506 + }, + { + "path": "dispatch/run-create.stdout", + "sha256": "9ee965eff898a62032de6f2cea227376e73de08606a5f6bfe94f1c453f2cf7dc", + "size_bytes": 569 + }, + { + "path": "dispatch/worker-start.stdout", + "sha256": "139acbafb14b19ad6de114df8d2dbad314a18cc5d038b812987b95941ce34a49", + "size_bytes": 2323 + }, + { + "path": "first-run-state.json", + "sha256": "7e6840655893af6d27eaa7b31306aff9af61209e43c75c243da517c5ee01cdeb", + "size_bytes": 611 + }, + { + "path": "host-before.json", + "sha256": "6d0176a7f3da708508eb865dd74efae495d5fb39dd1f017bf1d9d48165b56dc2", + "size_bytes": 1568 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "26b323166ad7f2827a6163baeda435aa2624fd1628573edf12b64931afbb5f3d", + "size_bytes": 1886 + }, + { + "path": "identity/host-probe.txt", + "sha256": "72648f40ecfb3adbb1027b1be9c8270f187d0706e5b5ddd41b5292b50abaa179", + "size_bytes": 1668 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "5d98e1c0fa7c596491842bb3d14e7a3e2e364de7acc8197d1f85240bd34678ee", + "size_bytes": 3181 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "b84d5ce115b6d7aa4f45f97fe89d56a01260f900b82f4f21b8f3c5c61a790ac7", + "size_bytes": 15053 + }, + { + "path": "logs/runner.log", + "sha256": "1ea53a14f619659363d09c4676a528f65b803b65c96683cc3f1f06cfc032ea6f", + "size_bytes": 2063 + }, + { + "path": "preflight.json", + "sha256": "cf001294a15d21827f09103ed1a6c9b70b3ae575ce52079ac8ac34335bdafea3", + "size_bytes": 928 + }, + { + "path": "run-before-cleanup.json", + "sha256": "895bcf02bc37659c7479b7062244fd563652b29c0490e0cf8540ef803d468eab", + "size_bytes": 36232 + }, + { + "path": "task-artifact/evidence.txt", + "sha256": "08b84ed9a0ffcce4c1802c042afa823ff70a713da509491790bf65fc294ab268", + "size_bytes": 17 + } + ], + "confirmed_at": "2026-09-15T18:41:26Z", + "detail": "20 artifact(s) re-read from the host; 0 unresolved", + "missing": [], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" + }, + "failure_classification": "", + "finished_at": "2026-09-15T18:41:34Z", + "first_run": { + "detail": "2 first-run state file(s) written into the per-run home, answering 4 question(s) the agent asks a home it has never run in", + "entries": [ + ".claude.json", + ".claude/settings.json" + ], + "questions": [ + "onboarding, which otherwise offers a theme and an interactive sign-in", + "trusting the project copy, by its path inside the environment", + "the external imports the project's own instructions declare", + "the permission mode the execution plane launches the agent with" + ], + "status": "OK", + "target": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home" + }, + "generated_at": "2026-09-15T18:41:34Z", + "host_after": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T182804Z-bbc5e6-366fa4a0", + "20260915T183739Z-bbc5e6-9088bc04", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011", + "20260915T230000Z-bbbbbb-00000012", + "20260916T000000Z-bbbbbb-00000013", + "20260916T010000Z-bbbbbb-00000014", + "20260916T020000Z-cccccc-00000001", + "20260916T030000Z-cccccc-00000002", + "20260916T040000Z-cccccc-00000003", + "20260916T050000Z-cccccc-00000004" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T18:41:34Z", + "extra": { + "backend": "distrobox" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "host_before": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T182804Z-bbc5e6-366fa4a0", + "20260915T183739Z-bbc5e6-9088bc04", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011", + "20260915T230000Z-bbbbbb-00000012", + "20260916T000000Z-bbbbbb-00000013", + "20260916T010000Z-bbbbbb-00000014", + "20260916T020000Z-cccccc-00000001", + "20260916T030000Z-cccccc-00000002", + "20260916T040000Z-cccccc-00000003", + "20260916T050000Z-cccccc-00000004" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T18:37:40Z", + "extra": { + "backend": "distrobox" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "identity": { + "environment": { + "container_marker": "containerenv", + "home": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home", + "hostname": "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "orca_fingerprint": "101026788:1592:1789327972", + "orca_path": "/opt/Orca/resources/bin/orca-ide", + "orca_version": "1.4.201", + "path": "/home//.local/bin:/home//.local/bin:/home//.polygraph/bin:/home//.local/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.config/orca/linux-orca-cli-shim:/home//.local/bin:/home//.polygraph/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.nvm/versions/node/v26.7.0/bin:/home//.volta/bin:/home//.asdf/shims:/home//.fnm/aliases/default/bin:/home//.local/share/mise/shims:/home//.yarn/bin:/home//.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/home//bin:/home//.nix-profile/bin:/home//.vite-plus/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin", + "pid": "1002977", + "project_real": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", + "uname": "Linux 7.2.5-200.fc44.x86_64 x86_64", + "user": "", + "virt": "docker" + }, + "host": { + "boot_id_digest": "de0d3220ff53399e", + "home_digest": "8fda3821c06d512d", + "hostname_digest": "bbc5e661e106c6dc", + "machine_id_digest": "907fa710d6d1c501", + "orca_fingerprint_digest": "5d53db3b52dba2dc", + "orca_path_digest": "7db47dea0d8ebe1f", + "orca_present": true, + "project_real_digest": "0d03c9e72dddfdae", + "user_digest": "3cc9cee71b0b967f", + "virt": "none" + }, + "markers": [ + "container-marker:containerenv", + "distinct-hostname", + "reported-virtualisation:docker" + ], + "orca_is_host_installation": false, + "orca_present": true, + "orca_runtime": { + "app_pid": 1003402, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "7d2eb076-9fdd-4c8b-a706-5a37d6b15a6d", + "runtime_state": "ready" + }, + "orca_status_exit_code": 0, + "orca_version": "1.4.201", + "reason": "the environment answered with its own identity, home and project copy", + "verdict": "ENVIRONMENT" + }, + "phases": [ + { + "commands": [], + "detail": "", + "elapsed_seconds": 1.291, + "finished_at": "2026-09-15T18:37:41Z", + "name": "prepare", + "started_at": "2026-09-15T18:37:40Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 84.838, + "finished_at": "2026-09-15T18:39:06Z", + "name": "create", + "started_at": "2026-09-15T18:37:41Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "set -eu; export DEBIAN_FRONTEND=noninteractive; sudo apt-get update -qq; sudo apt-get install -y -qq curl ca-certificates git nodejs npm xvfb x11-xserver-utils xdotool libgtk-3-0t64 libnss3 libasound2t64 libatk-bridge2.0-0t64 libcups2t64 libdrm2 libgbm1 libxkbcommon0 libpango-1.0-0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libsecret-1-0 xdg-utils > /dev/null; echo deps installed" + ], + "context": "environment", + "elapsed_seconds": 64.41, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:10Z", + "started_at": "2026-09-15T18:39:06Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "set -eu; curl -fsSL -o /tmp/orca.deb 'https://github.com/stablyai/orca/releases/download/v1.4.201/orca-ide_1.4.201_amd64.deb'; echo '9dab79a5223837fc91992e04ed54f92f45d6f4b486d7128c08166bfb1a15a573 /tmp/orca.deb' | sha256sum -c -; sudo apt-get install -y -qq /tmp/orca.deb > /dev/null; rm -f /tmp/orca.deb; test -x /opt/Orca/resources/bin/orca-ide; echo orca installed" + ], + "context": "environment", + "elapsed_seconds": 23.794, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:34Z", + "started_at": "2026-09-15T18:40:10Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "set -eu; sudo npm install -g --no-fund --no-audit --silent '@anthropic-ai/claude-code@2.1.272' > /dev/null; claude --version" + ], + "context": "environment", + "elapsed_seconds": 10.285, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:44Z", + "started_at": "2026-09-15T18:40:34Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "set -eu; if DISPLAY=:99 xset -q > /dev/null 2>&1; then echo 'virtual display already up'; exit 0; fi; setsid nohup Xvfb :99 -screen 0 1280x800x24 -nolisten tcp > /tmp/xvfb.log 2>&1 < /dev/null & waited=0; while ! DISPLAY=:99 xset -q > /dev/null 2>&1; do waited=$((waited + 1)); if [ $waited -ge 60 ]; then echo 'no virtual display' >&2; exit 3; fi; sleep 1; done; echo \"virtual display ready after ${waited}s\"" + ], + "context": "environment", + "elapsed_seconds": 1.476, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:46Z", + "started_at": "2026-09-15T18:40:44Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", + "init", + "-q", + "-b", + "main" + ], + "context": "environment", + "elapsed_seconds": 0.474, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:46Z", + "started_at": "2026-09-15T18:40:46Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", + "config", + "user.email", + "dely-cycle@invalid" + ], + "context": "environment", + "elapsed_seconds": 0.411, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:47Z", + "started_at": "2026-09-15T18:40:46Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", + "config", + "user.name", + "dely-cycle" + ], + "context": "environment", + "elapsed_seconds": 0.432, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:47Z", + "started_at": "2026-09-15T18:40:47Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", + "add", + "-A" + ], + "context": "environment", + "elapsed_seconds": 0.59, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:48Z", + "started_at": "2026-09-15T18:40:47Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "git", + "-C", + "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", + "commit", + "-q", + "-m", + "dely-cycle: the pinned revision as exported" + ], + "context": "environment", + "elapsed_seconds": 0.401, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:48Z", + "started_at": "2026-09-15T18:40:48Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "", + "elapsed_seconds": 102.301, + "finished_at": "2026-09-15T18:40:48Z", + "name": "bootstrap", + "started_at": "2026-09-15T18:39:06Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '')\"\norca_release=''\norca_fingerprint=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '%s' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '%i:%s:%Y' \"$orca_resolved\" 2>/dev/null || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id= \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'orca_fingerprint=%s\\n' \"$orca_fingerprint\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/project-source", + "/opt/Orca/resources/bin/orca-ide" + ], + "context": "host", + "elapsed_seconds": 0.434, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:48Z", + "started_at": "2026-09-15T18:40:48Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '')\"\norca_release=''\norca_fingerprint=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '%s' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '%i:%s:%Y' \"$orca_resolved\" 2>/dev/null || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id= \"$(first_line /etc/machine-id)\"\nprintf 'boot_id=%s\\n' \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'orca_fingerprint=%s\\n' \"$orca_fingerprint\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", + "/opt/Orca/resources/bin/orca-ide" + ], + "context": "environment", + "elapsed_seconds": 0.67, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:49Z", + "started_at": "2026-09-15T18:40:48Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "/opt/Orca/resources/bin/orca-ide", + "status", + "--json" + ], + "context": "environment", + "elapsed_seconds": 0.7, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:50Z", + "started_at": "2026-09-15T18:40:49Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "export DISPLAY=\"$1\"; shift; export XDG_RUNTIME_DIR=\"${XDG_RUNTIME_DIR:-/run/user/$(id -u)}\"; waited=0; while ! xset -q >/dev/null 2>&1; do waited=$((waited + 2)); if [ \"$waited\" -ge 120 ]; then printf \"no display after %ss\n\" \"$waited\" >&2; exit 3; fi; sleep 2; done; printf \"display ready after %ss\n\" \"$waited\"; rm -f \"$HOME/.config/orca/SingletonLock\" \"$HOME/.config/orca/SingletonCookie\"; nohup \"$@\" > \"$HOME/orca-app.log\" 2>&1 & printf \"started %s\n\" \"$!\"", + "orca-start", + ":99", + "/opt/Orca/orca-ide", + "--disable-gpu" + ], + "context": "environment", + "elapsed_seconds": 0.474, + "exit_code": 0, + "finished_at": "2026-09-15T18:40:51Z", + "started_at": "2026-09-15T18:40:50Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "orca runtime 7d2eb076-9fdd-4c8b-a706-5a37d6b15a6d is ready and the coordinator terminal is term_78d6aa0b-d97c-42d4-984c-fae24707c9b9", + "elapsed_seconds": 11.128, + "finished_at": "2026-09-15T18:40:59Z", + "name": "identity", + "started_at": "2026-09-15T18:40:48Z", + "status": "OK" + }, + { + "commands": [], + "detail": "the worker reported worker_done with outcome 'worker_done'", + "elapsed_seconds": 26.515, + "finished_at": "2026-09-15T18:41:26Z", + "name": "task", + "started_at": "2026-09-15T18:40:59Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "sh", + "-c", + "\nset -u\ntarget=\"$1\"\nexpected=\"$2\"\nif [ ! -f \"$target\" ]; then\n printf 'missing: %s\\n' \"$target\" >&2\n exit 3\nfi\nactual=\"$(cat \"$target\")\"\nif [ \"$actual\" = \"$expected\" ]; then\n printf 'marker matched at %s\\n' \"$target\"\n exit 0\nfi\nprintf 'marker mismatch at %s\\n' \"$target\" >&2\nexit 4\n", + "cycle-check", + "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project/evidence.txt", + "dely-cycle-marker" + ], + "context": "environment", + "elapsed_seconds": 0.738, + "exit_code": 0, + "finished_at": "2026-09-15T18:41:26Z", + "started_at": "2026-09-15T18:41:26Z", + "stderr_path": "check.stderr", + "stdout_path": "check.stdout", + "timed_out": false + } + ], + "detail": "", + "elapsed_seconds": 0.739, + "finished_at": "2026-09-15T18:41:26Z", + "name": "check", + "started_at": "2026-09-15T18:41:26Z", + "status": "OK" + }, + { + "commands": [], + "detail": "4 patch line(s) collected", + "elapsed_seconds": 0.068, + "finished_at": "2026-09-15T18:41:26Z", + "name": "collect", + "started_at": "2026-09-15T18:41:26Z", + "status": "OK" + }, + { + "commands": [], + "detail": "20 artifact(s) re-read from the host; 0 unresolved", + "elapsed_seconds": 0.05, + "finished_at": "2026-09-15T18:41:26Z", + "name": "export", + "started_at": "2026-09-15T18:41:26Z", + "status": "OK" + }, + { + "commands": [], + "detail": "every declared per-run resource is gone and every shared one remains", + "elapsed_seconds": 7.406, + "finished_at": "2026-09-15T18:41:34Z", + "name": "cleanup", + "started_at": "2026-09-15T18:41:26Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": null, + "finished_at": null, + "name": "close", + "started_at": "2026-09-15T18:41:34Z", + "status": "OK" + } + ], + "run_id": "20260915T183739Z-bbc5e6-9088bc04", + "schema_version": 1, + "started_at": "2026-09-15T18:37:40Z", + "status": "SETTLED", + "tested_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "timeout_seconds": 420, + "versions": { + "configured_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "distrobox": "distrobox: 1.8.2.5", + "git": "git version 2.55.0", + "podman": "podman version 5.8.4", + "project_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "pulumi": "v3.262.0", + "python": "3.14.7", + "qemu_img": "Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers", + "runner": "minimal-cycle", + "virsh": "12.0.0" + }, + "worker": { + "agent": "claude", + "commands": [ + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "orca", + "orchestration", + "run-create", + "--objective", + "dely minimal cycle", + "--json", + "--from", + "term_78d6aa0b-d97c-42d4-984c-fae24707c9b9" + ], + "context": "environment", + "elapsed_seconds": 0.727, + "exit_code": 0, + "finished_at": "2026-09-15T18:41:00Z", + "started_at": "2026-09-15T18:40:59Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "orca", + "orchestration", + "worker-start", + "--spec", + "Read the file /var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/dispatch-prompt.md in this environment and do exactly what it says.", + "--agent", + "claude", + "--model", + "claude-opus-5", + "--effort", + "high", + "--worktree", + "current", + "--timeout-ms", + "420000", + "--json", + "--run", + "run_5a5f2688c661", + "--from", + "term_78d6aa0b-d97c-42d4-984c-fae24707c9b9" + ], + "context": "environment", + "elapsed_seconds": 5.511, + "exit_code": 0, + "finished_at": "2026-09-15T18:41:05Z", + "started_at": "2026-09-15T18:41:00Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "distrobox", + "enter", + "--name", + "dely-cycle-20260915t183739z-bbc5e6-9088bc04", + "-T", + "--", + "sh", + "-c", + "for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"", + "cycle-isolate", + "--", + "orca", + "orchestration", + "check", + "--wait", + "--types", + "worker_done,escalation,question", + "--timeout-ms", + "420000", + "--json", + "--run", + "run_5a5f2688c661", + "--terminal", + "term_78d6aa0b-d97c-42d4-984c-fae24707c9b9" + ], + "context": "environment", + "elapsed_seconds": 20.258, + "exit_code": 0, + "finished_at": "2026-09-15T18:41:26Z", + "started_at": "2026-09-15T18:41:05Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "the worker reported worker_done with outcome 'worker_done'", + "dispatch_id": "ctx_68f16395149d", + "effort": "high", + "model": "claude-opus-5", + "outcome": "worker_done", + "run_id": "run_5a5f2688c661", + "status": "OK" + } +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/preflight.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/preflight.json new file mode 100644 index 0000000..6782bdd --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/preflight.json @@ -0,0 +1,36 @@ +{ + "backend": "distrobox", + "findings": [ + { + "detail": "/usr/bin/distrobox", + "name": "distrobox on the path", + "ok": true + }, + { + "detail": "podman", + "name": "container manager on the path", + "ok": true + }, + { + "detail": "docker.io/library/ubuntu:24.04 is present", + "name": "image present locally", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/state", + "name": "state root writable", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/artifacts", + "name": "artifact root writable", + "ok": true + }, + { + "detail": "distrobox mounts /home/ into the box and offers no flag to suppress it; the configuration acknowledged this, so the run records a declared compromise rather than claiming isolation", + "name": "host home reachable from the environment", + "ok": true + } + ], + "ok": true +} diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/task-artifact-evidence.txt b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/task-artifact-evidence.txt new file mode 100644 index 0000000..2945440 --- /dev/null +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/task-artifact-evidence.txt @@ -0,0 +1 @@ +dely-cycle-marker \ No newline at end of file diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index 5124ac9..4915a1d 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -71,7 +71,8 @@ '{"id": "request-fake", "ok": true, "result": {"run": {"id": "run-fake"}, ' '"runId": "run-fake", "taskId": "task-fake", "dispatchId": "dispatch-fake", ' '"state": "ready", ' - '"messages": [{"type": "worker_done", "outcome": "DONE"}], "count": 1}}' + '"messages": [{"type": "worker_done", ' + '"payload": "{\\"outcome\\": \\"DONE\\"}"}], "count": 1}}' ) #: A wait that woke with nothing to report, in the same shape. diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py index 86343e2..e7ccbb4 100644 --- a/experiments/minimal-cycle/tests/test_worker.py +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -43,7 +43,15 @@ def delivery(*messages): ) -SETTLED = delivery({"type": "worker_done", "outcome": "DONE"}) +SETTLED = delivery( + { + "type": "worker_done", + "subject": "evidence.txt created with marker", + # The worker's own outcome travels as a JSON string in the payload; + # the message itself carries no outcome field at all. + "payload": '{"dispatchId": "dispatch-fake", "outcome": "DONE"}', + } +) DEFAULT_SCRIPT = [ ("run-create", 0, READY, "", False), @@ -153,7 +161,7 @@ def test_an_escalation_settles_the_wait_without_claiming_success(self): ( "check --wait", 0, - delivery({"type": "escalation", "outcome": "BLOCKED"}), + delivery({"type": "escalation", "payload": '{"outcome": "BLOCKED"}'}), "", False, ), From 0ee077e2e2b258b4105f51af33b3daf1a0e22844 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 21:05:56 +0200 Subject: [PATCH 36/42] Read the agent's terminal when the plane could not see its turn start. An unverifiable dispatch says the turn was not observed and nothing more, and the same reply names the terminal it created for the agent and the command to read it. What that screen holds is the difference between an agent still thinking, one stopped at a question nobody can answer, and a launch line that was never submitted. The run now keeps it. --- experiments/minimal-cycle/counterexamples.py | 11 +++ .../minimal-cycle/cycle_runner/worker.py | 30 ++++++++ .../identity-environment-probe.txt | 4 +- .../identity-host-probe.txt | 6 +- .../distrobox-settled-cycle/manifest.json | 4 +- .../distrobox-settled-cycle/preflight.json | 2 +- .../minimal-cycle/tests/test_worker.py | 75 +++++++++++++++++++ 7 files changed, 124 insertions(+), 8 deletions(-) diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index ce9acfa..b5c0d59 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -38,6 +38,17 @@ class Counterexample: CASES: tuple[Counterexample, ...] = ( + Counterexample( + name="an-unverifiable-dispatch-keeps-its-terminal", + requirement=( + "A dispatch the plane could not verify keeps what its agent's " + "terminal held, which is the only thing that explains it" + ), + path="cycle_runner/worker.py", + original=' worker_terminal = agent_terminal(start_document)\n if worker_terminal and (settled.timed_out or state in UNVERIFIABLE_STATES):', + replacement=' worker_terminal = agent_terminal(start_document)\n if worker_terminal and False:', + instruments=("tests.test_worker.UnverifiedStartEvidenceTest",), + ), Counterexample( name="the-outcome-is-the-workers-own-verdict", requirement=( diff --git a/experiments/minimal-cycle/cycle_runner/worker.py b/experiments/minimal-cycle/cycle_runner/worker.py index 005ad23..2fa670b 100644 --- a/experiments/minimal-cycle/cycle_runner/worker.py +++ b/experiments/minimal-cycle/cycle_runner/worker.py @@ -222,6 +222,28 @@ def _settling_message(document: Mapping[str, Any]) -> dict[str, Any]: return {} +def agent_terminal(document: Mapping[str, Any]) -> str | None: + """Return the handle of the terminal the plane created for the agent. + + A dispatch whose turn start was never observed says so and nothing more. + What that terminal holds is the difference between an agent that is still + thinking, one waiting on a question nobody can answer, and a launch line + that was never submitted — and the plane names the handle here, and names + the command to read it in the same reply. + """ + effects = _result(document).get("effects") + if not isinstance(effects, list): + return None + for effect in effects: + if not isinstance(effect, Mapping): + continue + if effect.get("kind") == "terminal" and effect.get("role") == "agent": + handle = effect.get("id") + if isinstance(handle, str) and handle: + return handle + return None + + def _message_outcome(message: Mapping[str, Any]) -> str | None: """Return what the worker said it reached, from the message's payload. @@ -350,6 +372,14 @@ def remember(name: str, outcome) -> None: ) record.commands.append(settled.to_record()) remember("completion-wait", settled) + worker_terminal = agent_terminal(start_document) + if worker_terminal and (settled.timed_out or state in UNVERIFIABLE_STATES): + screen = adapter.execute( + ["orca", "terminal", "read", "--terminal", worker_terminal, "--screen"], + timeout=min(120, timeout_seconds), + ) + record.commands.append(screen.to_record()) + remember("worker-terminal", screen) if settled.timed_out: record.status = PhaseStatus.TIMEOUT record.detail = "the completion wait reached the run deadline before the worker settled" diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-environment-probe.txt b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-environment-probe.txt index 67ab937..93299b9 100644 --- a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-environment-probe.txt +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-environment-probe.txt @@ -2,8 +2,8 @@ hostname=dely-cycle-20260915t183739z-bbc5e6-9088bc04 machine_id= boot_id=b57c7e63-6a5c-4bac-92ba-eb33ff1305f8 home=/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home -user= -path=/home//.local/bin:/home//.local/bin:/home//.polygraph/bin:/home//.local/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.config/orca/linux-orca-cli-shim:/home//.local/bin:/home//.polygraph/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.nvm/versions/node/v26.7.0/bin:/home//.volta/bin:/home//.asdf/shims:/home//.fnm/aliases/default/bin:/home//.local/share/mise/shims:/home//.yarn/bin:/home//.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/home//bin:/home//.nix-profile/bin:/home//.vite-plus/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin pid=1002977 uname=Linux 7.2.5-200.fc44.x86_64 x86_64 container_marker=containerenv diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-host-probe.txt b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-host-probe.txt index 4f5d914..83fcc47 100644 --- a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-host-probe.txt +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/identity-host-probe.txt @@ -1,9 +1,9 @@ hostname=system machine_id= boot_id=b57c7e63-6a5c-4bac-92ba-eb33ff1305f8 -home=/home/ -user= -path=/home//.local/bin:/home//.local/bin:/home//.polygraph/bin:/home//.local/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.config/orca/linux-orca-cli-shim:/home//.local/bin:/home//.polygraph/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.nvm/versions/node/v26.7.0/bin:/home//.volta/bin:/home//.asdf/shims:/home//.fnm/aliases/default/bin:/home//.local/share/mise/shims:/home//.yarn/bin:/home//.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/home//bin:/home//.nix-profile/bin:/home//.vite-plus/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin +home= +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin pid=1002844 uname=Linux 7.2.5-200.fc44.x86_64 x86_64 container_marker= diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/manifest.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/manifest.json index c611926..de06452 100644 --- a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/manifest.json +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/manifest.json @@ -508,11 +508,11 @@ "orca_fingerprint": "101026788:1592:1789327972", "orca_path": "/opt/Orca/resources/bin/orca-ide", "orca_version": "1.4.201", - "path": "/home//.local/bin:/home//.local/bin:/home//.polygraph/bin:/home//.local/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.config/orca/linux-orca-cli-shim:/home//.local/bin:/home//.polygraph/bin:/home//.opencode/bin:/home//.kimi-code/bin:/usr/lib/ccache/bin/:/home//.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.nvm/versions/node/v26.7.0/bin:/home//.volta/bin:/home//.asdf/shims:/home//.fnm/aliases/default/bin:/home//.local/share/mise/shims:/home//.yarn/bin:/home//.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/home//bin:/home//.nix-profile/bin:/home//.vite-plus/bin:/home//.cargo/bin/:/home//.local/bin/:/home//.pulumi/bin:/home//.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/home//.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin", + "path": "/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin:/usr/local/sbin:/usr/sbin:/usr/local/bin:/usr/bin:/sbin:/bin:/usr/local/sbin:/usr/sbin:/sbin", "pid": "1002977", "project_real": "/var/tmp/dely-cycle/state/20260915T183739Z-bbc5e6-9088bc04/home/project", "uname": "Linux 7.2.5-200.fc44.x86_64 x86_64", - "user": "", + "user": "", "virt": "docker" }, "host": { diff --git a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/preflight.json b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/preflight.json index 6782bdd..78f504e 100644 --- a/experiments/minimal-cycle/evidence/distrobox-settled-cycle/preflight.json +++ b/experiments/minimal-cycle/evidence/distrobox-settled-cycle/preflight.json @@ -27,7 +27,7 @@ "ok": true }, { - "detail": "distrobox mounts /home/ into the box and offers no flag to suppress it; the configuration acknowledged this, so the run records a declared compromise rather than claiming isolation", + "detail": "distrobox mounts into the box and offers no flag to suppress it; the configuration acknowledged this, so the run records a declared compromise rather than claiming isolation", "name": "host home reachable from the environment", "ok": true } diff --git a/experiments/minimal-cycle/tests/test_worker.py b/experiments/minimal-cycle/tests/test_worker.py index e7ccbb4..c263734 100644 --- a/experiments/minimal-cycle/tests/test_worker.py +++ b/experiments/minimal-cycle/tests/test_worker.py @@ -476,3 +476,78 @@ def test_a_kept_reply_carries_no_value_the_overlay_passed(self): ) for name, text in kept.items(): self.assertNotIn("sk-ant-secret-value-here", text, name) + + +class UnverifiedStartEvidenceTest(WorkerTest): + """When the plane cannot see the turn start, read the terminal it names.""" + + UNVERIFIED = json.dumps( + { + "id": "request-abcdef", + "ok": True, + "result": { + "runId": "run-abcdef", + "taskId": "task-abcdef", + "dispatchId": "dispatch-abcdef", + "state": "outcome_unknown", + "stage": "turn_start_unobserved", + # The plane names the terminal it created for the agent here, + # and names the command to read it in `nextCommands`. + "effects": [ + {"kind": "worktree", "action": "reused", "id": "wt"}, + { + "kind": "terminal", + "role": "agent", + "action": "created", + "id": "term_worker_fake", + }, + ], + }, + } + ) + SCREEN = "cycle@guest:~/project$ claude --dangerously-skip-permissions" + + def launch_unverified(self): + kept = {} + adapter = ScriptedAdapter( + self.root / "run", + script=[ + ("run-create", 0, READY, "", False), + ("worker-start", 1, self.UNVERIFIED, "", False), + ("check --wait", 0, delivery(), "", False), + ("terminal read", 0, self.SCREEN, "", False), + ], + ) + handle = adapter.create() + record = worker.launch( + run_config=self.config, + adapter=adapter, + handle=handle, + timeout_seconds=self.config.timeout_seconds, + keep=lambda name, stdout, stderr: kept.__setitem__(name, stdout), + ) + return adapter, kept, record + + def test_the_agent_terminal_is_read_and_kept(self): + adapter, kept, _ = self.launch_unverified() + read = next( + (argv for argv in adapter.executed if "read" in argv and "--screen" in argv), + None, + ) + self.assertIsNotNone(read, "the agent terminal was never read") + self.assertIn("term_worker_fake", read) + self.assertIn("worker-terminal", kept) + self.assertIn("claude", kept["worker-terminal"]) + + def test_a_dispatch_that_reports_normally_is_not_read(self): + kept = {} + adapter = ScriptedAdapter(self.root / "run", script=DEFAULT_SCRIPT) + handle = adapter.create() + worker.launch( + run_config=self.config, + adapter=adapter, + handle=handle, + timeout_seconds=self.config.timeout_seconds, + keep=lambda name, stdout, stderr: kept.__setitem__(name, stdout), + ) + self.assertNotIn("worker-terminal", kept) From d84b0fbef836f6ef6bf34eb430fbf02e13a9217d Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 21:36:03 +0200 Subject: [PATCH 37/42] Make the coordinator terminal say which machine it is on before anything is dispatched from it. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A command line inside an environment can reach a runtime outside it. The binary, the user data path and the project path all look right, the reply carries a handle and says the surface is visible, and the terminal belongs to the host — which on this host meant tabs opening on the operator's own desktop for work the environment never did. Nothing in the reply says so. The terminal is now asked, once, and has to answer with the name of the environment it is supposed to be in. A terminal that answers with another machine, or does not answer at all, stops the run before a worker is dispatched to it. --- experiments/minimal-cycle/README.md | 21 +- experiments/minimal-cycle/counterexamples.py | 14 + .../minimal-cycle/cycle_runner/lifecycle.py | 1 + .../minimal-cycle/cycle_runner/orca.py | 73 +- .../evidence/vm-unobserved-turn/cleanup.json | 15 + .../dispatch-completion-wait.json | 0 .../dispatch-worker-start.json | 105 ++ .../dispatch-worker-terminal.txt | 9 + .../vm-unobserved-turn/first-run-state.json | 15 + .../identity-environment-probe.txt | 14 + .../identity-host-probe.txt | 14 + .../identity-orca-runtime.json | 90 ++ .../evidence/vm-unobserved-turn/manifest.json | 1081 +++++++++++++++++ .../vm-unobserved-turn/preflight.json | 81 ++ experiments/minimal-cycle/tests/fakes.py | 15 + .../minimal-cycle/tests/test_lifecycle.py | 13 + experiments/minimal-cycle/tests/test_orca.py | 60 + 17 files changed, 1616 insertions(+), 5 deletions(-) create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/cleanup.json create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-completion-wait.json create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-worker-start.json create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-worker-terminal.txt create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/first-run-state.json create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-environment-probe.txt create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-host-probe.txt create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-orca-runtime.json create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/manifest.json create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/preflight.json diff --git a/experiments/minimal-cycle/README.md b/experiments/minimal-cycle/README.md index 012e16d..dbe7768 100644 --- a/experiments/minimal-cycle/README.md +++ b/experiments/minimal-cycle/README.md @@ -93,6 +93,8 @@ $artifact_root// preflight.json what the host could and could not do backend-status.json the environment and its declared resources auth-receipt.json the method and its status, never its material + first-run-state.json the questions the agent was answered, by name + dispatch/ each orchestration reply, redacted, as the plane sent it host-before.json the host before the run host-after.json the host after it, and what changed identity/host-probe.txt the probe as the host answered it @@ -258,10 +260,21 @@ the table with the tests each row runs. The recorded sweep is in | An argument vector survives the transport intact | case `remote-command-is-quoted` | `evidence/counterexamples.txt` | | Creation waits for the guest to answer, not just to take an address | case `an-address-is-not-readiness` | `evidence/counterexamples.txt` | | A real machine cycle creates a domain, proves a distinct kernel, and destroys it | `./run-cycle run` on this host | `evidence/vm-blocked-on-absent-orca/manifest.json` | +| First-run state is keyed to the project copy the agent will actually open | case `first-run-state-names-the-environment-copy` | `evidence/distrobox-settled-cycle/first-run-state.json` | +| A receipt that names four answered questions answers four | case `first-run-answers-every-question-it-names` | `evidence/distrobox-settled-cycle/first-run-state.json` | +| A settling message is read from the delivery, not from the request envelope | case `a-settling-message-is-read-from-the-delivery` | `evidence/distrobox-settled-cycle/dispatch-completion-wait.json` | +| The reported outcome is the worker's own verdict, not the message type | case `the-outcome-is-the-workers-own-verdict` | `evidence/distrobox-settled-cycle/dispatch-completion-wait.json` | +| The reply that decided the run is exported beside the verdict | case `the-reply-that-decided-the-run-is-kept` | `evidence/distrobox-settled-cycle/dispatch-worker-start.json` | +| A dispatch the plane could not verify keeps what its agent's terminal held | case `an-unverifiable-dispatch-keeps-its-terminal` | `evidence/vm-unobserved-turn/` | +| A worker really does the task and an independent check agrees | `./run-cycle run` on this host | `evidence/distrobox-settled-cycle/` | ## What no instrument here observes -Orca starting inside either backend, a window bound to that instance, a Claude -Code worker driven through it, or a login surviving a run. Both backends reach -their identity gate and stop there, for the same honest reason: the image they run does not carry Orca. `evidence/README.md` says -exactly where this host stopped and what it did prove on the way. +A second, different task. A run on any host but this one. Isolation on the +container backend, which mounts the host home and says so in `preflight.json` +rather than claiming otherwise. + +The machine backend has not completed a cycle. It creates its domain, proves a +distinct kernel, starts Orca, opens a coordinator terminal and dispatches a +worker, and the worker's turn does not begin there. `evidence/README.md` says +what each run showed and where each one stopped. diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index b5c0d59..5cef3f2 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -38,6 +38,20 @@ class Counterexample: CASES: tuple[Counterexample, ...] = ( + Counterexample( + name="the-terminal-says-which-machine-it-is-on", + requirement=( + "A coordinator terminal opened on the host looks identical to one " + "opened in the environment until it is asked" + ), + path="cycle_runner/orca.py", + original=' if expected_host:\n inside, detail = confirm_terminal_is_inside(', + replacement=' if False:\n inside, detail = confirm_terminal_is_inside(', + instruments=( + "tests.test_orca.CoordinatorTerminalIsInsideTest", + "tests.test_lifecycle.TerminalOutsideTheEnvironmentTest", + ), + ), Counterexample( name="an-unverifiable-dispatch-keeps-its-terminal", requirement=( diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index 3866509..95d3c6c 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -427,6 +427,7 @@ def _identity(self) -> None: self.adapter, self.handle.project_path, timeout=min(300, self.config.timeout_seconds), + expected_host=self.handle.environment_id, ) except orca.OrcaSessionError as error: record.status = PhaseStatus.BLOCKED diff --git a/experiments/minimal-cycle/cycle_runner/orca.py b/experiments/minimal-cycle/cycle_runner/orca.py index 742373d..500ecf6 100644 --- a/experiments/minimal-cycle/cycle_runner/orca.py +++ b/experiments/minimal-cycle/cycle_runner/orca.py @@ -156,8 +156,71 @@ def start_application( return environment.execute(start_argv(app_argv, display), timeout=timeout) +#: Printed by the coordinator terminal and read back from its screen. A +#: terminal that answers with any other name is not in this environment. +MARKER_PREFIX = "cycle-terminal:" + + +def marker_command(marker: str) -> list[str]: + """Return the command that makes a terminal say where it is.""" + return [ + "orca", + "terminal", + "send", + "--text", + f"printf '{MARKER_PREFIX}%s\\n' \"$(hostname)\"", + "--enter", + "--json", + ] + + +def confirm_terminal_is_inside( + environment: Environment, + handle: str, + expected: str, + *, + timeout: float, + attempts: int = 10, + sleeper: Callable[[float], None] = time.sleep, +) -> tuple[bool, str]: + """Make the terminal name its own machine, and check that it is this one. + + A command line inside an environment can reach a runtime outside it: the + binary, the user data path and the project path can all look right while + the terminal that opens belongs to the host. Nothing in the reply says so. + Asking the terminal itself does, and costs one round trip. + """ + argv = marker_command(expected) + argv.extend(["--terminal", handle]) + environment.execute(argv, timeout=timeout) + expected_line = f"{MARKER_PREFIX}{expected}" + seen = "" + for attempt in range(attempts): + read = environment.execute( + ["orca", "terminal", "read", "--terminal", handle, "--screen"], + timeout=timeout, + ) + seen = read.stdout or "" + if expected_line in seen: + return True, f"the terminal answered with {expected}" + if MARKER_PREFIX in seen: + answered = seen.split(MARKER_PREFIX, 1)[1].splitlines()[0].strip() + return False, ( + f"the terminal answered with {answered!r}, not {expected!r}: it is " + "not in this environment" + ) + if attempt + 1 < attempts: + sleeper(1.0) + return False, "the terminal never said which machine it is on" + + def open_coordinator_terminal( - environment: Environment, project_path: str, *, timeout: float + environment: Environment, + project_path: str, + *, + timeout: float, + expected_host: str | None = None, + sleeper: Callable[[float], None] = time.sleep, ) -> str: """Register the project copy with Orca and open the terminal to send from.""" environment.execute( @@ -187,4 +250,12 @@ def open_coordinator_terminal( "orca terminal create returned no terminal handle, so nothing can be " "dispatched from it" ) + if expected_host: + inside, detail = confirm_terminal_is_inside( + environment, handle, expected_host, timeout=timeout, sleeper=sleeper + ) + if not inside: + raise OrcaSessionError( + "the coordinator terminal is not in this environment: " + detail + ) return handle diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/cleanup.json b/experiments/minimal-cycle/evidence/vm-unobserved-turn/cleanup.json new file mode 100644 index 0000000..517cf7a --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/cleanup.json @@ -0,0 +1,15 @@ +{ + "host_difference": {}, + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "domain:dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "path:/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8" + ], + "retained": [], + "shared_preserved": [ + "image:/var/tmp/dely-cycle/images/tool-image.qcow2" + ], + "status": "DESTROYED", + "stop_confirmed": true, + "verified": true +} diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-completion-wait.json b/experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-completion-wait.json new file mode 100644 index 0000000..e69de29 diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-worker-start.json b/experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-worker-start.json new file mode 100644 index 0000000..0774e0b --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-worker-start.json @@ -0,0 +1,105 @@ +{ + "id": "95446ad1-024d-440e-aaed-1c67b35c9ab1", + "ok": true, + "result": { + "runId": "run_37029bebbcb3", + "taskId": "task_5158753e15a3", + "dispatchId": "ctx_bdebab5ef0cd", + "state": "outcome_unknown", + "stage": "turn_start_unobserved", + "turnStart": "unobserved", + "lastError": "Dispatch input was written and submitted, but claude's turn start could not be verified during observation (up to 30s). This is unverifiable, not proof the worker is dead: the agent may still be starting, may be wedged (for example waiting on network), or may be holding the task unsent in its composer. If the worker recovers and reports, this Dispatch settles normally.", + "setup": { + "requested": "not_applicable", + "effective": "not_applicable", + "source": "existing_worktree", + "hookFound": false, + "startupPolicy": "start-immediately", + "state": "not_applicable" + }, + "launch": { + "requested": { + "agent": "claude", + "model": "claude-opus-5", + "effort": "high" + }, + "effective": { + "agent": "claude", + "model": "claude-opus-5", + "effort": "high" + } + }, + "mode": { + "mode": "terminal", + "preferred": "terminal", + "reason": "user_default", + "detail": "Started a terminal agent worker, the default for new agent tabs in your settings." + }, + "timeoutMs": 420000, + "effects": [ + { + "kind": "worktree", + "action": "reused", + "id": "1c12506b-2db6-41c7-9271-dc3491f41355::/home/cycle/project" + }, + { + "kind": "setup", + "action": "not_applicable", + "state": "not_applicable" + }, + { + "kind": "terminal", + "role": "agent", + "action": "created", + "id": "term_918910ba-c506-47fa-87bc-14517d44ca23", + "surface": "visible" + }, + { + "kind": "dispatch_input", + "role": "agent", + "id": "term_918910ba-c506-47fa-87bc-14517d44ca23", + "state": "accepted" + }, + { + "kind": "dispatch_input", + "role": "agent", + "id": "term_918910ba-c506-47fa-87bc-14517d44ca23", + "state": "turn_unobserved" + } + ], + "prompt": { + "requestId": "fa454659-6809-4203-a92b-5c58873113be", + "stages": [ + "input_accepted" + ], + "provider": "claude", + "observation": "supported", + "processIncarnation": "bd7eb2ed-9c6d-4934-ae59-6577736a25ca", + "generation": 2, + "baselineWorkingSequence": 0, + "baselineExplicitWorkingStartedAt": null, + "baselinePermissionSequence": 0 + }, + "residualResources": [ + { + "kind": "terminal", + "role": "agent", + "action": "created", + "id": "term_918910ba-c506-47fa-87bc-14517d44ca23", + "surface": "visible" + } + ], + "nextCommands": [ + "orca-ide orchestration worker-show --dispatch ctx_bdebab5ef0cd --json", + "orca-ide terminal read --terminal term_918910ba-c506-47fa-87bc-14517d44ca23 --screen", + "orca-ide orchestration worker-abandon --dispatch ctx_bdebab5ef0cd --json" + ], + "mutation": { + "requestId": "fa454659-6809-4203-a92b-5c58873113be", + "replayed": false + } + }, + "_meta": { + "runtimeId": "e885c3e7-8b3d-4fda-94ee-247976bed992" + } +} diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-worker-terminal.txt b/experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-worker-terminal.txt new file mode 100644 index 0000000..82fd38e --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/dispatch-worker-terminal.txt @@ -0,0 +1,9 @@ +handle: term_918910ba-c506-47fa-87bc-14517d44ca23 +status: running +source: screen +cursor: 1 +oldest cursor: 0 +latest cursor: 1 + +cycle@dely-cycle-20260915t190618z-bbc5e6-c588ade8:~/project$ claude '--dangerously-skip-permissions' '--model' 'claude-o +pus-5' '--effort' 'high' diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/first-run-state.json b/experiments/minimal-cycle/evidence/vm-unobserved-turn/first-run-state.json new file mode 100644 index 0000000..f6c9e45 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/first-run-state.json @@ -0,0 +1,15 @@ +{ + "detail": "2 first-run state file(s) written into the per-run home, answering 4 question(s) the agent asks a home it has never run in", + "entries": [ + ".claude.json", + ".claude/settings.json" + ], + "questions": [ + "onboarding, which otherwise offers a theme and an interactive sign-in", + "trusting the project copy, by its path inside the environment", + "the external imports the project's own instructions declare", + "the permission mode the execution plane launches the agent with" + ], + "status": "OK", + "target": "/home/cycle" +} diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-environment-probe.txt b/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-environment-probe.txt new file mode 100644 index 0000000..2280cd2 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-environment-probe.txt @@ -0,0 +1,14 @@ +hostname=dely-cycle-20260915t190618z-bbc5e6-c588ade8 +machine_id= +boot_id= +home=/home/cycle +user=cycle +path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin +pid=1886 +uname=Linux 6.8.0-139-generic x86_64 +container_marker= +virt=kvm +orca_path=/usr/local/bin/orca +orca_version=1.4.201 +orca_fingerprint=533628:1592:1789327972 +project_real=/home/cycle/project diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-host-probe.txt b/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-host-probe.txt new file mode 100644 index 0000000..5d9eeec --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-host-probe.txt @@ -0,0 +1,14 @@ +hostname=system +machine_id= +boot_id= +home= +user= +path=/.local/bin:/.local/bin:/.polygraph/bin:/.local/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.config/orca/linux-orca-cli-shim:/.local/bin:/.polygraph/bin:/.opencode/bin:/.kimi-code/bin:/usr/lib/ccache/bin/:/.local/share/pnpm:/usr/local/bin:/usr/bin:/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.nvm/versions/node/v26.7.0/bin:/.volta/bin:/.asdf/shims:/.fnm/aliases/default/bin:/.local/share/mise/shims:/.yarn/bin:/.bun/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/sbin:/snap/bin:/home/linuxbrew/.linuxbrew/bin:/nix/var/nix/profiles/default/bin:/bin:/.nix-profile/bin:/.vite-plus/bin:/.cargo/bin/:/.local/bin/:/.pulumi/bin:/.claude/plugins/cache/claude-plugins-official/superpowers/6.3.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-migration/1.0.0/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi/2.0.6/bin:/.claude/plugins/cache/pulumi-agent-skills/pulumi-delegation/1.0.0/bin +pid=1076502 +uname=Linux 7.2.5-200.fc44.x86_64 x86_64 +container_marker= +virt=none +orca_path=/.config/orca/linux-orca-cli-shim/orca +orca_version=1.4.201 +orca_fingerprint=99186908:65:1789370051 +project_real=/var/tmp/dely-cycle/project-source diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-orca-runtime.json b/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-orca-runtime.json new file mode 100644 index 0000000..37c6fc1 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/identity-orca-runtime.json @@ -0,0 +1,90 @@ +{ + "app_pid": 2112, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "e885c3e7-8b3d-4fda-94ee-247976bed992", + "runtime_state": "ready" +} diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/manifest.json b/experiments/minimal-cycle/evidence/vm-unobserved-turn/manifest.json new file mode 100644 index 0000000..64d0b3f --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/manifest.json @@ -0,0 +1,1081 @@ +{ + "artifact_root": "/var/tmp/dely-cycle/artifacts/20260915T190618Z-bbc5e6-c588ade8", + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "b3e1a9da8d4806448f08ecd637daaf22aac7688bafab6e6a98d0b082e2ada44b", + "size_bytes": 291 + }, + { + "path": "backend-status.json", + "sha256": "10ea36b61678def64e8f4df4da0dc217b473b4cc7df4dc641736327624ad3539", + "size_bytes": 2016 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "dispatch/completion-wait.stderr", + "sha256": "a8f4086903088b24c7a3ca9d38777f02724e655ba0e9d32569a1d9a8400475a4", + "size_bytes": 2105 + }, + { + "path": "dispatch/completion-wait.stdout", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "dispatch/run-create.stdout", + "sha256": "fabba7312aaef5ada4c691ec4f1926626f453f1b47e7869e8e17bfba97dd0f22", + "size_bytes": 569 + }, + { + "path": "dispatch/worker-start.stdout", + "sha256": "8ce4ea1a6dc4c6b8953b5af591afc3e1a71cccf4c3dd12bb7bea0b3aabbee998", + "size_bytes": 3302 + }, + { + "path": "dispatch/worker-terminal.stdout", + "sha256": "f0b64bf9b8d5487c38713ee540837e39c20bcd448a709c78f446b702f863f555", + "size_bytes": 272 + }, + { + "path": "first-run-state.json", + "sha256": "6c096bd1dcb7cce3ed7eb489b75dea9d16d7c6095d5b5b1b5c7b3cbb1e82a028", + "size_bytes": 559 + }, + { + "path": "host-before.json", + "sha256": "6c638f9220efffc52c75f39cc114d41e9ae888cda0516f8ba2cdca46803e9668", + "size_bytes": 1691 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "b6118363508a5fdaddcf9aa613fe647d9463232a671627e5f288ad9681963ad9", + "size_bytes": 471 + }, + { + "path": "identity/host-probe.txt", + "sha256": "36c8e0f02779690fd1312541a1c334196d30f9ccf99bffcddd74d59e6677e938", + "size_bytes": 1685 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "0ae804a368d6dfbc548baad3093fad59e6d864f5ce955913e6e5ea6d24abb46d", + "size_bytes": 3178 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "5f311eca0ab6d08b23576118e208935d90c5ca2f4202291df5d4b3468aa13596", + "size_bytes": 11393 + }, + { + "path": "logs/runner.log", + "sha256": "22592e75acf5a0727954863f8b887932d9f00aa852b68c4769d50b06da46c6c3", + "size_bytes": 1537 + }, + { + "path": "preflight.json", + "sha256": "a0a2864b989fabdb80b8dc5efcd4e301eff362033b3297114300ea6c75c24340", + "size_bytes": 1805 + }, + { + "path": "run-before-cleanup.json", + "sha256": "283130db421bec32dc09c35d463a2740ebdf1d77588d878aeec4fe404a842c7d", + "size_bytes": 29981 + } + ], + "auth": { + "detail": "the copied auth material was removed from the per-run home and verified gone", + "entries": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login", + "removed_after_run": true, + "status": "OK", + "target": "/home/cycle" + }, + "backend": "vm", + "check": { + "argv": [], + "detail": "not attempted", + "elapsed_seconds": null, + "exit_code": null, + "expected_marker": null, + "finished_at": null, + "started_at": null, + "status": "SKIPPED", + "stderr_path": null, + "stdout_path": null + }, + "cleanup": { + "reason": "every declared per-run resource is gone and every shared one remains", + "removed": [ + "domain:dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "path:/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8" + ], + "retained": [], + "shared_preserved": [ + "image:/var/tmp/dely-cycle/images/tool-image.qcow2" + ], + "status": "DESTROYED", + "verified": true + }, + "config": { + "artifact_root": "/var/tmp/dely-cycle/artifacts", + "auth": { + "allowlist": [ + ".claude/.credentials.json" + ], + "mode": "existing_login", + "reference": "workshop-host-login" + }, + "backend": "vm", + "check": { + "expected_marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "claude_code_version": "2.1.272", + "dely_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "orca": { + "agent": "claude", + "app_argv": [ + "/opt/Orca/orca-ide" + ], + "command": "orca", + "display": ":0", + "effort": "high", + "model": "claude-opus-5", + "ready_timeout_seconds": 420, + "run_objective": "dely minimal cycle", + "status_argv": [ + "orca", + "status", + "--json" + ], + "version": "1.4.201", + "version_argv": [ + "orca", + "--version" + ], + "worktree_selector": "current" + }, + "project": { + "environment_path": "project", + "revision": "497660005edd04cb399886334863ffe8fe5c31be", + "source": "/var/tmp/dely-cycle/project-source" + }, + "state_root": "/var/tmp/dely-cycle/state", + "task": { + "marker": "dely-cycle-marker", + "relative_path": "evidence.txt" + }, + "tested_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "timeout_seconds": 420, + "vm": { + "address_timeout_seconds": 600, + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_image_sha256": "1f11ab859bf049b3754909f32e09ea3dc1b71b4762ec24406dea661f9bf0123c", + "connect_uri": "qemu:///system", + "egress": true, + "graphics": "vnc", + "guest_user": "cycle", + "listen_address": "127.0.0.1", + "memory_mb": 4096, + "network": "default", + "overlay_size_bytes": 25769803776, + "pool": "dely-cycle", + "provider": "pulumi-libvirt", + "provider_version": "0.5.4", + "provision": [], + "pulumi_binary": "/.pulumi/bin/pulumi", + "qemu_agent": false, + "ssh_port": 22, + "stack_prefix": "dely-cycle", + "transport": "ssh", + "vcpus": 2, + "venv": "/var/tmp/dely-cycle/venv", + "video": "virtio" + } + }, + "dely_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "elapsed_seconds": 519.02, + "environment": { + "backend": { + "address": "192.168.122.127", + "backend": "vm", + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_image_sha256": "1f11ab859bf049b3754909f32e09ea3dc1b71b4762ec24406dea661f9bf0123c", + "base_volume": "tool-image.qcow2", + "connect_uri": "qemu:///system", + "domain": "dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "egress_interface": true, + "graphics": "vnc", + "listen_address": "127.0.0.1", + "network": "default", + "pool": "dely-cycle", + "provider": "pulumi-libvirt@0.5.4", + "stack": "dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "transport_mac": "52:54:00:b5:f8:0f", + "video": "virtio" + }, + "description": { + "address": "192.168.122.127", + "base_image": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "base_volume": "tool-image.qcow2", + "connect_uri": "qemu:///system", + "domain": "dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "egress": true, + "graphics": "vnc", + "network": "default", + "pool": "dely-cycle", + "provider": "pulumi-libvirt@0.5.4", + "stack": "/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/stack", + "transport_mac": "52:54:00:b5:f8:0f", + "video": "virtio" + }, + "environment_id": "dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "home_path": "/home/cycle", + "per_run_resources": [ + { + "identifier": "dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "kind": "domain" + }, + { + "identifier": "/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8", + "kind": "path" + }, + { + "identifier": "/var/tmp/dely-cycle/images/dely-cycle-20260915t190618z-bbc5e6-c588ade8-overlay.qcow2", + "kind": "volume" + }, + { + "identifier": "/var/tmp/dely-cycle/images/dely-cycle-20260915t190618z-bbc5e6-c588ade8-seed.iso", + "kind": "volume" + } + ], + "project_path": "/home/cycle/project", + "shared_resources": [ + { + "identifier": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "kind": "image" + } + ] + }, + "environment_id": "dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "export": { + "artifacts": [ + { + "path": "auth-receipt.json", + "sha256": "b3e1a9da8d4806448f08ecd637daaf22aac7688bafab6e6a98d0b082e2ada44b", + "size_bytes": 291 + }, + { + "path": "backend-status.json", + "sha256": "10ea36b61678def64e8f4df4da0dc217b473b4cc7df4dc641736327624ad3539", + "size_bytes": 2016 + }, + { + "path": "diff.patch", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "dispatch/completion-wait.stderr", + "sha256": "a8f4086903088b24c7a3ca9d38777f02724e655ba0e9d32569a1d9a8400475a4", + "size_bytes": 2105 + }, + { + "path": "dispatch/completion-wait.stdout", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size_bytes": 0 + }, + { + "path": "dispatch/run-create.stdout", + "sha256": "fabba7312aaef5ada4c691ec4f1926626f453f1b47e7869e8e17bfba97dd0f22", + "size_bytes": 569 + }, + { + "path": "dispatch/worker-start.stdout", + "sha256": "8ce4ea1a6dc4c6b8953b5af591afc3e1a71cccf4c3dd12bb7bea0b3aabbee998", + "size_bytes": 3302 + }, + { + "path": "dispatch/worker-terminal.stdout", + "sha256": "f0b64bf9b8d5487c38713ee540837e39c20bcd448a709c78f446b702f863f555", + "size_bytes": 272 + }, + { + "path": "first-run-state.json", + "sha256": "6c096bd1dcb7cce3ed7eb489b75dea9d16d7c6095d5b5b1b5c7b3cbb1e82a028", + "size_bytes": 559 + }, + { + "path": "host-before.json", + "sha256": "6c638f9220efffc52c75f39cc114d41e9ae888cda0516f8ba2cdca46803e9668", + "size_bytes": 1691 + }, + { + "path": "identity/environment-probe.txt", + "sha256": "b6118363508a5fdaddcf9aa613fe647d9463232a671627e5f288ad9681963ad9", + "size_bytes": 471 + }, + { + "path": "identity/host-probe.txt", + "sha256": "36c8e0f02779690fd1312541a1c334196d30f9ccf99bffcddd74d59e6677e938", + "size_bytes": 1685 + }, + { + "path": "identity/orca-runtime.json", + "sha256": "0ae804a368d6dfbc548baad3093fad59e6d864f5ce955913e6e5ea6d24abb46d", + "size_bytes": 3178 + }, + { + "path": "identity/orca-status.json", + "sha256": "86222f82dc556d15cb413be50082d0c6670a8d1d3c0dae8212c695c995a130e7", + "size_bytes": 363 + }, + { + "path": "logs/commands.jsonl", + "sha256": "5f311eca0ab6d08b23576118e208935d90c5ca2f4202291df5d4b3468aa13596", + "size_bytes": 11393 + }, + { + "path": "logs/runner.log", + "sha256": "22592e75acf5a0727954863f8b887932d9f00aa852b68c4769d50b06da46c6c3", + "size_bytes": 1537 + }, + { + "path": "preflight.json", + "sha256": "a0a2864b989fabdb80b8dc5efcd4e301eff362033b3297114300ea6c75c24340", + "size_bytes": 1805 + }, + { + "path": "run-before-cleanup.json", + "sha256": "283130db421bec32dc09c35d463a2740ebdf1d77588d878aeec4fe404a842c7d", + "size_bytes": 29981 + } + ], + "confirmed_at": "2026-09-15T19:14:48Z", + "detail": "18 artifact(s) re-read from the host; 1 unresolved", + "missing": [ + "task-artifact/evidence.txt (absent from the host)" + ], + "receipt_path": "export-receipt.json", + "status": "CONFIRMED" + }, + "failure_classification": "the task reached the configured deadline", + "finished_at": "2026-09-15T19:14:57Z", + "first_run": { + "detail": "2 first-run state file(s) written into the per-run home, answering 4 question(s) the agent asks a home it has never run in", + "entries": [ + ".claude.json", + ".claude/settings.json" + ], + "questions": [ + "onboarding, which otherwise offers a theme and an interactive sign-in", + "trusting the project copy, by its path inside the environment", + "the external imports the project's own instructions declare", + "the permission mode the execution plane launches the agent with" + ], + "status": "OK", + "target": "/home/cycle" + }, + "generated_at": "2026-09-15T19:14:57Z", + "host_after": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T182804Z-bbc5e6-366fa4a0", + "20260915T183739Z-bbc5e6-9088bc04", + "20260915T184612Z-bbc5e6-9fb2fb6e", + "20260915T190618Z-bbc5e6-c588ade8", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011", + "20260915T230000Z-bbbbbb-00000012", + "20260916T000000Z-bbbbbb-00000013", + "20260916T010000Z-bbbbbb-00000014", + "20260916T020000Z-cccccc-00000001", + "20260916T030000Z-cccccc-00000002", + "20260916T040000Z-cccccc-00000003", + "20260916T050000Z-cccccc-00000004" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T19:14:57Z", + "extra": { + "backend": "vm" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [ + "20260915T184600Z-bbc5e6-53139e11" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "host_before": { + "architecture": "x86_64", + "artifact_root": { + "entries": [ + "20260914T230000Z-aaaaaa-00000001", + "20260914T231500Z-aaaaaa-00000002", + "20260915T001500Z-aaaaaa-00000003", + "20260915T003000Z-aaaaaa-00000004", + "20260915T130000Z-bbbbbb-00000001", + "20260915T131500Z-bbbbbb-00000002", + "20260915T134500Z-bbbbbb-00000003", + "20260915T140000Z-bbbbbb-00000004", + "20260915T143000Z-bbbbbb-00000005", + "20260915T150000Z-bbbbbb-00000006", + "20260915T160000Z-bbbbbb-00000007", + "20260915T170000Z-bbbbbb-00000008", + "20260915T182804Z-bbc5e6-366fa4a0", + "20260915T183739Z-bbc5e6-9088bc04", + "20260915T184612Z-bbc5e6-9fb2fb6e", + "20260915T190618Z-bbc5e6-c588ade8", + "20260915T200000Z-bbbbbb-00000009", + "20260915T210000Z-bbbbbb-00000010", + "20260915T220000Z-bbbbbb-00000011", + "20260915T230000Z-bbbbbb-00000012", + "20260916T000000Z-bbbbbb-00000013", + "20260916T010000Z-bbbbbb-00000014", + "20260916T020000Z-cccccc-00000001", + "20260916T030000Z-cccccc-00000002", + "20260916T040000Z-cccccc-00000003", + "20260916T050000Z-cccccc-00000004" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/artifacts" + }, + "boot_id_digest": "de0d3220ff53399e", + "captured_at": "2026-09-15T19:06:18Z", + "extra": { + "backend": "vm" + }, + "hostname_digest": "bbc5e661e106c6dc", + "kernel": "7.2.5-200.fc44.x86_64", + "machine_id_digest": "907fa710d6d1c501", + "python_version": "3.14.7", + "state_root": { + "entries": [ + "20260915T184600Z-bbc5e6-53139e11" + ], + "exists": true, + "path": "/var/tmp/dely-cycle/state" + }, + "user_digest": "8fda3821c06d512d" + }, + "identity": { + "environment": { + "container_marker": "", + "home": "/home/cycle", + "hostname": "dely-cycle-20260915t190618z-bbc5e6-c588ade8", + "orca_fingerprint": "533628:1592:1789327972", + "orca_path": "/usr/local/bin/orca", + "orca_version": "1.4.201", + "path": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin", + "pid": "1886", + "project_real": "/home/cycle/project", + "uname": "Linux 6.8.0-139-generic x86_64", + "user": "cycle", + "virt": "kvm" + }, + "host": { + "boot_id_digest": "de0d3220ff53399e", + "home_digest": "8fda3821c06d512d", + "hostname_digest": "bbc5e661e106c6dc", + "machine_id_digest": "907fa710d6d1c501", + "orca_fingerprint_digest": "ea841899bf829047", + "orca_path_digest": "6784f600bbc7d6cf", + "orca_present": true, + "project_real_digest": "0d03c9e72dddfdae", + "user_digest": "3cc9cee71b0b967f", + "virt": "none" + }, + "markers": [ + "distinct-machine-identity", + "distinct-boot-identity", + "distinct-hostname", + "reported-virtualisation:kvm" + ], + "orca_is_host_installation": false, + "orca_present": true, + "orca_runtime": { + "app_pid": 2112, + "app_running": true, + "app_version": "1.4.201", + "capabilities": [ + "runtime.status.compat.v1", + "runtime.environments.v1", + "remote-runtime.shared-control.v1", + "orchestration.federation.v1", + "orchestration.federation-control-mail.v1", + "orchestration.federation-lifecycle-settlement.v1", + "orchestration.worker-stop-verdict.v1", + "orchestration.worker-launch-preferences.v1", + "orchestration.federation-structured-read.v1", + "orchestration.federation-fleet-snapshot.v1", + "orchestration.federation-release-archive.v1", + "orchestration.contract.v1", + "browser.screencast.v1", + "browser.tab-create-known-id.v1", + "browser.clientHost.v1", + "browser.clientHost.pageMetadata.v1", + "browser.clientHost.automation.v1", + "browser.clientHost.fileChannel.v1", + "network.browserTunnel.v1", + "network.browserTunnel.executionHosts.v1", + "terminal.binary-stream.v1", + "terminal.multiplex.v1", + "workspace-ports.v1", + "mobile.tasks.v1", + "project-host-setup.v1", + "task-source-context.v1", + "workspace-run-context.v1", + "worktree.linked-work-item-context.v1", + "worktree.github-pr-suppression.v1", + "folder-workspace.path-status.v1", + "linear.issue-attribute-filter.v1", + "jira.user-fields.v1", + "aiVault.v1", + "aiVault.session-titles.v1", + "terminal.query-reply-input.v1", + "terminal.prompt-delivery.v1", + "terminal.paired-parking.v1", + "terminal.quick-commands.v1", + "worktree.create-idempotency.v1", + "terminal.create-idempotency.v2", + "session-tabs.close-intent.v1", + "session-tabs.authoritative-inventory.v1", + "agent-session.session-boundary.v1", + "updater.remote-control.v1", + "agent-session.host-authority.v1", + "agent-session.omp-resume-path.v1", + "agent-session.structured.v1", + "agent-session.pending-send-result.v1", + "agent-session.structured.hold.v1", + "agent-session.structured.reveal.v1", + "agent-session.structured.resume-history.v1", + "agent-session.status-feed.v1", + "agent-session.rewind.v1", + "agent-session.background-task-stop.v1", + "agent-session.turn-item.v1", + "agent-session.background-task-row-stop.v1", + "agent-session.kimi-resume.v1", + "files.mutation-ownership.v1", + "github.markPRReadyForReview", + "gitlab.updateMR.readyForReview.v1", + "worktree.visibility-defaults.v1", + "worktree.visibility-source-defaults.v1", + "accounts.import-host-credentials.v1", + "accounts.codex-reset-credit.v1", + "skills.install.v1", + "skills.install.bundle.v1", + "skills.install-cancel.v1", + "skills.install-progress.v1", + "skills.install-result.v2", + "skills.upload.v1", + "skills.manage.v1", + "skills.install-providers.v1", + "skills.delete.v1", + "automation.list-host-scope.v1", + "automation.owner-fencing.v1", + "automation.create-idempotency.v1", + "notifications.remote-push.v1", + "browser.certificate-trust.v1" + ], + "desktop_window": "available", + "detail": "the runtime is ready and the application is running", + "ready": true, + "runtime_id": "e885c3e7-8b3d-4fda-94ee-247976bed992", + "runtime_state": "ready" + }, + "orca_status_exit_code": 0, + "orca_version": "1.4.201", + "reason": "the environment answered with its own identity, home and project copy", + "verdict": "ENVIRONMENT" + }, + "phases": [ + { + "commands": [], + "detail": "", + "elapsed_seconds": 6.518, + "finished_at": "2026-09-15T19:06:25Z", + "name": "prepare", + "started_at": "2026-09-15T19:06:18Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": 23.152, + "finished_at": "2026-09-15T19:06:48Z", + "name": "create", + "started_at": "2026-09-15T19:06:25Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- git -C /home/cycle/project init -q -b main" + ], + "context": "environment", + "elapsed_seconds": 0.257, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:49Z", + "started_at": "2026-09-15T19:06:49Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- git -C /home/cycle/project config user.email dely-cycle@invalid" + ], + "context": "environment", + "elapsed_seconds": 0.256, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:49Z", + "started_at": "2026-09-15T19:06:49Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- git -C /home/cycle/project config user.name dely-cycle" + ], + "context": "environment", + "elapsed_seconds": 0.239, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:49Z", + "started_at": "2026-09-15T19:06:49Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- git -C /home/cycle/project add -A" + ], + "context": "environment", + "elapsed_seconds": 0.309, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:50Z", + "started_at": "2026-09-15T19:06:49Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- git -C /home/cycle/project commit -q -m 'dely-cycle: the pinned revision as exported'" + ], + "context": "environment", + "elapsed_seconds": 0.268, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:50Z", + "started_at": "2026-09-15T19:06:50Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "", + "elapsed_seconds": 3.649, + "finished_at": "2026-09-15T19:06:52Z", + "name": "bootstrap", + "started_at": "2026-09-15T19:06:48Z", + "status": "OK" + }, + { + "commands": [ + { + "argv": [ + "sh", + "-c", + "\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf ''\n else\n printf ''\n fi\n}\n\nmarker=''\nif [ -f /run/.containerenv ]; then marker='containerenv'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='dockerenv'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='toolboxenv'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='unknown'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '')\"\norca_release=''\norca_fingerprint=''\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '%s' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '%i:%s:%Y' \"$orca_resolved\" 2>/dev/null || printf '')\"\nfi\n\nproject_resolved=''\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '')\"\nfi\n\nprintf 'hostname=%s\\n' \"$(uname -n 2>/dev/null || printf '')\"\nprintf 'machine_id= \"$(first_line /etc/machine-id)\"\nprintf 'boot_id= \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf 'home=%s\\n' \"${HOME:-}\"\nprintf 'user=%s\\n' \"$(id -un 2>/dev/null || printf '')\"\nprintf 'path=%s\\n' \"${PATH:-}\"\nprintf 'pid=%s\\n' \"$$\"\nprintf 'uname=%s\\n' \"$(uname -srm 2>/dev/null || printf '')\"\nprintf 'container_marker=%s\\n' \"$marker\"\nprintf 'virt=%s\\n' \"$virt\"\nprintf 'orca_path=%s\\n' \"$orca_binary\"\nprintf 'orca_version=%s\\n' \"$orca_release\"\nprintf 'orca_fingerprint=%s\\n' \"$orca_fingerprint\"\nprintf 'project_real=%s\\n' \"$project_resolved\"\n", + "cycle-probe", + "/var/tmp/dely-cycle/project-source", + "orca" + ], + "context": "host", + "elapsed_seconds": 0.201, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:52Z", + "started_at": "2026-09-15T19:06:52Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- sh -c '\nset -u\nproject_path=\"${1:-}\"\norca_command=\"${2:-orca}\"\n\nfirst_line() {\n if [ -r \"$1\" ]; then\n head -n 1 \"$1\" 2>/dev/null || printf '\"'\"''\"'\"'\n else\n printf '\"'\"''\"'\"'\n fi\n}\n\nmarker='\"'\"''\"'\"'\nif [ -f /run/.containerenv ]; then marker='\"'\"'containerenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -f /.dockerenv ]; then marker='\"'\"'dockerenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -f /run/.toolboxenv ]; then marker='\"'\"'toolboxenv'\"'\"'; fi\nif [ -z \"$marker\" ] && [ -n \"${container:-}\" ]; then marker=\"${container}\"; fi\n\nvirt='\"'\"'unknown'\"'\"'\nif command -v systemd-detect-virt >/dev/null 2>&1; then\n detected=\"$(systemd-detect-virt 2>/dev/null | head -n 1)\"\n if [ -n \"$detected\" ]; then virt=\"$detected\"; fi\nfi\n\norca_binary=\"$(command -v \"$orca_command\" 2>/dev/null || printf '\"'\"''\"'\"')\"\norca_release='\"'\"''\"'\"'\norca_fingerprint='\"'\"''\"'\"'\nif [ -n \"$orca_binary\" ]; then\n orca_release=\"$(\"$orca_binary\" --version 2>/dev/null | head -n 1 || printf '\"'\"''\"'\"')\"\n orca_resolved=\"$(readlink -f \"$orca_binary\" 2>/dev/null || printf '\"'\"'%s'\"'\"' \"$orca_binary\")\"\n orca_fingerprint=\"$(stat -c '\"'\"'%i:%s:%Y'\"'\"' \"$orca_resolved\" 2>/dev/null || printf '\"'\"''\"'\"')\"\nfi\n\nproject_resolved='\"'\"''\"'\"'\nif [ -n \"$project_path\" ] && [ -d \"$project_path\" ]; then\n project_resolved=\"$(cd \"$project_path\" 2>/dev/null && pwd -P || printf '\"'\"''\"'\"')\"\nfi\n\nprintf '\"'\"'hostname=%s\\n'\"'\"' \"$(uname -n 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'machine_id= \"$(first_line /etc/machine-id)\"\nprintf '\"'\"'boot_id= \"$(first_line /proc/sys/kernel/random/boot_id)\"\nprintf '\"'\"'home=%s\\n'\"'\"' \"${HOME:-}\"\nprintf '\"'\"'user=%s\\n'\"'\"' \"$(id -un 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'path=%s\\n'\"'\"' \"${PATH:-}\"\nprintf '\"'\"'pid=%s\\n'\"'\"' \"$$\"\nprintf '\"'\"'uname=%s\\n'\"'\"' \"$(uname -srm 2>/dev/null || printf '\"'\"''\"'\"')\"\nprintf '\"'\"'container_marker=%s\\n'\"'\"' \"$marker\"\nprintf '\"'\"'virt=%s\\n'\"'\"' \"$virt\"\nprintf '\"'\"'orca_path=%s\\n'\"'\"' \"$orca_binary\"\nprintf '\"'\"'orca_version=%s\\n'\"'\"' \"$orca_release\"\nprintf '\"'\"'orca_fingerprint=%s\\n'\"'\"' \"$orca_fingerprint\"\nprintf '\"'\"'project_real=%s\\n'\"'\"' \"$project_resolved\"\n' cycle-probe /home/cycle/project orca" + ], + "context": "environment", + "elapsed_seconds": 0.739, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:52Z", + "started_at": "2026-09-15T19:06:52Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- orca status --json" + ], + "context": "environment", + "elapsed_seconds": 0.765, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:53Z", + "started_at": "2026-09-15T19:06:52Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- sh -c 'export DISPLAY=\"$1\"; shift; export XDG_RUNTIME_DIR=\"${XDG_RUNTIME_DIR:-/run/user/$(id -u)}\"; waited=0; while ! xset -q >/dev/null 2>&1; do waited=$((waited + 2)); if [ \"$waited\" -ge 120 ]; then printf \"no display after %ss\n\" \"$waited\" >&2; exit 3; fi; sleep 2; done; printf \"display ready after %ss\n\" \"$waited\"; rm -f \"$HOME/.config/orca/SingletonLock\" \"$HOME/.config/orca/SingletonCookie\"; nohup \"$@\" > \"$HOME/orca-app.log\" 2>&1 & printf \"started %s\n\" \"$!\"' orca-start :0 /opt/Orca/orca-ide" + ], + "context": "environment", + "elapsed_seconds": 0.243, + "exit_code": 0, + "finished_at": "2026-09-15T19:06:54Z", + "started_at": "2026-09-15T19:06:54Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "orca runtime e885c3e7-8b3d-4fda-94ee-247976bed992 is ready and the coordinator terminal is term_cb481037-6209-42c9-9c6b-161272d1527e", + "elapsed_seconds": 10.602, + "finished_at": "2026-09-15T19:07:02Z", + "name": "identity", + "started_at": "2026-09-15T19:06:52Z", + "status": "OK" + }, + { + "commands": [], + "detail": "the completion wait reached the run deadline before the worker settled", + "elapsed_seconds": 464.874, + "finished_at": "2026-09-15T19:14:47Z", + "name": "task", + "started_at": "2026-09-15T19:07:02Z", + "status": "TIMEOUT" + }, + { + "commands": [], + "detail": "the task did not settle", + "elapsed_seconds": null, + "finished_at": null, + "name": "check", + "started_at": null, + "status": "SKIPPED" + }, + { + "commands": [], + "detail": "0 patch line(s) collected", + "elapsed_seconds": 1.115, + "finished_at": "2026-09-15T19:14:48Z", + "name": "collect", + "started_at": "2026-09-15T19:14:47Z", + "status": "OK" + }, + { + "commands": [], + "detail": "18 artifact(s) re-read from the host; 1 unresolved", + "elapsed_seconds": 0.051, + "finished_at": "2026-09-15T19:14:48Z", + "name": "export", + "started_at": "2026-09-15T19:14:48Z", + "status": "OK" + }, + { + "commands": [], + "detail": "every declared per-run resource is gone and every shared one remains", + "elapsed_seconds": 9.058, + "finished_at": "2026-09-15T19:14:57Z", + "name": "cleanup", + "started_at": "2026-09-15T19:14:48Z", + "status": "OK" + }, + { + "commands": [], + "detail": "", + "elapsed_seconds": null, + "finished_at": null, + "name": "close", + "started_at": "2026-09-15T19:14:57Z", + "status": "OK" + } + ], + "run_id": "20260915T190618Z-bbc5e6-c588ade8", + "schema_version": 1, + "started_at": "2026-09-15T19:06:18Z", + "status": "TIMEOUT", + "tested_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "timeout_seconds": 420, + "versions": { + "configured_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "distrobox": "distrobox: 1.8.2.5", + "git": "git version 2.55.0", + "podman": "podman version 5.8.4", + "project_revision": "497660005edd04cb399886334863ffe8fe5c31be", + "pulumi": "v3.262.0", + "python": "3.14.7", + "qemu_img": "Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers", + "runner": "minimal-cycle", + "virsh": "12.0.0" + }, + "worker": { + "agent": "claude", + "commands": [ + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- orca orchestration run-create --objective 'dely minimal cycle' --json --from term_cb481037-6209-42c9-9c6b-161272d1527e" + ], + "context": "environment", + "elapsed_seconds": 0.69, + "exit_code": 0, + "finished_at": "2026-09-15T19:07:03Z", + "started_at": "2026-09-15T19:07:02Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- orca orchestration worker-start --spec 'Read the file /home/cycle/dispatch-prompt.md in this environment and do exactly what it says.' --agent claude --model claude-opus-5 --effort high --worktree current --timeout-ms 420000 --json --run run_37029bebbcb3 --from term_cb481037-6209-42c9-9c6b-161272d1527e" + ], + "context": "environment", + "elapsed_seconds": 42.515, + "exit_code": 1, + "finished_at": "2026-09-15T19:07:46Z", + "started_at": "2026-09-15T19:07:03Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- orca orchestration check --wait --types worker_done,escalation,question --timeout-ms 420000 --json --run run_37029bebbcb3 --terminal term_cb481037-6209-42c9-9c6b-161272d1527e" + ], + "context": "environment", + "elapsed_seconds": 420.028, + "exit_code": null, + "finished_at": "2026-09-15T19:14:46Z", + "started_at": "2026-09-15T19:07:46Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": true + }, + { + "argv": [ + "ssh", + "-o", + "BatchMode=yes", + "-o", + "StrictHostKeyChecking=accept-new", + "-o", + "UserKnownHostsFile=/var/tmp/dely-cycle/state/20260915T190618Z-bbc5e6-c588ade8/known_hosts", + "-p", + "22", + "-i", + "", + "cycle@192.168.122.127", + "--", + "sh -c 'for prefix in \"$@\"; do case \"$prefix\" in --) shift; break;; esac; done; for name in $(env | sed -n \"s/^\\([A-Za-z_][A-Za-z0-9_]*\\)=.*/\\1/p\"); do case \"$name\" in ORCA_*) unset \"$name\";; esac; done; exec \"$@\"' cycle-isolate -- orca terminal read --terminal term_918910ba-c506-47fa-87bc-14517d44ca23 --screen" + ], + "context": "environment", + "elapsed_seconds": 0.974, + "exit_code": 0, + "finished_at": "2026-09-15T19:14:47Z", + "started_at": "2026-09-15T19:14:46Z", + "stderr_path": null, + "stdout_path": null, + "timed_out": false + } + ], + "detail": "the completion wait reached the run deadline before the worker settled", + "dispatch_id": "ctx_bdebab5ef0cd", + "effort": "high", + "model": "claude-opus-5", + "outcome": "outcome_unknown", + "run_id": "run_37029bebbcb3", + "status": "TIMEOUT" + } +} diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/preflight.json b/experiments/minimal-cycle/evidence/vm-unobserved-turn/preflight.json new file mode 100644 index 0000000..3e433a0 --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/preflight.json @@ -0,0 +1,81 @@ +{ + "backend": "vm", + "findings": [ + { + "detail": "/.pulumi/bin/pulumi", + "name": "pulumi on the path", + "ok": true + }, + { + "detail": "/usr/bin/qemu-img", + "name": "qemu-img on the path", + "ok": true + }, + { + "detail": "/usr/bin/virsh", + "name": "virsh on the path", + "ok": true + }, + { + "detail": "/usr/bin/ssh", + "name": "ssh on the path", + "ok": true + }, + { + "detail": "/usr/bin/scp", + "name": "scp on the path", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/venv/bin/python", + "name": "pinned environment present", + "ok": true + }, + { + "detail": "state lives at file://~", + "name": "pulumi state backend is local", + "ok": true + }, + { + "detail": "every class and field the program names exists in pulumi-libvirt@0.5.4", + "name": "provider schema verified", + "ok": true + }, + { + "detail": "/var/tmp/dely-cycle/images/tool-image.qcow2", + "name": "base image present", + "ok": true + }, + { + "detail": "the preserved base matches the pinned digest", + "name": "base image digest matches the pin", + "ok": true + }, + { + "detail": "/dev/kvm", + "name": "hardware acceleration device", + "ok": true + }, + { + "detail": "qemu:///system", + "name": "libvirt connection answers", + "ok": true + }, + { + "detail": "dely-cycle is active", + "name": "storage pool active", + "ok": true + }, + { + "detail": "default is active", + "name": "network active", + "ok": true + }, + { + "detail": "vnc is among sdl, vnc, dbus", + "name": "graphics type supported by this emulator", + "ok": true + } + ], + "ok": true +} diff --git a/experiments/minimal-cycle/tests/fakes.py b/experiments/minimal-cycle/tests/fakes.py index 4915a1d..4888cf0 100644 --- a/experiments/minimal-cycle/tests/fakes.py +++ b/experiments/minimal-cycle/tests/fakes.py @@ -106,6 +106,7 @@ def __init__( orca_path_from_host: bool = False, runtime_ready: bool = True, terminal_refused: bool = False, + terminal_answers: str | None = None, already_running: bool = False, refuse_repository: bool = False, dispatch_state: str | None = None, @@ -131,6 +132,7 @@ def __init__( self.orca_path_from_host = orca_path_from_host self.runtime_ready = runtime_ready self.terminal_refused = terminal_refused + self.terminal_answers = terminal_answers # The application is not running until something starts it, which is # what the real environment does too. self.app_started = already_running @@ -263,6 +265,19 @@ def execute( if self.terminal_refused: return self._outcome(argv, 1, "", "the runtime refused a terminal") return self._outcome(argv, 0, ORCA_TERMINAL_REPLY, "") + if "terminal send" in joined: + self.calls.append("terminal-send") + return self._outcome(argv, 0, '{"ok": true}', "") + if "terminal read" in joined: + # The runner asks the terminal which machine it is on. This fake + # answers as the environment unless a test asks it to answer as + # somewhere else, which is the shape of a terminal that opened + # outside the environment. + self.calls.append("terminal-read") + name = self.terminal_answers or f"fake-{self.root.name}" + return self._outcome( + argv, 0, f"handle: term_fake\nstatus: running\n\ncycle-terminal:{name}\n", "" + ) if "repo add" in joined: self.calls.append("repo-add") return self._outcome(argv, 0, '{"ok": true}', "") diff --git a/experiments/minimal-cycle/tests/test_lifecycle.py b/experiments/minimal-cycle/tests/test_lifecycle.py index a01bc90..bb1394f 100644 --- a/experiments/minimal-cycle/tests/test_lifecycle.py +++ b/experiments/minimal-cycle/tests/test_lifecycle.py @@ -463,3 +463,16 @@ def test_the_project_is_placed_before_either(self): i for i, c in enumerate(adapter.calls) if c.startswith("execute:git") ) self.assertLess(adapter.calls.index("put_tree"), first_git) + + +class TerminalOutsideTheEnvironmentTest(CycleTestCase): + """A command line inside an environment can still reach a runtime outside it.""" + + def test_a_coordinator_terminal_on_another_machine_blocks_the_run(self): + _, outcome = self.run_cycle(terminal_answers="workstation") + self.assertEqual(outcome.run_result.status, status.RunStatus.BLOCKED) + self.assertIn("not in this environment", outcome.run_result.failure_classification) + + def test_nothing_is_dispatched_to_a_terminal_that_is_not_ours(self): + adapter, _ = self.run_cycle(terminal_answers="workstation") + self.assertNotIn("worker-start", " ".join(adapter.calls)) diff --git a/experiments/minimal-cycle/tests/test_orca.py b/experiments/minimal-cycle/tests/test_orca.py index 6511893..2415c6c 100644 --- a/experiments/minimal-cycle/tests/test_orca.py +++ b/experiments/minimal-cycle/tests/test_orca.py @@ -188,3 +188,63 @@ def test_the_start_reports_what_it_ran(self): environment, ("/opt/Orca/orca-ide",), display=":0", timeout=30 ) self.assertEqual(outcome.context, "environment") + + +class CoordinatorTerminalIsInsideTest(unittest.TestCase): + """A terminal that opens outside the environment looks identical until asked.""" + + CREATED = json.dumps( + {"ok": True, "result": {"terminal": {"handle": "term_fake", "surface": "visible"}}} + ) + + def environment(self, screen): + return Environment( + [ + ("repo add", 0, '{"ok": true}', ""), + ("terminal create", 0, self.CREATED, ""), + ("terminal send", 0, '{"ok": true}', ""), + ("terminal read", 0, screen, ""), + ] + ) + + def open(self, environment, **options): + return orca.open_coordinator_terminal( + environment, + "/home/cycle/project", + timeout=5, + sleeper=lambda _seconds: None, + **options, + ) + + def test_a_terminal_that_answers_with_this_environment_is_accepted(self): + environment = self.environment( + "handle: term_fake\nstatus: running\n\ncycle-terminal:dely-cycle-abc\n" + ) + self.assertEqual( + self.open(environment, expected_host="dely-cycle-abc"), "term_fake" + ) + + def test_a_terminal_that_answers_with_another_machine_stops_the_run(self): + environment = self.environment( + "handle: term_fake\nstatus: running\n\ncycle-terminal:workstation\n" + ) + with self.assertRaises(orca.OrcaSessionError) as raised: + self.open(environment, expected_host="dely-cycle-abc") + self.assertIn("workstation", str(raised.exception)) + self.assertIn("not in this environment", str(raised.exception)) + + def test_a_terminal_that_never_answers_stops_the_run(self): + environment = self.environment("handle: term_fake\nstatus: running\n\n$ \n") + with self.assertRaises(orca.OrcaSessionError): + self.open( + environment, + expected_host="dely-cycle-abc", + ) + + def test_nothing_is_asked_when_no_machine_is_named(self): + environment = self.environment("") + self.assertEqual(self.open(environment), "term_fake") + self.assertFalse( + [argv for argv in environment.seen if "send" in " ".join(argv)], + "a run that names no machine should ask no question", + ) From 3b9f99d79d3d38d4bdeeaa661da7d0aa0508cb68 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 22:53:20 +0200 Subject: [PATCH 38/42] Correct a claim in the container evidence that a later run disproved. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The record said a virtual display left the host's desktop untouched. It did not. Distrobox shares the host's Wayland socket, the application chose it over the DISPLAY it was given, and because the container shares the host's PID namespace, destroying the container did not end those processes: instances from seven runs were found still alive on the host, drawing on the operator's screen. Setting DISPLAY is not isolation from a desktop, and nothing in that run observed where the application actually drew — which is why the claim stood. --- .../evidence/distrobox-full-cycle/README.md | 37 ++++++++++++++----- 1 file changed, 27 insertions(+), 10 deletions(-) diff --git a/experiments/minimal-cycle/evidence/distrobox-full-cycle/README.md b/experiments/minimal-cycle/evidence/distrobox-full-cycle/README.md index 4853404..1566e85 100644 --- a/experiments/minimal-cycle/evidence/distrobox-full-cycle/README.md +++ b/experiments/minimal-cycle/evidence/distrobox-full-cycle/README.md @@ -15,12 +15,24 @@ driven through the same lifecycle the machine backend uses. export: CONFIRMED cleanup: DESTROYED -## Nothing rendered on the host's desktop +## The virtual display did not keep the application off the host's desktop -Distrobox shares the host's display, so an application started in a box appears -on the host's own screen. This run gave the box a display of its own instead — -a virtual framebuffer on `:99`, provisioned into the container — and pointed -Orca at it. The host's desktop was untouched. +This run gave the box a display of its own — a virtual framebuffer on `:99`, +provisioned into the container — and pointed Orca at it. An earlier version of +this file claimed the host's desktop was therefore untouched. **That claim was +wrong, and a later session disproved it.** + +Distrobox shares the host's Wayland socket as well as its X socket, and the +application chose Wayland: it ran with `--ozone-platform=wayland` against the +host's own compositor, ignoring the `DISPLAY` it was given. Its windows appeared +on the operator's screen. Worse, because a Distrobox container shares the host's +PID namespace, destroying the container did not end those processes: instances +from seven separate runs were found still alive on the host hours later, each +holding `--user-data-dir` under its own destroyed per-run directory. + +Setting `DISPLAY` is not isolation from a desktop. Nothing in this run observed +where the application actually drew, which is why the claim survived as long as +it did. ## Two things this run found @@ -37,9 +49,14 @@ Provisioning also has to come before anything that needs what it installs: the repository initialisation ran first and failed with `executable file not found`, because git arrives with the provisioning steps. -## Where it stops +## Where it stops, and what came after + +This run reached a dispatch whose turn never began, and stopped there. The +deadline fired, the evidence was exported and confirmed, and the box and its +per-run home were removed. See `../vm-two-sessions/` for what the worker's +terminal held. -Exactly where the machine backend stops: the dispatch exists, the worker has a -terminal of its own, and its turn never begins. The deadline fired, the -evidence was exported and confirmed, and the box and its per-run home were -removed. See `../vm-two-sessions/` for what the worker's terminal holds. +Two defects were behind that, and both were found later: the agent was stopped +at first-run questions nobody could answer, and the completion wait was read +from the wrong half of the reply. `../distrobox-settled-cycle/` is the first +run on this backend that got past both. From 120d10456303b3b9b6ec7ec759a80a91605e9119 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 22:53:43 +0200 Subject: [PATCH 39/42] Rewrite the evidence index to say what each run showed and where the work stops. The index still said no Orca had started in either backend and no worker had been driven through one. Both are now false, and one backend completes a cycle while the other does not. The index says which is which, and keeps the earlier runs because each of them moved a rail. --- experiments/minimal-cycle/evidence/README.md | 157 ++++++++----------- 1 file changed, 67 insertions(+), 90 deletions(-) diff --git a/experiments/minimal-cycle/evidence/README.md b/experiments/minimal-cycle/evidence/README.md index 6536f83..5fd7a63 100644 --- a/experiments/minimal-cycle/evidence/README.md +++ b/experiments/minimal-cycle/evidence/README.md @@ -1,8 +1,8 @@ # What this runner actually did on one host -Everything here was produced by the committed code on a Fedora host with -rootless Podman, Distrobox 1.8.2.5, `qemu-img`, `virsh` answering -`qemu:///session`, `/dev/kvm` present, no `pulumi`, and no prepared tool image. +Everything here was produced by the committed code on one Fedora host with +rootless Podman, Distrobox, `qemu-img`, `virsh` on `qemu:///system`, `/dev/kvm`, +Pulumi 3.262.0 with a local `file://` backend, and a Packer-built tool image. Nothing here was written by hand. ## What was scrubbed, and what was not @@ -12,100 +12,77 @@ these: the host's home directory became ``, the host user name became ``, and the raw `machine_id` and `boot_id` lines in the probe captures became ``. Digests, sizes, timings, exit codes, statuses and reasons are untouched. The manifests already store host identifiers as digests -rather than as names, so nothing there needed scrubbing. - -## `preflight-distrobox.txt` - -The container backend on this host, usable. The last finding is the one that -matters: Distrobox mounts the host home into every box and has no flag to -suppress it, so the run is blocked until the configuration records that -compromise deliberately. That finding is not a claim — it is the result of -reading what `distrobox assemble create --dry-run` actually renders. - -## `preflight-vm.txt` - -The machine backend on this host, blocked, exit code 4. Three findings refuse: -`pulumi` is not installed, the provider schema has not been checked against a -pinned version, and there is no prepared tool image. The backend reports what is -missing rather than inventing a path around it. - -## `run-blocked-on-host-orca/` - -One real cycle, `20260915T001500Z-aaaaaa-00000003`, produced by the committed code. A fresh box was created from `docker.io/library/ubuntu:24.04` -with its own home, the project was exported at a pinned revision and copied in, -the identity probe ran on both sides, and the run stopped at the identity gate. -Every artifact was exported and confirmed, and the box and its per-run state -were then destroyed while the image was kept. - - status: BLOCKED - identity: ENVIRONMENT - export: CONFIRMED - cleanup: DESTROYED - why: the environment resolved orca to the host's own installation at - /.config/orca/linux-orca-cli-shim/orca; the run stops - rather than driving the host from inside a disposable environment - -Compare the two probe captures. The box answered with its own host name, its own -container marker and the per-run home — a real environment. It also answered -with the *host's* `orca_path` and an empty `orca_version`: Distrobox mounts the -host home and preserves `PATH`, so the host's launcher resolves inside the box -and cannot run there. - -**This is a defect the first run found.** Before it, the gate asked only whether -`command -v orca` resolved. It did, so the run proceeded and failed later at -`worker-start` with `/opt/Orca/resources/bin/orca-ide: No such file or -directory`. Had the container carried a working launcher pointing back at the -host, that run would have produced a complete green manifest for work the host -did. The gate now requires an Orca that is not the host's binary and that -reports a version inside the environment; `counterexamples.py` case +rather than as names. + +## Read these first + +### `distrobox-settled-cycle/` + +The one complete run. A worker read the prompt, wrote the marker byte for byte, +reported `worker_done` with outcome `succeeded`, and an independent check run +outside that session agreed. `SETTLED`, exported, destroyed. +`dispatch-completion-wait.json` carries the worker's own message. + +### `vm-unobserved-turn/` + +The machine backend, at the same commit, on the same host, not completing. +Domain created, distinct kernel proved, Orca ready, coordinator terminal open, +dispatch made — and the worker's turn never observed. +`dispatch-worker-terminal.txt` is what the agent's terminal actually held. +This is where the work currently stops, and it is not a green run. + +## Earlier runs, kept because each one moved a rail + +### `preflight-distrobox.txt`, `preflight-vm.txt` + +Both backends refusing on a host that could not yet run them. The container +finding that matters: Distrobox mounts the host home into every box and has no +flag to suppress it, so a run is blocked until the configuration records that +compromise deliberately. + +### `run-blocked-on-host-orca/` + +The first real cycle, stopped at the identity gate because the box resolved +`orca` to the *host's* launcher through the mounted host home. Presence of a +command is not presence of an installation. Counterexample `orca-is-the-environments-own` shows the old rule letting a host-driven run settle `SETTLED`. -## `vm-blocked-on-absent-orca/` - -One real machine cycle. Pulumi declared a per-run overlay over the preserved -Ubuntu base, a cloud-init seed and a domain; the guest booted, took an address, -answered, and settled; the project was copied in at a pinned revision; the probe -ran on both sides; and the run stopped at the identity gate. Everything was -exported and confirmed, and the domain, overlay, seed, stack and per-run state -were then destroyed while the base image stayed byte-identical. - - status: BLOCKED - identity: ENVIRONMENT - markers: distinct-machine-identity, distinct-boot-identity, - distinct-hostname, reported-virtualisation:kvm - export: CONFIRMED - cleanup: DESTROYED - why: orca is not present inside the environment; the run stops rather - than using the host's installation - -Compare the probe captures. The guest reported an Ubuntu kernel and `virt=kvm`; -the host reported a Fedora kernel and `virt=none`. The guest resolved the -project to `/home/cycle/project`, its own copy. That is a separate kernel, which -is what this backend buys and the container backend does not. - -Three defects were found by running this, not by reading it. The provider reads -interface addresses through the guest agent when the agent is declared, and the -base image does not run one, so declaring it failed the create. `ssh` -concatenates everything after the destination into one string that the guest's -shell re-parses, so the probe's positional argument was lost and the guest -resolved an empty project path. And libvirt hands out the lease while the guest -is still booting, so the first command hit a refused connection five -milliseconds after the address appeared. Each is now a rail with a -counterexample. - -## `counterexamples.txt` +### `vm-blocked-on-absent-orca/` + +The first real machine cycle: Pulumi declared the overlay, seed and domain, the +guest booted, answered and settled, and the run stopped because the image +carried no Orca. Three defects came out of running it — the guest agent broke +domain creation, `ssh` flattened the probe's argument vector, and a DHCP lease +is not readiness. + +### `orca-in-vm/`, `distrobox-own-orca/` + +Orca installed and reporting `ready` inside each backend, with its own runtime +identity rather than the host's. + +### `vm-two-sessions/`, `vm-dispatch-unverifiable/`, `vm-timeout/` + +Control and a separate coder session proved by identity rather than by layout; +a dispatch the plane could not verify; a deadline firing with every artifact +kept and the environment destroyed. + +### `distrobox-full-cycle/` + +One container cycle on a display of its own. Its own README corrects a claim it +originally made: the virtual display did **not** keep the application off the +host's desktop. + +### `counterexamples.txt` `python3 counterexamples.py`. Each line replaces one rail with an implementation that is present, runs and returns a pass, then runs the tests that are supposed to reject it. `RED` means the instrument rejected the wrong -implementation. A row that reported `GREEN` would be a row proving nothing. +implementation. A row reporting `GREEN` would be a row proving nothing. ## What none of this shows -No Orca has started inside either backend on this host, no window has been bound -to such an instance, no Claude Code worker has been driven through one, and no -login has survived a run. The machine backend has never created a domain here. -Those need a host carrying Orca in the chosen image, a built tool image and a -pinned libvirt provider, and they are the separate acceptance test this work -does not claim to have passed. +A second, different task. A run on any host but this one. A completed cycle on +the machine backend. Isolation on the container backend: it mounts the host +home, shares the host's PID namespace and the host's Wayland socket, and the +runner records that rather than claiming otherwise. From a819882d784e0ff96a2ffcde078355821f561d25 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 22:54:02 +0200 Subject: [PATCH 40/42] Record the machine backend not completing, and what that run ruled out. The same commit, host and task that settle in a container stop here at a worker whose turn is never observed. The capture is the agent's own terminal: the launch line and nothing after it. Network, shell, submission and the first-run questions are ruled out by measurement; the agent's own silence in that guest is a lead, and the measurement that would confirm it was interrupted, so it is written as a lead. --- .../evidence/vm-unobserved-turn/README.md | 52 +++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 experiments/minimal-cycle/evidence/vm-unobserved-turn/README.md diff --git a/experiments/minimal-cycle/evidence/vm-unobserved-turn/README.md b/experiments/minimal-cycle/evidence/vm-unobserved-turn/README.md new file mode 100644 index 0000000..2e3e2ea --- /dev/null +++ b/experiments/minimal-cycle/evidence/vm-unobserved-turn/README.md @@ -0,0 +1,52 @@ +# The machine backend, not completing + +Same commit, same host, same task as `../distrobox-settled-cycle/`. The +container settled; this did not. + + status: TIMEOUT + identity: ENVIRONMENT distinct machine identity, boot identity, hostname, virt=kvm + orca: runtime ready, window available + dispatch: made, then `outcome_unknown` / `turn_start_unobserved` + check: SKIPPED — nothing to check + export: CONFIRMED + cleanup: DESTROYED + +## How far it got + +Everything up to the worker. Pulumi declared the overlay, seed and domain over +a base image pinned by digest; the guest booted with a kernel of its own; +`first-run-state.json` shows the agent's four first-run questions answered in +the guest's home; Orca reported `ready` with a window; a coordinator terminal +opened, a Run was created and a Dispatch was made with the pinned model and +effort. + +## Where it stops, in the agent's own terminal + +`dispatch-worker-terminal.txt` is the screen the plane pointed at, captured by +the run itself: + + cycle@…:~/project$ claude '--dangerously-skip-permissions' '--model' 'claude-opus-5' '--effort' 'high' + +and nothing after it, at five, twenty and forty seconds. + +## What was ruled out, and what was not + +Ruled out by direct measurement in the guest during a separate session: + +- the network. DNS resolves `api.anthropic.com` and an HTTPS request completes. +- the shell. A plain terminal runs what is sent to it, and a terminal created + with a startup command runs that too — so the plane does submit here. +- the missing Enter. Sending one more Enter into that terminal changed nothing + at all, which a standing unsubmitted line would not do. +- the first-run questions. They are answered in this guest, and the same + answers are what let the container backend through. + +Not ruled out: the agent itself. `claude -p` in this guest produced no output +before its deadline, where the same command with the same token answers on the +host in about a second. That measurement was interrupted before it returned an +exit code, so it is a lead and not a finding. + +## What this does not show + +That the machine backend cannot work, or that the agent is at fault. It shows +one guest, on one host, where the agent was launched and never spoke. From a205df4005ec11b160e8179afb37996cd789aa46 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 23:03:26 +0200 Subject: [PATCH 41/42] End what a run started, and call a run with processes left over what it is. Cleanup declared a container and a directory and destroyed both, truthfully. It never looked for what the run had started inside them, and on a backend that shares the host's process namespace those survive: instances from six runs were found alive on the host, each holding a user data directory that no longer existed, each still drawing on the operator's screen. The survey reads more than argv, because the application names no path in its own: it reads the process environment, the working directory, and the tree under the identifier the start script reported. It matches paths, never program names, so it cannot reach the operator's own application. A process it cannot stop makes the run RESIDUE rather than DESTROYED. A process identifier only means something where the environment's process table is the host's, so the adapter declares that and the machine backend declares it false. The survey is passed in rather than defaulted, so a test that does not ask for it can never signal a process on the machine running the tests. --- docs/decisions.md | 75 ++++++ experiments/minimal-cycle/counterexamples.py | 22 ++ .../cycle_runner/adapters/base.py | 6 + .../cycle_runner/adapters/distrobox.py | 2 + .../minimal-cycle/cycle_runner/cleanup.py | 45 +++- experiments/minimal-cycle/cycle_runner/cli.py | 11 +- .../minimal-cycle/cycle_runner/lifecycle.py | 18 +- .../minimal-cycle/cycle_runner/orca.py | 15 ++ .../minimal-cycle/cycle_runner/processes.py | 220 ++++++++++++++++++ .../minimal-cycle/cycle_runner/result.py | 2 + .../minimal-cycle/tests/test_cleanup.py | 61 ++++- .../tests/test_doubles_are_contained.py | 24 ++ .../minimal-cycle/tests/test_processes.py | 159 +++++++++++++ 13 files changed, 655 insertions(+), 5 deletions(-) create mode 100644 experiments/minimal-cycle/cycle_runner/processes.py create mode 100644 experiments/minimal-cycle/tests/test_processes.py diff --git a/docs/decisions.md b/docs/decisions.md index c64632d..51abf97 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -9,6 +9,81 @@ Last updated 2026-09-15. ## Settled +### 2026-09-15 — A disposable environment is not disposable until nothing of it is still running on the host + +#### Context + +A container run gave its Orca a virtual framebuffer of its own and the evidence +recorded that the host's desktop was therefore untouched. It was not. Distrobox +shares the host's Wayland socket as well as its X socket, and the application +chose Wayland: it drew on the operator's own compositor and ignored the +`DISPLAY` it was handed. Because a Distrobox container also shares the host's +PID namespace, destroying the container did not end those processes. Instances +from seven separate runs were found alive on the host hours later, each holding +a `--user-data-dir` under its own destroyed per-run directory, and each still +putting windows on the operator's screen — twice, on two different days. + +Cleanup had reported `DESTROYED` every time, truthfully: every resource the +adapter *declared* was gone. The declaration named a container and a directory. +It did not name the processes the run had started, because nothing in the run +had ever looked for them. + +A second escape had the same shape. A command line inside an environment can +reach a runtime outside it. The binary, the user data path and the project path +all look right, the reply carries a terminal handle and reports its surface as +visible, and the terminal belongs to the host. Two such terminals were found in +the operator's own application, holding the host's shell in a per-run directory +that no longer existed. + +#### Decision + +An environment's resources include what it started. Cleanup surveys the host for +processes holding a path under this run's own state directory, stops them, and +reports what it stopped; a survivor is residue, not a detail. + +Before anything is dispatched from it, the coordinator terminal is asked which +machine it is on and has to answer with the name of the environment it is +supposed to be in. A terminal that answers with another machine, or does not +answer, stops the run. + +Neither check asks the backend to promise anything. Both read what is true after +the fact, which is the only form in which either of these was ever visible. + +#### Alternatives considered + +Trusting `DISPLAY`. It is what produced the false claim: an environment variable +records an intention, and the application is free to connect somewhere else. + +Reading the desktop to see whether a window appeared. It answers the question +for the run that is watching and not for the one that already exited, and it +needs a desktop the runner is not supposed to touch. + +Killing by process name. It would reach the operator's own application, which is +the failure this is meant to prevent, not a worse version of it. + +#### Consequences + +"Destroyed" now means something a reader can check, and it costs one survey per +run. A backend that shares the host's namespaces is recorded as sharing them: +the container backend mounts the host home, shares the host's PID namespace and +shares the host's Wayland socket, and the preflight and the evidence say so +rather than describing a sandbox. + +The general form is worth keeping. A disposable environment that shares any +namespace with its host can leave something behind that outlives the resource +the run declared, and a cleanup that verifies only its own declarations will +report success while that thing is still on the screen. + +#### Non-goals + +Making Distrobox an isolation boundary. Managing the operator's own +application. + +#### Deferred + +The same survey for the machine backend's guest-side processes, triggered by a +backend whose transport can outlive its domain. + ### 2026-09-15 — A test double that can run a command runs it on the developer's machine #### Context diff --git a/experiments/minimal-cycle/counterexamples.py b/experiments/minimal-cycle/counterexamples.py index 5cef3f2..de07360 100755 --- a/experiments/minimal-cycle/counterexamples.py +++ b/experiments/minimal-cycle/counterexamples.py @@ -38,6 +38,28 @@ class Counterexample: CASES: tuple[Counterexample, ...] = ( + Counterexample( + name="a-process-left-running-is-residue", + requirement=( + "A run whose declared resources are gone while its processes are " + "still on the host has not been destroyed" + ), + path="cycle_runner/cleanup.py", + original=' if not survey_report.clean:\n return CleanupRecord(\n status=CleanupStatus.RESIDUE,', + replacement=' if False:\n return CleanupRecord(\n status=CleanupStatus.RESIDUE,', + instruments=("tests.test_cleanup.ProcessSurveyTest",), + ), + Counterexample( + name="the-survey-matches-a-path-not-a-program", + requirement=( + "A survey that matches a program name reaches the operator's own " + "application instead of this run's" + ), + path="cycle_runner/processes.py", + original=' return any(path in field for field in (self.command, self.home, self.cwd))', + replacement=' return "orca" in self.command', + instruments=("tests.test_processes.SurveyTest",), + ), Counterexample( name="the-terminal-says-which-machine-it-is-on", requirement=( diff --git a/experiments/minimal-cycle/cycle_runner/adapters/base.py b/experiments/minimal-cycle/cycle_runner/adapters/base.py index d930ac2..2d09823 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/base.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/base.py @@ -170,6 +170,12 @@ def plan_handle(self) -> EnvironmentHandle | None: """ return None + #: Whether a process identifier from inside this environment means the same + #: thing on the host. Distrobox shares the host's process table, so it does; + #: a virtual machine has its own, so the same number there belongs to an + #: unrelated process here and must never be acted on. + shares_host_processes: bool = False + def describe(self) -> dict[str, Any]: """Return backend facts for the manifest.""" return {"backend": self.name} diff --git a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py index 789403a..1b3a1cf 100644 --- a/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py +++ b/experiments/minimal-cycle/cycle_runner/adapters/distrobox.py @@ -52,6 +52,8 @@ class DistroboxAdapter(BackendAdapter): """One disposable Distrobox, driven through Assemble and the command line.""" name = "distrobox" + # The container runs in the host's process namespace. + shares_host_processes = True def __init__( self, diff --git a/experiments/minimal-cycle/cycle_runner/cleanup.py b/experiments/minimal-cycle/cycle_runner/cleanup.py index d0554d5..49c3cc4 100644 --- a/experiments/minimal-cycle/cycle_runner/cleanup.py +++ b/experiments/minimal-cycle/cycle_runner/cleanup.py @@ -11,8 +11,9 @@ import shutil from pathlib import Path -from typing import Iterable, Sequence +from typing import Callable, Iterable, Sequence +from . import processes from .adapters.base import BackendAdapter, EnvironmentHandle, Resource from .result import CleanupRecord, ExportRecord from .status import CleanupStatus @@ -71,12 +72,23 @@ def safe_remove( target.unlink() +def run_paths(handle: EnvironmentHandle) -> list[str]: + """Return the paths that belong to this run and nothing else.""" + paths = [handle.home_path] + paths.extend( + item.identifier for item in handle.per_run_resources if item.kind == "path" + ) + return [path for path in dict.fromkeys(paths) if path] + + def perform( *, adapter: BackendAdapter, handle: EnvironmentHandle, export_record: ExportRecord, stop_confirmed: bool = True, + survey: Callable[..., processes.SurveyReport] | None = None, + started: Sequence[int] = (), ) -> CleanupRecord: """Destroy per-run resources only after a confirmed export and a confirmed stop.""" per_run = list(handle.per_run_resources) @@ -108,6 +120,15 @@ def perform( verified=False, ) + # Before the directory goes: a process still holding a path of this run + # outlives the resource that nominally contained it, keeps its window on the + # host's screen, and writes to a directory that is about to stop existing. + survey_report = ( + survey(run_paths(handle), roots=tuple(started)) + if survey is not None + else processes.SurveyReport(detail="no survey was asked for") + ) + report = adapter.destroy() surviving = [item for item in per_run if adapter.resource_exists(item)] @@ -125,6 +146,7 @@ def perform( retained=[str(item) for item in surviving], shared_preserved=[str(item) for item in preserved_shared], verified=False, + processes=survey_report.to_document(), ) if surviving: @@ -138,13 +160,32 @@ def perform( retained=[str(item) for item in surviving], shared_preserved=[str(item) for item in preserved_shared], verified=True, + processes=survey_report.to_document(), + ) + + if not survey_report.clean: + return CleanupRecord( + status=CleanupStatus.RESIDUE, + reason=( + "every declared per-run resource is gone, and " + + survey_report.detail + ), + removed=list(report.removed), + retained=[f"process:{pid}" for pid in survey_report.surviving], + shared_preserved=[str(item) for item in preserved_shared], + verified=True, + processes=survey_report.to_document(), ) return CleanupRecord( status=CleanupStatus.DESTROYED, - reason="every declared per-run resource is gone and every shared one remains", + reason=( + "every declared per-run resource is gone, every shared one remains, and " + + survey_report.detail + ), removed=list(report.removed), retained=[], shared_preserved=[str(item) for item in preserved_shared], verified=True, + processes=survey_report.to_document(), ) diff --git a/experiments/minimal-cycle/cycle_runner/cli.py b/experiments/minimal-cycle/cycle_runner/cli.py index 95ac297..04ab4c7 100644 --- a/experiments/minimal-cycle/cycle_runner/cli.py +++ b/experiments/minimal-cycle/cycle_runner/cli.py @@ -15,7 +15,15 @@ from pathlib import Path from typing import Sequence, TextIO -from . import adapters, config as config_module, ids, lifecycle, manifest, proc +from . import ( + adapters, + config as config_module, + ids, + lifecycle, + manifest, + proc, + processes, +) from .status import RunStatus, exit_code USAGE_EXIT = 2 @@ -81,6 +89,7 @@ def _run(arguments, stdout: TextIO, stderr: TextIO) -> int: adapter=adapter, run_id=run_id, tool_versions=_tool_versions(), + survey=processes.survey_and_stop, ) result = outcome.run_result if arguments.json: diff --git a/experiments/minimal-cycle/cycle_runner/lifecycle.py b/experiments/minimal-cycle/cycle_runner/lifecycle.py index 95d3c6c..bbfb322 100644 --- a/experiments/minimal-cycle/cycle_runner/lifecycle.py +++ b/experiments/minimal-cycle/cycle_runner/lifecycle.py @@ -156,7 +156,9 @@ def __init__( host_home: Path | None, environ: Mapping[str, str] | None, tool_versions: Mapping[str, Any] | None, + survey=None, ): + self.survey = survey self.config = run_config self.adapter = adapter self.run_id = run_id @@ -173,6 +175,7 @@ def __init__( dely_revision=run_config.dely_revision, ) self.handle: EnvironmentHandle | None = None + self.app_pid: int | None = None self.create_attempted = False self.blocked_reason = "" self.error_reason = "" @@ -407,6 +410,10 @@ def _identity(self) -> None: timeout=min(180, self.config.timeout_seconds), ) record.commands.append(started.to_record()) + # Only meaningful where the environment's process table is the + # host's; elsewhere the same number is somebody else's process. + if self.adapter.shares_host_processes: + self.app_pid = orca.started_pid(started) runtime = orca.wait_for_runtime( self.adapter, self.config.orca.status_argv, @@ -571,6 +578,8 @@ def _cleanup(self, export_record) -> CleanupRecord: handle=self.handle, export_record=export_record, stop_confirmed=self.stop_confirmed, + survey=self.survey, + started=[self.app_pid] if self.app_pid else [], ) self.result.cleanup = record phase_record.status = ( @@ -655,8 +664,14 @@ def run_cycle( host_home: Path | None = None, environ: Mapping[str, str] | None = None, tool_versions: Mapping[str, Any] | None = None, + survey=None, ) -> CycleOutcome: - """Run one minimal proof cycle and return its result and artifacts.""" + """Run one minimal proof cycle and return its result and artifacts. + + `survey` is how cleanup finds processes this run left on the host. It is + passed in rather than defaulted, so a test that does not ask for it can + never reach the machine running the tests. + """ cycle = _Cycle( run_config=run_config, adapter=adapter, @@ -664,6 +679,7 @@ def run_cycle( host_home=host_home, environ=environ, tool_versions=tool_versions, + survey=survey, ) with tempfile.TemporaryDirectory(prefix="dely-cycle-") as staging: return cycle.run(Path(staging)) diff --git a/experiments/minimal-cycle/cycle_runner/orca.py b/experiments/minimal-cycle/cycle_runner/orca.py index 500ecf6..e52e6bd 100644 --- a/experiments/minimal-cycle/cycle_runner/orca.py +++ b/experiments/minimal-cycle/cycle_runner/orca.py @@ -145,6 +145,21 @@ def start_argv(app_argv: Sequence[str], display: str) -> list[str]: return ["sh", "-c", START_SCRIPT, "orca-start", display, *[str(a) for a in app_argv]] +def started_pid(outcome: Any) -> int | None: + """Return the process identifier the start script reported, if it did. + + The application appears in the process table naming no path of its own, so + this is the only exact handle a run has on what it started. + """ + for line in (getattr(outcome, "stdout", "") or "").splitlines(): + if line.startswith("started "): + try: + return int(line.split()[1]) + except (IndexError, ValueError): + return None + return None + + def start_application( environment: Environment, app_argv: Sequence[str], diff --git a/experiments/minimal-cycle/cycle_runner/processes.py b/experiments/minimal-cycle/cycle_runner/processes.py new file mode 100644 index 0000000..8b1871d --- /dev/null +++ b/experiments/minimal-cycle/cycle_runner/processes.py @@ -0,0 +1,220 @@ +"""What the run started, and whether any of it is still on the host. + +A backend that shares namespaces with its host — Distrobox shares the process +table, the message bus and the display sockets — can leave a process running +after the resource that nominally contained it is gone. Removing the container +reports success, and the application it held keeps running, keeps its window on +the host's screen, and keeps writing to a directory the run has deleted. + +So a run surveys for processes still holding a path of its own, stops them, and +reports what it stopped. The survey is by path, never by program name: a name +would reach the operator's own application, which is the failure this exists to +prevent rather than a worse version of it. +""" + +from __future__ import annotations + +import os +import signal +import time +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Callable, Iterable, Sequence + +PROC_ROOT = Path("/proc") + + +@dataclass(frozen=True) +class Process: + """One process, and everything that can tie it to this run. + + The command line alone is not enough. The application this runner starts + appears in the process table as `orca-ide --disable-gpu` and names no path + at all; only its children carry `--user-data-dir`, and only its own + environment and working directory name the home it was given. A survey that + reads argv finds the children and leaves the parent running. + """ + + pid: int + command: str + parent: int = 0 + home: str = "" + cwd: str = "" + + def names(self, path: str) -> bool: + """Report whether this process is tied to the given path.""" + return any(path in field for field in (self.command, self.home, self.cwd)) + + +@dataclass +class SurveyReport: + """What was still running, what was stopped, and what would not stop.""" + + found: list[int] = field(default_factory=list) + stopped: list[int] = field(default_factory=list) + surviving: list[int] = field(default_factory=list) + detail: str = "no survey ran" + + @property + def clean(self) -> bool: + return not self.surviving + + def to_document(self) -> dict[str, Any]: + return { + "found": list(self.found), + "stopped": list(self.stopped), + "surviving": list(self.surviving), + "clean": self.clean, + "detail": self.detail, + } + + +def _read(pid: int, name: str) -> str: + try: + raw = (PROC_ROOT / str(pid) / name).read_bytes() + except (OSError, ValueError): + return "" + return raw.replace(b"\0", b" ").decode("utf-8", "replace").strip() + + +def _read_home(pid: int) -> str: + for pair in _read(pid, "environ").split(" "): + if pair.startswith("HOME="): + return pair[5:] + return "" + + +def _read_parent(pid: int) -> int: + for line in _read(pid, "status").splitlines(): + if line.startswith("PPid:"): + try: + return int(line.split()[1]) + except (IndexError, ValueError): + return 0 + return 0 + + +def holding( + paths: Sequence[str], + *, + lister: Callable[[], Iterable[Process]] | None = None, + exclude: Sequence[int] = (), + roots: Sequence[int] = (), +) -> list[Process]: + """Return the processes tied to these paths, and the trees under `roots`. + + A root is a process identifier the run recorded when it started something. + It is only meaningful on a backend that shares the host's process table, + so the caller decides whether to pass one: on a backend that does not, the + same number belongs to some unrelated process on this machine. + """ + wanted = [str(path) for path in paths if str(path).strip()] + skip = set(exclude) | {os.getpid(), os.getppid()} + processes = [ + process + for process in (list(lister()) if lister is not None else _live_processes()) + if process.pid not in skip + ] + matched = { + process.pid: process + for process in processes + if any(process.names(path) for path in wanted) + } + for root in roots: + if root in skip or root <= 1: + continue + for process in processes: + if process.pid == root: + matched[process.pid] = process + for process in _tree(processes, root): + matched[process.pid] = process + return [matched[pid] for pid in sorted(matched)] + + +def _tree(processes: Sequence[Process], root: int) -> list[Process]: + """Return every descendant of `root` in this table.""" + children: dict[int, list[Process]] = {} + for process in processes: + children.setdefault(process.parent, []).append(process) + found: list[Process] = [] + pending = [root] + seen = {root} + while pending: + current = pending.pop() + for child in children.get(current, ()): + if child.pid in seen: + continue + seen.add(child.pid) + found.append(child) + pending.append(child.pid) + return found + + +def _live_processes() -> list[Process]: + found = [] + for entry in PROC_ROOT.iterdir(): + if not entry.name.isdigit(): + continue + pid = int(entry.name) + command = _read(pid, "cmdline") + if not command: + continue + try: + cwd = os.readlink(PROC_ROOT / entry.name / "cwd") + except OSError: + cwd = "" + found.append( + Process( + pid=pid, + command=command, + parent=_read_parent(pid), + home=_read_home(pid), + cwd=cwd, + ) + ) + return found + + +def survey_and_stop( + paths: Sequence[str], + *, + lister: Callable[[], Iterable[Process]] | None = None, + signaller: Callable[[int, int], None] = os.kill, + sleeper: Callable[[float], None] = time.sleep, + grace_seconds: float = 5.0, + exclude: Sequence[int] = (), + roots: Sequence[int] = (), +) -> SurveyReport: + """Stop every process still holding one of this run's paths, and confirm it.""" + found = holding(paths, lister=lister, exclude=exclude, roots=roots) + report = SurveyReport(found=[process.pid for process in found]) + if not found: + report.detail = "no process on the host was still holding a path of this run" + return report + for process in found: + try: + signaller(process.pid, signal.SIGTERM) + except (OSError, ProcessLookupError): + continue + sleeper(grace_seconds) + still = holding(paths, lister=lister, exclude=exclude, roots=roots) + for process in still: + try: + signaller(process.pid, signal.SIGKILL) + except (OSError, ProcessLookupError): + continue + if still: + sleeper(1.0) + remaining = holding(paths, lister=lister, exclude=exclude, roots=roots) + report.surviving = [process.pid for process in remaining] + report.stopped = [pid for pid in report.found if pid not in report.surviving] + report.detail = ( + f"{len(report.stopped)} process(es) the run had started were still on the " + "host and were stopped" + if report.clean + else ( + f"{len(report.surviving)} process(es) the run had started would not stop " + "and are still on the host" + ) + ) + return report diff --git a/experiments/minimal-cycle/cycle_runner/result.py b/experiments/minimal-cycle/cycle_runner/result.py index ec7fb8d..0e4a424 100644 --- a/experiments/minimal-cycle/cycle_runner/result.py +++ b/experiments/minimal-cycle/cycle_runner/result.py @@ -197,6 +197,7 @@ class CleanupRecord: retained: list[str] = field(default_factory=list) shared_preserved: list[str] = field(default_factory=list) verified: bool = False + processes: dict[str, Any] = field(default_factory=dict) def to_document(self) -> dict[str, Any]: return { @@ -206,6 +207,7 @@ def to_document(self) -> dict[str, Any]: "retained": list(self.retained), "shared_preserved": list(self.shared_preserved), "verified": self.verified, + "processes": dict(self.processes), } diff --git a/experiments/minimal-cycle/tests/test_cleanup.py b/experiments/minimal-cycle/tests/test_cleanup.py index 722cc12..2e610c5 100644 --- a/experiments/minimal-cycle/tests/test_cleanup.py +++ b/experiments/minimal-cycle/tests/test_cleanup.py @@ -4,7 +4,7 @@ import unittest from pathlib import Path -from cycle_runner import cleanup, result, status +from cycle_runner import cleanup, processes, result, status from cycle_runner.adapters.base import Resource from tests.fakes import FakeAdapter @@ -178,3 +178,62 @@ def test_a_confirmed_stop_lets_destroy_run(self): stop_confirmed=True, ) self.assertEqual(record.status, status.CleanupStatus.DESTROYED) + + +class ProcessSurveyTest(unittest.TestCase): + """Cleanup reports what the run left running, and refuses to call it clean.""" + + def setUp(self): + self._tmp = tempfile.TemporaryDirectory() + self.root = Path(self._tmp.name) + self.adapter = FakeAdapter(self.root / "run") + self.handle = self.adapter.create() + self.addCleanup(self._tmp.cleanup) + + def perform(self, survey, **options): + return cleanup.perform( + adapter=self.adapter, + handle=self.handle, + export_record=export_record(status.ExportStatus.CONFIRMED), + survey=survey, + **options, + ) + + def test_a_clean_survey_is_named_in_the_reason(self): + record = self.perform( + lambda paths, roots=(): processes.SurveyReport(detail="no process on the host") + ) + self.assertEqual(record.status, status.CleanupStatus.DESTROYED) + self.assertIn("no process on the host", record.reason) + + def test_a_surviving_process_is_residue_even_when_every_resource_is_gone(self): + record = self.perform( + lambda paths, roots=(): processes.SurveyReport( + found=[7], surviving=[7], detail="1 process would not stop" + ) + ) + self.assertEqual(record.status, status.CleanupStatus.RESIDUE) + self.assertIn("process:7", record.retained) + self.assertEqual(record.processes["surviving"], [7]) + + def test_the_survey_is_given_this_runs_paths_and_no_others(self): + seen = [] + + def survey(paths, roots=()): + seen.append(list(paths)) + return processes.SurveyReport(detail="looked") + + self.perform(survey) + self.assertEqual(len(seen), 1) + for path in seen[0]: + self.assertTrue(path.startswith(str(self.root)), path) + + def test_what_the_run_started_is_handed_to_the_survey(self): + seen = {} + + def survey(paths, roots=()): + seen["roots"] = list(roots) + return processes.SurveyReport(detail="looked") + + self.perform(survey, started=[4242]) + self.assertEqual(seen["roots"], [4242]) diff --git a/experiments/minimal-cycle/tests/test_doubles_are_contained.py b/experiments/minimal-cycle/tests/test_doubles_are_contained.py index 898c293..751282a 100644 --- a/experiments/minimal-cycle/tests/test_doubles_are_contained.py +++ b/experiments/minimal-cycle/tests/test_doubles_are_contained.py @@ -110,3 +110,27 @@ def test_an_unlisted_program_is_answered_not_run(self): ) self.assertEqual(outcome.exit_code, 0) self.assertFalse(Path("/tmp/stub-should-not-exist").exists()) + + +class SurveyIsNeverDefaultedTest(unittest.TestCase): + """The survey signals real processes, so a test must have to ask for it.""" + + def test_cleanup_without_a_survey_reads_no_process_table(self): + import inspect + + from cycle_runner import cleanup, lifecycle + + self.assertIsNone( + inspect.signature(cleanup.perform).parameters["survey"].default, + "cleanup must not default to a survey that signals real processes", + ) + self.assertIsNone( + inspect.signature(lifecycle.run_cycle).parameters["survey"].default, + "a cycle must not default to a survey that signals real processes", + ) + + def test_the_command_line_is_what_asks_for_a_real_survey(self): + from pathlib import Path + + source = (Path(__file__).resolve().parent.parent / "cycle_runner" / "cli.py").read_text() + self.assertIn("survey=processes.survey_and_stop", source) diff --git a/experiments/minimal-cycle/tests/test_processes.py b/experiments/minimal-cycle/tests/test_processes.py new file mode 100644 index 0000000..246aff4 --- /dev/null +++ b/experiments/minimal-cycle/tests/test_processes.py @@ -0,0 +1,159 @@ +"""Removing the container is not ending what the run started inside it.""" + +import unittest + +from cycle_runner import processes + +RUN_PATH = "/var/tmp/dely-cycle/state/20260915T000000Z-aaaaaa-00000001" + + +def process(pid, command): + return processes.Process(pid=pid, command=command) + + +class Table: + """A stand-in process table that a signal really changes.""" + + def __init__(self, rows, stubborn=()): + self.rows = list(rows) + self.stubborn = set(stubborn) + self.signalled = [] + + def lister(self): + return list(self.rows) + + def signaller(self, pid, number): + self.signalled.append((pid, number)) + if pid in self.stubborn and number != 9: + return + self.rows = [row for row in self.rows if row.pid != pid] + + +class SurveyTest(unittest.TestCase): + def setUp(self): + self.slept = [] + + def run_survey(self, table, **options): + return processes.survey_and_stop( + [RUN_PATH], + lister=table.lister, + signaller=table.signaller, + sleeper=self.slept.append, + **options, + ) + + def test_a_process_holding_this_runs_path_is_found_and_stopped(self): + table = Table([ + process(101, f"/opt/Orca/orca-ide --user-data-dir={RUN_PATH}/home/.config/orca"), + process(102, "/opt/Orca/orca-ide --user-data-dir=/home/someone/.config/orca"), + ]) + report = self.run_survey(table) + self.assertEqual(report.found, [101]) + self.assertEqual(report.stopped, [101]) + self.assertTrue(report.clean) + + def test_the_operators_own_application_is_never_touched(self): + table = Table([ + process(201, "/opt/Orca/orca-ide --user-data-dir=/home/someone/.config/orca"), + process(202, "/opt/Orca/orca-ide"), + ]) + report = self.run_survey(table) + self.assertEqual(report.found, []) + self.assertEqual(table.signalled, []) + self.assertIn("no process", report.detail) + + def test_a_process_that_ignores_the_first_signal_is_killed(self): + table = Table([process(301, f"bash --rcfile {RUN_PATH}/home/.bashrc")], stubborn={301}) + report = self.run_survey(table) + self.assertEqual([number for _pid, number in table.signalled], [15, 9]) + self.assertTrue(report.clean) + + def test_a_process_that_will_not_stop_is_reported_as_surviving(self): + class Unkillable(Table): + def signaller(self, pid, number): + self.signalled.append((pid, number)) + + table = Unkillable([process(401, f"orca-ide --user-data-dir={RUN_PATH}/x")]) + report = self.run_survey(table) + self.assertEqual(report.surviving, [401]) + self.assertFalse(report.clean) + self.assertIn("would not stop", report.detail) + + def test_the_survey_never_signals_the_runner_itself(self): + import os + + table = Table([process(os.getpid(), f"python3 run-cycle {RUN_PATH}")]) + report = self.run_survey(table) + self.assertEqual(report.found, []) + self.assertEqual(table.signalled, []) + + def test_an_empty_path_list_surveys_nothing(self): + table = Table([process(501, f"orca-ide {RUN_PATH}")]) + report = processes.survey_and_stop( + [""], lister=table.lister, signaller=table.signaller, sleeper=self.slept.append + ) + self.assertEqual(report.found, []) + self.assertEqual(table.signalled, []) + + +if __name__ == "__main__": + unittest.main() + + +class ParentWithoutAMarkerTest(unittest.TestCase): + """The application this runner starts names no path in its own argv.""" + + def setUp(self): + self.slept = [] + + def table(self): + return Table([ + # As it really appears: the parent names nothing, the children do. + process(900, "/opt/Orca/orca-ide --disable-gpu"), + process(901, f"/opt/Orca/orca-ide --type=gpu-process --user-data-dir={RUN_PATH}/home/.config/orca"), + process(902, "/opt/Orca/orca-ide parcel-watcher-process-entry.js"), + process(999, "/opt/Orca/orca-ide"), + ]) + + def test_a_parent_that_names_nothing_is_left_when_only_argv_is_read(self): + table = self.table() + found = processes.holding([RUN_PATH], lister=table.lister) + self.assertEqual([p.pid for p in found], [901]) + + def test_its_home_ties_it_to_the_run(self): + table = Table([ + processes.Process( + pid=900, + command="/opt/Orca/orca-ide --disable-gpu", + home=f"{RUN_PATH}/home", + ), + processes.Process(pid=999, command="/opt/Orca/orca-ide", home="/home/someone"), + ]) + found = processes.holding([RUN_PATH], lister=table.lister) + self.assertEqual([p.pid for p in found], [900]) + + def test_a_recorded_identifier_takes_its_whole_tree(self): + table = Table([ + processes.Process(pid=900, command="/opt/Orca/orca-ide --disable-gpu", parent=1), + processes.Process(pid=901, command="helper", parent=900), + processes.Process(pid=902, command="helper of a helper", parent=901), + processes.Process(pid=999, command="/opt/Orca/orca-ide", parent=1), + ]) + found = processes.holding([], lister=table.lister, roots=[900]) + self.assertEqual([p.pid for p in found], [900, 901, 902]) + + def test_no_recorded_identifier_means_no_tree_is_taken(self): + table = Table([ + processes.Process(pid=900, command="/opt/Orca/orca-ide --disable-gpu", parent=1), + processes.Process(pid=901, command="helper", parent=900), + ]) + self.assertEqual(processes.holding([], lister=table.lister), []) + + def test_the_runner_is_never_a_root(self): + import os + + table = Table([ + processes.Process(pid=os.getpid(), command="the runner", parent=1), + processes.Process(pid=2, command="a child of the runner", parent=os.getpid()), + ]) + self.assertEqual(processes.holding([], lister=table.lister, roots=[os.getpid()]), []) From 1114607c560c898a4acae0abd94aed9f4b4e75d4 Mon Sep 17 00:00:00 2001 From: Kacper Paczos Date: Tue, 15 Sep 2026 23:03:44 +0200 Subject: [PATCH 42/42] Name the new rails in the acceptance table and stop the README implying a sandbox. --- experiments/minimal-cycle/README.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/experiments/minimal-cycle/README.md b/experiments/minimal-cycle/README.md index dbe7768..7951d4c 100644 --- a/experiments/minimal-cycle/README.md +++ b/experiments/minimal-cycle/README.md @@ -107,7 +107,8 @@ $artifact_root// logs/commands.jsonl every command, with timings and exit codes run-before-cleanup.json the result as it stood when the export was taken export-receipt.json each artifact re-read from the host, with its digest - cleanup.json what was removed, what was kept, and how that was checked + cleanup.json what was removed, what was kept, what the run left + running on the host, and how each was checked ``` `manifest.json`, `cleanup.json` and `host-after.json` are written after cleanup, @@ -267,12 +268,20 @@ the table with the tests each row runs. The recorded sweep is in | The reply that decided the run is exported beside the verdict | case `the-reply-that-decided-the-run-is-kept` | `evidence/distrobox-settled-cycle/dispatch-worker-start.json` | | A dispatch the plane could not verify keeps what its agent's terminal held | case `an-unverifiable-dispatch-keeps-its-terminal` | `evidence/vm-unobserved-turn/` | | A worker really does the task and an independent check agrees | `./run-cycle run` on this host | `evidence/distrobox-settled-cycle/` | +| A coordinator terminal has to say which machine it is on | case `the-terminal-says-which-machine-it-is-on` | `evidence/distrobox-settled-cycle/` | +| A run whose processes are still on the host is residue, not destroyed | case `a-process-left-running-is-residue` | `evidence/counterexamples.txt` | +| The survey matches this run's paths, never a program name | case `the-survey-matches-a-path-not-a-program` | `evidence/counterexamples.txt` | ## What no instrument here observes -A second, different task. A run on any host but this one. Isolation on the -container backend, which mounts the host home and says so in `preflight.json` -rather than claiming otherwise. +A second, different task. A run on any host but this one. + +Isolation on the container backend. Distrobox mounts the host home, shares the +host's process namespace and shares the host's display sockets. The runner +records each of these rather than claiming otherwise: `preflight.json` blocks +until the home mount is acknowledged, the cleanup survey exists because +processes outlive the container, and a virtual display did **not** keep the +application off the host's desktop when one run assumed it would. The machine backend has not completed a cycle. It creates its domain, proves a distinct kernel, starts Orca, opens a coordinator terminal and dispatches a