feat: durable media output history, one safe opener, and aether doctor v2 - #66
Merged
Conversation
…rm opener
`aether output open 101` could resolve to any of dozens of artifacts. The v1
index wrote `index: entries.length + 1`, and retention trims the array to 100,
so the 101st generation and every one after it all claimed index 101. A parse
failure returned `[]`, making a corrupt history indistinguishable from "no
generations yet". The whole file was rewritten with a bare writeFileSync, so an
interrupted write could lose every entry, and two Agent turns writing at once
would silently drop one side.
Schema v2 gives every artifact a UUID that never changes plus a persistent
monotonic `sequence` alias that survives trimming, so the convenient numeric
reference stays convenient without ever being reused. The counter is stored, not
derived from the retained window.
Writes now run one locked transaction: acquire an owner-stamped cross-process
lock, read the best valid generation, allocate under the lock, validate, write a
same-directory temp, fsync, stage a backup of the previous generation, rename
atomically, then re-read and confirm the committed generation before reporting
success. A crash at any phase leaves either the old or the new generation intact.
Reads recover in order — primary, backup, rebuild from the output directory —
and every degraded outcome returns a visible state (`recovered-backup`,
`rebuilt`, `degraded`) with a warning the CLI prints above the results. An
unreadable index is preserved as `.corrupt.<stamp>` instead of being overwritten,
and a document written by a newer Aether is left strictly alone.
Opening moves to one argument-array implementation shared by media, `auth login`
and `github connect`. The two old paths each built a shell string — vision.ts ran
execSync(`${cmd} "${filepath}"`) and browser.ts spawned `cmd /c start "" <url>` —
so a filename containing a quote, `&` or a backtick was a command-injection
primitive. Targets are validated first (http/https only, no embedded credentials,
files must exist), then handed to explorer.exe/open/xdg-open with shell disabled.
- v1 indexes migrate on read; duplicate 101s are repaired deterministically
before the document is committed, so the resolver never picks a first match
- ambiguous references now report their candidates instead of guessing
- 53 new tests: identity, retention, migration, four-writer cross-process
concurrency, crash injection at every persistence transition, future-schema
refusal, and a filename-injection regression
…answers `aether doctor` reported one axis — pass / warn / fail / skip. That made "the backend URL is well-formed" and "the backend answered just now" both read as a green pass, which is the exact confusion a health command exists to remove. A `skip` for an unrun check sat next to real passes in the same summary line. Every check now answers three questions independently: is it configured, is it reachable, and was it verified during this run. An axis nobody exercised reports `not-checked`, never a pass, and a `verified now` timestamp can only come from the current run. A surface this build genuinely does not have — Actions dispatch, Predator — reports `n/a` with the reason, instead of a green tick for something that does not exist. Four checks are new: the media output index (schema, retained count, next reference, recovery state), the opener (validated and command-selected without spawning anything), GitHub identity, and Protocol-C receipt persistence. Fast mode is now explicitly read-only: no network call, no model call, no session, no opener launch, no credential refresh, no write. `--deep` keeps meaning exactly what it meant and points at the new mode rather than silently becoming something that mutates. `--fix` is a closed allowlist, not a repair agent. It prints the exact scope, action, risk, reversibility and backup behaviour of every planned repair, and changes nothing without `--yes`. `--dry-run` shows the plan and stops. Each repair re-checks its own precondition at apply time, so a plan built against a stale report cannot force a mutation. The forbidden set — credential rotation, UVT spend, model invocation, source edits, git ref mutation, Actions dispatch, Predator runs, MCP write tools — is exported as data so every exclusion has a negative test. - allowlist: create missing state dirs, tighten Aether-owned permissions, remove provably abandoned locks and transaction temps, rebuild the media index preserving the corrupt original, prune worktree metadata git already reports gone - `--live` is declared and refuses with an explanation. The session, opener-callback and receipt proofs are not built yet, and reporting fast-mode results as verified would be the false green this change removes - 18 tests: axis defaults, ordering, redaction, n/a honesty, --deep contacting nothing, allowlist closure, live-lock survival, evidence preservation, and a negative test per forbidden operation
…ipts `--live` was declared but refused, because reporting fast-mode results as verified is exactly the false green this command exists to remove. It now runs a real proof, and still refuses to claim anything it did not exercise. The agent sequence is driven serially — create, frames, pause, resume, steer, sandboxed tool round trip, close — because a parallel version could not tell a missing acknowledgement from a racing one. Frame sequence numbers are checked for strict monotonicity, the steer carries the run nonce, and the tool call is answered by a write/read/compare/delete confined to a doctor-owned temp directory regardless of what the server asked for. No billed health probe. The create request carries purpose:"doctor" and max_uvt:0, and the loop runs only if the server echoes purpose:"doctor" with billable:false. Anything else is closed immediately and reported as unproven with the reason, rather than risking a probe that costs the user money. Every usage and done frame is accounted, and a run that was billed — or that left a session open — reports spend.none as an error. The opener proof serves a page on loopback and waits for that page to call back with the run nonce, so a process that spawns but renders nothing is not a pass. Headless reports skipped, never verified. Branch freshness compares local HEAD to the remote tip using ls-remote plus local object queries only. When origin has a commit this checkout has never seen, it says so instead of fetching to find out — a health command must not mutate the repository. A test asserts no mutating git verb is ever invoked. The MCP probe calls a tool only when the tool declares itself readOnly and doctorSafe. No such annotation exists on the broker today, so the honest result is "reachable, and here is why no tool proof was possible" rather than guessing which tool looks harmless. The Protocol-C proof persists a receipt, reads it back, verifies the commitment, and proves a replay is de-duplicated — through the production appendCustody/readCustodyLog path, but pointed at a doctor sandbox so the user's real receipt log is never written to. custody.ts gained an optional trailing path parameter for this; every existing call site is unchanged. - adds --no-ui for headless boxes, and --only to narrow a live run - every temp file lives under one sandbox directory, removed in finally - 22 new tests: sequence monotonicity, all seven branch states, billed-run and orphan-session detection, unacknowledged control, missing doctor-safe tool, signed-out, sandbox cleanup, and a no-credential-leak assertion - docs: COMMANDS.md doctor section rewritten, patch notes for 2026-08-14
…n exit server.close() only stops accepting new connections. A browser holding the loopback proof page open keeps the handle — and therefore the process — alive after the probe has already answered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
Three defects that share a shape: state you can lose without being told.
output open 101was ambiguous.vision.tswroteindex: entries.length + 1, and retention trims the array to 100 — so the 101st generation and every one after it all claimed101. The resolver returned whichever came first.A corrupt history looked like an empty one.
readLog()wascatch { return [] }. Parse failure, disk failure and "no generations yet" were indistinguishable.The index could be lost outright.
saveLog()was a barewriteFileSyncover the only copy — no lock, no fsync, no backup. Two Agent turns writing at once silently dropped one side; an interrupted write could truncate everything.Opening anything went through a shell.
execSync(`${cmd} "${entry.filepath}"`)for files,cmd /c start "" <url>for URLs. A filename containing",&or a backtick was a command-injection primitive. The Windows file path was also simply broken —startis acmdbuiltin, not an executable.aether doctorreported one axis. "The backend URL is well-formed" and "the backend answered just now" both rendered as a greenpass, and an unrun check showedskipbeside real passes.Implementation map
src/core/durable_store.ts(new),src/core/opener.ts(new)src/core/media_history.ts(new),src/core/media_history_store.ts(new),src/core/vision.ts,src/core/browser.tssrc/commands/output.ts,src/commands/media.ts,src/commands/slash_media.tssrc/core/health.ts(new),src/core/diagnostics.ts,src/core/doctor_live.ts(new),src/core/doctor_repair.ts(new),src/commands/doctor.tssrc/core/custody.ts(optional trailingpathparam; every existing call site unchanged)Media history v2
Root document is versioned:
schemaVersion,generation,nextSequence,updatedAt,entries[]. Each entry carries anartifactId(UUID, generated once, never changes) and asequence— a persistent monotonic alias serialised as a decimal string, sooutput open 285stays convenient without the counter ever being derived fromentries.length. Trimming happens after allocation, so retention can never lower the next reference.Migration. v1 arrays migrate on first read. An index that is a positive integer and unique across the whole legacy set keeps its value; duplicate, missing, zero, negative and non-integer indexes are reallocated above the survivors in stable chronological order. Repairs land before the document is committed, so the resolver never picks a first match. Repeated migrations of the same input are deterministic.
Durability. Every write is one locked transaction: acquire an owner-stamped cross-process lock → read the best valid generation → allocate under the lock → validate → write a same-directory temp → fsync → stage a backup of the previous generation → atomic rename → re-read and confirm the committed generation → release in
finally. The temp is always a sibling of the target, because a temp on another volume turns rename into copy+delete.Recovery. Reads try primary → backup → rebuild from the output directory, and return a state (
ok/migrated/recovered-backup/rebuilt/degraded) with a warning the CLI prints above results. An unreadable primary is preserved as.corrupt.<timestamp>; a document with a newerschemaVersionis left strictly alone and refuses both append and clear. Rebuilt entries derive stable IDs from file facts, so re-running a rebuild does not mint a fresh duplicate set, and they are labelledrecoveredwith prompt and model left empty rather than invented.Resolution. Sequence → full artifact ID → unique ID prefix → unique filename. Ambiguity lists its candidates instead of guessing.
Safe opener
One implementation for files and URLs, shared by media,
auth loginandgithub connect. Validates first — http/https only, no embedded credentials, files must exist and be regular files or directories — then hands the target toexplorer.exe/open/xdg-openas an argument array withshell: false. Paths are resolved to absolute, which also removes the leading-dash case.doctor --liveproves this exact code path.Doctor v2
Three axes per check. An axis nobody exercised reports
not-checked, never a pass;verified nowcan only be stamped from the current run; a surface this build genuinely lacks (Actions dispatch, Predator) reportsn/awith the reason.Fast is strictly read-only — no network, model, session, opener launch, credential refresh or write. New checks: media index, opener (validated and command-selected without spawning), GitHub, Protocol-C receipt storage.
--liveruns the real proof: catalog fetch; dev session; sequence-numbered frames checked for strict monotonicity; pause → resume → steer, in that order, carrying the run nonce; a sandboxed tool write/read/compare/delete confined to a doctor temp dir regardless of what the server asks for; clean session close; a browser open confirmed by a loopback callback; GitHub identity; branch freshness; MCP broker; Protocol-C receipt round trip.No billed probe. The create request carries
purpose: "doctor"andmax_uvt: 0. The loop runs only if the server echoespurpose: "doctor"withbillable: false; anything else is closed immediately and reported unproven with the reason. Everyusage/doneframe is accounted, and a run that was billed — or left a session open — reportsspend.noneas an error.Branch freshness never fetches.
ls-remoteplus local object queries. When origin has a commit this checkout has never seen, it says so rather than fetching to find out. A test asserts no mutating git verb is ever invoked.MCP. A tool is called only when it declares itself
readOnlyanddoctorSafe. No such annotation exists on the broker today, so the honest result is "reachable, and here is why no tool proof was possible".--fixis a closed allowlist: create missing state dirs, tighten Aether-owned permissions, remove provably abandoned locks and transaction temps, rebuild the media index preserving the corrupt original, prune worktree metadata git already reports gone. It prints exact scope / action / risk / reversibility / backup and changes nothing without--yes;--dry-runshows the plan and stops. Each repair re-checks its own precondition at apply time, so a plan built against a stale report cannot force a mutation. The forbidden set is exported as data so every exclusion has a negative test.--deepkeeps its original read-only meaning and points at--live.Threat cases covered
"&`$( )'and spaces cannot spawn a second process — asserted on the argument array, not a rendered stringxdg-openas a flagnew URL("C:\\x")parses with protocolc:)--fixhas a negative test per forbidden operationTests
npm run typecheck npm test922 pass, 0 fail (886 on the base commit; the two
--deepnetwork tests were replaced by read-only and live equivalents). New files:test/durable_store.test.ts,test/opener.test.ts,test/media_history.test.ts,test/media_history_store.test.ts,test/doctor_repair.test.ts,test/doctor_live.test.ts.Notable coverage: 250 generations at retention 100 (no reference reused,
nextSequencesurvives every trim); four concurrent writer processes spawned via realexecFileSyncchildren (no duplicate sequences, no lost entries, no leftover lock or temp); crash injection at every persistence transition; corrupt-primary → backup → rebuild → degraded; future-schema refusal; deterministic duplicate-101migration; all seven branch-freshness states; billed-run and orphan-session detection; sandbox cleanup.CI (
.github/workflows/ci.yml) runsubuntu-latestandwindows-latest, covering the atomic-replace, locking, opener and path behaviour on both.Migration and rollback
No action on upgrade.
output open <number>keeps working; v1 logs migrate on first read and stay visible. Rollback is asymmetric: an older binary reads a v2 document as an unparseable array and would report an empty history. The v2 file is not destroyed, but a downgrade should be paired with restoring.genlog.json.bakor letting the rebuild path repopulate from disk.The doctor JSON report is now
schemaVersion: 2; scripts should read the three axes rather than a single status field.Docs
COMMANDS.mddoctor section rewritten; the/outputrow documents the new reference forms. Patch notes inRELEASE_NOTES.mdanddocs/releases/2026-08-14.md, indexed indocs/releases/README.md.Known gap, stated rather than hidden
--live's agent-loop probes report unproven against today's server, because no non-billable doctor session exists yet. The client contract is implemented and documented (purpose: "doctor"+max_uvt: 0out,purpose: "doctor"+billable: falseback); the server half is an AETHER-CLOUD change (api/routes/agent_dev_session_routes.py) and is deliberately not in this PR. Until it lands, those checks say why they could not be proven instead of showing green.