Skip to content

feat(backends)!: replace gemini with agy; publish runner images weekly - #71

Merged
KE7 merged 8 commits into
mainfrom
feat/agy-backend-and-sandbox-fixes
Sep 11, 2026
Merged

feat(backends)!: replace gemini with agy; publish runner images weekly#71
KE7 merged 8 commits into
mainfrom
feat/agy-backend-and-sandbox-fixes

Conversation

@KE7

@KE7 KE7 commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Replaces the gemini mutation backend with agy (Google's Antigravity CLI),
fixes the runner-image publish workflow so the images actually get
refreshed, and adds a pull-request check that builds and probes every image.

Both changes are independent of the credential-projection work explored on
impl/auth-two-modes. That branch's central idea — giving each candidate a
private copy of the operator's credential — is not adopted here: nothing in
this branch changes how the sandbox authenticates. agy gets the same
whole-$HOME helix-auth-agy mount every other backend already gets.

agy replaces gemini

BREAKING: agent.backend = "gemini" is no longer accepted. Migrate to
"agy" and run helix sandbox login agy; existing helix-auth-gemini
volumes are no longer read.

The migration touches the backend registry, argv construction and output
parsing, the config Literal and CLI --backend choice, a new
docker/agy.Dockerfile, the publish-runners matrices, and every doc, skill,
and test reference. Unlike gemini, agy propagates agent.effort
(--effort low|medium|high) instead of warning that the knob is ignored.

Two things are deliberately claimed no more strongly than they have been
established:

  • The credential path the status probe tests
    ($HOME/.gemini/antigravity-cli/antigravity-oauth-token) was derived by
    tracing the CLI's own file opens, not confirmed against a completed
    sign-in. logout therefore removes only ~/.gemini/antigravity-cli,
    never the shared ~/.gemini root.
  • agy gets no BACKEND_AUTH_ENV row: naming ambient credential variables
    would assert a shape nobody here has observed. Its --output-format json
    envelope, on the other hand, has been captured (agy 1.1.27): a single
    object with conversation_id, status, response, error (on failure),
    duration_seconds, num_turns, and a usage block of input_tokens /
    output_tokens / thinking_tokens / cache_read_tokens / total_tokens.
    _normalise_usage_stats now maps thinking_tokens to the reasoning
    counter, cache_read_tokens to the cache-read counter, and
    conversation_id to session_id; before this those two token counts were
    always charged as 0. The envelope has no cost field, so cost_usd stays 0
    for agy, and no per-tool event list, so _TRANSCRIPT_TOOL_COUNTERS still
    has no agy entry. The budget tests' agy fixtures are the captured envelope
    rather than an illustrative one.

Argv grammar. agy's -p/--print (alias --prompt) takes the prompt
as its value; it is not a boolean flag like claude's. The argv therefore
puts every other flag first and ends with --print <prompt> as the final
pair. Verified against agy 1.1.27: the old order (--print --output-format json ... <prompt>) exits 2 with --print took "--output-format" as its prompt, while --output-format json --print "<prompt>" --bogus fails only
on --bogus, proving the prompt is consumed as the flag's value. The unit
test now pins --print as the second-to-last element and the prompt as the
last, instead of only asserting "--print" in args. (An earlier revision of
this description claimed --print was boolean after a probe against
agy 1.1.17, which silently ran "--output-format" as the prompt instead of
erroring; that claim was wrong.)

Print timeout. agy's --print-timeout is a Go time.Duration flag
defaulting to 5m0s; when it expires the headless run stops, and per agy's
changelog a mid-turn expiry returns partial output with only a stderr
warning, so a cut-off mutation can look like a finished one. There is no
disable value: 0 and -1s both fail immediately with timeout waiting for response, -1 without a unit is an invalid duration, and no environment
variable or settings key overrides the flag. The argv therefore passes the
largest duration Go can represent, 2562047h47m16.854775807s
(math.MaxInt64 nanoseconds, ~292 years), which agy 1.1.27 accepts and
completes a turn under; 2562048h is rejected, so this is the ceiling of
the flag's type. The maximum was chosen over a 24h cap because every other
backend runs its subprocess with no timeout at all
(test_no_timeout_in_subprocess), and this keeps agy on the same policy.
The flag sits among the leading flags, ahead of the closing --print <prompt> pair; the argv tests pin it to exactly one occurrence, immediately
followed by the value, before --print.

Runner images get published on a schedule

The images were only ever published on a v* tag. The last successful
publish was v0.2.1 on 2026-05-10, so latest had been drifting behind the
agent CLIs' own releases for months. The one attempt to refresh it, a manual
dispatch from feat/agy-backend on 2026-08-24, failed at the manifest step
with can't push with no tags specified, because a non-default branch
produced no tag to push.

Three changes to publish-runners.yml, no new files:

  • Weekly schedule trigger (Mondays 12:00 UTC, inside the
    "before 6am on monday" America/Los_Angeles window Renovate uses in chore: add Renovate configuration for automated dependency updates #54). A scheduled run is on
    main, so it publishes latest through the existing tagging rules.
  • Verify as node. The post-publish smoke test now runs each CLI's
    version probe as the unprivileged user every agent container runs as, not
    as root. Root cannot see an installer that leaves its binary under /root,
    which is exactly the bug agy.Dockerfile has to work around; this makes
    that class of bug fail the publish instead of the first real run.
  • Branch dispatch gets a tag. A single resolve-tag job now decides the
    tag every later job builds against and publishes: the version for a v*
    tag, latest on main, and the branch name (slashes replaced) anywhere
    else. It replaces the two copied shell snippets that previously made the
    same choice and never handled the branch case.

Runner images get a pull-request check

publish-runners.yml pushes latest before it verifies, so a broken
Dockerfile reaches latest before anything notices, and a backend added in
a PR (like agy here) has no published image to probe at all. A new
runner-images.yml workflow closes both gaps: on any PR touching
docker/**, src/helix/backends.py, or the workflow itself, each matrix
leg builds the base image and one backend image from this checkout, then
runs the CLI's version probe as node. No layer cache, so each run installs
the CLI versions a release build would. It is 42 lines of plain YAML with no
bake file or test module.

Gates

  • uv run python -m pytest985 passed
  • uv run ruff check src/ tests/ — clean
  • uv run mypy --strict src/helix/ — clean
  • actionlint .github/workflows/runner-images.yml — clean
  • actionlint .github/workflows/publish-runners.yml — no new findings (the
    four pre-existing SC2046 warnings on the imagetools create lines are
    unchanged from main)
  • git diff --shortstat origin/main...HEAD — 17 files changed, 426 insertions(+), 295 deletions(-)

Dockerfile build proof. docker/agy.Dockerfile was built from this
branch (--no-cache, linux/arm64) and exercised as the unprivileged user:

$ docker run --rm --user node --network none --security-opt no-new-privileges <image> sh -lc 'agy --version'
1.1.21
$ ... sh -lc 'id'
uid=1000(node) gid=1000(node) groups=1000(node)

The /opt copy is load-bearing, not cosmetic: /root is mode 0700, and
invoking the installer's own /root/.local/bin/agy as node gives
Permission denied.

🤖 Generated with Claude Code

https://claude.ai/code/session_015HBBoDVQK7baNMQBhgRkh4

…y CLI)

Google's Antigravity CLI (`agy`) replaces the `gemini` backend across the
registry (`BACKENDS`, `EFFORT_*`, `BACKEND_DISPLAY_NAMES`,
`DEFAULT_BACKEND_IMAGES`, `BACKEND_AUTH_COMMANDS` in `helix.backends`), argv
construction and output parsing (`_build_backend_args`,
`_parse_backend_output`, `_TRANSCRIPT_TOOL_COUNTERS` in `helix.mutator`), the
config `Literal` and CLI `--backend` choice, a new `docker/agy.Dockerfile`
modelled on cursor's curl-installer pattern, the publish-runners build /
merge / verify matrices, and every doc, skill, and test reference.

agy is effort-aware (`--effort low|medium|high`, the same values as claude),
unlike gemini, so it joins `EFFORT_AWARE_BACKENDS` and gets an
`EFFORT_VALID_VALUES` entry instead of warning that the knob is ignored.

Authentication is unchanged in shape: agy gets the same whole-`$HOME`
`helix-auth-agy` mount every other backend gets, so nothing backend-specific
is needed on the sandbox path. Two things are deliberately claimed no more
strongly than they have been established:

  * The credential path the `status` probe tests
    (`$HOME/.gemini/antigravity-cli/antigravity-oauth-token`) was derived by
    tracing the CLI's own file opens and has not been confirmed against a
    completed sign-in. `logout` therefore removes only
    `~/.gemini/antigravity-cli`, never the shared `~/.gemini` root.
  * agy gets no `BACKEND_AUTH_ENV` row and no `_TRANSCRIPT_TOOL_COUNTERS`
    entry. Naming ambient credential variables or transcript field names for
    it would assert a shape nobody here has observed; the generic
    `_normalise_usage_stats` walk covers usage stats in the meantime.

`docker/agy.Dockerfile` copies the installed binary out of `/root/.local/bin`
to `/opt` before symlinking it onto PATH, because `/root` is mode 0700 and
the agent container runs as the unprivileged `node` user (see `_docker_args`
in `helix.sandbox`). Verified by building the image and running
`agy --version` in it as `node`, not as root.

BREAKING: `agent.backend = "gemini"` is no longer accepted. Migrate to
`"agy"` and run `helix sandbox login agy` to create `helix-auth-agy`;
existing `helix-auth-gemini` volumes are no longer read.
Three small changes to publish-runners.yml, replacing a much larger
build-every-image-per-PR job that did not address publish cadence:

- Add a weekly `schedule` trigger so `latest` tracks the agent CLIs'
  own release cadence instead of waiting for the next `v*` tag. The
  last successful publish before this was v0.2.1 on 2026-05-10.
- Run the verify step's probe as `node`, the unprivileged user every
  agent container runs as. Probing as root cannot see an installer
  that leaves its binary under /root, which is exactly the bug
  agy.Dockerfile had to work around.
- Give a manual dispatch from a non-default branch a tag to push (the
  branch name, slashes replaced). The 2026-08-24 dispatch from
  feat/agy-backend failed at the manifest step with "can't push with
  no tags specified" because no tag applied. The tag choice now lives
  in one `resolve-tag` job instead of two copied shell snippets.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015HBBoDVQK7baNMQBhgRkh4
@KE7
KE7 force-pushed the feat/agy-backend-and-sandbox-fixes branch from 9e29c4f to 98fd252 Compare September 8, 2026 19:36
@KE7 KE7 changed the title feat(backends)!: replace gemini with agy, and cover the runner images in CI feat(backends)!: replace gemini with agy; publish runner images weekly Sep 8, 2026
KE7 and others added 3 commits September 8, 2026 17:22
publish-runners.yml pushes `latest` before it verifies, so a broken
Dockerfile reaches `latest` before anything notices, and a backend added
in a PR has no published image to probe at all. This job closes both
gaps with one paths-filtered workflow: each matrix leg builds the base
image and one backend image from this checkout, then runs the CLI's
version probe as `node`, the unprivileged user every agent container
runs as. No layer cache, so each run installs the CLI versions a release
build would.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015HBBoDVQK7baNMQBhgRkh4
agy's -p/--print (alias --prompt) is a string flag, not a boolean like
claude's: the prompt is its VALUE and a trailing positional is ignored.
The previous argv, `agy --dangerously-skip-permissions --print
--output-format json ... <prompt>`, made agy 1.1.27 exit 2 with
`--print took "--output-format" as its prompt`, so every agy mutation
failed before starting a turn.

Every other flag now goes first and `--print <prompt>` closes the argv
(verified locally: `agy --output-format json --print "..." --bogus`
fails only on `--bogus`, proving the prompt was consumed as the flag's
value). The stale comment claiming the grammar was unconfirmed is
replaced with the verified one.

The unit test only asserted `"--print" in args`, which is why CI stayed
green; it now pins `--print` as the second-to-last element, the prompt
as the last, exactly one `--print`, and no `--output-format` after it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015HBBoDVQK7baNMQBhgRkh4
Move the schedule from 06:00 UTC to 12:00 UTC on Mondays (04:00 PST /
05:00 PDT), which falls inside the "before 6am on monday"
America/Los_Angeles window the repo's Renovate config (PR #54) uses, so
CLI updates and the image rebuild land in the same slot.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015HBBoDVQK7baNMQBhgRkh4
KE7 added a commit that referenced this pull request Sep 9, 2026
The weekly rebuild (cron, node-user verification, resolve-tag job) now
lives in PR #71. This PR is Renovate-only again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015HBBoDVQK7baNMQBhgRkh4
Karim Elmaaroufi and others added 3 commits September 9, 2026 17:27
agy's --print-timeout is a Go time.Duration flag defaulting to 5m0s. When
it expires a headless run stops, and per agy's changelog a mid-turn expiry
returns partial output with only a stderr warning, so a cut-off mutation
can look like a finished one. There is no disable value: 0 and negative
durations fail immediately with "timeout waiting for response", -1 without
a unit is an invalid duration, and no env var or settings key overrides
the flag.

Pass the largest duration Go can represent (2562047h47m16.854775807s,
math.MaxInt64 nanoseconds, ~292 years). agy 1.1.27 accepts it and a turn
completes normally; 2562048h is rejected, so this is the ceiling of the
flag's type. Every other backend runs its subprocess with no timeout at
all (test_no_timeout_in_subprocess), so this keeps agy on the same policy.

The flag goes among the leading flags, ahead of the closing
`--print <prompt>` pair, which stays the last argv element. The argv tests
now pin --print-timeout to exactly one occurrence, immediately followed by
the constant, and before --print.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015HBBoDVQK7baNMQBhgRkh4
A real `agy --output-format json` envelope (agy 1.1.27) is a single object:
conversation_id, status, response, error (on failure), duration_seconds,
num_turns, and a usage block of input_tokens / output_tokens /
thinking_tokens / cache_read_tokens / total_tokens. Two of those usage keys
had no alias in _normalise_usage_stats, so agy's thinking and cache-read
counts were always charged as 0. Map thinking_tokens to the reasoning
counter, cache_read_tokens to the cache-read counter, and conversation_id
to session_id.

agy reports no cost field, so cost_usd stays 0 for this backend; the
envelope also has no per-tool event list, so _TRANSCRIPT_TOOL_COUNTERS
still has no agy entry. Its comment and the matching test no longer claim
the shape is unobserved, and the "illustrative" agy fixtures in
test_budget.py are replaced with the captured envelope.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015HBBoDVQK7baNMQBhgRkh4
Resolves the one conflict in src/helix/mutator.py: main's LF-only
line-splitting fix (#73) touched `_count_gemini_stdout_tool_events`,
which this branch removes along with the gemini backend. The deletion
wins; the split_lf_lines change is already applied to the remaining
per-backend counters.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0126UoDvKj2BN5SHLH81aqnW
@KE7
KE7 merged commit d6e0286 into main Sep 11, 2026
7 checks passed
KE7 pushed a commit that referenced this pull request Sep 11, 2026
Brings in #71 (gemini replaced by agy) and #73 (usage salvage before
parse). Conflict resolutions:

- README.md: kept main's one-line auth-volume list (helix-auth-agy,
  claude, codex, cursor, opencode) followed by our "Per-candidate agent
  state" section; dropped the gemini volume, added an agy row to the
  backend table and an agy bullet under "What still crosses candidates".
- src/helix/evolution.py: declared main's `merge_usage` sink before our
  try/except CredentialRefreshError around merge(), passed
  `record_usage=merge_usage.append` inside the call, kept our except
  clause. The except path sets `merged = None` and falls through to
  main's `if merged is None:` block, so a credential failure's tokens are
  still charged with source="merge_failed". merger.py's
  CredentialRefreshError handler now forwards `exc.usage` to
  `record_usage` like the MutationError and RateLimitError handlers
  (test_credential_error_hands_spent_usage_to_the_sink).
- src/helix/mutator.py: imports keep both CredentialRefreshError and
  HelixError; the `gemini` env block is gone and the cursor credential
  hazard warning plus our `sandbox_enabled` local stay. invoke_claude_code
  keeps our nested `_attempt(*, retried)` and now, per attempt: runs the
  backend, salvages `spent_usage` before anything can fail, classifies
  credential / rate-limit / max-turns outcomes as before, attaches
  `spent_usage` to any HelixError that lacks one, and writes artifacts in
  `finally` with `fallback_usage=spent_usage`.
- Retry usage accounting: when the first attempt raises a transient
  CredentialRefreshError (which now carries that attempt's usage) the
  retry's result is summed with it via the new `_combine_usage` helper --
  on success the returned UsageStats is the sum (session_id from the
  retry), on any HelixError from the retry the raised error's `usage` is
  the sum. Covered by TestRetryUsageAccounting.
- tests/unit/test_mutator.py: main's docstring wording with our variable
  ("OPENCODE_DB must NOT be injected for agy/claude/codex/cursor").

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0126UoDvKj2BN5SHLH81aqnW
KE7 pushed a commit that referenced this pull request Sep 11, 2026
main replaced the gemini backend with agy (#71); this branch's per-backend
tables still named gemini and lacked agy.

- helix.backends.CREDENTIAL_WARM_SKIP_REASONS: drop gemini; add agy. Its
  registered status probe is a file test on the OAuth token that touches no
  credential path, `agy models` exits 0 even when logged out and has not
  been measured for a refresh, and there is no agy credential to measure
  one against, so it is left unwarmed rather than warmed on a guess.
- helix.agent_state.UNRELOCATED_AGENT_STATE: drop gemini; add agy with the
  residue observed on agy 1.1.27 -- conversations/,
  conversation_summaries.db, brain/, cache/, history.jsonl, log/,
  knowledge/, presence/, settings.json, all under ~/.gemini/antigravity-cli/
  next to the OAuth token. No knob relocates the state without the
  credential (ANTIGRAVITY_EXECUTABLE_DATA_DIR is unverified and unused), so
  agy is not relocated, like claude.
- tests: gemini -> agy in test_agent_state and test_credential_warm
  parametrizations; the docker_integration suite gains AGY_IMAGE and an
  agy case that pins the no-knob contract and skips with the reason.
- docs/agent-state-isolation.md and README: agy row and residue; the
  README's host-credential paragraph now stays with the auth-volume list it
  describes instead of trailing the agent-state section.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0126UoDvKj2BN5SHLH81aqnW
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant