Skip to content

Run the app against a live host in one command, and fix the Grok working line it found - #181

Open
woosal1337 wants to merge 3 commits into
remaining-gapsfrom
hc-128-live-e2e
Open

woosal1337 wants to merge 3 commits into
remaining-gapsfrom
hc-128-live-e2e

Conversation

@woosal1337

@woosal1337 woosal1337 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The UI suite runs on the Demo, which plays every agent but is not one. This branch adds npm run e2e:live, which runs the app against the Mac it is on with real agents and cleans up after itself, and fixes the first bug it found, one commit each.

This branch sits on remaining-gaps (#177); merge it after #177.

Changes

  • npm run e2e:live (scripts/e2e-live.sh, HC-128).

    • It starts its own sshd on 127.0.0.1 with a key made for the run. ~/.ssh and Remote Login are not touched.
    • It adds the host in the app with add-server, which now takes a port and a name.
    • For each agent whose herdr integration is current, or those in LIVE_AGENTS, it starts a probe chat from New chat in a scratch folder (live/probe.yaml). The folder trust question is answered by its words, then one message is sent and the run waits for the agent's own reply. LIVE_PICTURE=1 also sends a picture (live/picture.yaml).
    • Pass or fail, it closes the scratch workspaces, deletes the sessions Claude and Grok filed for them, removes the host (live/remove-host.yaml), stops the sshd and deletes the key. Prompts go only to chats the run started.
    • Three traps found while making it are noted in the flows. A value in a flow's env: overrides -e, so the first run tested port 22, this Mac's own sshd, and was rejected. eraseText deletes back from where the tap put the cursor, so New chat's prefilled folder is cleared through Select All. Claude's trust menu lists "No, exit" first.
  • Grok's working screen is one working line (HC-134, found by the first live run). While Grok worked, the thread showed "⠋ Thinking… 0.0s 2.6s ⇣1.46k [stop]" under the reply as if Grok had written it, above the app's own "Writing…".

    • The preview anchors on the agent's spinner, and Grok's is a braille glyph the anchor did not know.
    • Read with herdr pane read on Grok 4.7, its screen while it works is blank rows with a scrollbar at their edge (the reply is scrolled out of view), the spinner and timer, and the composer box. The spinner check now takes a braille glyph, and a row that is only a scrollbar is chrome.
    • Three captured screens are fixtures. The Demo's Grok works under the same screen for "think it over", and regression/grok checks that only the app's line shows. Five new tests fail without the fix.
  • Antigravity in the live run, and a host removal that really removes (74e08a2). Running it with Antigravity found three problems in the run, none in the app:

    • With the keyboard up, Antigravity's row sat behind it, and the tap started Claude Code from the floating Start button. The probe now picks the agent before typing.
    • remove-host passed without removing anything: it looked for the host's name, but the row reads as one element, "Use ". The script now reads the host's id from the app's database, removes that row and checks the database.
    • agy keeps each probe under its conversation id and lists the prompt and the trusted folder by path; the cleanup removes all of them.
    • Two traces stay, and the script says so: Claude's trust of each scratch folder in ~/.claude.json, which every running Claude rewrites, and agy's summary database, which agy holds open.

Test plan

  • npx tsc --noEmit: no errors
  • npx expo lint: no errors, no warnings
  • npx jest: 2,069 passed, 14 skipped
  • python3 -m unittest discover -s scripts -p 'test_*.py': passes
  • npm run e2e:live on the iPhone 17 Pro simulator (iOS 27) against this Mac (herdr 0.9.5), with Claude, Grok and Antigravity (its integration installed for the run, then removed and its config checked against a copy): the host added, each probe answered "42", the host removed and confirmed gone from the app's database. Afterwards herdr's workspaces, ~/.claude/projects, ~/.grok/sessions and agy's conversations, history and settings were as before, and no sshd was running.
  • LIVE_AGENTS=grok LIVE_PICTURE=1 on the iPhone 18 Pro simulator: the picture was uploaded, shown in the bubble, and Grok read its title ("Spaces").
  • npm run e2e in Dark and Light on the iPhone 17 Pro simulator (iOS 27): 52 of 52, on a Release build of this branch together with the agent names branch (built with Xcode 27.1 from a fresh npm ci). Screenshots opened in both modes.

The Demo plays every agent the UI suite needs, and the live test of
2026-10-16 still found bugs it could not show: a keychain write an unsigned
build cannot make, a folder that is not on the host, a startup question that
failed New chat. Each needed a host set up by hand. `npm run e2e:live`
(scripts/e2e-live.sh) does it in one command (HC-128):

- A key and a host key made for the run in a temporary folder, and an sshd
  of its own on 127.0.0.1 that accepts that key only. `~/.ssh` and Remote
  Login are not touched.
- The host added in the app with add-server.yaml, which now takes a port and
  a name. Its defaults are `${PORT || '22'}`: a plain value in a flow's env
  wins over `-e`, so the first run tested port 22, this Mac's own sshd.
- For each agent whose herdr integration is current (or LIVE_AGENTS), a probe
  chat from New chat in a scratch folder (live/probe.yaml): the last folder
  New chat fills in cleared through Select All, the trust question answered
  by its words (Claude lists "No, exit" first), one message, and the agent's
  reply, which the question does not contain. LIVE_PICTURE=1 also sends a
  picture (live/picture.yaml); it is off by default because `simctl addmedia`
  hangs on some simulators.
- Pass or fail, the workspaces in the scratch folders closed, the sessions
  Claude and Grok filed for them deleted, the host removed from the app
  (live/remove-host.yaml), the sshd stopped and the folder deleted. Prompts
  go only to the chats the run started.

Against this Mac (herdr 0.9.5, Claude and Grok): every step passes, and
nothing of the run is left in herdr, ~/.claude/projects or ~/.grok/sessions.
Found by the first live run (scripts/e2e-live.sh): while Grok worked, the
thread showed "⠋ Thinking… 0.0s  2.6s ⇣1.46k [stop]" under the reply as if
Grok had written it, above the app's own "Writing…". Two working lines, one
of them Grok's status (HC-134).

The live preview anchors on the agent's spinner and shows the prose above it.
Grok's spinner is a braille glyph, which the anchor did not know, so the
status line itself was the last "prose" on the screen. Read with `herdr pane
read` on Grok 4.7, its screen while it works is blank rows with a scrollbar
at their right edge (it keeps the reply scrolled out of view), the spinner
and its timer, the composer box and a key hint. The spinner check now takes a
braille glyph too, and a row that is only a scrollbar is chrome.

livePreview.test.ts reads the three captured screens (waiting, writing with
the last list item in view, writing scrolled away) and the Demo's copy: none
gives a preview, and prose put above the spinner does. All five fail without
the fix. In the Demo, "think it over" in the Grok chat works under that
screen, and regression/grok checks that only the app's line shows.
Running `npm run e2e:live` with Antigravity found three problems in the run
itself, none in the app:

- The probe tapped the agent's row with the keyboard up. Antigravity's row,
  low in the picker, sat behind it, the tap landed on the floating Start
  button, and Claude Code started in the agy folder. The agent is now chosen
  first, while the keyboard is down, and its Start label checked again after
  the fields are filled.
- remove-host passed without removing anything. It looked for the host's
  name, but the row reads to VoiceOver as one element, "Use <name>", so the
  optional step never matched; seven hosts had piled up on the simulator.
  The script now reads the host's id from the app's database, the flow
  removes that row (the confirmation is the system alert, its Remove beside
  Cancel), and the script checks the database that it is gone.
- agy files each probe under its conversation id (brain/<id> and
  conversations/<id>.db) and lists the prompt and the trusted folder by
  path. The cleanup removes the conversation by the id herdr reported, and
  the scratch paths from history.jsonl and trustedWorkspaces, rewriting each
  only when something of the run is in it.

Against this Mac with Claude, Grok and Antigravity: every probe passes, and
afterwards herdr's workspaces, ~/.claude/projects, ~/.grok/sessions, agy's
brain, conversations, history and settings, and the app's hosts are as they
were. Two things stay, said in the script: Claude's trust of each scratch
folder in ~/.claude.json, which every running Claude rewrites, and agy's
summary database, which agy holds open.

@cobanov cobanov left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script is careful (its own sshd on 127.0.0.1, a key for the run only, every rm -rf bounded), and the Grok working-line fix is right. Two fixes in scripts/e2e-live.sh before it goes in:

  • Claude's probe transcripts are never deleted. Claude turns every character that isn't a letter or digit into -; the sed turns only / and ., and TMPDIR on macOS has _ in it (/var/folders/z1/ss_9fy…). sed 's|[^A-Za-z0-9]|-|g'.
  • A second Ctrl-C during cleanup stops it halfway. trap - EXIT INT TERM restores SIGINT before a 20–60 s Maestro flow, so a second Ctrl-C there leaves the host saved in the app, the key in $WORK, and possibly sshd listening. Ignore INT and TERM inside cleanup instead.

Smaller: assert the folder field holds ${CWD} before a probe starts, so one can never start in a real folder that cleanup then leaves alone; .*\b42\b.* also matches a row time like 09:42; a typo in LIVE_AGENTS isn't caught, because the pipeline returns cut's status; and the header's "left behind" list should name Codex, Pi and OMP sessions too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants