Repository navigation
Run the app against a live host in one command, and fix the Grok working line it found - #181
Open
woosal1337 wants to merge 3 commits into
Open
woosal1337 wants to merge 3 commits into
woosal1337 wants to merge 3 commits into
Conversation
The Demo plays every agent the UI suite needs, and the live test of
2026-10-16 still found bugs it could not show: a keychain write an unsigned
build cannot make, a folder that is not on the host, a startup question that
failed New chat. Each needed a host set up by hand. `npm run e2e:live`
(scripts/e2e-live.sh) does it in one command (HC-128):
- A key and a host key made for the run in a temporary folder, and an sshd
of its own on 127.0.0.1 that accepts that key only. `~/.ssh` and Remote
Login are not touched.
- The host added in the app with add-server.yaml, which now takes a port and
a name. Its defaults are `${PORT || '22'}`: a plain value in a flow's env
wins over `-e`, so the first run tested port 22, this Mac's own sshd.
- For each agent whose herdr integration is current (or LIVE_AGENTS), a probe
chat from New chat in a scratch folder (live/probe.yaml): the last folder
New chat fills in cleared through Select All, the trust question answered
by its words (Claude lists "No, exit" first), one message, and the agent's
reply, which the question does not contain. LIVE_PICTURE=1 also sends a
picture (live/picture.yaml); it is off by default because `simctl addmedia`
hangs on some simulators.
- Pass or fail, the workspaces in the scratch folders closed, the sessions
Claude and Grok filed for them deleted, the host removed from the app
(live/remove-host.yaml), the sshd stopped and the folder deleted. Prompts
go only to the chats the run started.
Against this Mac (herdr 0.9.5, Claude and Grok): every step passes, and
nothing of the run is left in herdr, ~/.claude/projects or ~/.grok/sessions.
Found by the first live run (scripts/e2e-live.sh): while Grok worked, the thread showed "⠋ Thinking… 0.0s 2.6s ⇣1.46k [stop]" under the reply as if Grok had written it, above the app's own "Writing…". Two working lines, one of them Grok's status (HC-134). The live preview anchors on the agent's spinner and shows the prose above it. Grok's spinner is a braille glyph, which the anchor did not know, so the status line itself was the last "prose" on the screen. Read with `herdr pane read` on Grok 4.7, its screen while it works is blank rows with a scrollbar at their right edge (it keeps the reply scrolled out of view), the spinner and its timer, the composer box and a key hint. The spinner check now takes a braille glyph too, and a row that is only a scrollbar is chrome. livePreview.test.ts reads the three captured screens (waiting, writing with the last list item in view, writing scrolled away) and the Demo's copy: none gives a preview, and prose put above the spinner does. All five fail without the fix. In the Demo, "think it over" in the Grok chat works under that screen, and regression/grok checks that only the app's line shows.
Running `npm run e2e:live` with Antigravity found three problems in the run itself, none in the app: - The probe tapped the agent's row with the keyboard up. Antigravity's row, low in the picker, sat behind it, the tap landed on the floating Start button, and Claude Code started in the agy folder. The agent is now chosen first, while the keyboard is down, and its Start label checked again after the fields are filled. - remove-host passed without removing anything. It looked for the host's name, but the row reads to VoiceOver as one element, "Use <name>", so the optional step never matched; seven hosts had piled up on the simulator. The script now reads the host's id from the app's database, the flow removes that row (the confirmation is the system alert, its Remove beside Cancel), and the script checks the database that it is gone. - agy files each probe under its conversation id (brain/<id> and conversations/<id>.db) and lists the prompt and the trusted folder by path. The cleanup removes the conversation by the id herdr reported, and the scratch paths from history.jsonl and trustedWorkspaces, rewriting each only when something of the run is in it. Against this Mac with Claude, Grok and Antigravity: every probe passes, and afterwards herdr's workspaces, ~/.claude/projects, ~/.grok/sessions, agy's brain, conversations, history and settings, and the app's hosts are as they were. Two things stay, said in the script: Claude's trust of each scratch folder in ~/.claude.json, which every running Claude rewrites, and agy's summary database, which agy holds open.
9 of 10 tasks
cobanov
requested changes
Oct 20, 2026
cobanov
left a comment
Owner
There was a problem hiding this comment.
The script is careful (its own sshd on 127.0.0.1, a key for the run only, every rm -rf bounded), and the Grok working-line fix is right. Two fixes in scripts/e2e-live.sh before it goes in:
- Claude's probe transcripts are never deleted. Claude turns every character that isn't a letter or digit into
-; the sed turns only/and., andTMPDIRon macOS has_in it (/var/folders/z1/ss_9fy…).sed 's|[^A-Za-z0-9]|-|g'. - A second Ctrl-C during cleanup stops it halfway.
trap - EXIT INT TERMrestores SIGINT before a 20–60 s Maestro flow, so a second Ctrl-C there leaves the host saved in the app, the key in$WORK, and possibly sshd listening. Ignore INT and TERM insidecleanupinstead.
Smaller: assert the folder field holds ${CWD} before a probe starts, so one can never start in a real folder that cleanup then leaves alone; .*\b42\b.* also matches a row time like 09:42; a typo in LIVE_AGENTS isn't caught, because the pipeline returns cut's status; and the header's "left behind" list should name Codex, Pi and OMP sessions too.
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The UI suite runs on the Demo, which plays every agent but is not one. This branch adds
npm run e2e:live, which runs the app against the Mac it is on with real agents and cleans up after itself, and fixes the first bug it found, one commit each.This branch sits on
remaining-gaps(#177); merge it after #177.Changes
npm run e2e:live(scripts/e2e-live.sh, HC-128).~/.sshand Remote Login are not touched.add-server, which now takes a port and a name.LIVE_AGENTS, it starts a probe chat from New chat in a scratch folder (live/probe.yaml). The folder trust question is answered by its words, then one message is sent and the run waits for the agent's own reply.LIVE_PICTURE=1also sends a picture (live/picture.yaml).live/remove-host.yaml), stops the sshd and deletes the key. Prompts go only to chats the run started.env:overrides-e, so the first run tested port 22, this Mac's own sshd, and was rejected.eraseTextdeletes back from where the tap put the cursor, so New chat's prefilled folder is cleared through Select All. Claude's trust menu lists "No, exit" first.Grok's working screen is one working line (HC-134, found by the first live run). While Grok worked, the thread showed "⠋ Thinking… 0.0s 2.6s ⇣1.46k [stop]" under the reply as if Grok had written it, above the app's own "Writing…".
herdr pane readon Grok 4.7, its screen while it works is blank rows with a scrollbar at their edge (the reply is scrolled out of view), the spinner and timer, and the composer box. The spinner check now takes a braille glyph, and a row that is only a scrollbar is chrome.regression/grokchecks that only the app's line shows. Five new tests fail without the fix.Antigravity in the live run, and a host removal that really removes (
74e08a2). Running it with Antigravity found three problems in the run, none in the app:remove-hostpassed without removing anything: it looked for the host's name, but the row reads as one element, "Use ". The script now reads the host's id from the app's database, removes that row and checks the database.~/.claude.json, which every running Claude rewrites, and agy's summary database, which agy holds open.Test plan
npx tsc --noEmit: no errorsnpx expo lint: no errors, no warningsnpx jest: 2,069 passed, 14 skippedpython3 -m unittest discover -s scripts -p 'test_*.py': passesnpm run e2e:liveon the iPhone 17 Pro simulator (iOS 27) against this Mac (herdr 0.9.5), with Claude, Grok and Antigravity (its integration installed for the run, then removed and its config checked against a copy): the host added, each probe answered "42", the host removed and confirmed gone from the app's database. Afterwards herdr's workspaces,~/.claude/projects,~/.grok/sessionsand agy's conversations, history and settings were as before, and no sshd was running.LIVE_AGENTS=grok LIVE_PICTURE=1on the iPhone 18 Pro simulator: the picture was uploaded, shown in the bubble, and Grok read its title ("Spaces").npm run e2ein Dark and Light on the iPhone 17 Pro simulator (iOS 27): 52 of 52, on a Release build of this branch together with the agent names branch (built with Xcode 27.1 from a freshnpm ci). Screenshots opened in both modes.