Skip to content

End omarchy update when its terminal closes at a prompt - #16045

Merged
ryanrhughes merged 3 commits into
omacom:quattrofrom
shbernal:fix/update-hangup-orphan
Nov 14, 2026
Merged

ryanrhughes merged 3 commits into
omacom:quattrofrom
shbernal:fix/update-hangup-orphan

Conversation

@shbernal

@shbernal shbernal commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #16044.

Closing the update terminal while omarchy update waits at its confirmation prompt can leave the updater and gum attached to a deleted PTY, still holding the update flock. A live occurrence survived almost seven days. The script ancestor was already gone; no package transaction had begun.

Fix

Keep a plain, untrapped Bash as the session leader inside script by following the updater command with ; exit $?. Without that trailing command, Bash can exec the updater: its signal traps then wait for the foreground prompt, which may never receive the hangup. With the plain shell leader, terminal loss ends the leader and the kernel hangs up the foreground process group.

Pin script's interpreter to Bash because the command uses Bash quoting. Restore the caller's exported SHELL inside that interpreter, preserving an absent export as absent, so hooks and package builds retain their environment. Preserve the updater's exit status.

Record the acquiring shell's PID in the lock file for diagnostics and report it on contention. Open contenders without truncating the recorded PID. The PID is not an unlock heuristic: PIDs can be reused and children can inherit the descriptor. flock remains the sole authority; no lock-file deletion, watchdog, polling service or timeout-based unlock is introduced.

Reproduction and tests

test/shell.d/update-hangup-test.sh runs real packaged gum, script, PTYs and the real lock, with all update/package/privileged steps stubbed. It covers:

  • Closing the sole outer PTY master normally.
  • Killing script outright before closing the terminal, reproducing the observed missing-ancestor state without its two-second fallback cleanup.
  • Both cases terminate the prompt and release the flock.
  • Preservation of exported/unexported SHELL and nonzero update status.

On unchanged quattro d81c2e2, ordinary terminal closure passes, but abrupt loss of script leaves real gum alive holding the lock. That regression passes with this patch. This does not establish why script originally vanished on affected machines.

The test uses a private runtime directory (no login-session requirement), bounded waits, zombie-aware exit checks, and PID-scoped subreaper cleanup. No reader child retains the PTY master and no global pkill pattern is used.

The existing lock test additionally verifies that rejected contenders preserve/report the acquiring PID and a stale diagnostic PID cannot block a free flock.

Validation

Passing locally:

  • update-hangup-test.sh (fails on unchanged base for the intended regression)
  • update-lock-test.sh
  • update-user-path-test.sh
  • update-sequence-test.sh
  • update-hook-security-test.sh
  • update-stay-awake-security-test.sh
  • ./test/cli
  • bash -n on changed files and git diff --check

Full VM CI remains required. No real package update or installed-file modification was performed.

Updated through maintainer edits against current quattro, preserving the original contribution without a force push. Related: #8081 addresses daemon-inherited flock descriptors after a completed updater, while #17020 addresses transcript truncation on contention. #14601 and #15683 touch the same logger boundary and may require coordination.

@greptile-apps

greptile-apps Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[High impact] The PR appears safe to merge, with a non-blocking timing issue in the hangup test.

Summary

The update keeps a plain Bash process above it so losing the terminal also ends a waiting prompt.

  • Closing the update terminal ends its prompt and releases the lock.
  • The update flow now restarts the shell for pending migrations and reports mise failures.
  • A blocked update now reports the process that recorded the lock.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Terminal closes] --> B[script loses its terminal]
  B --> C[Plain Bash session leader exits]
  C --> D[Foreground prompt receives hangup]
  D --> E[Update runs cleanup]
  E --> F[Update exits and releases lock]
Loading

Reviews (2) · Last reviewed commit: "Record the update lock acquiring PID for..." · Reviewed by Greptile

Comment thread test/shell.d/update-hangup-test.sh Resolved
@omarchybot omarchybot added the bug Something isn't working label Oct 21, 2026
@omarchybot

Copy link
Copy Markdown
Collaborator

Thanks for the careful write-up. I reviewed this, ran it on a worker, and had Codex Medium review it as a second opinion. The mechanism you describe is right, but on a stock install I could not reproduce the update staying stranded.

What I ran, on a disposable worker built from the Omarchy ISO (foot, gum 2.0.2, util-linux 2.42.4). I used the real omarchy-update, script, lock and gum confirm. Only the 10 GiB free-space check was stubbed, because the worker has less room than that.

  • On quattro, the process tree at the prompt is what you describe: bash -p …/omarchy-update leads the inner PTY's session, with omarchy-update-confirm and gum in its foreground group.
  • On quattro, when the terminal hangs up, gum keeps redrawing on EOF for about two seconds. Then script gives up on its child and exits, and gum and the lock go with it. Closing a real foot window through Hyprland cleared both within a second.
  • On this head, gum and the lock are gone within 0.3 seconds.
  • ./test/cli passes on the head, and so do the update shell tests (update-hangup, update-lock, update-sequence, update-user-path, update-hook-security, update-stay-awake-security, update-dev, update-disk-space, security-entrypoint-symlink).
  • Your new test fails on quattro and passes here. But its stand-in prompt loops on EOF forever, and real gum does not: it exits once script does. So the test shows the faster shutdown, not the stranding in Closing the update terminal at the confirm prompt leaves omarchy update holding its lock forever #16044.

Codex Medium agreed with the mechanism; its independence from my review is not guaranteed. It pointed out why base recovers here: util-linux script sends TERM to its child on a termination signal, waits two seconds, then sends KILL. A week-long stranding probably needs script itself killed outright before that fallback runs, or a prompt that survives the hangup. I could not produce either on a stock worker.

One question decides what happens next: on the machine where it sat for a week, which terminal emulator was the update window in, and what do gum --version and script --version print? With that I can try to reproduce it on a worker set up the same way.

Findings in the change, none pushed while the reproduction is open:

  • bin/omarchy-update:94 exports SHELL=/usr/bin/bash to the whole update, not just to script. Post-update hooks, mise and AUR builds now see Bash as the user's shell. Pin the update logger shell #14601 pins the same shell for script and restores the caller's SHELL inside the command, which avoids this.
  • test/shell.d/update-hangup-test.sh:12 hardcodes /run/user/$UID. A headless runner without a login session fails at the lock rather than at the assertion. A directory under the fixture's temp dir would avoid that.
  • In the test, the forked reader inherits the PTY master, so os.close(fd) does not close the terminal; only the explicit killpg hangs it up. waitpid after that has no deadline, and kill -0 counts a zombie as a live prompt.

Overlaps: #14601 and #15683 edit the same script line, so whichever lands second needs a rebase. After #15683 lands, the test's sed of /tmp/omarchy-update.log would stop isolating the transcript.

Waiting on your answer above. After that, the maintainer decides whether to take the faster shutdown on its own merits.

@Pillumz

Pillumz commented Nov 10, 2026

Copy link
Copy Markdown

Another reproduction with Ghostty 1.4.0+omarchy.1, gum 2.0.2, and script from util-linux 2.42.4. The update was launched through omarchy-launch-floating-terminal-with-presentation and remained at the initial confirmation for about 19 hours after its terminal disappeared. omarchy-update, omarchy-update-confirm, and gum were still alive and held the flock. Their stdin/stdout/stderr pointed to /dev/pts/10 (deleted), while the script process was already gone. No package steps had started. Sending TERM only to gum triggered the normal cancellation path and released the lock.

@tobi

tobi commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Another live occurrence on 2026-11-14: omarchy-update (PID 1725243) → omarchy-update-confirm (1725258) → gum confirm (1725299) had survived for 6 days 21 hours. The updater was reparented to PID 1515; all three were attached to /dev/pts/26 (deleted), and the updater/confirmation script still had fd 10 open on /run/user/1000/omarchy-update.lock. No script(1) ancestor or package transaction remained. Terminal emulator for the original launch is not established.

Terminating only the abandoned gum with SIGTERM let the confirmation/updater exit. Independently verified flock could then be acquired; the lock file was not removed.

Current installed packages: omarchy-dev 4.0.0.r7600.g6194f46-1, util-linux 2.42.4-1, gum 2.0.2-1, ghostty 1.4.0-1, foot 1.28.0-2. Important qualification: gum --version in the agent environment reports v0.17.0 (6045525), so PATH can differ from the packaged gum. The abandoned process executable/version was not captured before termination; do not treat installed package versions as proof of its executable version.

This supports the stranded-process failure mode, but is not a controlled reproduction or validation of the proposed patch.

@tobi tobi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and updated the contributor branch using maintainer edits (no force push, no merge of the PR).

Resolved against current quattro d81c2e2 while retaining the original contribution. The runtime fix remains a plain Bash PTY session leader plus trailing exit; there is no watchdog, stale-lock deletion, PID liveness heuristic or timeout-based unlock.

Addressed the earlier review findings:

  • Preserve the caller’s exported SHELL, including its absence, inside the trusted Bash wrapper. Merely checking whether the Bash variable exists is insufficient: Bash synthesizes an unexported SHELL on startup.
  • Private fixture runtime directory; no login-session /run/user requirement.
  • Real packaged gum, script, PTYs and flock instead of an EOF-loop prompt substitute.
  • Exactly one PTY-master owner; closing it really closes the terminal. All waits are bounded, zombies count as exited, and orphaned fixture children are adopted/reaped with Linux subreaper semantics. No global pkill-by-pattern cleanup.
  • Added a deterministic abrupt-script-loss case. On unchanged quattro, ordinary terminal closure passes but killing script outright leaves real gum holding the lock. With this patch, both cases pass. This reproduces the missing-script-ancestor state observed on affected machines without claiming to establish why their script process originally vanished.

Also added the operator-requested diagnostic acquiring PID to the lock file. Rejected contenders preserve it and report it. The PID is explicitly informational: only flock decides exclusion, and an old PID never blocks a free flock.

Validation: update-hangup, update-lock, update-user-path, update-sequence, update-hook-security, update-stay-awake-security, and ./test/cli all pass. bash -n and git diff --check pass. New hangup test fails on unchanged quattro specifically because real gum survives abrupt script loss. No live package update or installed-file modification was performed. Full VM CI remains required; this is not an independent approval of changes I authored.

@ryanrhughes
ryanrhughes changed the base branch from quattro to 1password-scale-factor November 14, 2026 18:35
@ryanrhughes
ryanrhughes changed the base branch from 1password-scale-factor to quattro November 14, 2026 18:35
@ryanrhughes
ryanrhughes merged commit 9ac0883 into omacom:quattro Nov 14, 2026
3 checks passed
ryanrhughes added a commit that referenced this pull request Nov 14, 2026
)

* End omarchy update when its terminal closes at a prompt

* Record the update lock acquiring PID for diagnostics

---------


(cherry picked from commit 9ac0883)

Co-authored-by: Santiago <shbernal.01@gmail.com>
Co-authored-by: Tobi Lutke <tobi@shopify.com>
omarchybot added a commit to kdriedger/omarchy that referenced this pull request Nov 16, 2026
quattro's omacom#16045 records the acquiring PID in the lock file. Under this branch the wrapper is what holds the flock, so it writes its own PID there, and the stale-PID test compares against the locked command's parent rather than the command itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Closing the update terminal at the confirm prompt leaves omarchy update holding its lock forever

5 participants