Skip to content

Start the scheduler service without a console session - #1

Merged
CerebralCoding merged 2 commits into
CerebralCoding:mainfrom
STRML:headless-service
Oct 28, 2026
Merged

CerebralCoding merged 2 commits into
CerebralCoding:mainfrom
STRML:headless-service

Conversation

@STRML

@STRML STRML commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

latch service start fails on a Mac nobody has logged into at the console. ServiceInstallation.domain is hardcoded to gui/<uid>, and that domain only exists while someone is logged in at the console. Our Studio rebooted after a power cut, has no auto-login, and is driven entirely over ssh:

$ latch service start
Bootstrap failed: 125: Domain does not support specified action
latch: launchctl bootstrap failed (125)

latch service run works fine in the same state (sensors read, jobs admit and run), so only the launchd plumbing needs the GUI.

before:  ssh, no console login ─▶ bootstrap gui/501 ─✗─▶ 125, service stays down
after:   ssh, no console login ─▶ bootstrap user/501 ─▶ running
         later console login   ─▶ gui/501 copy starts ─▶ waits on service.lock ─▶ takes over when the user/501 copy stops

Changes

Where Change
ServiceInstallation.domain gui/<uid> when it exists (launchctl print gui/<uid> succeeds), else user/<uid>
Launch agent plist adds LimitLoadToSessionType = [Aqua, Background]. Without Background, bootstrap user/<uid> fails with 5: Input/output error (tested on macOS 27)
stop / start stop boots out every domain holding the label; start kickstarts whichever one has it loaded
SchedulerService.run when launchd started it (XPC_SERVICE_NAME equals the label) and the latch is held, it waits on service.lock instead of exiting 75. Otherwise a console login would load a second copy into gui/<uid> that exits 75 and KeepAlive respawns it every 10 s. A manual latch service run still fails fast with 75

Heads-up: this changes what install writes to the plist (the new LimitLoadToSessionType key). Versions, the service revision and schemas are unchanged. A plist from an existing install doesn't have the key, so on those installs start with no console session still fails as it does today until the plist is reinstalled.

Verified

  • On the Studio, over ssh with no console session: a test agent with LimitLoadToSessionType = Background loads into user/501, keeps running across ssh sessions, and launchd sets XPC_SERVICE_NAME to its label. Without the key, the bootstrap fails with error 5.
  • Debug build against a scratch latch: with one service run holding it, a plain second service run exits 75 (unchanged). A second one with XPC_SERVICE_NAME=<label> logs another scheduler holds this latch; waiting, then takes over service.json within 3 s of the holder exiting.
  • New tests: domain selection, the launchd check, and the plist key. swift format lint --strict is clean.
  • swift test: my new tests pass in every run. Each full run had 1 to 3 different MCPTests failures on latch_wait / "MCP message timed out", and unmodified main (78da077) also failed one (protocol tasks and completed output survive reconnecting). MCPTests alone passes 51/51.

Not tested: an actual console login while the user/<uid> copy runs. I couldn't log in on that box, so the takeover is verified only with the simulated launchd environment above.

ServiceInstallation hardcoded gui/<uid>, which only exists while the user is logged in at the console. Over ssh after a reboot, install, start and update failed with 'launchctl bootstrap failed (125)'.

- install/start/update load into user/<uid> when gui/<uid> is absent; the plist adds LimitLoadToSessionType [Aqua, Background] so that domain accepts it
- stop boots out every domain that holds the label; start kickstarts the loaded one
- a launchd-started scheduler (XPC_SERVICE_NAME == label) that finds the latch held waits for it instead of exiting 75 into a KeepAlive respawn loop, so a later console login hands over cleanly; a manual service run still fails fast
@STRML
STRML requested a review from CerebralCoding as a code owner October 28, 2026 00:35

@CerebralCoding CerebralCoding left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution and the detailed reproduction and testing. The background-domain fallback looks reasonable, and retaining the scheduler lock as the single-instance guard keeps the change focused.

I reviewed the patch and ran the full test suite with Swift 6.4: all 219 tests passed, along with strict formatting. I didn’t reproduce the MCP failures in this run.

Before merging, I’d like to address the existing-installation limitation you noted. ServiceInstallation.configuration adds the required LimitLoadToSessionType setting, but ServiceUpdate.apply only replaces the executable and updates its receipt. The official installer also takes that update path for existing installations, so upgrading would leave those users unable to start headlessly.

Could you extend the update path to refresh the installed launch-agent configuration while preserving the selected queue? Please include a regression test for an installed plist missing this setting, and include the plist in failure recovery so a failed restart restores the original configuration alongside the executable.

The actual console-login transition remains an outstanding integration check. I haven’t exercised that here, so the passing tests shouldn’t be taken as verification of that transition. Thanks for explicitly calling out that limitation.

Updates replaced only the executable, so an installation made before LimitLoadToSessionType was added could not start without a console session. apply now rewrites the installed plist for the queue it already serves, restarts a loaded service when the plist changed, and restores the original plist alongside the executable if the restart fails.
@STRML

STRML commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 13f289d. update and rollback now rewrite the installed launch agent for the queue it already serves, so installs that predate LimitLoadToSessionType pick it up. A loaded service restarts when the plist changed, even with an unchanged binary. If the restart fails, the original plist is restored along with the executable.

8 new tests: stale plist with a changed and an unchanged binary, stopped service, identical plist (file untouched), failed-restart recovery for both binary cases, unwritable plist directory, and queue preservation. Full suite: 226 of 227 passed. The one failure, ordinary MCP jobs exceed two concurrent starts..., passes alone in 0.25s and failed once under full-suite load.

Not exercised: the real launchctl path in ServiceInstallation.update, and the console-login transition.

@CerebralCoding CerebralCoding left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @STRML—this addresses the requested update-path and failure-recovery changes. I reviewed 13f289d and ran the full suite: all 227 tests passed, and strict formatting passed. The real launchctl update path and console-login transition remain unverified. Approved.

@CerebralCoding
CerebralCoding merged commit 7defc81 into CerebralCoding:main Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants