Make the integrations mount survive unattended for months (auto-renew creds + supervision + watchdog) #330

Problem

The .integrations mount cannot run unattended for long. Its lifetime is bounded by short-lived delegated credentials (1h TTL) with no auto-renewal, no process supervision, and fragile scope/workspace resolution. When the cred expires the mount wedges (writeback-needs-attention, reconcile stops), which silently breaks factory + integration writebacks until a human re-auths.

This is the root cause behind a recurring pear destabilization: an unhealable mount gets force-restarted forever and the detached relayfile-mount daemons orphan and spin CPU. pear shipped a downstream mitigation in AgentWorkforce/pear#385 (suppress the resulting event-spam, reap orphaned mount daemons, surface a re-auth banner when restarts are exhausted) — but the mount still can't self-heal. That's this issue.

Evidence

• agent-relay cloud login refreshes the agent-relay session but not the relayfile delegated credential — login rotates the identity (new delegated-cred hash), so relayfile start then fails delegated relayfile credentials are required: …/<hash>/…json not found.
• Recovery today is manual: relayfile workspace join rw_7ccfea89 --write re-mints the delegated cred (read/write), then relayfile start.
• A prior mint failed 403 insufficient_scope — an earlier token was minted with only fs:write:/linear/** (no /slack), so Slack writebacks silently failed on scope.
• Symptom in prod (2026-06-22): mount stuck writeback-needs-attention, reconcile frozen; downstream agent_not_found event-spam + orphaned relayfile-mount procs (6 observed outliving their parent, load avg 5.3).

Root cause

Mount lifetime is bounded by short-lived delegated creds, with no auto-renewal, no supervision, and fragile scope/workspace resolution.

Work

1. Eliminate short-lived creds (load-bearing). Long-lived service credential for the mount, OR a silent refresh daemon that re-mints delegated creds before expiry from a durable refresh token. Dovetails with relayfile: long-lived operator session tokens (1-hour TTL too short for unattended mount + ad-hoc CLI writebacks) #284 (1h TTL).
2. Supervise the process. launchd (macOS) / systemd with KeepAlive wrapping a re-auth-on-failure relaunch. --background alone is unsupervised (and detaches from the host app — see pear#385 orphan reaping).
3. Health watchdog with auto-recovery. Periodically check reconcile freshness; on staleness, re-mint + restart; escalate to a human only when re-auth truly needs one.
4. Distinguish failure modes + back off. Expired-cred failures must not hot-loop (prior auto-mount respawn storm: relaunch every ~12-20s, CPU spin, orphans). Detect needs-reauth vs transient; exponential backoff; pause-for-human on the former. (Relates to relayfile-mount --state-dir state not reset/cleaned across remounts → degrades on repeated mount on the same box #219 — state not reset across repeated remounts degrades the box.)
5. Pin full scopes at mint. /linear/**, /slack/**, /github/**, ops:read — a narrow token must not silently break a subset of writebacks.
6. Observability. Emit reconcile_age + cred_expires_in; alert before expiry, not after a writeback fails.

Acceptance

Mount survives a simulated cred-expiry (auto-renews, no human), survives a kill (supervisor restarts), and a staleness watchdog recovers it; all writeback scopes (Linear + Slack) verified post-recovery.

Related

• relayfile: long-lived operator session tokens (1-hour TTL too short for unattended mount + ad-hoc CLI writebacks) #284 — relayfile: long-lived operator session tokens (1h TTL too short) — the load-bearing cred TTL piece
• relayfile start should repair missing delegated credential cache from valid cloud credentials #329 — relayfile start should repair missing delegated credential cache from valid cloud credentials
• relayfile-mount --state-dir state not reset/cleaned across remounts → degrades on repeated mount on the same box #219 — relayfile-mount --state-dir state not reset/cleaned across remounts
• fix(integrations): stop mount respawn-storm orphans + recipient inject-spam pear#385 — downstream pear-side hardening (event-spam suppression, orphan reaping, re-auth escalation)
• Repos in scope: relayfile (CLI + daemon), relayauth (mint/scope), relayfile-cloud (delegated-token route)