type: feature-ticket
id: daemon-turn-reliability
status: ready
zone: 1
pr:
Feature: daemon turn reliability — visible actions, interruptible turns, zero-loss restarts
Intent
Three reliability requirements for the Linear agent daemon, driven by live
incidents. (1) All tool-call progress is invisible in Linear: the daemon
posts action activities with a body field, but Linear requires action
and parameter — every one has always been rejected
(session_progress_failed on each tool call). (2) A user cannot get a
message to a working agent: prompts queue behind the running turn, which
can be hours for an implementer run. (3) Daemon restarts (= every deploy)
hard-kill in-flight turns with no recovery — a live run (TM-390) lost its
in-progress verification this way. The unifying principle: interruption
must be cheap and recoverable (conversation context and worktree files
already survive via --resume + SQLite; only in-flight process state is
ephemeral), and webhook ingress must never close.
Desired end state
- Each tool call an agent makes appears as an action activity in the
Linear session (name + detail), interleaved with thoughts.
- A user message to a session whose turn is running interrupts that turn;
the agent picks up the message and continues with full context. From
the user's side: agents are interruptible, like a colleague.
systemctl restart linear-agent-daemon mid-turn: the daemon exits
after a short grace, and on startup the interrupted turn auto-resumes
("continue where you left off") — no user action, no lost conversation,
only ephemeral runtime state (e.g. a booted dev stack) to redo.- Webhooks are accepted and durably logged at every moment the process is
alive, including during shutdown.
Key architecture directions
- D1 — activity mapping: action activities send Linear's required
shape: action = tool name, parameter = the detail describeTool
already computes, with a non-empty fallback ("running"). No body
field on actions. Clean replacement — the old shape never worked.
- D2 — ingress never closes: during shutdown the HTTP server keeps
accepting and persisting webhooks until the workers have stopped;
server close is the last step before exit. Events received during
shutdown are processed after restart via the existing queue.
- D3 — prompt interrupts the running turn: a prompted event for an
issue whose turn is running aborts that turn's claude process
(graceful signal, short grace), then runs the prompted turn with
--resume of the stored session id — the existing prompted-resume
path. The interrupted turn posts no error activity in this case; the
new turn is the continuation.
- D4 — restarts auto-resume: on shutdown, active turns get a short
grace (SHUTDOWN_GRACE_MS, default 60s) then abort and are marked
interrupted; the systemd unit sets TimeoutStopSec above the grace.
On startup, each interrupted turn with a stored claude session id is
re-queued as a synthetic continuation turn (resume + "continue"-style
prompt, deduped via turns.source_key so a crash loop can't spawn
duplicates). The long-drain alternative (15-min window) was rejected:
it only saves nearly-finished turns and creates a webhook blackout.
- D5 — honest loss boundary: interruption loses only in-flight
process state (running dev stacks, partially-run test suites); the
resumed turn is expected to redo that. No attempt to checkpoint or
restore runtime state.
Starting points (non-exhaustive)
daemon/src/linear.ts:12 — ProgressContent (wrong action shape);
postActivitydaemon/src/sessions.ts:25 — describeTool; :134 action content;
:324 immediate-abort stop(); per-turn AbortController mapdaemon/src/eventlog.ts — turn queue, source_key dedupe,
interruptStaleRunning (existing restart-interrupt path to build on)daemon/src/index.ts:28 — shutdown() ordering (server currently
closes before workers)daemon/ops/linear-agent-daemon.service — no TimeoutStopSec today
Verification criteria
- AC1 — WHEN a claude tool-use event is emitted during a turn, the
daemon shall post an ephemeral action activity whose content contains
non-empty action and parameter fields and no body field.
- AC2 — IF a tool-use event has empty or missing input, THEN the
posted activity's parameter shall be a non-empty string.
- AC3 — WHILE shutdown is in progress, the system shall accept and
durably persist incoming webhook deliveries until worker stop
completes.
- AC4 — WHEN a prompted event arrives for an issue whose turn is
running, the system shall abort the running turn and execute the
prompted turn with the stored claude session id (--resume).
- AC5 — IF a turn is aborted by an incoming prompt, THEN no error
activity shall post for the aborted turn.
- AC6 — WHEN the daemon starts with turns left in
running state,
the system shall enqueue one continuation turn per interrupted session
(resume of the stored session id), deduplicated across repeated
restarts.
- AC7 — IF shutdown occurs while a turn is running, THEN the process
shall exit within SHUTDOWN_GRACE_MS plus a fixed margin, and the
turn shall be marked interrupted.
- AC8 — WHEN the worker is stopped with no active turns, shutdown
shall complete without waiting for the grace window.
- AC9 — the systemd unit shall set
TimeoutStopSec greater than the
default SHUTDOWN_GRACE_MS.
| Criterion |
Method |
Command / flow |
✓ |
| AC1 |
automated |
pnpm test — capture GraphQL payload via test double, assert shape |
[ ] |
| AC2 |
automated |
pnpm test — tool-use event with input: {} |
[ ] |
| AC3 |
automated |
pnpm test — POST signed webhook mid-shutdown, assert row persisted |
[ ] |
| AC4 |
automated |
pnpm test — prompted event mid-turn → abort + --resume argv asserted |
[ ] |
| AC5 |
automated |
pnpm test — no error terminal activity for the aborted turn |
[ ] |
| AC6 |
automated |
pnpm test — reopen log with running turns → one deduped continuation each |
[ ] |
| AC7 |
automated |
pnpm test — fake clock, stubborn child, exit within grace + margin |
[ ] |
| AC8 |
automated |
pnpm test — idle stop returns promptly |
[ ] |
| AC9 |
automated |
test asserting TimeoutStopSec in the unit file exceeds default grace |
[ ] |
Prevention: AC1/AC2 form the standing contract test for Linear activity
content — future shape drift fails the suite instead of failing silently
in production logs.
Out of scope
- Reconcile-loop log spam / backoff (
reconcile_webhook_failed every minute)
- App-actor-ID auto-discovery
- Per-turn process-group cleanup of session-spawned children
- Progress verbosity configuration
- Checkpointing/restoring in-flight runtime state (D5 boundary)
- Replaying implementer replies missed while the process was fully down
Justification
- "A drain window saves in-flight work" — rejected in the gate: drain
only saves nearly-finished turns and blacks out ingress; interruption
was made recoverable instead (60s grace + startup auto-resume, deduped
via turns.source_key against crash loops).
- "Webhooks can pause during shutdown" — rejected: ingress persists until
workers stop and server close is the last act; user prompts interrupt
the running turn rather than queueing behind hours-long runs.
- Combined item shape — held: both fixes live in the same turn lifecycle;
one review covers them.
Open questions
[DEFERRED] Continuation-prompt wording and whether the resumed turn
posts a "resuming after restart" thought — left to /do.[DEFERRED] Startup ordering when a prompt persisted during shutdown
(D2) and a synthetic continuation turn (D4) both exist for one
session — queue serialization + D3 should self-resolve it; needs a
test case in /do's plan, not a design change (gate round 2).
References (in refs/)
refs/evidence-journal.md — live log excerpts: per-tool-call
session_progress_failed rejections; 16:32 restart aborting a running
turn
type: feature-ticket
id: daemon-turn-reliability
status: ready
zone: 1
pr:
Feature: daemon turn reliability — visible actions, interruptible turns, zero-loss restarts
Intent
Three reliability requirements for the Linear agent daemon, driven by live
incidents. (1) All tool-call progress is invisible in Linear: the daemon
posts action activities with a
bodyfield, but Linear requiresactionand
parameter— every one has always been rejected(
session_progress_failedon each tool call). (2) A user cannot get amessage to a working agent: prompts queue behind the running turn, which
can be hours for an implementer run. (3) Daemon restarts (= every deploy)
hard-kill in-flight turns with no recovery — a live run (TM-390) lost its
in-progress verification this way. The unifying principle: interruption
must be cheap and recoverable (conversation context and worktree files
already survive via
--resume+ SQLite; only in-flight process state isephemeral), and webhook ingress must never close.
Desired end state
Linear session (name + detail), interleaved with thoughts.
the agent picks up the message and continues with full context. From
the user's side: agents are interruptible, like a colleague.
systemctl restart linear-agent-daemonmid-turn: the daemon exitsafter a short grace, and on startup the interrupted turn auto-resumes
("continue where you left off") — no user action, no lost conversation,
only ephemeral runtime state (e.g. a booted dev stack) to redo.
alive, including during shutdown.
Key architecture directions
shape:
action= tool name,parameter= the detaildescribeToolalready computes, with a non-empty fallback (
"running"). Nobodyfield on actions. Clean replacement — the old shape never worked.
accepting and persisting webhooks until the workers have stopped;
server close is the last step before exit. Events received during
shutdown are processed after restart via the existing queue.
issue whose turn is running aborts that turn's claude process
(graceful signal, short grace), then runs the prompted turn with
--resumeof the stored session id — the existing prompted-resumepath. The interrupted turn posts no error activity in this case; the
new turn is the continuation.
grace (
SHUTDOWN_GRACE_MS, default 60s) then abort and are markedinterrupted; the systemd unit sets
TimeoutStopSecabove the grace.On startup, each interrupted turn with a stored claude session id is
re-queued as a synthetic continuation turn (resume + "continue"-style
prompt, deduped via
turns.source_keyso a crash loop can't spawnduplicates). The long-drain alternative (15-min window) was rejected:
it only saves nearly-finished turns and creates a webhook blackout.
process state (running dev stacks, partially-run test suites); the
resumed turn is expected to redo that. No attempt to checkpoint or
restore runtime state.
Starting points (non-exhaustive)
daemon/src/linear.ts:12—ProgressContent(wrong action shape);postActivitydaemon/src/sessions.ts:25—describeTool;:134action content;:324immediate-abortstop(); per-turnAbortControllermapdaemon/src/eventlog.ts— turn queue,source_keydedupe,interruptStaleRunning(existing restart-interrupt path to build on)daemon/src/index.ts:28—shutdown()ordering (server currentlycloses before workers)
daemon/ops/linear-agent-daemon.service— noTimeoutStopSectodayVerification criteria
daemon shall post an ephemeral action activity whose content contains
non-empty
actionandparameterfields and nobodyfield.posted activity's
parametershall be a non-empty string.durably persist incoming webhook deliveries until worker stop
completes.
running, the system shall abort the running turn and execute the
prompted turn with the stored claude session id (
--resume).activity shall post for the aborted turn.
runningstate,the system shall enqueue one continuation turn per interrupted session
(resume of the stored session id), deduplicated across repeated
restarts.
shall exit within
SHUTDOWN_GRACE_MSplus a fixed margin, and theturn shall be marked interrupted.
shall complete without waiting for the grace window.
TimeoutStopSecgreater than thedefault
SHUTDOWN_GRACE_MS.pnpm test— capture GraphQL payload via test double, assert shapepnpm test— tool-use event withinput: {}pnpm test— POST signed webhook mid-shutdown, assert row persistedpnpm test— prompted event mid-turn → abort +--resumeargv assertedpnpm test— no error terminal activity for the aborted turnpnpm test— reopen log with running turns → one deduped continuation eachpnpm test— fake clock, stubborn child, exit within grace + marginpnpm test— idle stop returns promptlyTimeoutStopSecin the unit file exceeds default gracePrevention: AC1/AC2 form the standing contract test for Linear activity
content — future shape drift fails the suite instead of failing silently
in production logs.
Out of scope
reconcile_webhook_failedevery minute)Justification
only saves nearly-finished turns and blacks out ingress; interruption
was made recoverable instead (60s grace + startup auto-resume, deduped
via
turns.source_keyagainst crash loops).workers stop and server close is the last act; user prompts interrupt
the running turn rather than queueing behind hours-long runs.
one review covers them.
Open questions
[DEFERRED]Continuation-prompt wording and whether the resumed turnposts a "resuming after restart" thought — left to
/do.[DEFERRED]Startup ordering when a prompt persisted during shutdown(D2) and a synthetic continuation turn (D4) both exist for one
session — queue serialization + D3 should self-resolve it; needs a
test case in
/do's plan, not a design change (gate round 2).References (in refs/)
refs/evidence-journal.md— live log excerpts: per-tool-callsession_progress_failedrejections; 16:32 restart aborting a runningturn