Skip to content

feat: add ZCode session hooks integration - #160

Draft
qinghuidaqzp wants to merge 3 commits into
reason-machines:mainfrom
qinghuidaqzp:feat/zcode-hooks-upstream
Draft

qinghuidaqzp wants to merge 3 commits into
reason-machines:mainfrom
qinghuidaqzp:feat/zcode-hooks-upstream

Conversation

@qinghuidaqzp

@qinghuidaqzp qinghuidaqzp commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

ZCode receives Reason messages after successful tools through native SessionStart/PostToolUse hooks, with the existing MCP inbox as recovery. Setup adds optional hooks beside the merged MCP integration; it preserves native/.agents precedence and user settings.

Delivery is bounded: each message can be offered by hooks at most twice (initial attempt plus one retry after its lease expires). The attempt counter is persisted with the message, including across daemon restarts. Hook stdout never marks it read, and exhausted messages remain recoverable through check_inbox. Agents are not asked to send acknowledgement-only replies.

A normal correlated reply confirms only the exact sender/session pair, including a terminal's machine-qualified address. Explicit inbox reads record the upstream handed_to_agent receipt. Confirmation bookkeeping cannot fail an already-stored send. Hooks connect only to an existing daemon and have a bounded request budget.

Every installed hook carries --network-home <resolved state directory>, including the default state directory. Older engines such as v0.12.17 reject that argument before entering their generic hook handler, keeping pending mail unread after a binary rollback. Re-running setup adds the guard to existing managed entries while preserving disabled flags.

Messages use the existing framing and attachment confinement. Hooks honor the smaller of the user hooks.maxOutputBytes and the 24 KiB safety cap, including the final newline. An oversized message at the head of the queue intentionally holds later hook mail until check_inbox reads it; later messages do not overtake it. A bounded recovery reminder is returned when it fits. If even that reminder cannot fit, stdout is empty and mail remains unread. There is no Stop hook, end-of-turn delivery or idle wake-up.

Per-call ZCode metadata routes the exact session/workspace. Older clients with no metadata retain an isolated per-process inbox. Partial metadata is rejected rather than guessing a workspace or another conversation.

Hook configuration is best-effort: disabled, malformed or unknown hooks do not gate MCP installation, path updates or removal. Global and individual disable flags are preserved, recognized managed hooks are removed around unknown groups, and empty managed containers are cleaned up. Non-UTF-8 executable paths cannot panic hook JSON construction.

Validation

Windows head 56fe7cc, based on upstream bcf045c (0.12.17):

  • cargo fmt --check
  • cargo clippy --all-targets --locked -- -D warnings
  • cargo test --locked: 270 passed
  • cargo build --locked
  • python tests/e2e/zcode_hooks.py: 53 checks passed against an isolated real daemon. The configured-executable comparison now resolves both paths, including macOS symlinked temporary roots.
  • python tests/e2e/zcode_rollback.py --old-bin <v0.12.17 executable>: 2 actual rollback scenarios passed, for default and custom state directories. Used the official Windows v0.12.17 release after SHA-256 verification. Each scenario installs hooks with this build, queues one previously offered and one never-offered message, replaces only the isolated fixture executable, starts the old daemon on the migrated store, and runs both installed hook commands. Both fail with exit 2 and empty stdout, all message rows stay unchanged, and the old MCP check_inbox recovers both messages.

Unit regressions cover exact machine-qualified terminal confirmation, repeated lease expiry, database reopen, preserved disable flags, cleanup, rollback guards and upgrading unguarded managed hooks. Unix-only path/write tests are exercised by CI. No new dependencies or peer wire identifier changes.

Historical GUI validation applies only to the original PR2 commit adb7568: ZCode Desktop 3.15.3 received PostToolUse messages across two conversations and replied through MCP; suspended-daemon recovery was exercised. GUI validation and a release build have not been repeated on this head. The current cross-platform CI result is in Checks. This remains Draft for maintainer review.

@larstalian

Copy link
Copy Markdown
Member

Thanks for this! It is in our review queue, and we will come back here with findings.

@larstalian

Copy link
Copy Markdown
Member

Sorry for the churn: main moved quite a bit in the last two days (v0.12.14 to v0.12.17, with changes in mail.rs, store.rs, proto.rs and the daemon), and GitHub now shows conflicts here. Could you rebase onto current main when you get a chance? We'll review the rebased head right away.

@qinghuidaqzp
qinghuidaqzp force-pushed the feat/zcode-hooks-upstream branch from 585f4df to ea0fc6e Compare October 30, 2026 08:21
@larstalian

Copy link
Copy Markdown
Member

Thanks for the rebase, and for the care that went into this. We reviewed head ea0fc6e and ran the full suite on it: fmt and clippy pass, 271 unit tests pass (main has 264, so 7 new and none removed), two_machines 150/150, reconnect 41/41, gemini 9/9, and your zcode_hooks.py 30/30. hooks.py shows the same 13 failures as main, so those are not yours.

The good news first. Nothing here changes behaviour for anyone who doesn't use ZCode. There are no new dependencies. The framing holds: PostToolUse and check_inbox both go through render::for_agent. The per-call workspace_path goes through the existing Confine, and folder opt-out is checked on every call and every hook. We found no way around any of that.

Before we can merge, a few things need changing. Items 1, 3, 4 and 5 we reproduced on an isolated daemon. Item 2 comes from reading the code.

1. Unanswered mail is re-injected forever. (src/hook/zcode.rs, src/daemon/sessions.rs:451-457)

  • The hooks only lease mail and never mark it read, and nothing caps the retries.
  • In our run, one terminal message was injected at t=0, 93 s and 185 s, and was still unread at the end.
  • Any message the agent doesn't answer with reply_to or read with check_inbox comes back after every 90 s lease, for the whole session. That covers answers to its own questions, FYIs and terminal instructions.
  • The SessionStart text (src/hook/zcode.rs:89) tells the agent to reply to confirm receipt. Every reply wakes the teammate's session, and our guidance to agents is to message only when something is needed.
  • Suggested fix: treat the same session's next hook or MCP call as confirmation of what was injected before it, or allow a single retry. Then drop the "reply to confirm" instruction.

2. A reply to a teammate's terminal never confirms. (src/store.rs:386-397, called from src/daemon/mail.rs:413-414)

  • The match compares sender with reply.to, which is Target::address() (alice/terminal).
  • Received terminal mail is stored as alice/terminal@machine (render::shown_from in receive), so the two never match.
  • Together with item 1, that message keeps coming back.
  • Suggested fix: compare against the recipient's shown address (res.recipients.last().address).

3. The ZCode hook sends through_reason: None. (src/hook/zcode.rs:62)

  • Every other hook sends Some(false) (src/hook.rs:101).
  • sessions.rs:424 copies the value onto the live session, and mod.rs:386 then never records the handoff.
  • In our run, after an explicit check_inbox read, the sender's receipt stayed at queued for a ZCode session whose hook had fired. A ZCode session that only used MCP showed handed_to_agent.
  • So message_status tells teammates the wrong thing. The fix is one line.

4. MCP fails outright without com.zcode/request-context. (src/mcp.rs:394-398)

  • On this head, list_team returns missing session_id/workspace_path metadata. On main the same call works.
  • Any ZCode build that doesn't send that key would lose every Reason tool as soon as the engine auto-updates, without anyone re-running setup.
  • Please fall back to the existing per-process session when the metadata is absent.

5. Setup and removal regress for existing ZCode users. (merge_hooks, now gating the MCP part: src/setup/zcode.rs:150, :44-57)

  • If the user has hooks but hooks.enabled is not true, ZCode is skipped entirely and the MCP server isn't installed. Main installs it. The test disabled_or_invalid_user_hooks_block_install_before_any_write asserts the new behaviour, and init/join --json go through the same path.
  • If hooks are switched off globally after install, the next setup refuses with "disabled user hooks", even when the only hooks are ours. The MCP command then keeps pointing at the old binary path after a move.
  • setup --remove fails with "ZCode hook group has no hooks array" whenever any user hook group has an unexpected shape, and the MCP server stays installed.
  • Hooks should be best-effort on top of the MCP install and removal, never a gate on them.

Smaller points. We'd like these too, but they don't block:

  • Removal leftovers (src/setup/zcode.rs:84):
    • Install then remove leaves "hooks":{"events":{},"enabled":true} in a file that had no hooks key.
    • An explicit hooks.enabled:false with no entries is flipped to true.
    • A Reason entry the user disabled is re-enabled, and hooks_wired still reports it as wired.
  • Wrong "end of turn" promise. README.md:54, the sender note in src/daemon/mail.rs:535 and the turn tier in list_team all say ZCode also receives at the end of a turn. ZCode has no Stop hook, so senders are told something that won't happen.
  • Oversized messages. A message over the 24 KiB output cap never arrives by hook, and later ones overtake it. ZCode also lets users lower hooks.maxOutputBytes, which the hardcoded cap doesn't follow.
  • A send can fail after storing. ack_zcode_reply(&msg)? (src/daemon/mail.rs:414) can fail a send after the message was already stored. The neighbouring bookkeeping uses let _ =.
  • A possible panic. json!({"command": exe}) (src/setup/zcode.rs:14) panics on an install path that isn't valid UTF-8.

Tests we'd like added:

  • a reply to a terminal or a two-machine reply confirms (item 2);
  • receipts for a ZCode recipient (item 3);
  • MCP calls without the metadata (item 4);
  • removal with odd hook shapes (item 5);
  • a bound on re-injection (item 1).

Happy to talk through any of these, especially the confirmation design in item 1.

@qinghuidaqzp

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed reproduction and review. Addressed in efc0ab1:

  1. Chose your single-retry option. Hook attempts are persisted on each message and capped at two total; neither lease expiry nor daemon restart resets the bound. Exhausted messages remain unread and recoverable with check_inbox. Removed the reply-to-confirm instruction. Tests exercise repeated lease expiry and reopen/restart recovery.
  2. Confirmation now compares against the resolved recipient's shown address. The regression verifies alice/terminal@laptop confirms while the unqualified address and @desktop do not.
  3. ZCode hooks send Some(false). The end-to-end test verifies stdout alone stays queued, then an explicit inbox read records handed_to_agent.
  4. Calls with no ZCode context fall back to the existing per-process mailbox. Tests cover list/send/inbox without metadata and ensure that mailbox cannot consume a metadata-identified conversation's mail. Partial metadata still fails safely.
  5. Optional hooks no longer gate MCP install/update/removal. Disabled settings are preserved; unknown hook groups are left intact while recognized Reason declarations can be removed. Tests cover moved executables, globally/individually disabled hooks, odd shapes, and optional hook-only write failures.

The smaller points are addressed too: empty managed containers are removed; disabled declarations are not reported hook-wired; README, directory text and sender notes say after successful tools with no end-of-turn delivery; output follows the user's lower cap and an oversized head blocks later hook delivery with a bounded inbox-recovery reminder; confirmation bookkeeping is best-effort; non-UTF-8 paths skip optional hook installation without a panic.

Local Windows validation: fmt and strict Clippy pass, 269 Rust tests pass, build passes, and 53 isolated-daemon ZCode checks pass. Unix-specific regressions are included for CI. I have not repeated the genuine Desktop GUI test or your full two-machine suite on this head; those are not claimed as current passes. The PR description now reflects the final behavior and validation.

@larstalian

Copy link
Copy Markdown
Member

Thank you, that was fast and thorough. We re-reviewed efc0ab1, re-checked every earlier finding, and reproduced each fix on an isolated daemon:

  • the retry cap holds across lease expiry and restart;
  • terminal replies confirm;
  • receipts reach handed_to_agent;
  • calls without metadata fall back;
  • setup and removal never block MCP.

The store migration also holds up:

  • 3,317 sends during an upgrade under load, each stored once;
  • v0.12.17 and v0.10.8 still open a store this head has migrated;
  • re-upgrading after a rollback is clean.

Nothing changes for non-ZCode users. Tests: 279 unit tests; two_machines, reconnect, codex_busy and gemini all match main; mixed_versions matches main.

One thing left before we can merge:

After an engine rollback, the installed hooks lose mail silently. (src/setup/zcode.rs:9-14)

  • The hooks are written as plain hook zcode session-start|post-tool-use.
  • We rolled the binary back to v0.12.17 on a store this head had migrated. The installed hook zcode post-tool-use then exited 0, printed plain text instead of JSON, and marked both pending messages read, including one that was never offered.
  • ZCode drops non-JSON hook output, so the agent never sees that mail, while the sender is told it was handed over.
  • A rollback can happen when a Reason CLI release pins an older engine, or when someone downgrades by hand.
  • Could the hook command always carry an argument that older engines reject? Always passing --network-home <home> would do it, or a small version flag. In our run, v0.12.17 given --network-home failed fast with exit 2 and consumed nothing, so ZCode records a recoverable hook failure and the mail stays for check_inbox.

Two small things while you're there:

  • tests/e2e/zcode_hooks.py:244 compares the configured command with str(BIN), but setup writes the resolved path. On stock macOS, where /tmp and /var are symlinks, the test fails at check 46. Comparing with the resolved path fixes it.
  • With an oversized message at the head of the queue, later mail waits for check_inbox rather than overtaking it. That's a fine choice; please say so in a line of the PR description, so it reads as intended.

@qinghuidaqzp

Copy link
Copy Markdown
Contributor Author

Thanks for the rollback reproduction. Fixed in 56fe7cc.

  • Both SessionStart and PostToolUse now always include --network-home, using the effective state directory when no override is set. Setup also upgrades existing three-argument managed declarations, preserves enabled:false, remains idempotent and still removes them correctly.
  • Added tests/e2e/zcode_rollback.py. On Windows I ran it against the official v0.12.17 binary after checking its release SHA-256, for both default and custom state directories. It executes the actual setup-written argv after replacing the isolated executable and opening the migrated store with the old daemon. Both hook events exit 2 with empty stdout; the offered and never-offered message rows stay unchanged and unread. The old engine's MCP check_inbox then recovers both.
  • The macOS test compares Path(configured_command).resolve() with BIN.resolve().
  • The PR description explicitly documents intentional oversized-head blocking until check_inbox.

Local validation: fmt/strict Clippy/build pass, 270 Rust tests, 53 ZCode end-to-end checks, and 2 real rollback scenarios pass. Genuine Desktop GUI validation and macOS end-to-end execution were not repeated locally; new cross-platform CI results are in Checks.

@larstalian

larstalian commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Thanks! The rollback guard works. We ran the setup-written argv against the official v0.12.17 and v0.10.8 binaries, with both default and custom state dirs and with paths containing spaces and non-ASCII characters. Both engines exit 2 in about 10 ms, print nothing, and leave every row untouched, and check_inbox recovers the mail. zcode_rollback.py and zcode_hooks.py pass on macOS with the default /tmp. The full suite matches main.

Sorry, though: the way we suggested doing it has two side effects. Baking the default state path into the hook means:

  • The hook can end up on a different network from MCP. If setup ran without an override and ZCode later starts with REASON_NETWORK_HOME set, the hook talks to the default network while MCP follows the override. Before this commit, both followed the environment.
  • The hook can go silent on long home paths. When HOME is spelled through a symlink at setup and by its real path at runtime, and the socket path reaches about 100 bytes, the hook can't find the daemon and does nothing, while MCP keeps working.

Also, if the default state dir ever moves, these hooks would be the only integration still pointing at the old one.

A smaller variant keeps the guard without these effects: pass --network-home only when an override is set, as the MCP entry does. Otherwise pass a flag with no value that only new engines accept, for example --v2. Old engines reject it in the same way.

That is the last thing from the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants