Skip to content

fix(hooks): count one delegate_task batch once - #1658

Merged
devseunggwan merged 1 commit into
mainfrom
issue-1657-fix-fan-out-delegate-batch
Nov 17, 2026
Merged

devseunggwan merged 1 commit into
mainfrom
issue-1657-fix-fan-out-delegate-batch

Conversation

@devseunggwan

@devseunggwan devseunggwan commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Summary

fan-out-scope-gate now counts one Hermes delegate_task batch as one delegation. Closes #1657

  • count_targets_in_turn: a tool_use block of the gated call's own host call is not a candidate. The Hermes adapter ids one call's tasks <id>, <id>#1, <id>#2 (_hermes.tool_use_id); _host_call strips a #<index> suffix (index >= 1) to find the call. A later host call in the turn still counts every task before it, so a second delegation asks, with an ordinal that counts the running workers.
  • Claude Code ids carry no such suffix, so counting there is unchanged.
  • Spec: one table row, one paragraph under Counting rule, one Known limits bullet (a single batch that widens the request by itself is no longer asked). _hermes.py docstrings and docs/hermes-adapter-dev.md name the id contract the gate now reads.

Why

The ask from the second task of one call is a human approval on Hermes (HUMAN_APPROVAL_HOOKS). In a headless session whose turn was opened by a plugin message injection, nothing answers it and the turn waits the host's approvals.timeout. Two such stalls were observed on one day, each a three-task delegate_task. Details in the issue.

Tests

  • tests/hooks/preflight-gate/test_fan_out_scope_gate.sh: the three-sibling cases now expect silent for every task of one call; new cases for a second call after a three-task call (both of its tasks ask as target chore(skills): remove unused and redundant skills #4), a second one-task call (ask as feat(skill): add pr-dev-to-prod release PR skill #2), and ids with a # that is not a task index (counted as separate calls). RED with main's impl.py: 3 fail. GREEN: 52 passed, 0 failed.
  • tests/hooks/_lib/test_hermes.py, end to end through the bridge with a temp state.db: one call with 1 / 2 / 3 tasks not asked; a second call after a three-task call asks as target chore(skills): remove unused and redundant skills #4. RED with main's impl.py: 2 fail (the two- and three-task cases; the one-task case is the control).
  • Replay of the count on a saved rebuilt transcript of one stalled turn: main passes the first task and asks on #1, #2; the branch passes all three.
  • Corpus replay over saved transcripts, every Agent tool_use cut where its hook would have seen it: Hermes caches, 79 calls: main asks on 16, the branch on 5; all 11 flips are ask -> pass on #<n> siblings, 0 pass -> ask. Claude Code transcripts, 14 calls: 0 differ.
  • scripts/run-tests.sh on tree ddce90e6 (this PR's tree; only the commit message changed since), python part (pytest 2879 passed, coverage floor, manifest and lint checks, ruff, mypy) and shell part: both ALL TESTS PASSED.
  • ruff check . (0.15.8) and markdownlint-cli2 on the changed docs: clean.

Caller chain verified: count_targets_in_turn <- main in fan-out-scope-gate/impl.py (only caller); _hermes.tool_use_id unchanged, docstring only (grep over hooks/ and plugins/).

Summary by CodeRabbit

  • Bug Fixes

    • The fan-out gate now counts tasks launched together in one Hermes delegation as a single target, rather than prompting partway through the batch. Subsequent delegations are still checked using the cumulative target count.
    • Task IDs are grouped only when their suffix indicates a numeric task index.
  • Documentation

    • Updated the Hermes development guide and gate documentation to clarify delegation counting behavior.

fan-out-scope-gate counted the earlier tasks of one Hermes delegate_task
call as targets already created, so every multi-task delegation asked
from its second task. That ask is a human approval on Hermes; in a
headless turn opened by a plugin message injection, the prompt waited
the host's approval timeout and the turn stalled.

- count_targets_in_turn: blocks of the gated call's own host call (ids
  that differ only by a `#<index>` suffix, as _hermes.tool_use_id writes
  them) are not candidates. A later host call in the turn still counts
  every task before it, so a second delegation asks as before.
- Claude Code ids carry no such suffix, so its counting is unchanged.
- spec, adapter docstrings and docs/hermes-adapter-dev.md say so; the
  spec lists the cost (one batch that widens the request passes).

Closes #1657
@devseunggwan devseunggwan self-assigned this Nov 17, 2026
@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The fan-out scope gate now treats indexed Agent tool-use IDs from one Hermes delegation as a single host call. It excludes tasks from that call when counting its targets, while counting tasks from earlier calls. Documentation and tests describe the rule and its limits.

Changes: Hermes fan-out scope gate

Layer / File(s) Summary
Group and count Hermes delegation batches
hooks/preflight-gate/fan-out-scope-gate/impl.py, hooks/preflight-gate/fan-out-scope-gate/spec.md, hooks/_lib/_hermes.py, docs/hermes-adapter-dev.md, tests/hooks/_lib/test_hermes.py, tests/hooks/preflight-gate/test_fan_out_scope_gate.sh
The gate identifies tool-use IDs that differ by a trailing positive-integer index as belonging to the same host call. It excludes the gated call’s batch from its target count and continues to count tasks from earlier calls. Tests cover batch sizes, later calls, and IDs with non-index suffixes. The spec and documentation describe the counting rule and its limits.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related PRs

  • devseunggwan/praxis#1608: Updates fan-out-scope-gate to use tool_use_id to identify calls. This PR extends the counting rule to exclude all indexed tasks from the gated Hermes host call.
Merge Risk: Minimal · up to 632ac

The change makes one Hermes delegate_task batch count as a single delegation, so multi-task batches no longer prompt on their second task. Later delegations in the same turn are still gated, and Claude Code counting is unchanged. No concrete merge-blocking risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "fix(hooks): count one delegate_task batch once" matches the main change. The PR makes fan-out-scope-gate count the tasks of one Hermes delegate_task batch as one delegation, and the diffs i…
Linked Issues check Passed Directly linked issue #1657 is the only active target. Its coding requirements are: (1) skip earlier candidate blocks of the gated call's own host call; (2) keep counting all tasks of earlier host cal…
Out of Scope Changes check Passed All changed files relate to the #1657 objective. impl.py holds the counting change. spec.md holds the counting rule, table row, and Known limits bullet for the batch tradeoff named in the issue. _herm…
Full details: Title check

Explanation

The title "fix(hooks): count one delegate_task batch once" matches the main change. The PR makes fan-out-scope-gate count the tasks of one Hermes delegate_task batch as one delegation, and the diffs in impl.py, spec.md, and the tests all implement that. The title is short and specific, and it uses the conventional fix(scope) form. It does not name the gate, but the hooks scope and delegate_task term identify the change.

Full details: Linked Issues check

Explanation

Directly linked issue #1657 is the only active target. Its coding requirements are: (1) skip earlier candidate blocks of the gated call's own host call; (2) keep counting all tasks of earlier host calls; (3) leave Claude Code counting unchanged; (4) add shell and end-to-end tests for 1/2/3-task single calls, a second call after a three-task call (target #4), and ids with a non-index '#'. The diff of hooks/preflight-gate/fan-out-scope-gate/impl.py at head shows: a new _host_call helper that strips a '#<positive integer>' suffix via _BATCH_SUFFIX_RE; an own_call value computed from current_id; and a loop check in count_targets_in_turn that skips a tool_use whose host call equals own_call. Later host calls still append all earlier candidates, and ids without a '#' suffix map to themselves, so Claude Code counting is unchanged. The spec, docstrings, and docs/hermes-adapter-dev.md describe the same contract. The test changes (test_fan_out_scope_gate.sh and tests/hooks/_lib/test_hermes.py) cover the required cases according to the PR description and the change summary. I did not execute the test suite; the test pass results rest on the author's reported runs and the diff is consistent with them.

Full details: Out of Scope Changes check

Explanation

All changed files relate to the #1657 objective. impl.py holds the counting change. spec.md holds the counting rule, table row, and Known limits bullet for the batch tradeoff named in the issue. _hermes.py changes are docstring-only and document the tool_use_id suffix contract the gate reads. docs/hermes-adapter-dev.md documents the same contract. The two test files cover the issue's stated test scope. No unrelated logic, dependency, or configuration change appears in the summary or in the impl.py diff.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 5b84ec2 and 632acb5.

Files selected for processing: 6

  • docs/hermes-adapter-dev.md
  • hooks/_lib/_hermes.py
  • hooks/preflight-gate/fan-out-scope-gate/impl.py
  • hooks/preflight-gate/fan-out-scope-gate/spec.md
  • tests/hooks/_lib/test_hermes.py
  • tests/hooks/preflight-gate/test_fan_out_scope_gate.sh

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@devseunggwan

devseunggwan commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner Author

Verification - 632acb5 (rev 1)

# Claim Result
0 The defect is real: on main, the 2nd and 3rd task of one stalled three-task delegate_task are counted as prior targets and asked PASS(live)
1 Gate shell test: every task of one host call is silent; a later host call asks with the full ordinal; a non-index # is its own call PASS(mock)
2 End to end through the bridge with a temp state.db: one call with 1/2/3 tasks not asked; a second call after a three-task one asks as target #4 PASS(mock)
3 RED: the new and changed cases fail with main's impl.py PASS(mock)
4 Corpus replay: Hermes asks drop only on same-call siblings (0 new asks); Claude Code counting unchanged PASS(mirror)
5 scripts/run-tests.sh passes on this PR's tree PASS(mock)
Unverified
  • The bridge checkout that Hermes runs is not updated yet; no live headless multi-task delegate_task has run against the branch.
  • No independent review round yet.

Carried: none - the live check is the bridge fast-forward after merge.

Evidence 0 - replay of the stalled turn's saved transcript, main vs branch

The same saved rebuilt transcript, cut at the call's assistant event: main asks on #1 and #2, the branch passes all three tasks. <transcript> is a copy of the host's rebuilt transcript of that session; the tool call id is shown as toolu_A in place of the real opaque id.

$ python3 fanout_replay_tree.py <main tree> <transcript> toolu_A
turn events: 11 Agent tool_use in turn: ['toolu_A', 'toolu_A#1', 'toolu_A#2']
toolu_A prior = 0 -> pass
toolu_A#1 prior = 1 -> ask
toolu_A#2 prior = 2 -> ask
$ python3 fanout_replay_tree.py <branch tree> <transcript> toolu_A
turn events: 11 Agent tool_use in turn: ['toolu_A', 'toolu_A#1', 'toolu_A#2']
toolu_A prior = 0 -> pass
toolu_A#1 prior = 0 -> pass
toolu_A#2 prior = 0 -> pass

(the tree: header lines, which print local paths, are omitted)

Evidence 1 - gate shell test on the branch

All 52 cases pass, including the seven multi-task and host-call cases.

$ bash tests/hooks/preflight-gate/test_fan_out_scope_gate.sh 2>&1 | grep -E "multi-task|host call|one-task host|'#'|passed,"
  PASS  multi-task delegation: 1st task (silent)
  PASS  multi-task delegation: 2nd task, same host call (silent)
  PASS  multi-task delegation: 3rd task, same host call (silent)
  PASS  2nd host call after a 3-task one: its 1st task (ask: target #4)
  PASS  2nd host call after a 3-task one: its 2nd task (ask: target #4)
  PASS  2nd one-task host call in a turn (ask: target #2)
  PASS  '#' in an id that is not a task index is its own call (ask: target #3)
  52 passed, 0 failed
Evidence 2 - bridge end to end

The delegation cases of test_hermes.py pass through the real bridge.

$ PYTHONDONTWRITEBYTECODE=1 uv run --no-project --with pytest --with PyYAML==6.0.3 python -m pytest -q -p no:cacheprovider tests/hooks/_lib/test_hermes.py -k "deleg or fan or batch" -rA 2>&1 | grep -E "^PASSED|passed|failed"
PASSED tests/hooks/_lib/test_hermes.py::test_delegate_task_spawn_is_one_agent_call_per_task
PASSED tests/hooks/_lib/test_hermes.py::test_per_call_gates_keep_a_call_specific_key[fan-out-scope-gate]
PASSED tests/hooks/_lib/test_hermes.py::test_first_delegation_is_not_its_own_prior_target
PASSED tests/hooks/_lib/test_hermes.py::test_second_delegation_in_a_turn_still_asks
PASSED tests/hooks/_lib/test_hermes.py::test_multi_task_delegation_is_one_target[goals0]
PASSED tests/hooks/_lib/test_hermes.py::test_multi_task_delegation_is_one_target[goals1]
PASSED tests/hooks/_lib/test_hermes.py::test_multi_task_delegation_is_one_target[goals2]
PASSED tests/hooks/_lib/test_hermes.py::test_second_multi_task_delegation_in_a_turn_still_asks
PASSED tests/hooks/_lib/test_hermes.py::test_retry_after_a_failed_multi_task_delegation_is_not_asked
PASSED tests/hooks/_lib/test_hermes.py::test_failed_batch_result_reaches_every_generated_tool_use
PASSED tests/hooks/_lib/test_hermes.py::test_successful_batch_result_keeps_one_block
PASSED tests/hooks/_lib/test_hermes.py::test_host_notification_row_is_not_a_human_message[async_delegation_complete]
12 passed, 77 deselected, 1782 warnings in 0.49s
Evidence 3 - RED with main's impl.py

The script copies main's impl.py over the branch file, runs both suites, and restores it (cmp). Shell: 3 fail; pytest: the two- and three-task cases fail, the one-task control passes.

$ bash praxis-fanout-red.sh <worktree>
=== shell test vs origin/main impl ===
  FAIL  multi-task delegation: 2nd task, same host call (silent) (rc=0, expected=silent)
  FAIL  multi-task delegation: 3rd task, same host call (silent) (rc=0, expected=silent)
  FAIL  2nd host call after a 3-task one: its 2nd task (ask: target #4) (rc=0, expected=ask)
  49 passed, 3 failed
=== test_hermes vs origin/main impl ===
FAILED tests/hooks/_lib/test_hermes.py::test_multi_task_delegation_is_one_target[goals1]
FAILED tests/hooks/_lib/test_hermes.py::test_multi_task_delegation_is_one_target[goals2]
2 failed, 10 passed, 77 deselected, 1782 warnings in 0.50s
impl restored

(the first FAIL's stdout dump is omitted)

Evidence 4 - corpus replay, main vs branch

Every Agent tool_use in the saved transcripts, cut at its own assistant event: on Hermes the 11 dropped asks are all same-call siblings and nothing newly asks; on Claude Code nothing differs.

$ python3 fanout_corpus.py <main tree> <branch tree> <hermes transcript cache>
transcripts=627 agent_calls=79 asked_base=16 asked_branch=5
flips: {'ask->pass': 11, 'pass->ask': 0} ask->pass by id shape: {'same-call sibling (#n)': 11}
$ python3 fanout_corpus.py <main tree> <branch tree> <claude code projects dir>
transcripts=659 agent_calls=14 asked_base=6 asked_branch=6
flips: {'ask->pass': 0, 'pass->ask': 0} ask->pass by id shape: {}

Claude Code transcripts at rest hold the gated call, unlike at hook time, so this measures only that the new sibling rule never matches a Claude Code id.

Evidence 5 - run-tests.sh

Both parts pass on tree ddce90e6, the tree of 632acb5 (only the commit message changed after the run). Each part's output went to a log file; the lines below are excerpts from those logs, ... marks skipped lines.

$ PYTHONDONTWRITEBYTECODE=1 PRAXIS_TEST_PART=python uv run --no-project --with pytest --with PyYAML==6.0.3 --with coverage==7.16.0 --with mypy==1.20.0 --with types-PyYAML==6.0.12.20260815 bash scripts/run-tests.sh
2879 passed, 71858 warnings in 141.82s (0:02:21)
...
=== mypy ===
Success: no issues found in 49 source files

ALL TESTS PASSED
$ PRAXIS_TEST_PART=shell bash scripts/run-tests.sh
...
Results: 5 pass, 0 fail

ALL TESTS PASSED
History
  • rev 1: 632acb5, first anchor.
  • rev 1 edit: the stalled session's real tool call id replaced by toolu_A in Evidence 0; no result changed.

@devseunggwan
devseunggwan merged commit f103dbc into main Nov 17, 2026
12 checks passed
@devseunggwan
devseunggwan deleted the issue-1657-fix-fan-out-delegate-batch branch November 17, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(hooks): fan-out asks inside one delegate batch

1 participant