Skip to content

Improve agent prompt determinism - #8

Open
snipextt wants to merge 2 commits into
mainfrom
improve/agent-engine-prompt-determinism
Open

Improve agent prompt determinism#8
snipextt wants to merge 2 commits into
mainfrom
improve/agent-engine-prompt-determinism

Conversation

@snipextt

@snipextt snipextt commented Aug 1, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Tighten Wacht coordinator, executor, reviewer, conversation, memory, workspace, and live-context prompt contracts.
  • Replace stale task workspace guidance with the current JOURNAL.md, artifacts/, audit/, and review/ paths.
  • Clarify tool-schema authority, lifecycle ownership, completion, abort, verification, and runtime-signal handling.
  • Add an LlmRequestFailed runtime signal so retries receive corrective guidance after a failed model request.

Validation

  • cargo check --workspace --locked
  • git diff --check

No tests were added or run per scope. No deployment or service restart was included.

Summary by CodeRabbit

  • Bug Fixes

    • Improved handling of AI request failures with clearer runtime warnings.
    • Improved detection and reporting of unknown tools, repeated tool-call cycles, and persistent failures.
  • Improvements

    • Strengthened task execution with clearer progress tracking, completion requirements, blocker handling, and deliverable reporting.
    • Expanded support for project tools, subscriptions, file and web operations, memory updates, and workspace artifacts.
    • Enhanced safety guidance and refined review and verification workflows for more reliable task completion.

@coderabbitai

coderabbitai Bot commented Aug 1, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The change adds runtime signals for LLM failures, unknown tools, tool failures, and tool-call cycles. It also revises agent prompt contracts for live tools, memory, journaling, lifecycle ownership, termination, conversation behavior, and input safety.

Changes

Runtime and agent operating contracts

Layer / File(s) Summary
Runtime signals and tool-call cycle detection
agent-engine/src/executor/...
The executor emits structured warning signals for LLM request failures, unknown tools, sustained tool failures, and recurring tool-call signatures. It retains adjacent-repeat detection and adds a bounded recent-signature window.
Workspace and journal completion contract
templatekit/src/templates/task_workspace_brief.hbs, templatekit/src/prompts/{delegated_execution_system.md,service_execution_system.md}
Execution requires task instructions, concrete journal entries, durable artifact paths, and recorded blockers. Clean termination is preferred over abort_task.
Tool and role execution contracts
templatekit/src/prompts/{coordinator_system.md,delegated_execution_system.md,service_execution_system.md,reviewer_system.md,shared_operating_style.md,memory_discipline.md}
Prompts now use explicit live tool schemas, define execute_command usage, add memory updates, require verification and progress, and refine lifecycle and abort rules by role.
Conversation and runtime-context guidance
templatekit/src/prompts/conversation_agent_system.md, templatekit/src/templates/agent_loop_live_context.hbs
Conversation agents receive subscription and task-field rules, user-question guidance, subject-based communication rules, and internal runtime-signal and input-safety handling guidance.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Sequence Diagram(s)

sequenceDiagram
  participant AgentLoop
  participant AgentExecutor
  participant RuntimeSignal
  AgentLoop->>AgentExecutor: process model response and tool calls
  AgentExecutor->>RuntimeSignal: emit failure, unknown-tool, escalation, or cycle signal
  AgentExecutor-->>AgentLoop: continue execution or return the original LLM error
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the primary change: improving determinism across agent prompt contracts and runtime guidance.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch improve/agent-engine-prompt-determinism

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
templatekit/src/prompts/delegated_execution_system.md (1)

152-157: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Align executor termination rules with the runtime outcome contract.

terminate_loop accepts blockers/next_actions but does not update assignment_result_status as a non-completed terminal handoff, while abort_task(blocked) is the only runtime path that records blocked assignment/result status. Reconcile the shared/delegated/service prompts so clean blocked termination actually persists the blocked state with a result summary, and keep abort_task(blocked) only for impossible/unrouteable cases.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@templatekit/src/prompts/delegated_execution_system.md` around lines 152 -
157, Update the terminal rules in
templatekit/src/prompts/delegated_execution_system.md (lines 152-157) and
templatekit/src/prompts/service_execution_system.md (lines 170-178) so clean
blocked termination uses terminate_loop with blockers/next_actions and records a
blocked assignment/result status plus a concise result summary; reserve
abort_task(blocked) for impossible or unrouteable cases where the loop cannot
exit cleanly. Align both prompt definitions with the runtime outcome contract
and preserve the single terminate_loop terminal shape.
🧹 Nitpick comments (1)
agent-engine/src/executor/core.rs (1)

35-35: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add focused tests for the new RuntimeSignal contract.

The change affects enum construction, key(), message(), is_warning(), and render(). Add assertions for the llm_request_failed key, recovery message, warning classification, and rendered output. cargo check does not validate these runtime behaviors.

Also applies to: 55-55, 82-82, 94-94

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@agent-engine/src/executor/core.rs` at line 35, Add focused tests for the
RuntimeSignal enum covering the LlmRequestFailed variant and its key(),
message(), is_warning(), and render() behavior. Assert the expected
llm_request_failed key, recovery message, warning classification, and rendered
output, including the other changed variants identified in the diff where
applicable.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@templatekit/src/prompts/conversation_agent_system.md`:
- Around line 85-87: Update update_scope_for_conversation so lifecycle fields
remain coordinator-owned without exception: remove permission for the
conversation agent to send status, schedule, result_summary, artifacts,
findings, cautions, or next even when explicitly requested. Align the
corresponding guidance around the coordinator-owned fields so all policy-field
changes are routed through the coordinator.
- Line 192: Update the first-turn guidance around [first_turn].must_include and
no_status_narration so their requirements do not conflict: permit the first
tool-call companion text only when it is a clarifying question or non-progress
acknowledgement, or explicitly define a narrowly scoped first-turn exception.
Remove progress-style narration from the required examples while preserving the
prohibition on routine status updates.

In `@templatekit/src/prompts/reviewer_system.md`:
- Around line 57-61: Update the [timeline.tool_output_preservation] verification
guidance to prohibit replaying arbitrary executor tool inputs, including
mutating, destructive, credential-bearing, or network commands. Replace the
current rerun requirement with reviewer-authored, explicitly allow-listed
checks, and update the repeated rule at the referenced verification guidance to
match.

In `@templatekit/src/prompts/shared_operating_style.md`:
- Around line 230-232: Update the only_exits template rule so it applies only to
non-conversation threads, or explicitly permits the required plain-text/no-tool
completion for conversation threads. Preserve the existing terminate_loop,
ask_user, abort_task, and notify_user conditions for non-conversation runs.

In `@templatekit/src/templates/task_workspace_brief.hbs`:
- Around line 18-21: Update the task workspace instructions in
task_workspace_brief.hbs to explicitly identify /task/audit/ as a runtime-owned,
read-only tool-call log that agents must not write evidence or review files
into; direct agent-created evidence and review material to the appropriate
writable location, while preserving the existing /task/review/ guidance.

---

Outside diff comments:
In `@templatekit/src/prompts/delegated_execution_system.md`:
- Around line 152-157: Update the terminal rules in
templatekit/src/prompts/delegated_execution_system.md (lines 152-157) and
templatekit/src/prompts/service_execution_system.md (lines 170-178) so clean
blocked termination uses terminate_loop with blockers/next_actions and records a
blocked assignment/result status plus a concise result summary; reserve
abort_task(blocked) for impossible or unrouteable cases where the loop cannot
exit cleanly. Align both prompt definitions with the runtime outcome contract
and preserve the single terminate_loop terminal shape.

---

Nitpick comments:
In `@agent-engine/src/executor/core.rs`:
- Line 35: Add focused tests for the RuntimeSignal enum covering the
LlmRequestFailed variant and its key(), message(), is_warning(), and render()
behavior. Assert the expected llm_request_failed key, recovery message, warning
classification, and rendered output, including the other changed variants
identified in the diff where applicable.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 55f38ca1-c8a8-4a18-bd73-7ee6f4fc2f73

📥 Commits

Reviewing files that changed from the base of the PR and between 11c95b4 and 1eadbc4.

📒 Files selected for processing (11)
  • agent-engine/src/executor/agent_loop/mod.rs
  • agent-engine/src/executor/core.rs
  • templatekit/src/prompts/conversation_agent_system.md
  • templatekit/src/prompts/coordinator_system.md
  • templatekit/src/prompts/delegated_execution_system.md
  • templatekit/src/prompts/memory_discipline.md
  • templatekit/src/prompts/reviewer_system.md
  • templatekit/src/prompts/service_execution_system.md
  • templatekit/src/prompts/shared_operating_style.md
  • templatekit/src/templates/agent_loop_live_context.hbs
  • templatekit/src/templates/task_workspace_brief.hbs

Comment on lines +85 to +87
tools = ["create_project_task", "delegate_task", "update_project_task", "get_project_task", "subscribe_to_task", "unsubscribe_from_task"]
tool_authority = "The current tool schema and live available-tools context are authoritative; never invent a tool name. Discover optional integrations with search_tools, then load_tools with exact names."
update_scope_for_conversation = "Title and description edits are the normal conversation path. Lifecycle fields are coordinator-owned by policy; do not send status, schedule, result_summary, artifacts, findings, cautions, or next unless the user explicitly asks and the lifecycle requires it."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Keep lifecycle updates coordinator-owned.

Line 87 permits the conversation agent to send status, schedule, result_summary, artifacts, findings, cautions, or next when the user asks and the lifecycle requires it. Lines 134-135 identify these fields as coordinator-owned. Line 141 also says a conversation thread cannot change task status.

Remove the exception. Route all policy-field changes through the coordinator.

Proposed wording
-update_scope_for_conversation = "Title and description edits are the normal conversation path. Lifecycle fields are coordinator-owned by policy; do not send status, schedule, result_summary, artifacts, findings, cautions, or next unless the user explicitly asks and the lifecycle requires it."
+update_scope_for_conversation = "Title and description edits are the normal conversation path. Status, schedule, result_summary, artifacts, findings, cautions, and next are coordinator-owned. A conversation agent must not send them."

Also applies to: 132-135

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@templatekit/src/prompts/conversation_agent_system.md` around lines 85 - 87,
Update update_scope_for_conversation so lifecycle fields remain
coordinator-owned without exception: remove permission for the conversation
agent to send status, schedule, result_summary, artifacts, findings, cautions,
or next even when explicitly requested. Align the corresponding guidance around
the coordinator-owned fields so all policy-field changes are routed through the
coordinator.

forbidden_words = ["milestones", "audit trails", "operational handoffs"]
sentence_form = "short sentences, full words, no jargon the user did not use first"
narration = "never narrate the control framework — say intent, not mechanism"
no_status_narration = "do not announce routine activity or progress mechanics — no 'I'm checking', 'still working', 'let me continue', or 'I'll now'. Use text for a question, blocker, or delivery; tool calls show the work"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Make first-turn text rules consistent.

[first_turn].must_include requires a text line beside the first tool call and lists progress-style examples. no_status_narration now forbids routine progress lines, including those examples.

Define a first-turn exception, or allow only a clarifying question or non-progress acknowledgement.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@templatekit/src/prompts/conversation_agent_system.md` at line 192, Update the
first-turn guidance around [first_turn].must_include and no_status_narration so
their requirements do not conflict: permit the first tool-call companion text
only when it is a clarifying question or non-progress acknowledgement, or
explicitly define a narrowly scoped first-turn exception. Remove progress-style
narration from the required examples while preserving the prohibition on routine
status updates.

Comment on lines 57 to 61
[timeline.tool_output_preservation]
current_execution = "your full tool inputs + outputs (working memory)"
past_executions = "input only; tagged [output not preserved in timeline view — re-run this tool yourself if you need the content]"
required_for_verification = "re-run the tool yourself (read_file the path, bash the test/build, diff against expected)"
required_for_verification = "re-run the tool yourself (read_file the path, execute_command for the build or diff, compare against expected)"
trust_rule = "do not trust journal claims that lack a corresponding tool call in the timeline; flag as unsound method"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Do not replay executor commands as verification.

Line 60 requires the reviewer to rerun executor tool inputs. Line 134 repeats this rule. An audit log can contain mutating commands, destructive paths, credentials, or network calls. Blind replay can change task state or expose secrets.

Restrict verification to reviewer-authored, allow-listed checks. Do not replay arbitrary executor inputs.

Proposed safer wording
-required_for_verification = "re-run the tool yourself (read_file the path, execute_command for the build or diff, compare against expected)"
+required_for_verification = "re-run only safe, reviewer-authored checks; never replay arbitrary executor inputs"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
[timeline.tool_output_preservation]
current_execution = "your full tool inputs + outputs (working memory)"
past_executions = "input only; tagged [output not preserved in timeline view — re-run this tool yourself if you need the content]"
required_for_verification = "re-run the tool yourself (read_file the path, bash the test/build, diff against expected)"
required_for_verification = "re-run the tool yourself (read_file the path, execute_command for the build or diff, compare against expected)"
trust_rule = "do not trust journal claims that lack a corresponding tool call in the timeline; flag as unsound method"
[timeline.tool_output_preservation]
current_execution = "your full tool inputs + outputs (working memory)"
past_executions = "input only; tagged [output not preserved in timeline view — re-run this tool yourself if you need the content]"
required_for_verification = "re-run only safe, reviewer-authored checks; never replay arbitrary executor inputs"
trust_rule = "do not trust journal claims that lack a corresponding tool call in the timeline; flag as unsound method"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@templatekit/src/prompts/reviewer_system.md` around lines 57 - 61, Update the
[timeline.tool_output_preservation] verification guidance to prohibit replaying
arbitrary executor tool inputs, including mutating, destructive,
credential-bearing, or network commands. Replace the current rerun requirement
with reviewer-authored, explicitly allow-listed checks, and update the repeated
rule at the referenced verification guidance to match.

Comment on lines +230 to +232
only_exits = "the run ends ONLY through `terminate_loop`{{#if resources.enabled_tools.ask_user}}, `ask_user`{{/if}}{{#if resources.enabled_tools.abort_task}}, or `abort_task`{{/if}}{{#if resources.enabled_tools.notify_user}}, `notify_user`{{/if}}; nothing else stops the loop"
terminate_loop = "for service and coordinator work, terminate_loop must be the only tool call in the response and must carry a concrete summary"
abort_task = "last resort for a genuinely stuck, impossible, or cancelled execution; do not use it for ordinary difficulty or a recoverable block"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Scope only_exits to non-conversation threads.

Line 230 says a run ends only through a tool. Lines 301-302 define plain text with no tool calls as the required completion for conversation threads. The unqualified rule gives conversation threads two termination contracts.

Make only_exits conditional on non-conversation threads, or include the plain-reply exception in the same rule.

Proposed wording
-only_exits = "the run ends ONLY through `terminate_loop`..."
+only_exits = "for non-conversation threads, the run ends ONLY through `terminate_loop`..."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@templatekit/src/prompts/shared_operating_style.md` around lines 230 - 232,
Update the only_exits template rule so it applies only to non-conversation
threads, or explicitly permits the required plain-text/no-tool completion for
conversation threads. Preserve the existing terminate_loop, ask_user,
abort_task, and notify_user conditions for non-conversation runs.

Comment on lines +18 to +21
- Read `/task/TASK.md` before acting.
- Append durable progress and handoff notes to `/task/JOURNAL.md`.
- Put deliverables in `/task/artifacts/` unless the brief names another mount.
- Use `/task/audit/` and `/task/review/` for evidence or review material when appropriate.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n -C 8 \
  'task_workspace_brief|delegated_execution_system|/task/artifacts/|/delegated_workspace/|/task/audit/|thread\.purpose' \
  templatekit agent-engine || true

Repository: wacht-platform/platform

Length of output: 50379


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== template files =="
for f in templatekit/src/templates/task_workspace_brief.hbs templatekit/src/templates/worker_assignment_execution_context.hbs templatekit/src/prompts/delegated_execution_system.md templatekit/src/prompts/service_execution_system.md templatekit/src/prompts/reviewer_system.md templatekit/src/prompts/sandbox_environment.md; do
  echo "--- $f"
  sed -n '1,220p' "$f"
done

echo "== prompt loader relevant section =="
sed -n '1,90p' templatekit/src/prompt_loader.rs

echo "== system prompt selection relevant section =="
sed -n '600,635p' agent-engine/src/executor/core.rs

echo "== call sites for task_workspace_brief rendering =="
rg -n -C 4 'TASK_WORKSPACE_BRIEF|task_workspace_brief|render.*task_workspace|render_template_with_prompt\([^)]*TASK_WORKSPACE_BRIEF' agent-engine templatekit || true

echo "== task creation/delegation fields around delegate =="
rg -n -C 4 'delegate_task|is_delegated|delegated_workspace|delegated_inputs|assignment_role|task_workspace_brief|templatekit' agent-engine src packages templates || true

Repository: wacht-platform/platform

Length of output: 50380


Clarify /task/audit/ as runtime-owned and read-only.

/task/audit/ is only mentioned as “evidence or review material” in the generic brief. It is a per-lane runtime tool-call log written by the runner and should not be used for agent-made evidence or review files.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@templatekit/src/templates/task_workspace_brief.hbs` around lines 18 - 21,
Update the task workspace instructions in task_workspace_brief.hbs to explicitly
identify /task/audit/ as a runtime-owned, read-only tool-call log that agents
must not write evidence or review files into; direct agent-created evidence and
review material to the appropriate writable location, while preserving the
existing /task/review/ guidance.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@agent-engine/src/executor/agent_loop/mod.rs`:
- Around line 1459-1486: Update the non-adjacent cycle detection loop in the
recent_tool_call_signatures scan to iterate earlier entries from newest to
oldest, excluding the current entry. Preserve the existing cycle_len calculation
and signal behavior so the first match reports the nearest recurrence and
tightest cycle period.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 28d1a5bb-73c0-497e-9436-5986cc87eebf

📥 Commits

Reviewing files that changed from the base of the PR and between 1eadbc4 and fda714c.

📒 Files selected for processing (6)
  • agent-engine/src/executor/agent_loop/mod.rs
  • agent-engine/src/executor/core.rs
  • templatekit/src/prompts/coordinator_system.md
  • templatekit/src/prompts/delegated_execution_system.md
  • templatekit/src/prompts/service_execution_system.md
  • templatekit/src/prompts/shared_operating_style.md
🚧 Files skipped from review as they are similar to previous changes (3)
  • templatekit/src/prompts/delegated_execution_system.md
  • templatekit/src/prompts/coordinator_system.md
  • templatekit/src/prompts/service_execution_system.md

Comment on lines +1459 to +1486
// --- Non-adjacent cycle detection: bounded window of recent signatures ---
// Push the current signature into the bounded window. Then scan the window
// (excluding the most-recent entry, which is the adjacent case above) for
// any earlier occurrence of the same signature. This catches alternating
// patterns like A→B→A→B or A→B→C→A without firing when the agent is
// making genuine progress (different arguments ⇒ different signature).
const TOOL_CALL_CYCLE_WINDOW: usize = 6;
self.recent_tool_call_signatures
.push_back(signature.clone());
while self.recent_tool_call_signatures.len() > TOOL_CALL_CYCLE_WINDOW {
self.recent_tool_call_signatures.pop_front();
}
if self.repeated_tool_call_count < 2 {
// The adjacent case already fires at count >= 2; only scan the wider
// window when the adjacent check did NOT trigger.
let window_len = self.recent_tool_call_signatures.len();
if window_len >= 3 {
// Check all entries except the last one (current turn).
for earlier_idx in 0..window_len - 1 {
if self.recent_tool_call_signatures[earlier_idx] == signature {
// Compute cycle window length (turns since first occurrence).
let cycle_len = window_len - earlier_idx;
self.signal(core::RuntimeSignal::ToolCallCycle { cycle_len });
break;
}
}
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Scan the recent-signature window from newest to oldest.

The loop at Line 1477 scans earlier_idx from 0 (oldest) to window_len - 2 (nearest to current) and stops at the first match. For a period-2 alternating pattern such as A, B, A, B, A, this reports the earliest occurrence of A (index 0), producing cycle_len = 5, even though A actually recurred just 2 turns earlier (index 2). The message "the same tool-call signature recurred after {cycle_len} turns" then overstates how far back the recurrence is, understating how tight the loop actually is.

Scan the window in reverse so the nearest match is found first. This reports the tightest recurrence, which better reflects the actual cycle period and gives the model a more accurate corrective signal.

🔁 Proposed fix to scan from most recent to oldest
             if window_len >= 3 {
-                // Check all entries except the last one (current turn).
-                for earlier_idx in 0..window_len - 1 {
+                // Check all entries except the last one (current turn), scanning
+                // from most recent to oldest so the reported cycle length is the
+                // tightest recurrence rather than the earliest one in the window.
+                for earlier_idx in (0..window_len - 1).rev() {
                     if self.recent_tool_call_signatures[earlier_idx] == signature {
                         // Compute cycle window length (turns since first occurrence).
                         let cycle_len = window_len - earlier_idx;
                         self.signal(core::RuntimeSignal::ToolCallCycle { cycle_len });
                         break;
                     }
                 }
             }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// --- Non-adjacent cycle detection: bounded window of recent signatures ---
// Push the current signature into the bounded window. Then scan the window
// (excluding the most-recent entry, which is the adjacent case above) for
// any earlier occurrence of the same signature. This catches alternating
// patterns like A→B→A→B or A→B→C→A without firing when the agent is
// making genuine progress (different arguments ⇒ different signature).
const TOOL_CALL_CYCLE_WINDOW: usize = 6;
self.recent_tool_call_signatures
.push_back(signature.clone());
while self.recent_tool_call_signatures.len() > TOOL_CALL_CYCLE_WINDOW {
self.recent_tool_call_signatures.pop_front();
}
if self.repeated_tool_call_count < 2 {
// The adjacent case already fires at count >= 2; only scan the wider
// window when the adjacent check did NOT trigger.
let window_len = self.recent_tool_call_signatures.len();
if window_len >= 3 {
// Check all entries except the last one (current turn).
for earlier_idx in 0..window_len - 1 {
if self.recent_tool_call_signatures[earlier_idx] == signature {
// Compute cycle window length (turns since first occurrence).
let cycle_len = window_len - earlier_idx;
self.signal(core::RuntimeSignal::ToolCallCycle { cycle_len });
break;
}
}
}
}
// --- Non-adjacent cycle detection: bounded window of recent signatures ---
// Push the current signature into the bounded window. Then scan the window
// (excluding the most-recent entry, which is the adjacent case above) for
// any earlier occurrence of the same signature. This catches alternating
// patterns like A→B→A→B or A→B→C→A without firing when the agent is
// making genuine progress (different arguments ⇒ different signature).
const TOOL_CALL_CYCLE_WINDOW: usize = 6;
self.recent_tool_call_signatures
.push_back(signature.clone());
while self.recent_tool_call_signatures.len() > TOOL_CALL_CYCLE_WINDOW {
self.recent_tool_call_signatures.pop_front();
}
if self.repeated_tool_call_count < 2 {
// The adjacent case already fires at count >= 2; only scan the wider
// window when the adjacent check did NOT trigger.
let window_len = self.recent_tool_call_signatures.len();
if window_len >= 3 {
// Check all entries except the last one (current turn), scanning
// from most recent to oldest so the reported cycle length is the
// tightest recurrence rather than the earliest one in the window.
for earlier_idx in (0..window_len - 1).rev() {
if self.recent_tool_call_signatures[earlier_idx] == signature {
// Compute cycle window length (turns since first occurrence).
let cycle_len = window_len - earlier_idx;
self.signal(core::RuntimeSignal::ToolCallCycle { cycle_len });
break;
}
}
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@agent-engine/src/executor/agent_loop/mod.rs` around lines 1459 - 1486, Update
the non-adjacent cycle detection loop in the recent_tool_call_signatures scan to
iterate earlier entries from newest to oldest, excluding the current entry.
Preserve the existing cycle_len calculation and signal behavior so the first
match reports the nearest recurrence and tightest cycle period.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant