Skip to content

feat(core): note an identical call with an identical result to the model - #317

Merged
Max17190 merged 1 commit into
mainfrom
note-repeated-identical-results
Sep 8, 2026
Merged

Max17190 merged 1 commit into
mainfrom
note-repeated-identical-results

Conversation

@Max17190

@Max17190 Max17190 commented Sep 7, 2026 •

Copy link
Copy Markdown
Owner

Why

A model that has finished can fail to stop: its reasoning says the task is complete, its reply is the same verification call again, the output is the same, and the turn runs to the iteration cap. On one full run of the release binary a fifth of all model turns repeated the previous turn's calls and results byte for byte, and one trial in eight ran to the cap that way. #225 removed a veto keyed on identical arguments because arguments say nothing about results and a stuck model is bounded by budgets; that reasoning stands, and this change vetoes nothing.

Summary

  • RepeatTracker hashes each iteration's tool calls and the tool replies they drew. Every call still executes. From the third consecutive byte-identical pair, the tool message the model reads next carries an advisory note with the count that names both ways out: keep waiting, or stop calling tools and answer. A reply with no calls resets the count.
  • The note is appended after the pair is hashed, so it never enters the next comparison, and it lands at the tail of the prompt, so no cached prefix is invalidated.
  • The effect on a model's behavior is not measured here; the tests pin the mechanism (when the note appears, with what count, and that nothing is vetoed), and the next full run measures the rest.

Test Plan

  • New identical_calls_with_identical_results_draw_a_note_from_the_third: five identical read_file calls; all five execute, replies one and two carry no note, three through five carry the note with the right count. Fails on main at the third reply.
  • New identical_calls_with_changing_results_draw_no_note: one bash command with fixed arguments whose output moves; no note on any reply.
  • cargo test --workspace --locked green; cargo +1.97.0 clippy --workspace --all-targets --locked -- -D warnings clean.

Greptile Summary

Summary

  • Adds an advisory message when consecutive tool-call iterations have identical calls and identical results.
  • Keeps execution unchanged and avoids the advisory when repeated calls return different output.
  • Focused and multi-tool behavior checks passed; safe to merge.

Confidence Score: 5/5

Safe to merge: the repeated-call advisory behaves as intended for identical single-tool and multi-tool iterations, while changing results remain unannotated.

No issues were found. Executed checks covered the third-and-later advisory threshold, continued detection after prior advisory text was appended, multi-tool iterations, and changing-output polling.

Files Needing Attention: None.

T-Rex T-Rex Logs

What T-Rex did

  • Verified that repeated single-tool call/result sequences trigger an advisory starting on the third consecutive iteration.
  • Verified with a local streaming endpoint that byte-identical multi-tool call/result iterations receive the advisory on iterations 3 through 5.
  • Confirmed that detecting later iterations remains effective after an earlier advisory has been appended to the transcript.
  • Observed that RepeatTracker::observe runs after tool execution and before append_and_emit_note, preserving detection across later iterations despite the appended note.
  • Validated that the contract behavior aligns with expectations at crates/core/src/agent.rs:1657-1671 and crates/core/src/agent.rs:3489-3491.

View all artifacts

T-Rex Ran code and verified through T-Rex

Reviews (1): Last reviewed commit: "feat(core): note an identical call with ..." | Re-trigger Greptile

A model that has finished can fail to stop: its reasoning says the task is
complete, its reply is the same verification call again, the output is the
same, and the turn runs to the iteration cap. Identical arguments alone
mean nothing (#225), so nothing here vetoes a call and every call executes.
What is tracked is the pair of call and result: from the third consecutive
byte-identical pair, the tool message the model reads next carries an
advisory note with the count that names both ways out, keep waiting or
stop calling tools and answer. A reply with no calls resets the count.
@Max17190
Max17190 merged commit 797bbe5 into main Sep 8, 2026
11 checks passed
@Max17190
Max17190 deleted the note-repeated-identical-results branch September 8, 2026 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant