fix(pipeline): raise turnHardCap 600s -> 1800s - #9
Merged
Conversation
Canyon lost a turn to the hard cap at exactly 10m0.002s: six iterations deep (read_page x4, update_page x2 over 10k-char pages), killed on iteration 5 of 10. 600s was chosen when chat ran on mainframe. Since that box was retired furnace carries chat AND research on the same six slots, decoding at roughly 33-43 tok/s, so one iteration writing ~2.3k tokens costs 55-67s and six iterations simply does not fit. Two wikiknowledge extracts in the same turn also failed with context deadline exceeded, which is the backend saying it is saturated. The failure mode is worse than a timeout. Tool side effects commit as each tool runs, but the transcript is only persisted at end of turn - so both page updates landed, with revisions at 01:44:32 and 01:49:25, while the user saw an empty reply and reasonably concluded nothing had happened. Work done, no record of it. This is a stopgap and the comment says so. A fixed ceiling cannot tell a turn doing steady useful work from one wedged on a hung backend; the right shape is an idle watchdog that resets on progress (each completed iteration or tool dispatch), killing a stuck turn in ~2min while letting a productive one run as long as it keeps earning it. Until that exists, err long: a killed turn loses the transcript but keeps the side effects, which is the confusing outcome. Not addressed here: persisting partial turn state on cap-cancellation, which would make even a killed turn leave a record.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Canyon lost a turn to the hard cap at exactly 10m0.002s: six iterations deep
(read_page x4, update_page x2 over 10k-char pages), killed on iteration 5 of 10.
600s was chosen when chat ran on mainframe. Since that box was retired furnace
carries chat AND research on the same six slots, decoding at roughly 33-43
tok/s, so one iteration writing ~2.3k tokens costs 55-67s and six iterations
simply does not fit. Two wikiknowledge extracts in the same turn also failed
with context deadline exceeded, which is the backend saying it is saturated.
The failure mode is worse than a timeout. Tool side effects commit as each tool
runs, but the transcript is only persisted at end of turn - so both page updates
landed, with revisions at 01:44:32 and 01:49:25, while the user saw an empty
reply and reasonably concluded nothing had happened. Work done, no record of it.
This is a stopgap and the comment says so. A fixed ceiling cannot tell a turn
doing steady useful work from one wedged on a hung backend; the right shape is
an idle watchdog that resets on progress (each completed iteration or tool
dispatch), killing a stuck turn in ~2min while letting a productive one run as
long as it keeps earning it. Until that exists, err long: a killed turn loses
the transcript but keeps the side effects, which is the confusing outcome.
Not addressed here: persisting partial turn state on cap-cancellation, which
would make even a killed turn leave a record.
Opened automatically after the
tierscheck passed on7d18b4cf.The check is attached to this branch head, so this is mergeable now.