From 7d18b4cfa373126473bf0fa017322aa5be80c013 Mon Sep 17 00:00:00 2001 From: sixvolts Date: Wed, 2 Sep 2026 16:41:31 +0000 Subject: [PATCH] fix(pipeline): raise turnHardCap 600s -> 1800s Canyon lost a turn to the hard cap at exactly 10m0.002s: six iterations deep (read_page x4, update_page x2 over 10k-char pages), killed on iteration 5 of 10. 600s was chosen when chat ran on mainframe. Since that box was retired furnace carries chat AND research on the same six slots, decoding at roughly 33-43 tok/s, so one iteration writing ~2.3k tokens costs 55-67s and six iterations simply does not fit. Two wikiknowledge extracts in the same turn also failed with context deadline exceeded, which is the backend saying it is saturated. The failure mode is worse than a timeout. Tool side effects commit as each tool runs, but the transcript is only persisted at end of turn - so both page updates landed, with revisions at 01:44:32 and 01:49:25, while the user saw an empty reply and reasonably concluded nothing had happened. Work done, no record of it. This is a stopgap and the comment says so. A fixed ceiling cannot tell a turn doing steady useful work from one wedged on a hung backend; the right shape is an idle watchdog that resets on progress (each completed iteration or tool dispatch), killing a stuck turn in ~2min while letting a productive one run as long as it keeps earning it. Until that exists, err long: a killed turn loses the transcript but keeps the side effects, which is the confusing outcome. Not addressed here: persisting partial turn state on cap-cancellation, which would make even a killed turn leave a record. --- .../internal/pipeline/pipeline.go | 22 ++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/familiar-gateway/internal/pipeline/pipeline.go b/familiar-gateway/internal/pipeline/pipeline.go index e9b0dbd..d39a86b 100644 --- a/familiar-gateway/internal/pipeline/pipeline.go +++ b/familiar-gateway/internal/pipeline/pipeline.go @@ -271,7 +271,27 @@ func (p *Pipeline) TurnRunning(sessID string) bool { // turnHardCap bounds one turn's total generation + tool work once it's // detached from the request context. Matches the per-LLM-call ceiling; // a turn that blows it is a stuck model, not a slow one. -const turnHardCap = 600 * time.Second +// Raised 600s -> 1800s on 2026-09-02. 600 was chosen when chat ran on +// mainframe; since that box was retired, furnace carries chat AND research on +// the same six slots and decodes at roughly 33-43 tok/s, so a single tool +// iteration writing a 2.3k-token page costs 55-67s. A real turn of Canyon's +// (read_page x4, update_page x2 over 10k-char pages) was guillotined at +// exactly 10m0.002s on iteration 5 of 10. +// +// That failure is worse than it looks: the tool side effects had already +// committed (both page updates landed, with revisions) while the transcript is +// only persisted at end of turn, so the user saw an empty reply and reasonably +// concluded nothing had happened — while their pages had in fact been +// rewritten. +// +// A fixed ceiling is the wrong shape for this and 1800s is a stopgap, not a +// fix: it still cannot distinguish a turn doing steady useful work from one +// wedged on a hung backend, and the honest answer is an idle watchdog that +// resets on progress (each completed iteration or tool dispatch), so a stuck +// turn dies in ~2min while a productive one runs as long as it keeps earning +// it. Until then, prefer erring long — a killed turn loses the transcript but +// keeps the side effects, which is the confusing outcome. +const turnHardCap = 1800 * time.Second // SetLifetime wires the gateway's root (shutdown) context. Call once at // startup, before serving. It's the cancellation source for detached