What happens
The Codewhale review required check fails on PRs with large diffs, and it fails
without producing a review — so a required gate blocks the merge while telling
the author nothing about their code.
Observed on PR #6281 and PR #6284 (both currently blocked by this, both with
otherwise-green required checks):
Review key: BYOK provider secret (provider: deepseek)
Review limits: 200000 characters per pass, at most 1 passes
Output budget: CLI automatic cap (no override set)
error: Review pass 1/1 incomplete: provider stop reason `length`;
the partial review was not accepted or posted.;
publication: not_attempted; completed review passes: 0/1;
accumulated usage: {"input_tokens":35839,"output_tokens":65536,
"prompt_cache_hit_tokens":640,"prompt_cache_miss_tokens":35199,
"reasoning_tokens":65536}
Run: https://github.com/Hmbown/Codewhale/actions/runs/35069645353
The actual defect
output_tokens: 65536 and reasoning_tokens: 65536 are the same number. The
model spent its entire output allowance on reasoning and emitted zero
review text, then hit length and the action discarded the (empty) partial.
So the budget is being accounted in the wrong unit for a reasoning model: the
cap has to leave room for the answer after reasoning, or reasoning has to be
bounded separately. This is the same shape as the sub-agent hand-back reserve
problem in #6194 — a reserve sized as if only the visible output costs
anything.
Two independent things are wrong:
- It fails closed with no signal. A gate that blocks a merge must say why
the code is wrong. "The reviewer ran out of budget" is the gate's problem,
not the author's, and it should not read as a failed review. Either surface
it as a distinct neutral/skipped outcome, or retry with a reduced scope.
- One pass, no degradation path.
at most 1 passes with a 200k-character
input and no chunking means the gate gets less reliable exactly as a PR
gets bigger — the case where review matters most.
Acceptance criteria
Notes
Copilot and Codex reviewers on the same PRs are also unavailable (quota), so
this gate is currently the only automated reviewer, which raises the cost of it
failing closed.
What happens
The
Codewhale reviewrequired check fails on PRs with large diffs, and it failswithout producing a review — so a required gate blocks the merge while telling
the author nothing about their code.
Observed on PR #6281 and PR #6284 (both currently blocked by this, both with
otherwise-green required checks):
Run: https://github.com/Hmbown/Codewhale/actions/runs/35069645353
The actual defect
output_tokens: 65536andreasoning_tokens: 65536are the same number. Themodel spent its entire output allowance on reasoning and emitted zero
review text, then hit
lengthand the action discarded the (empty) partial.So the budget is being accounted in the wrong unit for a reasoning model: the
cap has to leave room for the answer after reasoning, or reasoning has to be
bounded separately. This is the same shape as the sub-agent hand-back reserve
problem in #6194 — a reserve sized as if only the visible output costs
anything.
Two independent things are wrong:
the code is wrong. "The reviewer ran out of budget" is the gate's problem,
not the author's, and it should not read as a failed review. Either surface
it as a distinct neutral/skipped outcome, or retry with a reduced scope.
at most 1 passeswith a 200k-characterinput and no chunking means the gate gets less reliable exactly as a PR
gets bigger — the case where review matters most.
Acceptance criteria
rather than letting reasoning consume the whole cap; the reserve is sized
off the model's reasoning behaviour, not a flat constant
from "this PR failed review", and does not silently block a merge with no
findings
files and says what it skipped) instead of returning nothing
Notes
Copilot and Codex reviewers on the same PRs are also unavailable (quota), so
this gate is currently the only automated reviewer, which raises the cost of it
failing closed.