fix(llm): populate usage for the Claude Code local-agent path - #140
Conversation
LocalAgentAdapter.chat() never set `usage` on its LLMResponse, so AgentLoop's token budget check (tokensUsed >= maxTokens) silently never fired for Claude Code missions. maxIterations was the only brake, and it bounds turn count, not actual spend. Claude Code's --output-format json (instead of text) returns exact per-call token counts, including prompt-cache creation/read tokens that a text-length estimate has no way to see. localAgentChat now requests that format for the claude agent id only. LocalAgentAdapter parses the envelope into usage.promptTokens/completionTokens, with a character-based estimate as a fallback if parsing fails, so usage never goes back to undefined. Verified directly against the CLI: a live run now reports promptTokens 24458, completionTokens 1662 instead of undefined/0. Full suite: 675/675, no regressions. Fixes elder-plinius#139
|
Maintainer courtesy cleanup pushed as I kept the change narrow:
Verification on a synthetic merge into current |
jmagly
left a comment
There was a problem hiding this comment.
Reviewed at 11e3d4c884e05931e8728a427a6333888da86513.
No blocking findings after maintainer-courtesy cleanup. Real Claude JSON usage is aggregated correctly; absent or invalid usage retains the parsed result and fails over to a non-zero estimate; non-Claude paths remain unchanged. Behavioral regression coverage exercises the reported budget-accounting boundary and JSON-wrapped tool calls.
Verification: hosted CI passed all gates; synthetic merge into current main passed typecheck, focused 33/33, full 748/748 Vitest plus ops/model checks, and lint with zero errors.
Fixes #139.
LocalAgentAdapter.chat()never setusageon itsLLMResponsefor the Claude Code local-agent path, soAgentLoop's per-task token budget check always compared against zero and never fired.localAgentChat()now requests--output-format jsonfor theclaudeagent id only;LocalAgentAdapterparses the real envelope intousage.promptTokens/completionTokens, with a character-based estimate as a fallback if parsing fails. Codex and Hermes are untouched.Contribution Receipt
--output-format json) soAgentLoop's token budget check can fire.npm run typecheck-> passnpm test-> pass (675/675 vitest, 11/11 ops-preflight, 19/19 model-matrix)npm run doctor-> pass (30/33, 3 warnings: semgrep/promptfoo missing, API health offline with the server not running — no blockers)npm run verify-claims-> pass (27/27, no headline numbers changed)claude -p --output-format jsoncalls confirming the envelope shape.claudeonly). The character-based fallback estimate (used only if the JSON envelope fails to parse) is a rough approximation, not exact.agentFailureOutput's plain-text failure detection inlocal-agents.tsis not updated for JSON-wrapped error envelopes (pre-existing behavior, unchanged here).