fix: 6 infrastructure fixes (LLM retry, cache, token counting, tests) - #166
Merged
Conversation
…paction overwrite
Includes 14 tests covering Config defaults, state management, history restore, agent registration, delegation error handling, and sequential pipeline ordering. Uses stub provider classes to avoid real LLM calls.
Provider layer (openai_provider.cpp, anthropic_provider.cpp) already handles all transient failures with 3 attempts and exponential backoff. The AgentLoop retry was redundant and introduced two bugs: 1. Duplicate stream chunks: the failed attempt already emitted partial text via the on_chunk callback (flushed to UI). The retry re-emitted the same content from the start. 2. Response.text corruption: on final failure, response.text contained debris from the last failed attempt.
…vider Provider layer handles all transient errors (5xx, 429, curl errors) with 3 attempts and exponential backoff. AgentLoop now only catches for logging before propagating the exception upstream. Context-window retry was also removed: all callers (fork_skill_tool, worldbuilding_tools) already catch exceptions from sub_loop.run() and return errors as tool results. The heuristic-based compact+retry was unreliable — if token counting underestimates, compaction won't help either.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix 6 infrastructure issues across LLM Provider, Agent Loop, Token Counter, and SubAgentRunner subsystems.
Closes #1 — LLM HTTP Retry (exponential backoff)
Closes #2 — Apply CacheAwareContext::split() Result
Closes #6 — Token Counting: Hybrid Running Count
Closes #9 — Separate System Prompt from session_history_
Closes #10 — fan_out() Parallelism Hard Cap
Closes #4 — Test Coverage
Changes
#1 — LLM HTTP Retry (exponential backoff)
RetryConfigstruct tollm_provider.hppopenai_provider.cppandanthropic_provider.cppwith retry loops#2 — Apply CacheAwareContext::split() Result
split()result was only debug-logged, never applied#6 — Token Counting: Hybrid Running Count
update_authoritative()toTokenCounter, tracking APIusage.prompt_tokensTokenCounterin AgentLoop, reset on history restore#9 — Separate System Prompt from session_history_
system_prompt_andcompaction_summaries_members to AgentLoopset_system_prompt()build_context()uses effective system prompt with fallback, prepends summaries#10 — fan_out() Parallelism Hard Cap
std::asyncspawning with batched processingmin(4, hardware_concurrency), preventing thread exhaustion#4 — Test Coverage
test_openai_provider.cpp: 4 tests (build_messages, build_tools)test_anthropic_provider.cpp: 5 tests (cache_control, tool_use, thinking config)test_agent_loop.cpp: 14 tests (config defaults, state management, system prompt isolation)test_sub_agent_runner.cpp: 5 tests (register, delegate, sequential, fan_out)Files Changed
Test Plan
prompts)