feat(client): configurable model-call retries, deeper by default - #313
Open
lawrencechen98 wants to merge 3 commits into
Open
feat(client): configurable model-call retries, deeper by default#313lawrencechen98 wants to merge 3 commits into
lawrencechen98 wants to merge 3 commits into
Conversation
…p turns Returns the actor's exact next Chat Completions request (system prompt, windowed messages, sampling fields, request chaining), optionally with extra chat messages appended, without advancing the loop or mutating the trajectory. Lets a harness implement conventions like a step-cap stop-and-summarize probe on public surface instead of reaching into _prepare_completion_messages/_resolve_completions.
…_request() _predict_step now builds its api_kwargs via the public helper (items override), keeping one assembly path for messages, sampling fields, the historical completion_kwargs merge order, and request chaining.
Navigator model calls go through the bundled OpenAI client, which retries connection errors, timeouts and 429/5xx with exponential backoff. Two things were wrong for agent workloads: the depth was the vendor default of 2 (under ~2s of backoff in total) and it was not reachable from the SDK surface at all. A long-horizon agent run issues hundreds of sequential model calls, so one unretried upstream blip ends the whole run. Measured on 2026-08-31: three ~2-minute bursts of gateway upstream_error 5xx cost an eval 20 of 108 tasks, each dying on its first attempt. Adds max_retries to both clients (default 4, from config.DEFAULT_MAX_RETRIES), threaded into the chat namespace's client. Retried requests are idempotent, so the ceiling is caller patience; unattended batch work can raise it further. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QQn7mTTxR8tBuK7G4SSiKE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Navigator model calls go through the bundled OpenAI client, which retries connection errors, timeouts and 429/5xx with exponential backoff. Two things were wrong for agent workloads:
max_retrieswas not reachable from the SDK surface at all — the chat namespace's docstring even said so.A long-horizon agent run issues hundreds of sequential model calls, so a single unretried upstream blip ends the whole run.
Measured on 2026-08-31: three ~2-minute bursts of gateway
{'type': 'upstream_error'}5xx cost an OSWorld-V2 eval 20 of its 108 tasks, every one dying on its first attempt.What
config.DEFAULT_MAX_RETRIES = 4, documented with the reasoning above.max_retriesonYutoriClientandAsyncYutoriClient, threaded into the chat namespace's OpenAI client.max_retries=0.Retried requests are idempotent, so the ceiling is caller patience, not correctness; unattended batch work can raise it further.
Test
715 passed, 3 skipped— full suite. ruff check + format clean on the touched files.Note
Low Risk
Retry behavior changes default resilience for Navigator chat only; retried completions are documented as idempotent. N2 exposes a read-only request builder without changing loop semantics.
Overview
Navigator
client.chatmodel calls now use a SDK default of 4 retries (viaDEFAULT_MAX_RETRIES) instead of the bundled OpenAI client’s 2, with exponential backoff on transient failures.max_retriesis exposed onYutoriClientandAsyncYutoriClientand passed into the lazy chat namespace; callers can raise it for long batch runs or set0to disable retries. Chat namespace docstrings no longer claim retry depth is fixed.N2ComputerAgent.completion_request()is added so harnesses can obtain the same kwargs the loop would send (windowed messages, sampling,prev_request_id) plus optional extra chat messages—e.g. a step-cap “stop and summarize”—without advancing the loop or mutatingtrajectory. The internal_predict_steppath now builds requests throughcompletion_request(items=...).Tests cover default/override/disable retries and the public
completion_requestsurface.Reviewed by Cursor Bugbot for commit 7a5e49b. Bugbot is set up for automated code reviews on this repo. Configure here.