Skip to content

feat(client): configurable model-call retries, deeper by default - #313

Open
lawrencechen98 wants to merge 3 commits into
mainfrom
lawrence/n2-model-call-retries
Open

feat(client): configurable model-call retries, deeper by default#313
lawrencechen98 wants to merge 3 commits into
mainfrom
lawrence/n2-model-call-retries

Conversation

@lawrencechen98

@lawrencechen98 lawrencechen98 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Why

Navigator model calls go through the bundled OpenAI client, which retries connection errors, timeouts and 429/5xx with exponential backoff. Two things were wrong for agent workloads:

  • the depth was the vendor default of 2 (under ~2s of total backoff), and
  • max_retries was not reachable from the SDK surface at all — the chat namespace's docstring even said so.

A long-horizon agent run issues hundreds of sequential model calls, so a single unretried upstream blip ends the whole run.

Measured on 2026-08-31: three ~2-minute bursts of gateway {'type': 'upstream_error'} 5xx cost an OSWorld-V2 eval 20 of its 108 tasks, every one dying on its first attempt.

What

  • config.DEFAULT_MAX_RETRIES = 4, documented with the reasoning above.
  • max_retries on YutoriClient and AsyncYutoriClient, threaded into the chat namespace's OpenAI client.
  • Docstrings on both chat namespaces corrected — they claimed the depth "is not configurable through the SDK surface".
  • Tests pin the default, an override, and max_retries=0.

Retried requests are idempotent, so the ceiling is caller patience, not correctness; unattended batch work can raise it further.

Test

715 passed, 3 skipped — full suite. ruff check + format clean on the touched files.


Note

Low Risk
Retry behavior changes default resilience for Navigator chat only; retried completions are documented as idempotent. N2 exposes a read-only request builder without changing loop semantics.

Overview
Navigator client.chat model calls now use a SDK default of 4 retries (via DEFAULT_MAX_RETRIES) instead of the bundled OpenAI client’s 2, with exponential backoff on transient failures. max_retries is exposed on YutoriClient and AsyncYutoriClient and passed into the lazy chat namespace; callers can raise it for long batch runs or set 0 to disable retries. Chat namespace docstrings no longer claim retry depth is fixed.

N2ComputerAgent.completion_request() is added so harnesses can obtain the same kwargs the loop would send (windowed messages, sampling, prev_request_id) plus optional extra chat messages—e.g. a step-cap “stop and summarize”—without advancing the loop or mutating trajectory. The internal _predict_step path now builds requests through completion_request(items=...).

Tests cover default/override/disable retries and the public completion_request surface.

Reviewed by Cursor Bugbot for commit 7a5e49b. Bugbot is set up for automated code reviews on this repo. Configure here.

lawrencechen98 and others added 3 commits August 30, 2026 07:14
…p turns

Returns the actor's exact next Chat Completions request (system prompt, windowed
messages, sampling fields, request chaining), optionally with extra chat messages
appended, without advancing the loop or mutating the trajectory. Lets a harness
implement conventions like a step-cap stop-and-summarize probe on public surface
instead of reaching into _prepare_completion_messages/_resolve_completions.
…_request()

_predict_step now builds its api_kwargs via the public helper (items override),
keeping one assembly path for messages, sampling fields, the historical
completion_kwargs merge order, and request chaining.
Navigator model calls go through the bundled OpenAI client, which retries
connection errors, timeouts and 429/5xx with exponential backoff. Two things were
wrong for agent workloads: the depth was the vendor default of 2 (under ~2s of
backoff in total) and it was not reachable from the SDK surface at all.

A long-horizon agent run issues hundreds of sequential model calls, so one
unretried upstream blip ends the whole run. Measured on 2026-08-31: three ~2-minute
bursts of gateway upstream_error 5xx cost an eval 20 of 108 tasks, each dying on
its first attempt.

Adds max_retries to both clients (default 4, from config.DEFAULT_MAX_RETRIES),
threaded into the chat namespace's client. Retried requests are idempotent, so the
ceiling is caller patience; unattended batch work can raise it further.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QQn7mTTxR8tBuK7G4SSiKE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant