Skip to content

Bound tool image requests and clarify oversized payload errors - #1989

Open
lihongguang-0014 wants to merge 7 commits into
mainfrom
fix/bound-image-history-requests
Open

lihongguang-0014 wants to merge 7 commits into
mainfrom
fix/bound-image-history-requests

Conversation

@lihongguang-0014

@lihongguang-0014 lihongguang-0014 commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

  • Prepare tool-produced images before delivery (2048 px long edge, 1.5 MiB base64 each) and bound each model request to the four most recent tool images (6 MiB aggregate). State omitted-frame counts in the request view while preserving the stored session and legacy replay compatibility.
  • Guard the final provider JSON request body at 24 MiB by default, distinguish local pre-dispatch rejection from upstream HTTP 413, and show actionable errors in the Web UI and terminal. The limit is configurable with OPENSQUILLA_PROVIDER_REQUEST_MAX_BODY_BYTES.
  • Keep native-image acknowledgment tied to a successfully accepted physical image request. Exclude locally rejected, unsent requests from billable or unknown provider usage while retaining an audit entry.
  • Preserve vision routing for unseen tool media and existing image attachments.

Target and issue

  • Target: main (verified against freshly fetched origin/main).
  • Linked issue: None.

Release note

Yes. Oversized image requests now carry a bounded set of recent tool images, and HTTP 413 reports a clear request-size explanation. Gateway operators can adjust the local body guard through the environment variable above.

Tests

  • Focused Python image, routing, provider, terminal, and usage suite: 644 passed; final request-context regression: 27 passed. CI regression files, including a 12-frame text-to-vision route with a constrained candidate window: 203 passed. Ruff and mypy (1,643 source files) passed.
  • Web UI production build and i18n/architecture guards passed. A full unit sweep reached 11,754 passed, two test timeouts, and nine worker-start errors under concurrent load; isolated rerun of both affected files passed 44/44.
  • Desktop local-build contract tests: 24 passed. Python wheel and Gateway built. The latest macOS app passed verify:prepared, local signing, verify:package, and deep strict code-signature verification. The installed app started with the existing user profile, loaded the affected chat, and passed native sidebar collapse/reopen and model-route-menu checks.
  • Isolated sidebar exercise: text routed to a lightweight model; attaching an image and following up routed to a vision model with one image on the wire; synthetic upstream HTTP 413 displayed the request-size error and unknown-usage status.
  • Latest full CI run completed successfully: 146 jobs passed, two were skipped, and none failed. This includes Windows high-risk SQLite shards, packaged first-interaction (40 synthetic sends, renderer checks, and natural Gateway Quit), and the dependency audit. This change modifies no dependency manifests or lock files.

Safety and compatibility

  • Tool-image reduction is request-only. Stored transcripts and public RPC contracts remain compatible; old image history is covered by regression tests.
  • A real upstream provider may impose a body limit below the local 24 MiB guard, so it can still return HTTP 413. Omitted frames must be requested again when their detail matters.
  • Tests and browser exercises used synthetic media and prompts. No real session data, keys, or machine-specific paths are included. The logic is platform-neutral; local macOS delivery uses the maintained stable-signing build path.

Third-party origin

Read-only reference research informed the behavior and test shape. The implementation, documentation, and fixtures were written independently; no reference code or assets were copied.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant