Repository navigation
Console API calls from the sandbox get their own passthrough limits instead of internet-fetch limits (OPT-1586) - #23
Merged
Conversation
…r-run budget (OPT-1586)
A passthrough host may carry httpsPassthroughLimits {headersTimeoutMs,
maxFetchesPerGrant, maxAggregateBytesPerGrant}. It applies when the signed
policy asks for it and the deployment grants it (smaller value of each);
passthrough requests to that host then wait for headers up to its own
timeout and are metered in a per-host ledger budget, apart from the
grant-wide external-fetch budget. Internet fetches keep the frozen limits.
Policies without the field verify and behave exactly as before.
… than the grant fetch budget (OPT-1586 review)
…led ledger, not trailers (Bun 1.3.14 client exposes none)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Engine half of OPT-1586. Console API traffic from the sandbox gets its own passthrough limits (host field
httpsPassthroughLimits, granted by the deployment policy and requested by the Console-signed policy, smaller of each), metered in its own per-host budget; internet fetches keep the current limits.Files.
Why. OPT-1586 (Figent agent stalls F2, spec /home/thor/optale-artifacts/FIGENT-AGENT-STALLS-2026-10-13/diagnosis-spec.md RC2):
optale-agent tool callfrom code leaves through the sandbox egress gateway, which held calls to our own Console API to internet-fetch limits (5 s to headers, 8 fetches per run). Figent on 2026-10-13: 27 FETCH_TIMEOUT and 32 FETCH_BUDGET_EXCEEDED.Production engine today (read-only on AX41). Checked read-only on AX41 (65.109.115.37) over ssh with ~/.ssh/optale-ci-microvm-proof. api, worker, egress_gateway and file_server run code-interpreter e9ad47c (main before #20), created 2026-10-04; sandbox-runner is 0b2e6a5. The mounted policy /etc/optale/code-interpreter/external-fetch-policy.v2.json has global limits of 5 s to headers, 15 s total and 8 fetches per run. console.optale.com, figent.optale.com and console-staging.optale.com are HTTPS passthrough hosts with a 300 s total, 8 fetches and 2 MiB per response. A passthrough call therefore waited at most 5 s for headers and shared the 8-fetch budget with internet fetches; that is RC2 as the spec describes it. A separate limit also bites: api and worker run with EXECUTION_MANIFEST_MAX_REQUESTS=100, which caps all gateway requests in one code run.
Root cause. service/src/external-fetch-policy.ts:82-91 (origin/main) sets HARD_EXTERNAL_FETCH_LIMITS to 5 s headers, 15 s total and 8 fetches per run. Every host's limits were capped at the global limits (parseLimits, intersectExternalFetchPolicies), so a passthrough host could never raise them. service/src/external-fetch.ts:399 waited for passthrough headers with target.policy.limits.headersTimeoutMs, which is 5 s. service/src/egress-gateway.ts:1913-1916 and service/src/egress-ledger.ts:353-363 consumed the single run-wide fetch_count, shared with internet fetches. On the Console side, ops/optale-agent/transport.cjs:137-143 (main) printed the gateway's raw JSON {"error":"FETCH_TIMEOUT","message":"External fetch timed out"}.
Real-stack proof (KS-7). KS-7 through seat-heavy, run3, 2026-10-13 18:04-18:22Z. Evidence in /home/thor/optale-artifacts/foundation/OPT-1586/ (README.md, scripts/, evidence/run3/).
Not proven.
Risks and rollout order.