Provider Retry Proxy

An OMP/Pi extension that runs a local HTTP retry proxy for LLM provider requests, with configurable per-provider triangular backoff and infinite retry support.

Designed for scenarios where model provider resources are scarce and frequently return 503/429/529 errors — the proxy aggressively retries with a short, capped delay to seize the provider's available capacity window.

How it works

OMP agent ──→ 127.0.0.1:7878/<provider>/<path> ──→ Retry Proxy ──→ upstream provider API
                    │
                    ├─ Transparent passthrough (SSE streaming, headers, body)
                    ├─ Triangular backoff retry on 429/502/503/504/529
                    ├─ Network error retry (ETIMEDOUT, ECONNRESET, terminated, ...)
                    ├─ Permanent error exclusion (daily/monthly quota, billing, context-length)
                    └─ Infinite retry mode (maxAttempts: -1, 30-min elapsed guard)

The proxy runs in-process within one OMP session, shared by all concurrent sessions:

• Multiple concurrent OMP sessions share the same proxy instance (first session to start owns it).
• When the owning session exits, other sessions detect the proxy is gone via turn_start and restart it in-process — no timing gap.
• A 2s background health probe also catches proxy death while a session is idle.
• No external runtime dependency; the proxy lives inside the OMP process.

Triangular backoff

The delay formula (ported from GCMP, MIT):

delay = min(initialDelayMs * triangular(n), maxDelayMs)
where triangular(n) = n * (n + 1) / 2

With initialDelayMs: 20, maxDelayMs: 50:

Attempt	Calculation	Delay
1	20 * 1	20ms
2	20 * 3 = 60 -> cap	50ms
3	20 * 6 = 120 -> cap	50ms
4+	-> cap	50ms

From the 2nd retry onward, the delay is a constant 50ms — fast enough to seize a capacity window without hammering.

Error classification

The proxy retries the following error types:

Category	Signals
Rate limit	HTTP 429/529, rate_limit_exceeded, too_many_requests, quota_exceeded, resource_exhausted, throttled
Server error	HTTP 502/503/504, EngineInternalError, "service unavailable", "system is busy"
Network error	terminated, ETIMEDOUT, ECONNRESET, ECONNREFUSED, fetch failed, socket hang up, EOF

Permanent errors are excluded even if they carry a 429 status code:

• Daily/monthly quota exhaustion ("per day", "daily quota")
• Billing/plan issues ("billing", "upgrade your plan")
• Context length exceeded ("context length", "prompt too long")

Installation

Via omp plugin link (local development)

cd /path/to/provider_retry_proxy
bun build src/index.ts --outdir dist --target bun
bun build src/proxy.ts --outdir dist --target bun
omp plugin link .

Via npm (when published)

pi install npm:provider-retry-proxy

Configuration

1. Proxy config — ~/.omp/agent/retry-proxy.json

Copy retry-proxy.example.jsonc to ~/.omp/agent/retry-proxy.json and fill in your provider's upstream URL:

cp retry-proxy.example.jsonc ~/.omp/agent/retry-proxy.json

2. OMP models config — ~/.omp/agent/models.yml

Change the provider's baseUrl to point at the proxy, prefixing the path with the provider key:

providers:
  xfyun-anthropic:
    baseUrl: http://127.0.0.1:7878/xfyun   # was: https://your-provider-host/path
    # ... all other fields unchanged

3. OMP retry config — ~/.omp/agent/config.yml

Keep OMP's built-in retry enabled as a fallback for mid-stream disconnects (the proxy can only retry before streaming begins):

retry:
  enabled: true       # Fallback for stream-mid-disconnect errors
  maxRetries: 3       # Proxy handles most 503s; OMP only catches stream breaks
  baseDelayMs: 1000   # Stream-break retry needs full re-send, slower is fine
  maxDelayMs: 0       # Disable OMP's hard-fail cap (0 = no cap)

How retry layers interact

Request flow:
  OMP -> Proxy (fast retry: 50ms, handles 503/429/529/network)
           | (2xx Response obtained)
        Upstream (SSE streaming begins)
           | (if stream breaks mid-way)
        OMP retry (slow fallback: 1s, re-sends full conversation)

Error scenario	Handled by	Why
Provider returns 503/429/529	Proxy (50ms)	Fast retry before OMP even sees an error
Network timeout / connection refused	Proxy (50ms)	Same
200 OK, then SSE stream breaks	OMP retry (1s)	Proxy cannot retry mid-stream without duplicating tokens
Daily/monthly quota exhausted	Neither	Permanent error, classified and excluded from retry

Client cancellation

If the client aborts the request (e.g. the user stops a turn), the proxy stops immediately — it does not keep hammering upstream:

• The in-flight upstream attempt is aborted via the client's AbortSignal; an aborted attempt is never retried (AbortError is excluded from retry classification).
• Pending backoff delays are cancelled within 100ms (shouldCancel poll).
• Mid-stream disconnects propagate upstream: the SSE body is torn down and the upstream connection closed.

Bun's default 10s per-connection idle timeout is disabled per-request (server.timeout(req, 0)), so long quiet periods — retry storms or a model thinking mid-stream — do not look like client cancellation.

Standalone usage

The proxy can run independently without OMP:

RETRY_PROXY_CONFIG=~/.omp/agent/retry-proxy.json bun run dist/proxy.js

Acknowledgements

The retry classification logic (retryClassifier.ts) and triangular backoff retry manager (retryManager.ts) are ported from GCMP by VicBilibily, licensed under the MIT License. See the file headers and LICENSE file for the full MIT notice.

License

MIT