Skip to content

feat(sdk): classify transient Postgres failures and jitter retries - #84

Open
andrewkcchung wants to merge 1 commit into
mainfrom
upstream/u01-cms-retry
Open

andrewkcchung wants to merge 1 commit into
mainfrom
upstream/u01-cms-retry

Conversation

@andrewkcchung

@andrewkcchung andrewkcchung commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Why

Any PostgreSQL-backed PilotSwarm deployment can encounter short-lived connection saturation, failover, serialization, deadlock, cancellation, or network failures. A shared bounded classifier and jittered retry policy prevents each caller from inventing inconsistent retry behavior and reduces synchronized retry storms without encoding deployment-specific policy.

What changed

  • Classify a bounded set of transient PostgreSQL SQLSTATE and network error codes, preferring structured codes over message text.
  • Apply exponential backoff with jitter and explicit retry limits.
  • Keep critical operations fail-fast for non-transient errors and preserve the original error after exhaustion.
  • Emit actionable retry diagnostics without logging credentials or query contents.

Behavioral proof

Command:

npm run build --workspace=packages/sdk
node --test packages/sdk/test/unit/cms-retry.test.mjs
  • Red — test-only patch on main: The test failed because main did not expose the transient classifier or bounded retry behavior exercised by the new suite.
  • Green — this branch: SDK build succeeded; 5 tests passed.

Risk assessment

Relative risk: Low

The change is localized to an existing retry helper and uses a narrow structured-code allowlist, bounded attempts, and bounded jitter. It does not change database schemas or transaction semantics. The principal risk is accidentally retrying deterministic failures or increasing load during saturation, which is constrained by precedence and retry-schedule tests.

Review focus

  • Confirm the structured-code allowlist is narrow enough that deterministic failures are never retried.
  • Review the message fallback patterns and precedence when both a code and message are present.
  • Confirm jitter and attempt bounds are appropriate for both best-effort and critical callers.
  • Confirm exhaustion preserves the original failure rather than replacing it with retry bookkeeping.

Add bounded transient failure categories, structured-code precedence, exponential backoff with jitter, saturation handling, and actionable retry diagnostics.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant