Skip to content

Recover Redis-backed admission after connection loss - #14

Merged
austin-smith merged 2 commits into
mainfrom
recover-redis-admission
Jul 29, 2026
Merged

austin-smith merged 2 commits into
mainfrom
recover-redis-admission

Conversation

@austin-smith

Copy link
Copy Markdown
Owner

What Changed

  • Replaced the permanently cached Redis client with a lifecycle-managed rate-limit store that fails fast during outages, deduplicates reconnects, rotates unhealthy clients, and never retries a failed mutating Lua command.
  • Added structured 503 Service Unavailable responses with Retry-After across endpoint creation, webhook capture, event streams, and request replay.
  • Added bounded browser recovery for endpoint creation and cleanup of partially acquired event-stream leases.
  • Added regression coverage for Redis connection failure, recovery, route responses, lease cleanup, and browser retry behavior.

Why

The rate-limit layer cached one Redis client promise for the lifetime of the web process. When that client lost its connection, commands could accumulate in the default offline queue and fail after a long timeout while the cached client remained unusable. Existing logged-in sessions could appear healthy because they reused persisted endpoints, while brand-new sessions immediately exercised the broken endpoint-creation path.

This change makes Redis availability an explicit, recoverable dependency state instead of requiring an application restart.

Validation

  • pnpm verify
  • 26/26 Turbo tasks passed
  • Web: 39 test files and 255 tests passed
  • Webhooks server: 15 test files and 91 tests passed
  • git diff --cached --check

@austin-smith
austin-smith marked this pull request as ready for review July 29, 2026 03:57
@cursor

cursor Bot commented Jul 29, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挕 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d0cc495689

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/web/components/webhook-inspector/endpoint-session/transport.ts Outdated
Comment thread apps/web/components/webhook-inspector/endpoint-session/transport.ts Outdated
@austin-smith

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 22a63fd520

鈩癸笍 About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 馃憤.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@austin-smith
austin-smith merged commit 7b38427 into main Jul 29, 2026
1 check passed
@austin-smith
austin-smith deleted the recover-redis-admission branch July 29, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant