Skip to content

perf: tenant cache, single-flight cache reads, and pool env config - #1198

Open
realcodesiman wants to merge 5 commits into
mainfrom
perf/cache-pool-tuning
Open

realcodesiman wants to merge 5 commits into
mainfrom
perf/cache-pool-tuning

Conversation

@realcodesiman

Copy link
Copy Markdown
Contributor

Implements P0/P1 items A3, A4, A5, B11 (env-toggle) from the system-design-review audit.

A3 — tenant status cache (refined scope)

The audit assumed both the user→tenantId lookup and the tenant-status lookup were uncached. In reality QuotaEnforcementService.resolveContext already cached the tenantId lookup (60s TTL). The real uncached extra round-trip was tenantService.findById(tenantId) (fetches status/ownerId on every pooled-quota check) — wrapped it with withCache (30s TTL, tenants:<id> tag), invalidated on every tenant mutation (upsertByOwner, upsertById, setStatusByOwner).

Deliberately NOT implemented: caching WorkspaceMemberService.findMembership/isMember. That code carries an explicit comment that membership revocation must take effect immediately — caching it, even for 5-10s, is a real security regression. Investigated whether resolveWorkspaceAccess duplicates the membership lookup already done by the caller — it doesn't (it only re-fetches the workspace when realMember.workspace isn't already attached, which it always is on the real member path), so there was nothing to memoize there either.

A4 — single-flight for withCache

packages/redis/src/cache-utils.ts's withCache had no stampede protection — N concurrent callers on a cold key all missed and all hit the DB. Added in-process promise de-duplication (a Map<string, Promise<T>>, cleaned up via try/finally). Deliberately in-process only, not a cross-process Redis lock (that needs its own failure semantics for comparatively little extra benefit here).

A5 — connection pool config

packages/database/src/client.ts hardcoded pg Pool options. Added env-configurable DATABASE_POOL_MAX/_MIN, DATABASE_STATEMENT_TIMEOUT_MS, DATABASE_IDLE_IN_TRANSACTION_TIMEOUT_MS with defaults matching today's behavior (max: 10, no timeouts). Added a short PgBouncer recommendation note referencing the existing SET LOCAL compatibility comment.

B11 — read-replica env toggle

packages/database/src/sharding/message/connection-manager.ts hardcoded READ_REPLICAS_ENABLED = false. Replaced with an env-driven default (SHARD_READ_REPLICAS_ENABLED), still defaulting to false — no behavior change today, just no longer a code change to opt in. No replica infrastructure exists in this environment to test failover against; that remains a staging-only follow-up per the audit's own recommendation.

Verification

  • pnpm --filter @chatbotx.io/database check-types && test, pnpm --filter @chatbotx.io/redis check-types && test, pnpm --filter @chatbotx.io/business check-types && test, pnpm --filter builder check-types — all green

@github-actions github-actions Bot added the improvement Refactor or performance improvement label Sep 16, 2026
try {
return await sourcePromise
} finally {
if (inFlightCacheMisses.get(cacheKey) === sourcePromise) {
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Refactor or performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant