Skip to content

Stampede protection beyond a single process #62

Description

@camcima

Deferred from the architecture review (architectural observation #3) — see ARCHITECTURE_REVIEW.md.

Problem

Coalescing is per-process. CacheManager deduplicates concurrent misses through an in-memory Map (packages/core/src/cache-manager.ts:21), which is exactly right within one instance and does nothing across instances. With N pods, a hot key expiring still produces N factory calls — one per process, not one overall.

That is a reasonable v0 scope, but the README currently says:

Stampede protection — Concurrent cache misses for the same key coalesce into a single factory call. 100 simultaneous requests = 1 database query.

True on one instance; not true of a deployment, which is where stampedes actually hurt. The doc caveat is the immediately actionable part of this issue and worth doing on its own even if the rest waits.

Sketch

The standard escalation, cheapest first — each is independently useful, and the first two need no coordination at all:

  1. TTL jitter — an opt-in jitterRatio spreading expiry across a window so keys written together stop expiring together. Cheapest real mitigation for the synchronized-expiry case.
  2. Probabilistic early refresh (stale-while-revalidate) — serve the cached value past a soft threshold while one caller refreshes in the background. Removes the latency cliff entirely, not just the herd. Composes with wrapWrites: "background" added in fix: resolve architecture review findings (H1-H3, M1-M6, L1-L6) #56.
  3. Distributed lock — an optional adapter capability (tryLock/unlock, SET NX PX on Redis) so one instance computes and the rest wait or serve stale. Strongest guarantee, most operational complexity: lock TTLs, holder death, clock assumptions. Only worth it if 1 and 2 prove insufficient.

Done when

At minimum, the README says what the guarantee actually covers. Beyond that, whichever of the three the roadmap wants — they are independent, and each should be opt-in so the zero-config path stays what it is today.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions