You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Deferred from the architecture review (architectural observation #3) — see ARCHITECTURE_REVIEW.md.
Problem
Coalescing is per-process. CacheManager deduplicates concurrent misses through an in-memory Map (packages/core/src/cache-manager.ts:21), which is exactly right within one instance and does nothing across instances. With N pods, a hot key expiring still produces N factory calls — one per process, not one overall.
That is a reasonable v0 scope, but the README currently says:
Stampede protection — Concurrent cache misses for the same key coalesce into a single factory call. 100 simultaneous requests = 1 database query.
True on one instance; not true of a deployment, which is where stampedes actually hurt. The doc caveat is the immediately actionable part of this issue and worth doing on its own even if the rest waits.
Sketch
The standard escalation, cheapest first — each is independently useful, and the first two need no coordination at all:
TTL jitter — an opt-in jitterRatio spreading expiry across a window so keys written together stop expiring together. Cheapest real mitigation for the synchronized-expiry case.
Probabilistic early refresh (stale-while-revalidate) — serve the cached value past a soft threshold while one caller refreshes in the background. Removes the latency cliff entirely, not just the herd. Composes with wrapWrites: "background" added in fix: resolve architecture review findings (H1-H3, M1-M6, L1-L6) #56.
Distributed lock — an optional adapter capability (tryLock/unlock, SET NX PX on Redis) so one instance computes and the rest wait or serve stale. Strongest guarantee, most operational complexity: lock TTLs, holder death, clock assumptions. Only worth it if 1 and 2 prove insufficient.
Done when
At minimum, the README says what the guarantee actually covers. Beyond that, whichever of the three the roadmap wants — they are independent, and each should be opt-in so the zero-config path stays what it is today.
Deferred from the architecture review (architectural observation #3) — see
ARCHITECTURE_REVIEW.md.Problem
Coalescing is per-process.
CacheManagerdeduplicates concurrent misses through an in-memoryMap(packages/core/src/cache-manager.ts:21), which is exactly right within one instance and does nothing across instances. With N pods, a hot key expiring still produces N factory calls — one per process, not one overall.That is a reasonable v0 scope, but the README currently says:
True on one instance; not true of a deployment, which is where stampedes actually hurt. The doc caveat is the immediately actionable part of this issue and worth doing on its own even if the rest waits.
Sketch
The standard escalation, cheapest first — each is independently useful, and the first two need no coordination at all:
jitterRatiospreading expiry across a window so keys written together stop expiring together. Cheapest real mitigation for the synchronized-expiry case.wrapWrites: "background"added in fix: resolve architecture review findings (H1-H3, M1-M6, L1-L6) #56.tryLock/unlock,SET NX PXon Redis) so one instance computes and the rest wait or serve stale. Strongest guarantee, most operational complexity: lock TTLs, holder death, clock assumptions. Only worth it if 1 and 2 prove insufficient.Done when
At minimum, the README says what the guarantee actually covers. Beyond that, whichever of the three the roadmap wants — they are independent, and each should be opt-in so the zero-config path stays what it is today.