Skip to content

spec: throttle aggregate Redis fan-out to 12 concurrent - #1289

Merged
Flotapponnier merged 1 commit into
mainfrom
fix/throttle-aggregate-reads
Jul 18, 2026
Merged

spec: throttle aggregate Redis fan-out to 12 concurrent#1289
Flotapponnier merged 1 commit into
mainfrom
fix/throttle-aggregate-reads

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

Root cause found

Homepage "Awaiting samples" persistent state: the aggregate loader fires Promise.allSettled(specs.map(loadOne)) = 68 read chains at once. Each chain does 2-3 Redis GETs (pointer → blob → optional LKG), so ~150-200 concurrent Redis ops per aggregate cycle. SRH's request pool saturates and the alphabetically-later specs (perp-open-interestzksync-rpc, exactly 31 slugs) systematically time out at 8s and render as draft placeholders, even though their blobs are sitting in Redis the whole time (v=2, correct pointers, valid LKG).

Deterministic pattern: sorted slug list, cutoff at perp-funding-stability / perp-open-interest.

Fix

Wrap the fan-out in a small limitedAllSettled (12 concurrent workers, same result shape as Promise.allSettled). All 68 chains complete in ~4-5 s worst-case wall time instead of 37 completing in 200 ms + 31 timing out.

Behind the unstable_cache layer (revalidate 300 s) so the slightly-longer cache-miss wall time is invisible to end users. On cache hit (99% of requests) zero difference.

Follow-up work still open

  • (later, architectural) worker → Vercel Blob broadcast, remove SRH from read path entirely
  • (later) batch pointer reads via MGET so the 68 chains become 1 pipelined request
    Both would replace this throttle but aren't cheap.

…ing alphabetically-later specs to timeout as draft)
@Flotapponnier
Flotapponnier merged commit 427c684 into main Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant