Phase 1 blob broadcast: worker → Vercel Blob + revalidate hook (no reader switch) - #1297
Closed
Flotapponnier wants to merge 1 commit into
Closed
Phase 1 blob broadcast: worker → Vercel Blob + revalidate hook (no reader switch)#1297Flotapponnier wants to merge 1 commit into
Flotapponnier wants to merge 1 commit into
Conversation
…er tier A + revalidate hook on site (no reader switch yet, gated on BLOB_READ_WRITE_TOKEN)
Collaborator
Author
|
Pivoting to VPS-served aggregate (nginx static file) — self-hosted, $0, no vendor lock-in, no Blob propagation delay. New PR incoming with worker writing local JSON + nginx vhost. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 1 of the SRH replacement roadmap
Goal: eliminate SRH from the site's hot read path by having the materialize worker publish an aggregate JSON snapshot to Vercel Blob every 60 s, and having the site read that Blob (CDN-cached) instead of fanning out ~150 concurrent Redis GETs on every homepage render. This PR ships the write side only. The read side flips in a follow-up PR under a feature flag once we've observed writes are healthy.
What this PR does
worker/publish-blob.ts(new) — reads all 68 unfiltered bench snapshots back from Redis (worker uses TCP direct viaOCB_REDIS_URL, never SRH), assembles a single{ v, builtAt, benches[] }envelope,put()to Vercel Blob atbench-aggregate/latest.json(access: public,cacheControlMaxAge: 60,addRandomSuffix: false,allowOverwrite: true). Then POSTs the site's revalidate hook.worker/index.ts— callspublishAggregateBlob(specs)after tier A completes. Gated onBLOB_READ_WRITE_TOKENbeing set so the change is a no-op until the env is provisioned.src/app/api/internal/revalidate-aggregate/route.ts(new) — POST handler that verifies a bearer token (REVALIDATE_TOKEN) withtimingSafeEqual, then callsrevalidateTag('bench-aggregate', 'default'). Purges the CDN cache tag immediately so the site doesn't wait the ~60 s Blob overwrite propagation window.@vercel/blobadded todependencies(v1).pnpm-lock.yamlregenerated.Zero user-visible effect until step 2 ships
loadAllBenchmarksSafe→ Redis via SRH). Even if the Blob write silently fails every cycle, nothing on prod breaks.Env vars to provision after merge
VPS worker (
/run/ocb/.env.materialize-worker):BLOB_READ_WRITE_TOKEN— from Vercel Blob store (create via Vercel Dashboard → Storage → Blob)SITE_URL—https://openchainbench.comREVALIDATE_TOKEN— shared secret with the site (any random 32-byte hex)Vercel site (
vercel env add):REVALIDATE_TOKEN— same value as the workerThen rebuild worker container on VPS:
ssh -p 8822 rescue@57.130.61.201 "cd /opt/ocb && docker compose up -d --build materialize-worker".Observation checklist
Once envs are in place and worker restarted, watch:
docker logs -f ocb-materialize-worker | grep 'blob published'— should log every 60 sbench-aggregate/latest.jsonrefreshingPOST /api/internal/revalidate-aggregate 200every 60 sFollow-up PRs (roadmap)
loadAggregateFromBlob()behindAGG_SOURCE=blobflag, shadow-mode logging for parityOCB_REDIS_URLon Vercel, unset SRH env, retire SRH container + auto-heal cron