Skip to content

feat(data-api): add the account_events_daily rollup write path to Postgres#4840

Merged
JSONbored merged 2 commits into
mainfrom
fix/account-events-daily-postgres-rollup
Jul 11, 2026
Merged

feat(data-api): add the account_events_daily rollup write path to Postgres#4840
JSONbored merged 2 commits into
mainfrom
fix/account-events-daily-postgres-rollup

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • Gap-closure from the fix(data-api): 25 more D1-only handlers — some serving frozen data since the streamer stopped #4832 completeness sweep: account_events_daily's Postgres table already existed (schema mirrors D1) but nothing wrote to it, since indexer-rs writes account_events directly and continuously — unlike the daily neurons snapshot (Build the metagraph/neuron pipeline's missing Postgres write path (neurons + neuron_daily) #4771/feat(data-api): migrate account_position_daily to Postgres #4839), there's no existing write request to piggyback a rollup onto.
  • Adds POST /api/v1/internal/rollup-account-events-daily (workers/data-api.mjs): re-rolls the two active UTC days from account_events into account_events_daily, mirroring D1's rollupAccountEventsDaily (src/account-events.mjs) exactly. Proxied through the main Worker the same way neurons-sync is — refactored handleNeuronsSyncProxy into a shared proxyToDataApi helper since the two routes are otherwise identical (same behavior preserved; the existing tests/neurons-sync-proxy.test.mjs passes unchanged).
  • Adds a new hourly GitHub Actions workflow (rollup-account-events-daily.yml) to trigger it, matching D1's own hourly cadence.
  • Adds the read route GET /api/v1/accounts/:ss58/history (netuid/from/to filters + cursor pagination, same shape as its D1 sibling) and wires tryPostgresTier into handleAccountHistory.

Test plan

  • npx vitest run tests/data-api.test.mjs tests/request-handlers-entities.test.mjs tests/neurons-sync-proxy.test.mjs tests/rollup-account-events-daily-proxy.test.mjs — 199 + 490 + 5 + 5 tests pass (new write-route tests cover auth/not-provisioned/hyperdrive-unavailable/success/DB-failure; new read-route tests cover the happy path, netuid/date-range filters, cursor seek, and full-page next_cursor emission; new wiring tests confirm postgres-wins/falls-back-to-D1)
  • npx vitest run (full repo) — 260 files / 7391+ tests pass
  • npm run lint / npx prettier --check on touched files — clean
  • npm run scan:public-safety — clean
  • npm run test:coverage + manual branch-coverage isolation on the diff — every uncovered branch confirmed pre-existing/outside the patch

Deploy note

After merge: provision ROLLUP_SYNC_SECRET via wrangler secret put ROLLUP_SYNC_SECRET -c wrangler.data.jsonc and add the same value as a ROLLUP_SYNC_SECRET GitHub Actions secret, then manually trigger the new workflow once to verify live before its first scheduled run.

Refs #4832

…tgres

Gap-closure from the #4832 completeness sweep: account_events_daily's
Postgres table already existed (mirroring D1's schema) but nothing
wrote to it, since indexer-rs writes account_events directly and
continuously (unlike the daily neurons snapshot, there's no existing
write request to piggyback a rollup onto).

- POST /api/v1/internal/rollup-account-events-daily (data-api.mjs):
  re-rolls the two active UTC days from account_events into
  account_events_daily, mirroring D1's rollupAccountEventsDaily exactly.
  Proxied through the main Worker (api.mjs) the same way neurons-sync
  is; refactored the neurons-sync proxy into a shared proxyToDataApi
  helper since the two routes are otherwise identical.
- New hourly GitHub Actions workflow (rollup-account-events-daily.yml)
  triggers it -- matches D1's own hourly cadence.
- Adds the read route (GET /api/v1/accounts/:ss58/history) with the
  same cursor/date-range/netuid filtering as its D1 sibling, and wires
  tryPostgresTier into handleAccountHistory.

Refs #4832
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
metagraphed-data-api 004e829 Commit Preview URL

Branch Preview URL
Jul 11 2026, 05:01 AM

@superagent-security

Copy link
Copy Markdown

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.78%. Comparing base (46a0e13) to head (874dd7c).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4840   +/-   ##
=======================================
  Coverage   97.77%   97.78%           
=======================================
  Files         162      162           
  Lines       19137    19189   +52     
  Branches     7188     7208   +20     
=======================================
+ Hits        18712    18764   +52     
  Misses         59       59           
  Partials      366      366           
Files with missing lines Coverage Δ
workers/api.mjs 95.16% <100.00%> (+0.01%) ⬆️
workers/data-api.mjs 99.60% <100.00%> (+0.02%) ⬆️
workers/request-handlers/entities.mjs 99.34% <100.00%> (+<0.01%) ⬆️
🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

validate-workflows.mjs requires every workflow to include
actions/checkout -- the new rollup-account-events-daily.yml didn't
need the repo checked out for its bare curl trigger, but the gate
doesn't special-case that, so add it like every other workflow.
@JSONbored JSONbored merged commit 7414a8e into main Jul 11, 2026
13 checks passed
@JSONbored JSONbored deleted the fix/account-events-daily-postgres-rollup branch July 11, 2026 05:08
@github-actions github-actions Bot mentioned this pull request Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant