feat: implement persistent 1-minute price history snapshots with rete…#70
Open
githoboman wants to merge 1 commit into
Open
feat: implement persistent 1-minute price history snapshots with rete…#70githoboman wants to merge 1 commit into
githoboman wants to merge 1 commit into
Conversation
…ntion and query API
|
@githoboman Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implemented the historical price snapshots feature on the Add-historical-price-snapshots-endpoint branch.
Closes #55
Schema (sql/schema.sql, prisma/schema.prisma)
New price_snapshots(pair, ts, price, volume) table, PRIMARY KEY (pair, ts) plus an index on (pair, ts). Added to both the SQL file and the Prisma model — the latter is what db push actually applies at startup. (The issue mentioned a Supabase migration, but this repo has no migrations dir; its real mechanism is Prisma db push + sql/schema.sql, so I followed that.)
Snapshot loop (src/ingesters/snapshot.ts)
Aligns to the wall-clock minute, then inserts one row per active pair: latest known price + base volume traded in the minute that just closed, in a single query. Idempotent via ON CONFLICT (pair, ts) DO NOTHING (safe across restarts). Wired into src/index.ts as a fault-isolated ingester.
Endpoint (src/api/history.ts)
GET /prices/history?pair=…&from=…&to=…&interval=1m|5m|1h. Epoch-aligned bucket aggregation (close price + summed volume), defaults to last 24h at 1m, with validation and a MAX_HISTORY_POINTS guard against unbounded scans. Note: the path matches the x402 /price prefix, so it's payment-gated like the other price endpoints.
Retention (src/jobs/snapshotRetention.ts)
Hourly bullmq job pruning rows older than 30 days, following the existing aggregateRefresh pattern.
Tests — 22 new across 3 files (history aggregation/validation, floorToMinute boundaries + append idempotency, retention). Full suite: 101 passed. Typecheck is clean except two pre-existing webhookDispatcher.ts errors I confirmed exist on the base commit.
Docs — README table row, OpenAPI path, and a minor changeset.
Two notes for your review:
I had to run npm ci + prisma generate since node_modules wasn't installed.
The acceptance criterion "returns expected count for a window" is satisfied by the count field + count-matching test, against the snapshot table (not raw price_points).Summary
Implemented the historical price snapshots feature on the Add-historical-price-snapshots-endpoint branch.
Schema (sql/schema.sql, prisma/schema.prisma)
New price_snapshots(pair, ts, price, volume) table, PRIMARY KEY (pair, ts) plus an index on (pair, ts). Added to both the SQL file and the Prisma model — the latter is what db push actually applies at startup. (The issue mentioned a Supabase migration, but this repo has no migrations dir; its real mechanism is Prisma db push + sql/schema.sql, so I followed that.)
Snapshot loop (src/ingesters/snapshot.ts)
Aligns to the wall-clock minute, then inserts one row per active pair: latest known price + base volume traded in the minute that just closed, in a single query. Idempotent via ON CONFLICT (pair, ts) DO NOTHING (safe across restarts). Wired into src/index.ts as a fault-isolated ingester.
Endpoint (src/api/history.ts)
GET /prices/history?pair=…&from=…&to=…&interval=1m|5m|1h. Epoch-aligned bucket aggregation (close price + summed volume), defaults to last 24h at 1m, with validation and a MAX_HISTORY_POINTS guard against unbounded scans. Note: the path matches the x402 /price prefix, so it's payment-gated like the other price endpoints.
Retention (src/jobs/snapshotRetention.ts)
Hourly bullmq job pruning rows older than 30 days, following the existing aggregateRefresh pattern.
Tests — 22 new across 3 files (history aggregation/validation, floorToMinute boundaries + append idempotency, retention). Full suite: 101 passed. Typecheck is clean except two pre-existing webhookDispatcher.ts errors I confirmed exist on the base commit.
Docs — README table row, OpenAPI path, and a minor changeset.
Two notes for your review:
I had to run npm ci + prisma generate since node_modules wasn't installed.
The acceptance criterion "returns expected count for a window" is satisfied by the count field + count-matching test, against the snapshot table (not raw price_points).