Skip to content

fix(ohlcv): bound pool OHLC queries to recent candles for perf (#582) - #583

Merged
DenisCarriere merged 1 commit into
mainfrom
fix/ohlcv-window-bound-perf
Jul 9, 2026
Merged

fix(ohlcv): bound pool OHLC queries to recent candles for perf (#582)#583
DenisCarriere merged 1 commit into
mainfrom
fix/ohlcv-window-bound-perf

Conversation

@DenisCarriere

Copy link
Copy Markdown
Contributor

Fixes the slow pool-OHLC queries and intermittent 500s reported in #582 (customer charting use case).

Problem

The pool OHLC endpoints aggregate AggregateFunction state rows via a blocking GROUP BY. With no lower time bound, the query merges the pool's entire history before ORDER BY … LIMIT can discard it. High-activity pools therefore read GBs and take multiple seconds — the cause of the >10s calls and 500 internal_server_error timeouts in the report (especially under the client's 5× concurrency).

This affects SVM, EVM, and TVM (TVM reuses evm.sql). EVM was actually the worst: a single hot-pool query read 1.56 GiB / 5.1 GB RAM.

Fix

Add a lightweight window_bound CTE that finds the timestamp cutoff of the most-recent (limit+offset) candles by scanning only the cheap timestamp column of state_ohlc_prices, then constrain the heavy aggregation with timestamp >= cutoff.

  • Results are byte-identical (verified on production across 5 presets/pools), including for sparse pools whose candles span a much wider wall-clock range than interval×limit — so this is not the unsafe "default a time window" approach.
  • Same query parameters, no handler/schema/API-contract change.
  • The excluded-protocol filter is applied inside window_bound (EVM) so every counted bucket has a surviving row.

Benchmarks (production, FORMAT Null, use_query_cache=0)

query before after
SVM 1h · 1m/60 830–1240 ms · 210 MiB · 815 MiB RAM 120–200 ms · 16 MiB · 42 MiB
SVM 1d · 5m/288 980–1450 ms 160–250 ms
SVM 1m · 4h/180 1189 ms 259 ms
EVM 1m/100 (hot pool) 3270 ms · 1.56 GiB · 5133 MiB RAM 155 ms · 262 MiB · 314 MiB

~7–21× faster, ~6–16× less memory. The per-query memory collapse is what relieves the concurrency contention producing the 500s.

Trade-off: coarse intervals (1h/1w) with little history regress by tens of ms (extra timestamp pass) but stay far under budget. SETTINGS optimize_aggregation_in_order=1 was tried and rejected — it doesn't prune (bytes unchanged) and made the 1h case slower.

Also

  • Bump dbs-config.yaml.example dex versions to match production: svm-dex@v0.5.2, mainnet:evm-dex@v0.5.0, tron:evm-dex@v0.5.0.

Validation notes

🤖 Generated with Claude Code

Pool OHLC endpoints (svm, evm, tvm) aggregate AggregateFunction state
rows with a blocking GROUP BY. With no lower time bound the query merges
the pool's ENTIRE history before ORDER BY ... LIMIT can discard it, so
high-activity pools read GBs and take multiple seconds — the cause of the
>10s calls and 500 timeouts reported in #582.

Add a lightweight `window_bound` CTE that finds the timestamp cutoff of
the most-recent (limit+offset) candles by scanning only the cheap
`timestamp` column of `state_ohlc_prices`, then constrain the heavy
aggregation to `timestamp >= cutoff`. Results are byte-identical
(verified on production), including for sparse pools whose candles span a
much wider wall-clock range than interval*limit.

Benchmarked on production against outlier pools from #582:
- SVM 1h 1m/60:  ~830-1240ms/210MiB/815MiB-RAM -> ~120-200ms/16MiB/42MiB
- SVM 1d 5m/288: ~980-1450ms          -> ~160-250ms
- EVM 1m/100:    3270ms/1.56GiB/5.1GB-RAM -> 155ms/262MiB/314MiB

evm.sql is shared by the TVM route, so this covers svm/evm/tvm. Coarse
intervals with little history regress by tens of ms (extra timestamp
pass) but stay well under budget.

Also bump dbs-config.yaml.example dex versions to match production
(svm-dex v0.5.2, evm-dex v0.5.0, tron evm-dex v0.5.0).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HGQ61P2deQE3fRCVLRbo1S
@YaroShkvorets
YaroShkvorets temporarily deployed to fix/ohlcv-window-bound-perf - token-api PR #583 July 9, 2026 18:14 — with Render Destroyed
@DenisCarriere
DenisCarriere merged commit a715dfe into main Jul 9, 2026
2 of 3 checks passed
@DenisCarriere
DenisCarriere deleted the fix/ohlcv-window-bound-perf branch July 9, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants