[knudtty]

Summary

"All values" lookups that read a text index now return each key's most frequent values with topKWeighted, and run in parallel batches of at most 10 keys, so a key with very many values delays only its own batch.

Stacked on #3614.

The problem

getMapTextIndexKeyValues (Map key=value indexes, e.g. idx_log_attr_items) and getTextIndexKeyValues (text indexes on plain columns, e.g. TraceId):

1. Kept the alphabetically first values (groupUniqArray(limit) over a sorted dictionary), not the common ones.
2. Ran up to 100 keys in one query. A key with a value per row (request IDs, UUIDs) has a token per row, and every other key in that query waited on it. The shared 3M-row cap (read_overflow_mode: 'break') also stops the sorted read partway, so keys that sort after a high-cardinality key came back empty.

The fix

• topKWeighted(limit)(value, cardinality). cardinality is the token's row count in the part, so the values are the most frequent, most frequent first. limit is inlined as a literal like the other parametric aggregates in this file.
• Map key=value lookups are split per Map column into queries of at most 10 keys (TEXT_INDEX_KEYS_PER_QUERY), and plain-column text indexes into queries of at most 10 columns. They already stream in as each query finishes. Materialized-view and raw-table lookups keep batches of 100, because each raw-table batch scans the table.

What I tried and dropped: one branch per key

Giving each key its own UNION ALL branch with LIMIT 100000 cut tokens read about 12× on a local benchmark. On ClickHouse Cloud it was much slower. Each branch reads the index of every part separately, and an open has a fixed cost no matter how few tokens it reads. On a 26-part otel_logs table with a warm cache:

Query	ms	Dictionary blocks	CPU
One key	804	26	4.9 s
A key that doesn't exist	772	26	4.7 s
system.parts lookup alone	2	0	0 s

That's about 0.18 CPU-s and 10–30 MiB of filesystem-cache reads per part per open, apparently because mergeTreeTextIndex() loads and decodes a per-part structure sized to the whole dictionary, and doesn't use the text index header cache. With per-key branches, a page load did about (keys × parts) ≈ 2,600 opens instead of (Map columns × parts × batches), and the server ran out of CPU. The fixed cost per open should be raised upstream.

Cost and known limits

• More opens than main. Batches of 10 mean up to 10× more queries, and each query opens every part of the index once per Map column. That's still far fewer opens than per-key branches. The batch size trades total CPU against how long one slow key can hold up the others.
• topKWeighted uses about 1.6× the CPU per token of groupUniqArray. Memory is the same.
• The 3M-row cap still applies per query. Within a batch, a high-cardinality key can still use it up, and keys sorting after it come back empty. That now affects at most 9 other keys instead of 99.

How to reproduce

Starting point: yarn dev against ClickHouse 26.3+, with the OTel otel_logs schema (key=value text indexes idx_log_attr_items, idx_res_attr_items), and a Map key with one rare value that sorts first and one common value, e.g. map('freq.key', if(number = 0, 'a-rare', 'z-common')) from numbers(4).

On main:

1. On Search, turn on showing all values in the filter panel. freq.key lists a-rare first, and system.query_log shows one values query for all keys.

On this branch:

1. Repeat step 1. freq.key lists z-common first, and there is one values query per 10 keys.

References

• Related PRs: perf: share one Map key read between concurrent callers #3611, perf: find the most frequent Map keys with topKWeighted #3614 (base of this stack)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
hyperdx-oss	Ready	Preview	Nov 11, 2026 3:18am UTC
hyperdx-storybook	Ready	Preview	Nov 11, 2026 3:18am UTC

[changeset-bot]

🦋 Changeset detected

Latest commit: 6525149

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@hyperdx/common-utils	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[github-actions]

Duplicate code analysis

1 new clone introduced by this PR.

• packages/common-utils/src/core/metadata.ts:1221-1241 ↔ packages/common-utils/src/core/metadata.ts:1269-1286 — 21 lines

If one of these duplicates a helper that already exists, import it instead
of redefining it. If the duplication is deliberate, ignore this comment —
the check is advisory and never fails the build.

---

jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 110 in total (1.20%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

[github-actions]

E2E Test Results

✅ All tests passed • 464 passed • 1 skipped • 1130s

Status	Count
✅ Passed	464
❌ Failed	0
⚠️ Flaky	2
⏭️ Skipped	1

Tests ran across 4 shards in parallel.

View full report →