Filter-aware aliases: make filtered_stats cost track the view, and cache it - #235
Draft
DarshitChanpura wants to merge 6 commits into
Draft
DarshitChanpura wants to merge 6 commits into
DarshitChanpura wants to merge 6 commits into
Conversation
Add FilteredStatsCache, a per-index cache of the visible-doc bitset and the per-field / per-term BM25 statistics used by the filtered_stats path. Entries are keyed by the segment core CacheKey plus the alias filter and evicted via a segment close listener, so a refresh/merge that replaces a segment drops its entry and the next query recomputes -- correct freshness with no explicit invalidation. Cached bytes are reported to the fielddata circuit breaker via addWithoutBreaking (accounting only; the cache never rejects) and returned on eviction. ContextIndexSearcher now serves getVisibleDocsPerSegment and the filtered collection/term statistics from the cache, falling back to inline computation when no cache is available (e.g. a bare test context). This turns the per-query cost from O(total corpus) -- a field-wide postings walk on every query -- into an amortized O(number of segments) summation after the first query per (segment, filter), which is what makes ranked (filtered_stats) secured views viable at scale. Wire the cache through IndexCache / IndexService and expose SearchContext.filteredStatsCache() with a null-safe default. Verified: server compiles; FilterAwareAliasIT green with -Dopensearch.filter_aware_alias.filtered_stats=true (visible-subset scores still match a physically-filtered index); new FilteredStatsCacheTests covers compute-once caching, null (no-visible-docs) caching, field/term caching, and segment-close eviction. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
Filtered statistics cost O(visible postings) to build the first time a segment is queried, so on a very large view that first query can be slow. Add a configurable budget (opensearch.filter_aware_alias.filtered_stats.max_visible_docs, default 1M, -1 to disable): above it, useFilteredStatistics() returns false and the request falls back to the default constant_score behavior, which is still leak-free and flat but unranked. The estimate uses ScorerSupplier#cost(), an upper bound from postings metadata, so the guardrail does not iterate the filter and runs before the expensive build it protects against. The decision is memoized per context because it is read twice per request -- when choosing the query shape and when computing statistics -- and those must agree: a filtered-stats query shape scored with whole-shard statistics would reintroduce the whole-corpus IDF the pre-filter excludes. Estimation failure falls back to constant_score (degrades ranking, not correctness). Verified: FilterAwareAliasIT gains a test asserting both sides of the switch -- real BM25 within budget, flat 1.0 (and still leak-free) once the guardrail trips; 5/5 green with the filtered_stats gate on. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
The filtered-stats cache is per-segment, so a refresh or merge that produces a new segment leaves its visible-subset statistics uncomputed and the first query to touch it pays the O(visible postings) build. That is bounded -- existing segments stay warm -- but shows up as an occasional tail-latency spike on a write-heavy index, and repeats on every node that builds its own cache. Add FilteredStatsWarmer, an IndexWarmer.Listener that pre-computes the visible bitset and per-field statistics for newly visible segments on the warmer thread pool, following the existing eager-load pattern used for the bitset filter cache. It warms only the (filter, field) pairs the cache has actually been asked about -- tracked in FilteredStatsCache and bounded -- so an index that never queries through a filtered alias does no speculative work. Gated by the index setting index.filter_aware_alias.warm_filtered_stats (default false). Warming is best-effort: a failure is logged and the query path recomputes on a miss. To avoid duplicating the build logic, the visible-bitset and field-contribution computations move from ContextIndexSearcher into FilteredStatsCache as static helpers and the searcher now delegates to them. Verified: FilteredStatsCacheTests gains a test that warms a newly visible segment and asserts both the bitset and the field contribution are then served from cache (the recompute would fail the test); 5/5 unit and FilterAwareAliasIT + ConstantScoreFilteredAliasIT green. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
A visible-doc bitset is maxDoc bits per (segment, filter) regardless of how selective the filter is, so cache memory scales with the number of distinct views rather than with the visible fraction. That matters for capacity planning -- dozens of views over a large index is hundreds of MB per node -- so assert it rather than leaving it to be rediscovered: a 1-doc view costs the same as an all-docs view, and ten views cost ten times one view. If the representation later becomes sparse or shared across views, this test should be updated deliberately instead of the characteristic drifting silently. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
…tatistics Computing visible-subset collection statistics walked the field's entire term dictionary and every posting in it, making the first query on a segment cost O(all tokens in the field) -- measured at a constant ~5.3 ns/token, so ~880 ms for 165M tokens and on the order of 15-27 s at 100M documents. That is the dominant cost of the ranked path and it is spent on a value BM25 never reads. BM25 takes exactly two numbers from CollectionStatistics: docCount, and sumTotalTermFreq solely to derive avgFieldLength = sumTotalTermFreq / docCount. Neither needs the term dictionary. Norms already carry one length value per document, so a single pass over the visible documents yields both in O(visible docs). sumDocFreq -- the value that forces the full walk -- is not read by BM25, so estimate it by scaling the shard-wide field metadata (free) by the visible fraction and clamp to Lucene's CollectionStatistics invariants. Fields indexed without norms keep the exact walk. Also leapfrog per-term statistics: intersect the term's postings with the visible bitset through a conjunction instead of scanning the full postings list and testing each hit, so skip lists do the work and the cost is O(min(visible, docFreq)). Selective views -- the tenancy case -- get cheaper rather than more expensive. Caveat: avgFieldLength now derives from the same lossy per-document lengths BM25 itself normalizes with, rather than from exact term frequencies, so it is an approximation. It is not part of the leak-free property, which rests on per-term docFreq and docCount remaining strictly visible-only. Both are unchanged. Verified: FilterAwareAliasIT 5/5 with the gate on, including the assertion that filtered_stats scores still equal a physically filtered index; FilteredStatsCacheTests 6/6. Cold cost on a 27.5-token/doc corpus drops from 158/438/880 ms to 54/23/93 ms at 1M/3M/6M docs, and ns-per-token stops being constant -- the cost is no longer a function of token count. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
…e view The existing equivalence test runs a ~10% view. Since the cost of gathering visible-subset statistics now scales with the size of the view rather than the size of the index, the ratio of visible to total documents is worth covering at more than one point, so add the same assertion at ~1% visible: 21 visible documents of 2,021. Found while measuring whether reading per-document structures sequentially and testing the visible bitset beats seeking to each visible document. On identical segments it does not -- 16% worse at a 10% view on 10M documents, 50% worse at 100M, and better only above roughly 40% visible, which is not a restricted view in any useful sense. That change is not included; this test is what was worth keeping from it. Signed-off-by: Darshit Chanpura <dchanp@amazon.com>
DarshitChanpura
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Two related changes to the filter-aware-alias
filtered_statspath, which computes BM25 collection/term statistics over only the documents an alias filter admits:FilteredStatsCache— a per-index cache of those visible-subset statistics, so they are computed once per (segment, filter) rather than on every query. Keyed by the segment coreCacheKeyplus the alias filter, evicted via a segment close listener (refresh/merge → the next query recomputes). Cached bytes are reported to the fielddata circuit breaker withaddWithoutBreaking(accounting only; the cache never rejects).ContextIndexSearcherserves the visible bitset and the filtered statistics from the cache, falling back to inline computation when no cache is present.A cost-model fix for the build itself. The build previously walked the field's entire term dictionary and every posting in it, so the first query on a segment cost O(total text in the field) — a function of corpus size, not of how many documents the view exposes.
BM25Similarityreads onlydocCountandsumTotalTermFreq(the latter solely to deriveavgFieldLength); it never readssumDocFreq, andsumDocFreqis what forced the walk. Both values it does read are derivable from the field's norms in a single pass over the visible documents, so the build is now O(visible documents).sumDocFreqis filled in by scaling free shard-wide field metadata, then clamped to Lucene'sCollectionStatisticsinvariants. Per-termdocFreqleapfrogs the term's postings against the visible bitset rather than scanning them, costingO(min(visible, docFreq)). Fields indexed without norms keep the exact term-dictionary walk.Also included: a small-view guardrail that falls back to
constant_scoreabove a configurable visible-doc budget, andFilteredStatsWarmer, which precomputes statistics for newly visible segments on the warmer thread pool.Stacked on the
filter_aware_aliaswork (fork branchfeat/filter-aware-aliases/ PR #226) — this PR is based on that branch; review it after #226.Related Issues
Builds on the filter-aware-alias
filtered_statsfeature (fork PR #226).Performance
Measured on one host (Xeon 8488C, 8 GiB heap) against the same Lucene files — corpora built once, force-merged to a single segment, node binary swapped between builds. 27.5 tokens/doc, 10 % visible. "First query" is the build for a (segment, filter) not seen before.
The more important change is what the cost now depends on. Holding the index at 10M documents and varying the fraction the view admits:
The "before" column barely responds to view size — a 2× spread across a 50× range — because cost tracked the index rather than the view. It now tracks the view at roughly 110–150 ns per visible document, so a more selective view is cheaper, which is the opposite of the previous behaviour.
Holding documents fixed and scaling token density 12× confirms the model change directly: the old cost model predicts 53 → 636 ms, and the measurement is flat at 11 ms. Token volume is no longer in the cost model.
Why this is needed
Without the cost-model fix, the first
filtered_statsquery against a new segment cost seconds at scale (19 s at 100M here), which makes ranked search over a filtered view impractical and leaves a separate physical copy of the visible subset as the only option.The cache is complementary and its value is narrower now that the build is cheap: it removes a repeat of that build on every query. The work it removes is provably redundant — Lucene segments are immutable and a view's filter is fixed, so for a given (segment, filter) the recomputed answer is bit-identical every time. This is not an accuracy/speed trade: scores, hits, and ranking are unchanged (
FilterAwareAliasITassertsfiltered_statsstill equals a physically filtered index), only latency moves. It follows the existingBitsetFilterCachepattern (per-segment, keyed on segment identity, close-listener eviction, breaker accounting), so there is no new caching machinery to reason about.One approximation, stated explicitly
avgFieldLengthis now derived from norms — the same lossy per-document lengths BM25 already normalizes against — rather than from exact term frequencies, so it is approximate.docCountand per-termdocFreqremain exact and strictly visible-only, which is what the feature's scoring property depends on.sumDocFreqis approximated from shard-wide metadata, which is safe because BM25 never reads it; approximating a value that does reach the score was considered and rejected.Cache scope and behavior on ingest
Scope. One cache per index, per node (built in
IndexService), keyed by the segment core cache key and sub-keyed by the alias-filterQuery. Because a segment belongs to exactly one shard, entries are naturally partitioned by shard without a shard key. The cache is JVM-heap and node-local: each node builds its own.Ingest. The design leans on segment immutability, so correctness needs no invalidation logic:
Measured under continuous indexing (~23k docs/s, 1 s refresh, not force-merged): p50/p90 1.1 / 1.7 ms, with only 0.4% of queries above 10 ms and a worst case of ~187 ms on a large merge's first query — which is what the warmer in this PR removes.
Known caveat: the bitset is built against
liveDocsat build time, so deletes/updates to documents in an already-cached segment leave that segment's statistics slightly stale until it is merged away. Stock BM25 collection statistics already include deleted documents until merge, so this is consistent with existing behavior rather than a regression.Known gap
The cache evicts only when a segment closes: there is no size cap or LRU, and breaker accounting is advisory. Memory is a bitset per (segment, filter) regardless of filter selectivity, so it scales with the number of views. This is adequate at the scales measured but would need a bounded eviction policy before many-view deployments.
Check List
FilteredStatsCacheTests6/6 (compute-once, no-visible-docs caching, field/term caching, warm pre-population, segment-close eviction, memory scaling);FilterAwareAliasIT6/6 with-Dopensearch.filter_aware_alias.filtered_stats=true, including score equality against a physically filtered index at both ~10 % and ~1 % visible, and the guardrail's fallback toconstant_score.--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.