Filter-aware aliases: pre_filter enforcement + filtered BM25 statistics

[DarshitChanpura]

Summary

Adds filter-aware aliases to OpenSearch: a filtered alias can now enforce its
filter before query statistics and term enumeration are computed, instead of
only post-filtering the result set. When enabled, this closes a scoring /
term-dictionary side-channel that plain filtered aliases — and, by extension, the
security plugin's DLS post-filtering — leave open, at zero storage and full
freshness.

Personal feature branch on my fork. Opening this PR against my own main for
review/history, not (yet) upstream.

Background — the side-channel

A filtered alias narrows the result set to matching documents, but the query
still executes against the whole shard:

• BM25 scoring uses corpus-wide collection statistics
  (idf = log(1 + (N - df + 0.5)/(df + 0.5)), where N/df span every document,
  including hidden ones). A term appearing only in hidden documents has an
  inflated df, which depresses its score. A restricted user who can write a
  document and read back its score can detect hidden terms — the ExactOracle.
• Term enumeration (prefix/wildcard/fuzzy/match_phrase_prefix) expands
  against the full term dictionary — the PrefixOracle.

Both exist because the filter is applied after statistics and the term
dictionary have been consulted over all documents.

What this adds

A new enforcement setting on an alias add action:

Value	Behavior
post_filter (default)	Historical behavior — filter applied post-collection; statistics run over the whole shard. Unchanged.
pre_filter	Filter applied before scoring; statistics reflect only the visible subset, so the side-channel does not leak.

POST /_aliases
{ "actions": [ { "add": {
    "index": "patients", "alias": "cardiology-view",
    "filter": { "term": { "dept": "cardiology" } },
    "enforcement": "pre_filter"
} } ] }

pre_filter has two scoring sub-behaviors:

1. constant_score (default for pre_filter) — the query is wrapped in
   ConstantScoreQuery(BooleanQuery(MUST=query, FILTER=aliasFilter)); no IDF is
   computed, so there is no scoring channel to leak. Fits the common security
   case where ranking is not required.
2. filtered_stats — real BM25 scoring, but CollectionStatistics
   (N_f, docCount_f) and TermStatistics (df_f, ttf_f) are computed over
   the visible-document bitset, so ranking works over the visible subset and still
   leaks nothing. Opt-in via the
   opensearch.filter_aware_alias.filtered_stats system property (read per
   request), keeping constant_score the default while this sub-behavior is
   experimental.

Design / data flow

REST enforcement -> IndicesAliasesRequest.AliasActions   (parse + version-gated wire)
  -> AliasAction.Add -> AliasMetadata                    (cluster state, version-gated)
  -> IndicesService.buildAliasFilter                     (metadata -> AliasFilter.Enforcement)
  -> AliasFilter.Enforcement (PRE_FILTER)
  -> DefaultSearchContext                                (constant-score wrap OR plain filter clause)
  -> ContextIndexSearcher                                (filtered CollectionStatistics/TermStatistics)

• The enforcement field is on the wire only between nodes at V_3_8_0+; older
  nodes never see it and behave as post_filter, so mixed-version clusters are
  unaffected.
• AliasFilter.Enforcement is @ExperimentalApi.
• dfs_query_then_fetch: the DFS phase's per-shard statistics use the same
  filtered overrides, so coordinator-aggregated stats are filtered too. The
  filtered branch is evaluated before the aggregatedDfs branch; the three cases
  (non-dfs query, dfs-phase compute, dfs query-phase read) are documented in
  ContextIndexSearcher.

Testing

FilterAwareAliasIT proves the behavior on a live cluster:

• testPreFilterClosesScoringSideChannelWhilePostFilterLeaks — through a
  post_filter alias a hidden-only term scores below a control term (the
  leak); through pre_filter they score identically (closed). Both views
  return the same visible set.
• testPreFilterClosesPrefixScoringChannel — same contrast for a
  match_phrase_prefix expansion (the PrefixOracle scoring path).
• testFilteredStatisticsMatchPhysicallyFilteredIndex — the filtered_stats
  BM25 score equals the score from a physically-filtered visible-only index
  (ground-truth df/N correctness, not merely "lower").

Regression: 101 tests across alias / rollover / dfs / ContextIndexSearcher
green; AliasFilterTests covers serialization + pre-3.8 wire BWC.

Scope / follow-ups (not in this PR)

• Raw term-dictionary APIs (A5b) — terms / suggest / _termvectors with
  term statistics bypass the per-request query filter and are not covered;
  they need the enumeration itself intersected with the alias bitset. Documented
  in FILTER_AWARE_ALIASES.md.
• Per-user templated filters + DLS-role bridge (A7/A8) — live in the
  opensearch-project/security repo; design is sketched in
  FILTER_AWARE_ALIASES.md. This PR provides the core mechanism they build on.
• filtered_stats as a first-class wire-level scoring mode — currently gated
  behind a system property rather than an alias field.

Backward compatibility

Default is post_filter — existing aliases, queries, and mixed-version clusters
behave exactly as before. No default path changes.

See server/.../action/admin/indices/alias/FILTER_AWARE_ALIASES.md for the full
design note.

[DarshitChanpura]

End-to-end flow (one line per hop)

POST /_aliases {enforcement: pre_filter}
  → IndicesAliasesRequest.AliasActions.enforcement          [API: parse]
  → TransportIndicesAliasesAction → AliasAction.Add         [API → cluster state]
  → AliasMetadata.enforcement (persisted in cluster state)  [cluster state]
  ── later, a search arrives on the alias ──
  → IndicesService.buildAliasFilter: "pre_filter" → PRE_FILTER   [bridge: string → enum]
  → AliasFilter(enforcement=PRE_FILTER) travels to each shard    [wire, version-gated]
  → DefaultSearchContext: fork on enforcement + filtered_stats   [behavior]
       ├─ post_filter          → filters.add(aliasFilter)        (unchanged; default)
       ├─ pre_filter + constant_score → ConstantScoreQuery wrap  (no IDF computed)
       └─ pre_filter + filtered_stats → filter clause + ContextIndexSearcher filtered N/df
  → (dfs_query_then_fetch) DfsPhase computes per-shard stats via the same filtered overrides

Where the substance is: most files just carry the enforcement flag from the REST
API down to the shard. The actual behavior lives in three places:

• DefaultSearchContext — the fork above.
• ContextIndexSearcher — builds a visible-doc bitset from the alias filter and overrides
  collectionStatistics() / termStatistics() to compute N/df over only that bitset
  (the filtered_stats path).
• DfsPhase — makes the per-shard DFS statistics use those same filtered overrides so
  dfs_query_then_fetch aggregates visible-only numbers across shards.

Default is post_filter — existing aliases, queries, and mixed-version clusters are
unaffected. pre_filter's two scoring sub-behaviors have different cost profiles:
constant_score skips IDF entirely; filtered_stats does per-term postings∩bitset work
and is not yet performance-benchmarked.