Surfaced by the 2026-06-29 maintenance audit (PR #2). Documented as a "v1" design choice in CLAUDE.md, but it's a real, observable divergence from Upstash on the dense surface up-vector claims to fully support.
Problem
In src/routes/query.ts (~lines 97-100), when a filter is present the server fetches min(topK*3, 1000) nearest-by-vector candidates from RediSearch, then applies the metadata filter in JS and trims to topK. Upstash applies the filter inside the index and returns up to topK matching results regardless of scan depth.
Consequences:
- For a selective filter (say 5% match rate), up-vector returns ~
0.05 × 3 × topK results instead of topK.
- For
topK ≥ 334 (any topK*3 > 1000) there is effectively no over-fetch headroom.
Suggested direction
Push filters into RediSearch: index metadata as TAG/NUMERIC schema fields and translate the filter AST into an FT.SEARCH filter expression. Alternatively, loop/deepen the KNN fetch until topK matches are found or the namespace is exhausted. Needs a design pass (the filter grammar is rich — GLOB/IN/CONTAINS/HAS FIELD/AND/OR/NOT — and not all map cleanly to RediSearch).
Acceptance criteria
- A selective filter returns up to
topK matching results, matching Upstash semantics, including for large topK.
- The full filter operator set (
tests/unit/filter.test.ts) remains supported end-to-end.
Surfaced by the 2026-06-29 maintenance audit (PR #2). Documented as a "v1" design choice in CLAUDE.md, but it's a real, observable divergence from Upstash on the dense surface up-vector claims to fully support.
Problem
In
src/routes/query.ts(~lines 97-100), when afilteris present the server fetchesmin(topK*3, 1000)nearest-by-vector candidates from RediSearch, then applies the metadata filter in JS and trims totopK. Upstash applies the filter inside the index and returns up totopKmatching results regardless of scan depth.Consequences:
0.05 × 3 × topKresults instead oftopK.topK ≥ 334(anytopK*3 > 1000) there is effectively no over-fetch headroom.Suggested direction
Push filters into RediSearch: index metadata as
TAG/NUMERICschema fields and translate the filter AST into anFT.SEARCHfilter expression. Alternatively, loop/deepen the KNN fetch untiltopKmatches are found or the namespace is exhausted. Needs a design pass (the filter grammar is rich — GLOB/IN/CONTAINS/HAS FIELD/AND/OR/NOT — and not all map cleanly to RediSearch).Acceptance criteria
topKmatching results, matching Upstash semantics, including for largetopK.tests/unit/filter.test.ts) remains supported end-to-end.