fix(store): pre-filtered KNN for collection-scoped vector search

[brettdavies]

Collection-scoped vector search as a pre-filtered KNN statement

Problem

searchVec with a collection filter collects the scope's hash_seq keys and evaluates hash_seq IN (400 keys) against vectors_vec in chunks (#847). vec0 uses its key index only for equality, so every chunk is a full pass over the table. On a 793k-vector index (12.6 GB) a 3.8k-key collection pays ten passes of about 0.7s per vector query: qmd vsearch -c solutions spends 30s of a 33s run there, and a default scope of 13 collections recurses once per collection and takes 152s. BM25 stays under a second for the same scopes.

Scopes above 20k keys take the other branch, a capped global top-k with a post-filter, which is the starvation the exact scan was added to avoid. On this index vault holds 20,844 active chunks, so a default-scope search returned one vault document while stars (761k keys) filled the top-k.

Fix

One KNN statement per vector query, whatever the scope:

SELECT hash_seq, distance
FROM vectors_vec
WHERE embedding MATCH ? AND k = ?
  AND hash_seq IN (
    SELECT r.id
    FROM content_vectors cv
    JOIN documents d ON d.hash = cv.hash AND d.active = 1
    JOIN vectors_vec_rowids r ON r.id = cv.hash || '_' || cv.seq
    WHERE d.collection IN (?, ...)
  )

vec0 applies the IN list on its primary key before selecting k, so the result is an exact top-k over the scope and a small collection is never crowded out by a larger one, in or out of scope. The subselect reads content_vectors, documents, and vec0's own key table vectors_vec_rowids; it never touches the vectors_vec virtual table, so the two-statement rule from #23 still holds. The join on vectors_vec_rowids is load-bearing: vec0 0.1.9 answers a KNN whose IN list names a key it does not hold with no rows at all, and a content_vectors row whose vector is missing (the embed path writes the two tables in separate statements) would otherwise blank every scoped search that includes that document. Drawing the list from the shadow table narrows the scope instead.

A scope that holds at least half of the active documents skips the pre-filter: the global KNN runs with an over-fetch of four times k over the scope's share, its rows are post-filtered to the scope, and the result is kept when at least k in-scope rows came back (or the table ran out). The in-scope rows of a global top-k' are the exact in-scope top-m, so that answer is identical to the pre-filtered one; when the scan comes back short the pre-filtered statement runs. This matters because vec0 resolves every key of an IN list one by one, which for a 761k-key scope costs more than the brute-force pass itself. A 19ms document count picks the route, and the global attempt is skipped when its over-fetch would not fit under the 4096 cap, so a large -n or --all never pays both scans.

Without a scope the statement is the unchanged global KNN. Step 2 binds the same names as an IN list. k is three chunks per requested result for each collection in scope, pooled into the one statement and capped at 4096, so a long document in one collection cannot own a union's whole pool; a single scope keeps the previous k of three times the limit. Result assembly is unchanged: one row per file at its best chunk distance, score is one minus distance.

exactVecScanByHashSeq, its two constants, and the per-collection recursion in searchVec are gone. searchFTS and structuredSearch keep their per-collection loops, so RRF weighting does not change.

Measured on a live index

12.6 GB index, 793k vectors; stars holds 761k keys, the 13 default collections 36k, solutions 3.8k. Best of the runs taken outside the index's five-minute update and embed windows. Both columns use the same warm model process for embedding and query expansion, so the difference is the vector stage.

Command	Before	After
qmd vsearch "slow query" -c solutions -n 3	33.3s	6.8s
qmd query "slow query" -c solutions -n 3	32.0s	6.9s
qmd vsearch "slow query" -n 3 (13 collections)	152.0s	7.4s
qmd vsearch "slow query" -c stars -n 3 (761k keys)	18.2s	14.1s

searchVec alone, one precomputed query embedding, read-only connection:

Scope	Before	After
solutions, limit 20	5.6s	1.6s
13 default collections, limit 20	29.4s	2.0s
stars, limit 3	3.8s	3.4s

The stars scope goes through the global-scan route; a pre-filtered statement over its 761k keys measures 5.2s per vector query because vec0 resolves each key in its shadow table one by one, and the old code paid 3.8s collecting the same keys in JavaScript before its global scan.

Results before and after

Ordered result keys from searchVec with the same embedding are identical for solutions at limits 3 and 20, for the default scope at limit 3, and for stars at limit 3. The default scope at limit 20 shares its first eight rows; the old tail held rows scoring 0.541 to 0.547, the new tail holds eight vault files scoring 0.553 to 0.561 that the old code never fetched because vault's 20,844 chunks put it on the global top-k branch.

Tests

test/store.test.ts, "Vector Search collection filter": the #791/#803 and #775 tests pass with their assertions unchanged. Added: two of three collections in scope; a hash shared with an out-of-scope collection returned once under the scoped name; a hash whose only in-scope row is inactive; an active plus an inactive row for one hash; a scoped collection with no vectors; an unknown name; a limit above the k cap; an empty list equals unscoped; chunk collapse across a union; result row fields; a 20,001-chunk scope behind 100 nearer out-of-scope vectors, which returns nothing on main and passes here; a scoped collection that still answers when one chunk row has no vector; a union that keeps a sibling collection reachable behind a long in-scope document; a majority collection served from the global scan, both below and above the over-fetch, plus the same scope not starved when the nearest vectors sit outside it; and a routing test that pins the global scan to a majority scope and the vectors_vec_rowids pre-filter to a minority scope through the prepared statements.

CI=true vitest and bun suites green under Node 26 and Bun 1.4; tsc --noEmit and oxlint clean.

Known limits

• A document that owns every one of a union's pooled candidate chunks still collapses the result to fewer files than -n; the README note names -n and --all as the lever.
• A limit above 1365 loses part of its three-per-result over-fetch because sqlite-vec caps k at 4096.
• Every route still pays the brute-force pass over all stored chunks (about 1.8s on this index). Going below that floor needs a schema change, out of scope here: a vec0 partition key on collection so a scoped scan reads only its own chunks, an integer rowid key so the pre-filter binds rowids instead of text keys, or int8 vectors to shrink the pass.

Related issues

#775, #791, #803 were closed by #847's exact scan, which this replaces. #918 is the keyword-side counterpart for searchFTS.

[OldEricJi]

Independent synthetic reproduction against pinned main 04e4dbd8245c527a88f1a8f0bda547aef9ca81fb (Node 24.15.0, Linux). No real corpus, credentials, or models are involved. I have not tested this PR branch, so this is supporting evidence for the baseline defect, not a claim that this PR passes the reproducer.

This PR addresses the provider-side behavior we need. We reproduced a sharp eligible-set threshold on the baseline using 3D synthetic vectors and a precomputed query:

Eligible chunks	Closer ineligible chunks	Requested documents	Returned
20,000	200	5	5
20,001	200	5	0

The eligible vectors are [0,1,0], excluded vectors and query are [1,0,0]. The >20k branch reverts to global KNN followed by filtering. An explicit scope/budget limitation would also be preferable to a successful false-empty result.

There is a second, distinct document-top-k edge worth retaining when changing this path: one eligible document with 20 close chunks and another eligible document with one farther chunk, requested limit=2, returns only one document on the baseline exact-scan path. Truncating to limit*3 chunks before file deduplication can underfill document results even after collection/metadata prefiltering. This is not a request for multiple passages per file.

Our small exact-scan candidate retains the best chunk per content hash before truncation; downstream filepath mapping and metadata filtering remain intact. It fixes that second case, including a noneligible path sharing the content hash and the best chunk crossing the 400-key batch boundary. It deliberately does not fix the >20k branch, and may not apply directly after this PR removes the exact-scan helper. Sharing it as a test/design reference, not a proposed competing long-term fork.

Diagnostic instructions: save the script as vector-scope-probe.mjs in a prepared checkout; use isolated QMD_CONFIG_DIR/XDG_CACHE_HOME and run QMD_UPSTREAM_DIR="$PWD" node --import tsx vector-scope-probe.mjs. --assert-chunks checks the chunk-dedup candidate; --assert-complete should still fail with only that candidate applied until the large-scope problem is addressed. No embed/download command is used; synthetic dimensions do not establish real embedding quality or production latency.

Self-contained vector diagnostic

import assert from 'node:assert/strict';
import { pathToFileURL } from 'node:url';
import { resolve } from 'node:path';
import { performance } from 'node:perf_hooks';
const root=process.env.QMD_UPSTREAM_DIR;
if(!root) throw new Error('Set QMD_UPSTREAM_DIR to the isolated pinned QMD checkout.');
const {createStore,searchVec,insertContent,insertDocument,insertEmbedding,hashContent,DEFAULT_EMBED_MODEL}=await import(pathToFileURL(resolve(root,'src/store.ts')).href);
const {replaceDocumentMetadata}=await import(pathToFileURL(resolve(root,'src/metadata-store.ts')).href);
const {METADATA_EXTRACTION_VERSION}=await import(pathToFileURL(resolve(root,'src/metadata.ts')).href);
const now='2026-09-16T00:00:00Z', filter={key:'eligible',operator:'eq',value:true};
async function doc(store,name,eligible,embedding,chunks=1) {
  const body=`# ${name}`, hash=await hashContent(body);
  insertContent(store.db,hash,body,now);
  const id=insertDocument(store.db,'book',`${name}.md`,name,hash,now,now);
  replaceDocumentMetadata(store.db,id,{metadata:{eligible},extractionVersion:METADATA_EXTRACTION_VERSION});
  for(let seq=0;seq<chunks;seq++) insertEmbedding(store.db,hash,seq,seq*100,new Float32Array(embedding),DEFAULT_EMBED_MODEL,now,chunks);
  return hash;
}
async function search(store,limit) {
  const start=performance.now();
  const rows=await searchVec(store.db,'synthetic',DEFAULT_EMBED_MODEL,limit,'book',undefined,[1,0,0],undefined,filter);
  assert.ok(rows.every(row=>row.metadata.eligible===true),'scope correctness must hold');
  return {count:rows.length,files:rows.map(row=>row.displayPath),ms:Math.round(performance.now()-start)};
}
const large=createStore(':memory:');
const chunked=createStore(':memory:');
try {
  large.ensureVecTable(3); chunked.ensureVecTable(3);
  large.db.exec('BEGIN');
  for(let i=0;i<20000;i++) await doc(large,`eligible-${i}`,true,[0,1,0]);
  for(let i=0;i<200;i++) await doc(large,`future-${i}`,false,[1,0,0]);
  large.db.exec('COMMIT');
  const at20000=await search(large,5);
  assert.equal(at20000.count,5);
  await doc(large,'eligible-20000',true,[0,1,0]);
  const at20001=await search(large,5);
  const sharedHash=await doc(chunked,'many-chunks',true,[1,0,0],20);
  const excludedId=insertDocument(chunked.db,'book','excluded-copy.md','excluded-copy',sharedHash,now,now);
  replaceDocumentMetadata(chunked.db,excludedId,{metadata:{eligible:false},extractionVersion:METADATA_EXTRACTION_VERSION});
  await doc(chunked,'second-document',true,[0,1,0]);
  const chunkTop2=await search(chunked,2);
  assert.ok(!chunkTop2.files.includes('book/excluded-copy.md'));
  if(process.argv.includes('--assert-chunks')) assert.equal(chunkTop2.count,2);
  for(let seq=0;seq<20;seq++) insertEmbedding(chunked.db,sharedHash,seq,seq*100,new Float32Array(seq===19?[1,0,0]:[0,0,1]),DEFAULT_EMBED_MODEL,now,20);
  const bestChunk=await searchVec(chunked.db,'synthetic',DEFAULT_EMBED_MODEL,2,'book',undefined,[1,0,0],undefined,filter);
  assert.equal(bestChunk[0].chunkPos,1900,'best chunk position must survive deduplication');
  for(let seq=0;seq<450;seq++) insertEmbedding(chunked.db,sharedHash,seq,seq*100,new Float32Array(seq===449?[1,0,0]:[0,0,1]),DEFAULT_EMBED_MODEL,now,450);
  const acrossBatch=await searchVec(chunked.db,'synthetic',DEFAULT_EMBED_MODEL,2,'book',undefined,[1,0,0],undefined,filter);
  assert.equal(acrossBatch[0].chunkPos,44900,'best chunk across the 400-row scan batch must survive');
  if(process.argv.includes('--assert-chunks')||process.argv.includes('--assert-complete')) assert.equal(acrossBatch.length,2);
  console.log(JSON.stringify({at20000,at20001,chunkTop2,acrossBatch:{count:acrossBatch.length,bestPosition:acrossBatch[0].chunkPos},expected:{large:5,chunked:2},modelsDownloaded:false},null,2));
  if(process.argv.includes('--assert-complete')) {
    assert.equal(at20001.count,5,'eligible-set threshold must not change recall correctness');
    assert.equal(chunkTop2.count,2,'chunks of one document must not starve another eligible document');
  }
} finally {large.close();chunked.close();}

Isolated exact-scan document-dedup candidate

diff --git a/src/store.ts b/src/store.ts
index 3a45bbc..acea152 100644
--- a/src/store.ts
+++ b/src/store.ts
@@ -4201,9 +4191,9 @@ function exactVecScanByHashSeq(
   if (hashSeqs.length === 0 || limit <= 0) return [];
 
   const queryVec = new Float32Array(embedding);
-  // Over-fetch a bit so multi-chunk docs can still yield `limit` unique files.
-  const fetchLimit = Math.max(limit * 3, limit);
-  const scored: { hash_seq: string; distance: number }[] = [];
+  // Preserve the best chunk per content before truncating candidates. A long
+  // document must not occupy the entire window with near-identical chunks.
+  const best = new Map<string, { hash_seq: string; distance: number }>();
 
   for (let i = 0; i < hashSeqs.length; i += VEC_HASH_SEQ_IN_CHUNK) {
     const chunk = hashSeqs.slice(i, i + VEC_HASH_SEQ_IN_CHUNK);
@@ -4213,11 +4203,14 @@ function exactVecScanByHashSeq(
       FROM vectors_vec
       WHERE hash_seq IN (${placeholders})
     `).all(queryVec, ...chunk) as { hash_seq: string; distance: number }[];
-    scored.push(...rows);
+    for (const row of rows) {
+      const hash = row.hash_seq.slice(0, row.hash_seq.lastIndexOf("_"));
+      const previous = best.get(hash);
+      if (!previous || row.distance < previous.distance) best.set(hash, row);
+    }
   }
 
-  scored.sort((a, b) => a.distance - b.distance);
-  return scored.slice(0, fetchLimit);
+  return [...best.values()].sort((a, b) => a.distance - b.distance).slice(0, limit);
 }
 
 function annVecScan(