Summary
qmd vsearch takes ~70 s on a single ~11k-vector collection. Two things multiply:
Collection-scoped searches use the exact-scan path (exactVecScanByHashSeq, added for bug: Vector search returns nothing for small collections: global ANN candidates are filtered *after* retrieval #791 vsearch -c <collection> returns nothing for small collections (filter applied after global ranking) #803 COLLECTION_VEC_EXACT_SCAN_MAX (20,000) vectors. It computes vec_distance_cosine(embedding, ?) over hash_seq IN (…) lists, so sqlite-vec reads each vector individually from the chunk blobs via overflow pages.Expanded queries are not deduplicated. vectorSearchQuery runs the exact scan once per expansion. For my query, the cached expansion had 31 vec/hyde entries, only 9 of them distinct, so 32 scans instead of 10.
Environment
qmd 2.8.3 (bun global), node-llama-cpp 3.20.0, Apple M5 Max, macOS 26.6.2
Default collections: akasha (11,390 vectors, 5,787 docs) and fumu-ch (56 vectors)
vectors_vec holds 52,877 rows in total (the rest belong to inactive/old documents); index.sqlite 290 MB
Measurement
qmd vsearch "Delegieren" -n 4, no other qmd process running, expansion already cached. Timestamps per stderr/stdout line:
0.1s Searching 32 vector queries...
0.3s [node-llama-cpp] ggml_metal_library_init_from_source: error compiling source
70.9s qmd://akasha/... (first result)
So ~70 s is spent in the vector phase, ~2.2 s per query. Embedding the queries themselves takes a few ms each on Metal.
A sample of the process during that phase shows the main thread almost entirely in sqlite-vec column reads:
sqlite3_step
sqlite3VdbeExec
vec0Column
vec0_get_vector_data
blobReadWrite
accessPayload
getOverflowPage
btreeGetPage → readDbPage → unixRead → pread
Possible directions (just ideas)
Related: #791 , #803 .
Summary
qmd vsearchtakes ~70 s on a single ~11k-vector collection. Two things multiply:exactVecScanByHashSeq, added for bug: Vector search returns nothing for small collections: global ANN candidates are filtered *after* retrieval #791/vsearch -c <collection> returns nothing for small collections (filter applied after global ranking) #803) whenever the collection has ≤COLLECTION_VEC_EXACT_SCAN_MAX(20,000) vectors. It computesvec_distance_cosine(embedding, ?)overhash_seq IN (…)lists, so sqlite-vec reads each vector individually from the chunk blobs via overflow pages.vectorSearchQueryruns the exact scan once per expansion. For my query, the cached expansion had 31 vec/hyde entries, only 9 of them distinct, so 32 scans instead of 10.Environment
akasha(11,390 vectors, 5,787 docs) andfumu-ch(56 vectors)vectors_vecholds 52,877 rows in total (the rest belong to inactive/old documents); index.sqlite 290 MBMeasurement
qmd vsearch "Delegieren" -n 4, no other qmd process running, expansion already cached. Timestamps per stderr/stdout line:So ~70 s is spent in the vector phase, ~2.2 s per query. Embedding the queries themselves takes a few ms each on Metal.
A
sampleof the process during that phase shows the main thread almost entirely in sqlite-vec column reads:Possible directions (just ideas)
queryTextsinvectorSearchQuery(and probably in the hybrid path) before searching.kup to 4096) plus post-filter first, and fall back to the exact scan only when that yields fewer thanlimitresults for the collection. That keeps the bug: Vector search returns nothing for small collections: global ANN candidates are filtered *after* retrieval #791/vsearch -c <collection> returns nothing for small collections (filter applied after global ranking) #803 fix for small collections without paying the full scan for larger ones.vec0are much more expensive than the ANN scan.Related: #791, #803.