Problem
_cosine() is a pure-Python loop over 384 floats, called once per stored memory per search. It also recomputes the norm of every stored vector on every query, even though those norms never change.
This may be the entire scaling problem — and fixing it is far simpler than adding a vector index (#22).
Ideas, roughly in order of effort
- Normalise vectors at write time. If stored vectors are unit length, cosine similarity is just a dot product. This removes a square root and a full pass per row, per query, and costs one column or a convention.
- Vectorise the scan. Load embeddings into a single array and score with one matrix multiply rather than N Python loops.
- Skip the round trip. Unpacking every blob into a Python list allocates heavily; operating on buffers directly avoids it.
Idea 1 is likely the best ratio of gain to complexity, and it's backward-compatible if existing vectors are normalised on read.
Acceptance criteria
Why this matters more than it looks
If this makes a 100k-row scan fast enough, #22 can be closed and the project keeps its exact search, its single-file property, and its dependency-light install. That's a considerably better outcome than adding an index.
Pointers
src/localmem_mcp/store.py — _cosine(), _pack(), _unpack(), and the loop in search()
Problem
_cosine()is a pure-Python loop over 384 floats, called once per stored memory per search. It also recomputes the norm of every stored vector on every query, even though those norms never change.This may be the entire scaling problem — and fixing it is far simpler than adding a vector index (#22).
Ideas, roughly in order of effort
Idea 1 is likely the best ratio of gain to complexity, and it's backward-compatible if existing vectors are normalised on read.
Acceptance criteria
Why this matters more than it looks
If this makes a 100k-row scan fast enough, #22 can be closed and the project keeps its exact search, its single-file property, and its dependency-light install. That's a considerably better outcome than adding an index.
Pointers
src/localmem_mcp/store.py—_cosine(),_pack(),_unpack(), and the loop insearch()