Detect near-duplicate memories at store time

[himanshu231204]

Problem

Agents restate things. Over a few months, a memory store accumulates half a dozen near-identical versions of "we use SQLite", which crowd out other results and make search worse the longer you use the tool.

The embedding needed to spot this is already computed on every add() — the similarity check is nearly free.

What to build

At store time, compare the new memory's embedding against existing ones. If similarity exceeds a threshold, surface it rather than silently storing another copy.

Acceptance criteria

• add() optionally checks for near-duplicates using the embedding it already computes
• store_memory returns any near-duplicate it found — id, content, and similarity — alongside the stored memory
• Configurable threshold, off or permissive by default
• Adds no second embedding call, and no full extra scan when the store is large — reuse the search path
• Tests: exact restatement detected, paraphrase detected, genuinely different memory not flagged, threshold respected
• Docs in docs/guide/mcp-tools.md plus CHANGELOG.md

Design question

Warn, or refuse?

I'd argue warn. Refusing means an agent's store_memory silently fails and it has no idea its memory wasn't kept — worse than a duplicate. Returning duplicate_of: 12 lets a well-behaved agent decide: skip it, or call update_memory (#17) to refine the existing one instead.

That framing makes this genuinely useful rather than annoying — the tool docstring can tell the model that a returned duplicate is an invitation to update rather than restate.

Notes

The threshold matters. Too low and every related memory looks like a duplicate; too high and paraphrases slip through. Worth reporting the similarity distribution from a real corpus in the PR rather than picking a number.

Pointers

• src/localmem_mcp/store.py — add() computes the vector; search() has the scoring machinery to reuse

[himanshu231204]

/claim

[github-actions]

🎉 Thanks @himanshu231204!

This issue has been assigned to you.

Happy coding! 🚀