Skip to content

Add add_many for batch embedding #24

Description

@himanshu231204

Problem

add() embeds one string at a time. fastembed's embed() takes an iterable and batches internally, so storing 1,000 memories one by one pays per-call overhead a thousand times over.

This blocks bulk import (#20) from being usable at any real size, and makes the benchmark harness (#21) slow to set up.

What to build

MemoryStore.add_many(items: Sequence[MemoryInput]) -> list[Memory]

One embedding call for the whole batch, one transaction for the inserts.

Acceptance criteria

  • Single embed() call for the batch, not one per item
  • Single transaction — a partial batch on failure is worse than none
  • Returns stored Memory records in input order, with their ids
  • Same validation as add(): empty content rejected, tags normalized
  • Configurable batch size, so a 100k import doesn't try to embed everything at once and exhaust memory
  • Tests: ordering preserved, all records findable afterwards, empty input handled, rollback leaves the database unchanged
  • Benchmark showing the improvement over looping add(), posted on the issue
  • Docs in docs/guide/python-library.md plus CHANGELOG.md

Notes

Deliberately library-only for now — no MCP tool. An agent storing memories one at a time is the normal case, and a bulk tool invites an agent to dump a conversation into memory wholesale, which is exactly the low-value noise that degrades search. Bulk import is a human operation.

Pointers

  • src/localmem_mcp/store.pyadd(); note self.embedder.embed() already accepts a sequence
  • The _lock and with self._conn transaction pattern in add()

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: python-apiMemoryStore and the importable libraryenhancementNew feature or requestperformanceSpeed, memory use, scalingv0.3: scaleBenchmarks, vector index, batch embedding

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions