Add GoodMemory adapter (LoCoMo / LongMemEval / BEAM)

[hjqcan]

Add GoodMemory adapter (LoCoMo / LongMemEval / BEAM)

What this adds

A drop-in adapter so the benchmark suite can evaluate GoodMemory alongside Mem0.

GoodMemory is served over its HTTP bridge — a single-process sidecar (Bun,
bearer-token auth required by default). The new client duck-types the existing
Mem0Client async interface (add / search / delete_user), so the three
runners select it via an environment switch and are otherwise unchanged.

File	Change
benchmarks/common/goodmemory_client.py	new — HTTP-bridge client with Mem0-parity interface
benchmarks/{locomo,longmemeval,beam}/run.py	MEMORY_SYSTEM=goodmemory switch at client init (identical 12-line block)
README.md	Quick-start "Option C: GoodMemory"

No new dependencies (reuses aiohttp / aiolimiter). Runner logic, prompts,
metrics, and output schema are untouched.

How to run

Start the bridge with the full recall stack, then point any benchmark at it:

GOODMEMORY_HTTP_BRIDGE_TOKEN=your-token \
GOODMEMORY_STORAGE_PROVIDER=memory \
GOODMEMORY_HTTP_BRIDGE_RETRIEVAL_PRESET=recommended \
GOODMEMORY_ASSISTED_EXTRACTOR_PROVIDER=openai \
GOODMEMORY_ASSISTED_EXTRACTOR_MODEL=gpt-4o-mini \
GOODMEMORY_ASSISTED_EXTRACTOR_API_KEY=$OPENAI_API_KEY \
GOODMEMORY_EMBEDDING_PROVIDER=openai \
GOODMEMORY_EMBEDDING_MODEL=text-embedding-3-small \
GOODMEMORY_EMBEDDING_API_KEY=$OPENAI_API_KEY \
  bun scripts/goodmemory-http-bridge.ts --port 8739

export MEMORY_SYSTEM=goodmemory
export GOODMEMORY_BRIDGE_HOST=http://localhost:8739
export GOODMEMORY_HTTP_BRIDGE_TOKEN=your-token
python -m benchmarks.locomo.run --project-name goodmemory-test

The four bridge pieces all matter (see README Option C): a well-formed LLM
extractor, an embedding endpoint, the recommended retrieval preset
(GoodMemory's semantic candidate union), and in-memory storage (its pure-JS
vector index needs no native libraries). Omitting any of them makes recall fall
back to a lexical floor and under-reports.

Design notes

• add → POST /memory/remember with extractionStrategy: llm-assisted
  (the bridge defaults to a deterministic rules-only floor).
• search → POST /memory/recall-context with strategy: hybrid — this
  activates GoodMemory's semantic recall (embedding + BM25 + candidate union).
  GoodMemory returns items already ordered by its recall ranking; the adapter
  assigns rank-descending scores so the harness's sorted(score desc)[:cutoff]
  slice measures GoodMemory's own ranking rather than reordering by a
  memory-quality confidence (not a retrieval-relevance signal). maxTokens is
  sized from top_k.
• delete_user is best-effort (export + per-memory forget); the runners do
  not call it.
• Scope is carried per request via x-goodmemory-* headers and a matching
  scope body object; the bearer token is the security boundary.

Verification

• Adapter contract — live protocol (add drives real llm-assisted
  extraction, search round-trips mem0-shaped {memory, score, id} with
  non-increasing scores, delete_user round-trips) plus a deterministic unit
  check of the ranking/slicing map ([1.0, 0.8, 0.6, 0.4, 0.2], honored
  top_k, distinct cutoff slices). All pass.
• End-to-end relevance — against a fully-configured local bridge, cross-topic
  probes surfaced the query-relevant fact at the top of recall in every case
  (4/4: "what car" → the car fact, "dietary" → the allergy fact, etc.), with
  routing.resolvedStrategy = hybrid.

Requires a GoodMemory build whose bridge supports
GOODMEMORY_HTTP_BRIDGE_RETRIEVAL_PRESET. A full scored suite run (with the
LoCoMo/LongMemEval/BEAM datasets and answer+judge model keys) is left to the
maintainers / a follow-up.

🤖 Generated with Claude Code

[hjqcan]

Closing as a duplicate of #17, which adds the same GoodMemory backend and predates this one. Improvements from this branch (hybrid-retrieval recall path + the bridge's semantic-union preset) will be folded into #17 instead. Apologies for the noise.