Add GoodMemory adapter (LoCoMo / LongMemEval / BEAM)

README.md

@@ -76,6 +76,59 @@ python -m benchmarks.beam.run --project-name my-first-test --chat-sizes 100K --c
 
 By default, the OSS server uses OpenAI for fact extraction (`gpt-4o-mini`) and embeddings (`text-embedding-3-small`). See [Custom Models](#custom-models) for using Azure, Ollama, or other providers.
 
+### Option C: GoodMemory (Self-Hosted Bridge)
+
+[GoodMemory](https://github.com/hjqcan/GoodMemory) runs as a single-process HTTP
+bridge (Bun, bearer-token auth required by default). The benchmark numbers come
+from GoodMemory's LLM-assisted extraction **plus** its semantic candidate union,
+so the bridge must be started with all four pieces — an LLM extractor, an
+embedding endpoint, the `recommended` retrieval preset (the semantic union), and
+in-memory storage (its pure-JS vector index needs no native libraries). A bare
+rules-only / no-embedding / no-preset bridge under-reports badly:
+
+```bash
+# From the GoodMemory repo. Any OpenAI-compatible endpoint works (OpenAI,
+# OpenRouter, Azure, …); pick an extractor model that emits well-formed
+# extractions (e.g. gpt-4o-mini) — a weak proxy can yield empty facts.
+GOODMEMORY_HTTP_BRIDGE_TOKEN=your-token \
+GOODMEMORY_STORAGE_PROVIDER=memory \
+GOODMEMORY_HTTP_BRIDGE_RETRIEVAL_PRESET=recommended \
+GOODMEMORY_ASSISTED_EXTRACTOR_PROVIDER=openai \
+GOODMEMORY_ASSISTED_EXTRACTOR_MODEL=gpt-4o-mini \
+GOODMEMORY_ASSISTED_EXTRACTOR_API_KEY=$OPENAI_API_KEY \
+GOODMEMORY_ASSISTED_EXTRACTOR_BASE_URL=https://api.openai.com/v1 \
+GOODMEMORY_EMBEDDING_PROVIDER=openai \
+GOODMEMORY_EMBEDDING_MODEL=text-embedding-3-small \
+GOODMEMORY_EMBEDDING_API_KEY=$OPENAI_API_KEY \
+GOODMEMORY_EMBEDDING_BASE_URL=https://api.openai.com/v1 \
+  bun scripts/goodmemory-http-bridge.ts --port 8739
+# Bridge:  http://localhost:8739  (health: /healthz)
+```
+
+Then point any benchmark at it with `MEMORY_SYSTEM=goodmemory`:
+
+```bash
+export MEMORY_SYSTEM=goodmemory
+export GOODMEMORY_BRIDGE_HOST=http://localhost:8739
+export GOODMEMORY_HTTP_BRIDGE_TOKEN=your-token
+
+python -m benchmarks.locomo.run --project-name goodmemory-test
+python -m benchmarks.longmemeval.run --project-name goodmemory-test --all-questions
+python -m benchmarks.beam.run --project-name goodmemory-test --chat-sizes 100K --conversations 0-9
+```
+
+The adapter (`benchmarks/common/goodmemory_client.py`) speaks GoodMemory's HTTP
+bridge contract and returns the same result shape as the Mem0 client, so the
+runners are otherwise unchanged. `add` drives GoodMemory's LLM-assisted
+extraction; `search` requests hybrid retrieval and maps GoodMemory's ranked
+recall to per-memory results with rank-descending scores, so the harness's
+cutoff slicing measures GoodMemory's own ranking. Set
+`GOODMEMORY_BRIDGE_EXTRACTION_STRATEGY=rules-only` only to measure the
+deterministic floor.
+
+> Requires a GoodMemory build whose bridge supports
+> `GOODMEMORY_HTTP_BRIDGE_RETRIEVAL_PRESET` (the semantic-union preset flag).
+
 ### View results in the UI
 
 ```bash

benchmarks/beam/run.py

@@ -1021,12 +1021,23 @@ async def async_main() -> None:
 
     # Init clients
     backend = os.getenv("MEM0_BACKEND", args.backend)
-    mem0 = Mem0Client(
-        mode=backend,
-        host=args.mem0_host,
-        api_key=args.mem0_api_key if backend == "cloud" else None,
-        rpm=args.rpm,
-    )
+    # MEMORY_SYSTEM selects the backend memory system (default: mem0). GoodMemory
+    # is served over its HTTP bridge; the client duck-types Mem0Client, so the
+    # rest of the runner is unchanged. See benchmarks/common/goodmemory_client.py.
+    if os.getenv("MEMORY_SYSTEM", "mem0").lower() == "goodmemory":
+        from benchmarks.common.goodmemory_client import GoodMemoryClient
+        mem0 = GoodMemoryClient(
+            host=os.getenv("GOODMEMORY_BRIDGE_HOST"),
+            token=os.getenv("GOODMEMORY_HTTP_BRIDGE_TOKEN") or os.getenv("GOODMEMORY_BRIDGE_TOKEN"),
+            rpm=args.rpm,
+        )
+    else:
+        mem0 = Mem0Client(
+            mode=backend,
+            host=args.mem0_host,
+            api_key=args.mem0_api_key if backend == "cloud" else None,
+            rpm=args.rpm,
+        )
     answerer = LLMClient(
         model=args.answerer_model, provider=args.provider, rpm=args.rpm
     )