Add GoodMemory as a third backend (local HTTP bridge)#17
Open
hjqcan wants to merge 3 commits into
Open
Conversation
Adds benchmarks/common/goodmemory_client.py - an async client for the GoodMemory HTTP bridge (npm i -g goodmemory; goodmemory-http-bridge) exposing the same add/search/delete_user interface as Mem0Client - and wires --backend goodmemory (+ --goodmemory-host) into the LOCOMO, LongMemEval, and BEAM runners. Notes: - Bearer-token auth (GOODMEMORY_HTTP_BRIDGE_TOKEN) and the bridge's per-request caller header are handled automatically. - Benchmark seeding writes every turn deterministically (verified fact annotations + rules-only extraction), mirroring how GoodMemory's own benchmark harnesses ingest conversations; assistant turns are stored as user-scoped facts with the original role preserved in the content, since assistant-authored writes are blocked by the product write policy. - Runs are isolated by scope (user ids embed the project name); the bridge has no bulk delete endpoint, so delete_user relies on fresh project names. - Smoke-tested end to end against a live bridge (add -> search round trip).
…tack The bridge coerces any non-hybrid recall strategy (including auto) to a rules-only lexical floor, which cannot surface semantically-relevant facts. Default recall_strategy to hybrid so representative recall works out of the box (override via GOODMEMORY_RECALL_STRATEGY), and document the bridge config it needs: an embedding endpoint, the recommended retrieval preset (semantic candidate union), and in-memory storage. Deterministic verbatim seeding is unchanged (no LLM extractor required). Validated 4/4 on cross-topic recall probes (the query-relevant fact ranks first).
Author
|
Refreshed the GoodMemory adapter in commit Validation:
The PR remains ready for maintainer review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds GoodMemory — a local-first memory layer for AI products and coding agents — as a third backend next to Mem0 Cloud/OSS.
What's included
benchmarks/common/goodmemory_client.py: async client for GoodMemory's packaged HTTP bridge (npm i -g goodmemory && goodmemory-http-bridge, no Docker), exposing the sameadd/search/delete_userinterface asMem0Client.--backend goodmemory(+--goodmemory-host) wired into the LOCOMO, LongMemEval, and BEAM runners.Design notes
GOODMEMORY_HTTP_BRIDGE_TOKEN, picked up automatically) plus a per-request caller header — both handled inside the client.--project-name); the bridge intentionally has no bulk delete endpoint, sodelete_userdocuments that and relies on fresh project names.Testing
Smoke-tested end to end against a live local bridge: two-turn add → search round trip recalls the seeded facts (including the role-prefixed assistant turn), with token auth active.