Nominate independently operated memory systems for Phase 3 experiment

[thomaswillner]

Phase 3 system nominations

LLM Errata is seeking nominations for three independently operated AI-memory
systems for a synthetic-data interoperability experiment.

Repository: https://github.com/thomaswillner/llm-errata

Current verdict: NOT_PROD_READY. The experiment has not started.

Required experiment shape

One synthetic memory root must receive a correction, supersession, and erasure
across three independently operated systems. The set must include:

• one intentionally nonconforming importer;
• one system whose coverage is genuinely incomplete or opaque; and
• at least one system capable of recording derivation across mixed artifacts.

Measurements include observation-to-quarantine time, known-descendant coverage,
stale-behavior rate, replacement activation, collateral retention, stale-reimport
resistance, opaque coverage, operator effort, and user-visible friction.

Nomination requirements

Please name:

• system, operator, public documentation, and version;
• memory stores and derivation surfaces exposed;
• API, account, cost, and data-residency requirements;
• deletion, correction, export, and audit capabilities;
• known opaque stores or unsupported coverage; and
• whether the operator can authorize a synthetic experiment and publish results.

Nomination is not approval. Every system, account, integration, cost, and
interaction requires explicit owner authorization before the experiment begins.

[thomaswillner]

Phase 3 nomination contract is now included in draft PR #8:

#8

Exact current implementation target: 35052868cde8b6f16fc6c20b4136ae0277f9a00e.

No experiment has started. Three independently operated systems, two independent adapters, a separate validator, owner authorization for synthetic data/account/cost, and external review remain required before execution. Please nominate systems using the fields in PHASE3_SYSTEMS.md, including opaque stores and authorization/publication constraints.

Status remains NOT_PROD_READY; a nomination is not approval or interoperability evidence.

[thomaswillner]

@kartik-mem0 — inviting Mem0 as a candidate independently operated system for a later synthetic-data LLM Errata interoperability experiment.

Why this fit is specific: Mem0 is a large, actively maintained memory layer with multiple SDKs/integrations and vector/graph-backed behavior. The experiment needs real systems where correction, descendant coverage, stale recall, and opaque provider-controlled surfaces can be measured rather than assumed.

Nomination contract: https://github.com/thomaswillner/llm-errata/blob/agent/g2-publication/PHASE3_SYSTEMS.md
Draft program: #8

This is only a nomination request:

• Phase 2 is incomplete and no experiment has started.
• No account, data, cost, or integration is authorized by this invitation.
• A Mem0 operator would need to name version, stores, correction/deletion/export/audit capabilities, opaque surfaces, and whether synthetic results may be published.
• System operation must remain separate from validator authorship.
• Status remains NOT_PROD_READY.

If this is of interest, a brief nomination naming OSS or hosted scope and an operator contact is enough for the next planning gate.

[thomaswillner]

@Vasilije1990 — inviting Cognee as a candidate independently operated system for the later three-system LLM Errata experiment.

Why this fit is specific: Cognee spans knowledge graph, vector, relational/session, and multiple storage backends, including explicitly demo-vs-production boundaries. That makes it useful for measuring known-descendant coverage and honestly reporting stores that cannot be fully inspected or repaired.

Nomination contract: https://github.com/thomaswillner/llm-errata/blob/agent/g2-publication/PHASE3_SYSTEMS.md
Draft program: #8

This is only a nomination request:

• Phase 2 is incomplete and no experiment has started.
• No account, API key, cost, data transfer, or deployment is authorized.
• Please name candidate version/backend, operator, exposed lineage, correction/deletion/export/audit behavior, opaque stores, and publication constraints.
• Operated-system evidence must be independent of validator authorship.
• Status remains NOT_PROD_READY.

A preliminary “possible/impossible and why” response would already improve experiment design.

[thomaswillner]

Final Phase 2 target available for three-system nominations

Internal Phase 2 is complete and frozen at:

• commit 50e895fbfec544b16c94caa07bf2d1f4049a42e2
• canonical G2 surface digest 9547aec8328b601489dda067c6e62f287229b2b24a413dac2c9e7be98e429804
• Phase 3 contract: https://github.com/thomaswillner/llm-errata/blob/50e895fbfec544b16c94caa07bf2d1f4049a42e2/PHASE3_SYSTEMS.md
• draft program: Add provider-neutral semantic probes and independent validation calls #8

Three independently operated real memory systems are still needed for a synthetic-data experiment. A nomination should identify system and exact version, operator, stores and lineage visibility, correction/deletion/export/audit capabilities, opaque or provider-controlled surfaces, account and cost requirements, data constraints, and whether raw artifacts and results may be published.

No experiment has started. Nomination does not authorize account creation, credentials, cost, deployment, data transfer, or publication. Execution additionally requires two independent adapters, a separate validator, qualifying external review, and explicit operator authorization for every system.

G5 remains BLOCKED; repository remains NOT_PROD_READY. Zero systems have supplied qualifying authorization or experiment evidence so far.

[thomaswillner]

Corrected Phase 3 target and operator nomination request

The earlier frozen target in this thread is historical. The current experiment contract is:

• commit 08b95263c9ed700c43aea0b285696956cc23e878
• canonical G2 digest 03abc492319b875a7d528e0e8de05714bc5a7219b42031fc7c3f42cff1f0bf14
• Phase 3 system contract
• draft PR #8

@kartik-mem0 — Mem0 remains relevant because its hosted and open-source surfaces expose real vector/graph-backed memory behavior and provider-controlled boundaries. Would you kindly nominate a specific Mem0 version and OSS or hosted scope that an independent operator could use for the synthetic-data experiment, or state why the experiment is impractical?

@Vasilije1990 — Cognee remains relevant because its graph, vector, relational/session, and multiple-backend surfaces can test descendant coverage and honest reporting of opaque stores. Would you kindly nominate a specific Cognee version/backend and operator scope, or state why the experiment is impractical?

A nomination is not authorization. No account creation, credential use, cost, deployment, data transfer, or experiment is requested by this comment. A later run requires explicit operator authorization, synthetic data, independent adapters, a separate validator, and publishable raw evidence. Invitations do not satisfy G5. Current verdict remains NOT_PROD_READY; G5 remains BLOCKED.