Skip to content

fix: ChromaUris could neither write nor read uris on a real chromadb (review of #5) - #6

Merged
thorwhalen merged 1 commit into
masterfrom
fix/chromauris-real-chromadb
Sep 22, 2026
Merged

thorwhalen merged 1 commit into
masterfrom
fix/chromauris-real-chromadb

Conversation

@thorwhalen

Copy link
Copy Markdown
Member

Post-merge adversarial review of #5, run against a real chromadb 1.5.9 in a scratch venv. chromadb is unpinned, so this is what users install.

Defects found (the fix in #5 did not make ChromaUris work)

With a real PersistentClient collection created with data_loader=FileLoader():

ChromaUris(col)["k"] = "/path/a.txt"   -> ValueError: Exactly one of documents, images must be provided in upsert.
ChromaUris(col).append("/path/b.txt")  -> same
ChromaUris(col)["k"]                   -> None, even for a record that has uris
  1. Write. In chromadb, upsert and update embed only documents or images. Only add embeds from uris through the data loader (_validate_and_prepare_upsert_request passes embeddable_fields={"documents", "images"}, while add uses the default set, which includes uris). So fix: ChromaUris was shadowed by a 'metadata' class, so every uris write raised #5's docstring ("needs a collection created with a data_loader") was not enough: the write fails either way. fix: ChromaUris was shadowed by a 'metadata' class, so every uris write raised #5's tests only exercised a recording fake that accepts any kwarg.
  2. Read. A default collection.get(k) includes only documents and metadatas. uris comes back None, so the codec always returned None.

Fix

  • ChromaCollection.get_include is a new class attribute. It defaults to None, which keeps chromadb's default, so ChromaCollection and ChromaDocuments reads are unchanged. ChromaUris sets it to ("uris",).
  • ChromaUris.__setitem__ calls add for new keys. For existing keys it does delete + add, keeping the record's metadata the way upsert does. If the add fails (for example when the loader can't read the uri), it restores the old record with its embeddings. It also turns chromadb's read-back {} metadata into None, since chromadb rejects {} on write.
  • The recording fake now records add too and filters get by id, so the existing kwarg assertions still check the same thing.

Tests

These two fail on master and pass here:

  • test_chroma_uris_round_trips_against_real_chromadb: set, get, overwrite with and without metadata, and append, all on a real collection.
  • test_chroma_uris_overwrite_that_fails_keeps_the_old_record

Locally: 13 passed (py3.12, chromadb 1.5.9). Ruff is clean.

Design concerns (not changed here)

Self-reviewed only (this reviewer was told not to spawn a sub-agent).

🤖 Generated with Claude Code

Post-merge review of #5, against chromadb 1.5.9.

- Writes: upsert embeds only documents or images, so an upsert carrying only
  uris raises "Exactly one of documents, images must be provided", even with
  a data_loader. Only add embeds uris through the data loader. ChromaUris now
  adds new keys, and replaces existing ones (delete + add, keeping metadata
  as upsert would). If the add fails, it restores the old record.
- Reads: a default collection.get leaves uris out, so ChromaUris[k] was
  always None. ChromaCollection gains a get_include seam; ChromaUris sets it
  to ("uris",).
- The recording fake also records add and filters get by id.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@thorwhalen
thorwhalen merged commit ddc30a4 into master Sep 22, 2026
12 checks passed
@thorwhalen
thorwhalen deleted the fix/chromauris-real-chromadb branch September 22, 2026 14:44
@thorwhalen

Copy link
Copy Markdown
Member Author

round-3 review: no defect found. Checked on real chromadb 1.5.9: a failed overwrite of a doc-only record and of a doc+uri+metadata record restores embedding, document, uri and metadata exactly; a collection with no data loader restores after the add fails; a multi-key write with mixed existing and new keys keeps metadata; a missing-key read returns [] (the pre-existing behaviour, already noted).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant