Skip to content

fix: apply appendable outside the value codec so append/extend work - #3

Merged
thorwhalen merged 1 commit into
masterfrom
fix/appendable-outside-value-codec
Sep 22, 2026
Merged

thorwhalen merged 1 commit into
masterfrom
fix/appendable-outside-value-codec

Conversation

@thorwhalen

Copy link
Copy Markdown
Member

Summary

ChromaDocuments/ChromaUris stacked @appendable(...) inside @ValueCodecs.single_nested_value(...). dol's class-wrap re-installs the inner append/extend as DelegatedAttributes bound to the unwrapped leaf store, so .append/.extend wrote raw values straight to ChromaCollection.__setitem__, which expects a mapping — every call raised TypeError. The feature was entirely dead (its own demo was commented out in __init__.py and the old test file). Root cause: i2mint/dol#18, this repo's instance: #2.

Fix: apply appendable outside the value codec on all three field-scoped classes, so append/extend go through the wrapper's __setitem__ (and therefore the codec) like store[k] = v already did. Added AppendableChromaCollection for callers who want to append raw chromadb kwargs mappings instead of a single field's value.

No dependents of chromadol in the fleet manifest. Not a breaking change in practice — .append/.extend raised unconditionally before this fix, so nothing could have been relying on their old behaviour.

(Pre-existing, unrelated bug noticed in passing and left untouched: the two decorator stacks for uris and metadata both define a class named ChromaUris at module scope, so the second silently shadows the first — chromadol.base.ChromaUris currently resolves to the metadata-field class, and the uris-field class is unreachable by name. Out of scope for this branch; happy to file a follow-up if wanted.)

Test plan

  • wads ci-local: ruff format + lint, pytest on py3.10/py3.12 (7 passed, incl. 2 new tests pinning append/extend go through the codec and AppendableChromaCollection takes raw kwargs), build — all green.
  • Not covered by the local gate: Windows matrix, any Linux-only job, secrets-dependent jobs (none apply to this change).

Closes #2.

🤖 Generated with Claude Code

`ChromaDocuments` and `ChromaUris` stacked `appendable` *inside* the
`ValueCodecs.single_nested_value` wrapper. dol's class-wrapping then
re-installs `append`/`extend` as delegated attributes bound to the
un-codec'd leaf store, so appended values skipped the codec and reached
`ChromaCollection.__setitem__` raw -- which unpacks them as `**v`:

    TypeError: Collection.upsert() argument after ** must be a mapping, not str

Swapping the two decorators puts `appendable` outermost, so `append` and
`extend` now write the same values `__setitem__` writes. A comment records
the ordering constraint so it doesn't get flipped back.

Adds `AppendableChromaCollection` as the escape hatch for auto-keyed
appends of raw chromadb kwargs (what `ChromaDocuments.append` accidentally
did for mapping arguments before, minus the codec inconsistency).

Verified every other observable surface is byte-identical before/after
(name/MRO/dir, get/set/del/len/contains/iter/items/values, `.collection`,
`.store`). Top-level `chromadol` exports are unchanged.

Closes #2

Claude-Session: https://claude.ai/code/session_01L1aQPB34n7PU7jmbztSjBe
@thorwhalen
thorwhalen merged commit 5411dcb into master Sep 22, 2026
12 checks passed
@thorwhalen
thorwhalen deleted the fix/appendable-outside-value-codec branch September 22, 2026 13:23
thorwhalen added a commit that referenced this pull request Sep 22, 2026
base.py defined `ChromaUris` twice; the second (codec field "metadata",
which is not a chromadb kwarg) replaced the first, so every ChromaUris
write/append raised TypeError. #3's test covered "ChromaUris" only through
a fake that accepts any kwarg, so it passed. The second class is now
`ChromaMetadatas` on the real "metadatas" field, with no `append` since
chromadb refuses an upsert that has metadata but no documents.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
@thorwhalen

Copy link
Copy Markdown
Member Author

Post-merge refute review: the ChromaUris parametrization in test_append_writes_what_setitem_writes was vacuous. base.py defined ChromaUris twice, and the second (field "metadata", which is not a chromadb kwarg) won, so every real ChromaUris write or append raised TypeError. The recording fake accepts any kwarg, so the test passed anyway. Fixed in #5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

.append/.extend on ChromaDocuments/ChromaUris bypass the value codec (dol delegation, i2mint/dol#18)

1 participant