📋 Pre-flight Checks
📝 Bug Description
Server chunk-ingest (WriteChunk) materializes only sessions/observations/prompts and silently drops relation mutations that ride inside the chunk. A relation uploaded via the chunk path (engram sync --cloud) lands in cloud_chunks.payload but never reaches cloud_mutations, so consumers that read via mutation-pull never see it.
materializedChunkMutations (internal/cloud/cloudstore/cloudstore.go:357-398) builds rows only from chunk.Sessions, chunk.Observations, chunk.Prompts — it never reads chunk.Mutations, where relations ride. The mutation-push path does the opposite: InsertMutationBatch (cloudstore.go:727) writes the entire batch unfiltered (:751), relations included.
This is the chunk-ingest sibling of the mutation-push path that #379 already fixed; #497 did not touch this code.
Root cause
#379 (commit 436c03f) added store.SyncEntityRelation to isChunkMaterializableMutationEntity (cloudstore.go:1057-1064) and taught the mutation-push materializer materializedMutationBatchChunk (cloudstore.go:975) to carry relations, but never updated materializedChunkMutations (the chunk-ingest path used by WriteChunk). #497 did not touch cloudstore.go. docs/codebase/sync-and-cloud.md:31 documents that chunks are supposed to carry the non-orphaned relations graph, so this contradicts the contract — an omission, not a deliberate scope cut.
🔄 Steps to Reproduce
- Self-hosted engram-cloud built from
main (docker-compose.cloud.yml).
- Isolated client DB: 2 observations + a judged relation between them, present in the chunk (so it rides as a
relation mutation in chunk.Mutations).
engram sync --cloud --project P (chunk path).
- Server SQL: relation present in
cloud_chunks.payload->mutations, but SELECT COUNT(*) FROM cloud_mutations WHERE entity='relation' AND entity_key='<rel>' returns 0; sessions/observations from the same chunk DID materialize (control).
- Second client pulling via
/sync/mutations/pull → relation absent. Then engram sync --cloud --import → relation recovered (chunk payload intact; the gap is in chunk-ingest materialization).
Unit-level: extend the WriteChunk materialize test in cloudstore_test.go with a chunk whose payload.mutations[] contains a relation, then assert a cloud_mutations relation row exists — currently fails.
✅ Expected Behavior
A relation uploaded via the chunk path should be materialized into cloud_mutations (same as the mutation-push path, and as the documented chunk relation contract at sync-and-cloud.md:31), so mutation-pull consumers see it.
❌ Actual Behavior
The relation persists in cloud_chunks.payload but is never written to cloud_mutations; mutation-pull consumers never see it. The client also acks the local sync_mutations row even though the server never materialized it, so the origin won't auto-resend (recovery needs a re-judge). No data loss: the relation stays in cloud_chunks.payload and is recoverable via engram sync --cloud --import.
Operating System
macOS
Engram Version
main @ 36c0819 (PR #497 merged), client built from source (1.16.4-dev)
Agent / Client
Claude Code
📋 Relevant Logs
# server-side, read-only (self-hosted cloud from main)
# relation IS in the uploaded chunk:
SELECT m->>'entity_key' FROM cloud_chunks c, jsonb_array_elements(c.payload->'mutations') m
WHERE m->>'entity'='relation'; -- returns the relation
# but it never materialized into cloud_mutations:
SELECT COUNT(*) FROM cloud_mutations WHERE entity='relation' AND entity_key='<rel>'; -- 0
# control: sessions/observations from the same chunk DID materialize
💡 Additional Context
Blast radius (narrow): autosync cross-machine sync uses /sync/mutations/push|pull (internal/cloud/autosync/manager.go:525/:575), which materializes relations correctly — that's the #496/#379 path. The gap only bites relations that travel via the chunk-upload path (manual engram sync --cloud / file-based chunk versioning) and are consumed via mutation-pull. No data loss (recoverable via --import).
Suggested fix: mirror #379 in materializedChunkMutations — emit relation MutationEntry rows from chunk.Mutations with the same validation as the push path, so both server-side ingest paths stay consistent. Optionally add a cloud_chunks→cloud_mutations reconciliation to back-materialize relations already dropped.
Related follow-up: the local chunk-import path has a sibling gap — ApplyPulledChunk (internal/store/store.go:4245) lacks the ErrRelationFKMissing deferral that ApplyPulledMutation (store.go:4196-4226) has, so a chunk relation whose observations are absent can abort the import (conditional on a partial/pruned clone). Happy to file separately if useful.
Severity: medium — no data loss, narrow trigger, but breaks the documented chunk relation contract and leaves the origin's relation acked-but-unmaterialized.
📋 Pre-flight Checks
status:approvedbefore a PR can be opened📝 Bug Description
Server chunk-ingest (
WriteChunk) materializes only sessions/observations/prompts and silently drops relation mutations that ride inside the chunk. A relation uploaded via the chunk path (engram sync --cloud) lands incloud_chunks.payloadbut never reachescloud_mutations, so consumers that read via mutation-pull never see it.materializedChunkMutations(internal/cloud/cloudstore/cloudstore.go:357-398) builds rows only fromchunk.Sessions,chunk.Observations,chunk.Prompts— it never readschunk.Mutations, where relations ride. The mutation-push path does the opposite:InsertMutationBatch(cloudstore.go:727) writes the entire batch unfiltered (:751), relations included.This is the chunk-ingest sibling of the mutation-push path that #379 already fixed; #497 did not touch this code.
Root cause
#379 (commit
436c03f) addedstore.SyncEntityRelationtoisChunkMaterializableMutationEntity(cloudstore.go:1057-1064) and taught the mutation-push materializermaterializedMutationBatchChunk(cloudstore.go:975) to carry relations, but never updatedmaterializedChunkMutations(the chunk-ingest path used byWriteChunk). #497 did not touchcloudstore.go.docs/codebase/sync-and-cloud.md:31documents that chunks are supposed to carry the non-orphaned relations graph, so this contradicts the contract — an omission, not a deliberate scope cut.🔄 Steps to Reproduce
main(docker-compose.cloud.yml).relationmutation inchunk.Mutations).engram sync --cloud --project P(chunk path).cloud_chunks.payload->mutations, butSELECT COUNT(*) FROM cloud_mutations WHERE entity='relation' AND entity_key='<rel>'returns0; sessions/observations from the same chunk DID materialize (control)./sync/mutations/pull→ relation absent. Thenengram sync --cloud --import→ relation recovered (chunk payload intact; the gap is in chunk-ingest materialization).Unit-level: extend the
WriteChunkmaterialize test incloudstore_test.gowith a chunk whosepayload.mutations[]contains a relation, then assert acloud_mutationsrelation row exists — currently fails.✅ Expected Behavior
A relation uploaded via the chunk path should be materialized into
cloud_mutations(same as the mutation-push path, and as the documented chunk relation contract atsync-and-cloud.md:31), so mutation-pull consumers see it.❌ Actual Behavior
The relation persists in
cloud_chunks.payloadbut is never written tocloud_mutations; mutation-pull consumers never see it. The client also acks the localsync_mutationsrow even though the server never materialized it, so the origin won't auto-resend (recovery needs a re-judge). No data loss: the relation stays incloud_chunks.payloadand is recoverable viaengram sync --cloud --import.Operating System
macOS
Engram Version
main@36c0819(PR #497 merged), client built from source (1.16.4-dev)Agent / Client
Claude Code
📋 Relevant Logs
💡 Additional Context
Blast radius (narrow): autosync cross-machine sync uses
/sync/mutations/push|pull(internal/cloud/autosync/manager.go:525/:575), which materializes relations correctly — that's the #496/#379 path. The gap only bites relations that travel via the chunk-upload path (manualengram sync --cloud/ file-based chunk versioning) and are consumed via mutation-pull. No data loss (recoverable via--import).Suggested fix: mirror #379 in
materializedChunkMutations— emit relationMutationEntryrows fromchunk.Mutationswith the same validation as the push path, so both server-side ingest paths stay consistent. Optionally add acloud_chunks→cloud_mutationsreconciliation to back-materialize relations already dropped.Related follow-up: the local chunk-import path has a sibling gap —
ApplyPulledChunk(internal/store/store.go:4245) lacks theErrRelationFKMissingdeferral thatApplyPulledMutation(store.go:4196-4226) has, so a chunk relation whose observations are absent can abort the import (conditional on a partial/pruned clone). Happy to file separately if useful.Severity: medium — no data loss, narrow trigger, but breaks the documented chunk relation contract and leaves the origin's relation acked-but-unmaterialized.