When slug is omitted, the save path derives one from the opening words of the content. Two saves
whose openings agree resolve to the same file, and the second overwrites the first — with no
error, no warning, and a success receipt identical to a fresh create.
Reproduced on current main with three saves that share an opening line and differ afterwards:
save 0: rel_path=insights/user-thanks-that-helps-a-lot.md
save 1: rel_path=insights/user-thanks-that-helps-a-lot.md
save 2: rel_path=insights/user-thanks-that-helps-a-lot.md
distinct rel_paths: 1
files on disk: ['user-thanks-that-helps-a-lot.md']
surviving file mentions ALPHA: False | BRAVO: False | CHARLIE: True
Two memories are gone from the queryable store. They are recoverable from git history — three
commits to one path — but nothing told the caller anything happened. Exposure scales with write
volume and with how formulaic the content is: human-authored saves rarely collide, while anything
saving conversational turns, logs, or telemetry collides constantly.
There is a precedent in the codebase, but do not copy it.
palinode/ingest/pipeline.py:311-314 attempts this job for research ingestion:
if os.path.exists(filepath):
slug += f"-{stable_md5_hexdigest(content[:100])[:6]}"
filename = f"{today}-{slug}.md"
filepath = os.path.join(config.palinode_dir, "research", filename)
It has two defects, and copying it verbatim would reproduce the bug you are fixing:
- It hashes only
content[:100]. Two memories that agree for their first 100 characters and
differ afterwards hash identically, so they land on the same disambiguated filename and the
second still overwrites the first. Confirmed — with two bodies sharing a 120-character prefix,
both produce the suffix 13365e.
- It checks existence once. If the disambiguated name is also taken, it overwrites without
looking.
What to change in palinode/core/save.py, for the derived-slug path only:
- Disambiguate using a hash of the full content, not a prefix — or loop until the candidate
filename is unused (-2, -3, …). Either is fine; a single prefix-hash check is not.
- If the existing file's content is byte-identical, keeping the current overwrite is correct — that
is a re-save of the same memory, not a collision.
Tests to include:
- Two saves sharing an opening line, differing afterwards → two distinct files, both retrievable.
- Two saves whose content is identical for the first 100 characters and differs only after
character 100 → still two distinct files. This is the case a prefix hash silently fails, so it is
the test that proves the fix rather than the pattern.
Two things to leave alone, both deliberate:
- An explicitly-passed
slug must keep overwriting. That is the documented escape hatch and
callers rely on it for idempotent updates. Only the derived case should change.
- Do not extract a shared helper into
palinode/ingest/pipeline.py, or fix its version here.
Keep your change inside palinode/core/save.py. There is another issue open against that file's
docstrings and we would rather not make either of you rebase.
update_policy is not the lever here, in case it looks like one: it is sticky frontmatter recording
write semantics, not a branch that decides whether the file gets replaced.
When
slugis omitted, the save path derives one from the opening words of the content. Two saveswhose openings agree resolve to the same file, and the second overwrites the first — with no
error, no warning, and a success receipt identical to a fresh create.
Reproduced on current
mainwith three saves that share an opening line and differ afterwards:Two memories are gone from the queryable store. They are recoverable from git history — three
commits to one path — but nothing told the caller anything happened. Exposure scales with write
volume and with how formulaic the content is: human-authored saves rarely collide, while anything
saving conversational turns, logs, or telemetry collides constantly.
There is a precedent in the codebase, but do not copy it.
palinode/ingest/pipeline.py:311-314attempts this job for research ingestion:It has two defects, and copying it verbatim would reproduce the bug you are fixing:
content[:100]. Two memories that agree for their first 100 characters anddiffer afterwards hash identically, so they land on the same disambiguated filename and the
second still overwrites the first. Confirmed — with two bodies sharing a 120-character prefix,
both produce the suffix
13365e.looking.
What to change in
palinode/core/save.py, for the derived-slug path only:filename is unused (
-2,-3, …). Either is fine; a single prefix-hash check is not.is a re-save of the same memory, not a collision.
Tests to include:
character 100 → still two distinct files. This is the case a prefix hash silently fails, so it is
the test that proves the fix rather than the pattern.
Two things to leave alone, both deliberate:
slugmust keep overwriting. That is the documented escape hatch andcallers rely on it for idempotent updates. Only the derived case should change.
palinode/ingest/pipeline.py, or fix its version here.Keep your change inside
palinode/core/save.py. There is another issue open against that file'sdocstrings and we would rather not make either of you rebase.
update_policyis not the lever here, in case it looks like one: it is sticky frontmatter recordingwrite semantics, not a branch that decides whether the file gets replaced.