Skip to content

save silently overwrites an existing memory when the slug is derived from content #129

Description

@Paul-Kyle

When slug is omitted, the save path derives one from the opening words of the content. Two saves
whose openings agree resolve to the same file, and the second overwrites the first — with no
error, no warning, and a success receipt identical to a fresh create.

Reproduced on current main with three saves that share an opening line and differ afterwards:

save 0: rel_path=insights/user-thanks-that-helps-a-lot.md
save 1: rel_path=insights/user-thanks-that-helps-a-lot.md
save 2: rel_path=insights/user-thanks-that-helps-a-lot.md
distinct rel_paths: 1
files on disk: ['user-thanks-that-helps-a-lot.md']
surviving file mentions  ALPHA: False | BRAVO: False | CHARLIE: True

Two memories are gone from the queryable store. They are recoverable from git history — three
commits to one path — but nothing told the caller anything happened. Exposure scales with write
volume and with how formulaic the content is: human-authored saves rarely collide, while anything
saving conversational turns, logs, or telemetry collides constantly.

There is a precedent in the codebase, but do not copy it.
palinode/ingest/pipeline.py:311-314 attempts this job for research ingestion:

if os.path.exists(filepath):
    slug += f"-{stable_md5_hexdigest(content[:100])[:6]}"
    filename = f"{today}-{slug}.md"
    filepath = os.path.join(config.palinode_dir, "research", filename)

It has two defects, and copying it verbatim would reproduce the bug you are fixing:

  1. It hashes only content[:100]. Two memories that agree for their first 100 characters and
    differ afterwards hash identically, so they land on the same disambiguated filename and the
    second still overwrites the first. Confirmed — with two bodies sharing a 120-character prefix,
    both produce the suffix 13365e.
  2. It checks existence once. If the disambiguated name is also taken, it overwrites without
    looking.

What to change in palinode/core/save.py, for the derived-slug path only:

  • Disambiguate using a hash of the full content, not a prefix — or loop until the candidate
    filename is unused (-2, -3, …). Either is fine; a single prefix-hash check is not.
  • If the existing file's content is byte-identical, keeping the current overwrite is correct — that
    is a re-save of the same memory, not a collision.

Tests to include:

  • Two saves sharing an opening line, differing afterwards → two distinct files, both retrievable.
  • Two saves whose content is identical for the first 100 characters and differs only after
    character 100 → still two distinct files.
    This is the case a prefix hash silently fails, so it is
    the test that proves the fix rather than the pattern.

Two things to leave alone, both deliberate:

  1. An explicitly-passed slug must keep overwriting. That is the documented escape hatch and
    callers rely on it for idempotent updates. Only the derived case should change.
  2. Do not extract a shared helper into palinode/ingest/pipeline.py, or fix its version here.
    Keep your change inside palinode/core/save.py. There is another issue open against that file's
    docstrings and we would rather not make either of you rebase.

update_policy is not the lever here, in case it looks like one: it is sticky frontmatter recording
write semantics, not a branch that decides whether the file gets replaced.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinggood first issueGood for newcomers

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions