Skip to content

Asset-level evidence has no retirement path: superseded passes/beats/words accumulate #25

Open

Description

@thorwhalen

Found while reviewing #23 (the evidence layer). Not a defect in what landed — a gap that landing it created, and the honest half of the fix for that PR's cross-guide-pruning bug.

What happens

ASSET_SCOPED_TIERS (source.pass, beat, transcript.word) are deliberately never pruned. Their bodies carry no doc_id because they describe the asset, not one guide over it, and two guides share them; deleting one out from under a sibling guide's was_derived_from would leave that guide's grid and steps pointing at nothing. That exemption is right.

The cost is that superseded asset-level evidence has no retirement path at all. Re-run to_store with a different measurement and the counts only ever go up:

tier before after a re-measure
source.pass 2 4
beat 3 4
transcript.word 2 3

Keys on these tiers are content-derived, so a moved boundary is a new row rather than an overwrite — which is what keeps the old lineage edges resolving, and is exactly why nothing is deleted.

What it does and does not break

  • The document is unaffected. from_store never reads these tiers, so the projection is identical either way. This is not a correctness bug in to_store / from_store.
  • Provenance still says which run used which. grid and step rows cite the pass they were actually derived from (PR Evidence layer: persist analysis into a lacing store (#4) #23 made that hold even when the value digest is unchanged), so "which split produced this grid" stays answerable.
  • A consumer reading a tier directly gets superimposed generations. Iterate transcript.word for the asset after two runs and you get two transcripts interleaved, with nothing in the row saying which is current. Nothing in paces does this yet; anything outside paces reasonably would.

The options

  1. A GC verb — something like retire_superseded(store, *, asset_id, tier) that drops asset-level rows nothing's was_derived_from still cites. Reachable, explicit, and safe by construction (it only removes unreferenced rows), but "unreferenced" is only meaningful once every consumer records lineage.
  2. A current-run marker — a body field or a separate annotation naming the run that produced a generation, so a reader can filter to the newest without anything being deleted. Non-destructive and cheap to read; adds a field to two bodies plus lacing's word/v1, which paces does not own.
  3. Say it is intended — asset-level evidence is append-only by design, and a consumer that wants one generation walks in from the grid / step rows' was_derived_from rather than reading the tier directly. Costs nothing, and makes the lineage edges load-bearing rather than decorative.

Option 3 is close to what the code already does; the question is whether that is a decision or an accident. It should be a decision, written down.

Why it is not urgent

Nothing consumes these tiers today, and the store is regenerable by definition (docs/07 §6.0: gitignored, project.annot). It becomes real at the first direct consumer — most likely whatever answers #24, since a verifying-trace writer needs to know which generation it is attesting to.

Sibling of #24. Referenced from ASSET_SCOPED_TIERS' docstring in paces/evidence.py.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    discoveredFound while doing other work; not part of that issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions