Skip to content

bug: Claims have no content-version tracking — audit anchors and Freshness chart don't detect stale facts after code changes #91

Description

@priyamkarn

Summary

Greplica stores claims about the code (text, kind, truth, code_anchors)
but never records what the anchored code actually contained at the time the
claim was written. As a result, once a symbol's implementation changes while
its name/signature stays the same, there is no mechanism anywhere in the
system that detects the claim is now stale — the agent keeps getting fed the
outdated fact.

Reproduction

  1. Bootstrap/apply a proposal with a claim anchored to a function, e.g.
    {"file": "src/foo.ts", "symbol": "computeTotal"}, whose text describes
    current behavior (e.g. "computeTotal applies a flat 5% discount").
  2. Change the body of computeTotal so the claim is now false (e.g. remove
    the discount), without renaming the function or moving the file.
  3. Run greplica graph audit anchors — it reports the anchor as healthy
    (resolved), because the symbol still exists.
  4. Open greplica graph view — the Freshness chart still shows the claim as
    100% active.
  5. Run greplica graph context "computeTotal" — the agent is served the
    original, now-incorrect claim text with no warning.

Root cause

  • libs/knowledge-graph/code-anchors/audit.ts / resolver.ts only check
    structural resolution (file exists, symbol exists, symbol is
    unambiguous). They re-resolve against the current file on every run but
    never compare content against anything recorded at claim-creation time.
  • Claim / ClaimCodeAnchor (libs/knowledge-graph/claim.ts) have no field
    for a content hash, commit SHA, or line snapshot — so even if the audit
    wanted to detect drift, there is nothing persisted to diff against.
  • Confirmed at the schema level (libs/storage/sqlite/schema.ts): the
    claims table has no versioning column of any kind.
  • The "Freshness" concept surfaced in graph view
    (libs/knowledge-graph/graph-view/build-graph-view.ts) is unrelated to
    code drift — it's active vs superseded, driven entirely by whether a
    claim has been explicitly superseded by a later proposal. A claim can be
    100% stale relative to the code and still show as 100% "active" freshness.

Impact

  • Silent factual drift: an agent can act on confidently-presented but false
    information, with both of the tool's own health signals (anchor audit,
    freshness chart) reporting green.
  • This undermines the core value proposition of the graph (durable, trusted
    memory) more than a missing feature would — a wrong fact that looks
    verified is worse than a known gap.

Suggested direction

  • Store a content fingerprint per anchor at write time — e.g. a hash of the
    resolved symbol's source text (or the anchored line range) — as part of
    code_anchors ({file, symbol, content_hash}).
  • On graph audit anchors, recompute the hash for the currently resolved
    symbol and compare; introduce a new status (e.g. stale_content) distinct
    from missing_symbol/resolved for "symbol still exists but its content
    changed since this claim was recorded."
  • Surface stale_content claims distinctly in graph view (or fold real
    code-drift detection into what "Freshness" measures, rather than only
    supersession state) so the chart isn't misleading about what it verifies.
  • Out of scope for a first pass: full historical versioning/diffing — a
    binary "matches / drifted" signal per anchor would already close the gap
    described here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions