Retrieval drift: recency is effectively unweighted in episode ranking

[Wynelson94]

Deferred deliberately. The fix needs a larger corpus to validate — see "Why this is not being fixed yet" below. Filing now so the analysis and the baseline aren't lost.

Raised by a reader comment on the 1.0 write-up:

The failure mode I would watch next is retrieval drift after the index gets large, where the tool finds a plausible old session and quietly treats it as current.

The finding

_rank_score in longhand/recall/recall_pipeline.py scores episodes as:

semantic_boost(0–20) + keyword_hits × 10 + confidence(0–1) + recency_boost × 0.5

Recency contributes at most 0.5 points against 10 per keyword hit. An eleven-month-old episode with one extra keyword match outranks today's work by 20×. recency_boost is 1.0 - (days_ago / 365), so it is also fully saturated at one year — everything older than that ranks identically on recency.

Segments have no recency term at all; only episodes get even the 0.5.

Not silent, though — _humanize_timestamp is called at all 8 render sites in narrative.py, so every result carries "today" / "3 days ago" / "1 month ago". The age is always stated. But stating an age is not the same as flagging it: a reader skims past "1 month ago" when their question implied "now."

Measurement (2026-08-12)

Observed by accident while re-baselining scripts/recall_diff.py for the 1.0 release. Against a July 11 baseline, after the live corpus grew 387 → 433 sessions (+12%):

• 4 of 8 fixed queries shifted their rankings
• 2 sessions that climbed into the top results predated the baseline — i.e. old sessions rising as the index grew, not new ones displacing old
• One query (what did we ship in v0.6) turned over its entire top-3 segment set

Ruled out code drift: checked out the v0.13.0 tag into a worktree and ran the identical script against the same store — byte-identical diff. Nothing in PRs #74–#77 touches recall_pipeline, ranking, embeddings, or extraction. This is retrieval drift from index growth alone, at a 12% increase.

Why this is not being fixed yet

A ranking change validated on a corpus that barely exhibits the problem is a guess wearing a test. The honest move is to let the index grow until the drift is unmistakable, then fix against real signal.

Baseline captured at filing (2026-08-12):

sessions	436
episodes	1,154
segments	4,400
recall_diff baseline	scripts/recall_diff_baseline.json @ 1129ca9

Revisit trigger: re-run python3 scripts/recall_diff.py at roughly 600 and 800 sessions without re-baselining in between. Record how many of the 8 queries shift at each checkpoint. If the shift rate climbs with corpus size, that is the signal — and the numbers become the test case for whatever fix lands.

What the fix is probably NOT

Increasing the recency weight. "What did I do last year" queries genuinely need old results to win, and a heavier recency term would break time-scoped recall to fix time-agnostic recall.

The shape more likely to be right, and consistent with Promise 5 in COMPATIBILITY.md: when a query carries no time phrase and the top hit is substantially older than the runner-up candidates, surface that gap explicitly rather than relying on the reader to notice a timestamp. Make the tool say "the best match here is much older than the alternatives" instead of quietly ranking it first.

That is a smaller, more honest change than re-tuning the scorer, and it fails safe: it adds information rather than reordering results.