Skip to content

Seed-survival: measure the 0.414 threshold or stop claiming it #104

Description

@schmug

Task

SEED_TETHER_MIN = 0.414 in public/drift/position.js is presented as proof that a surfaced card is still about the seed. It is not, on two counts: the number is not computed anywhere in this repository, and this repo's own committed evidence contains counterexamples above it. Replace the claim with something measured, or remove the claim. After this is done, either drift enforces a seed-survival criterion backed by a committed harness and a labelled set, or it enforces nothing and says so.

Context

Found by critic cycle 3 (critic-reports/cycle-03.json, blocker + two majors). Seed retention is the finding that chose projection over translation in the first place, so this is the load-bearing claim of the whole surface.

The two problems

1. The number has no provenance here. 0.414 is described in position.js and in two measurement entries as "the measured bge-m3 anisotropy — the mean cosine between two unrelated phrases". It comes from an external run described in a design document. npm run axis-walk, axis-projection and axis-power neither contain an unrelated-pair corpus nor compute that value. grep -rn "0\.414" scripts/ returns only a comment.

2. This repo's own evidence contradicts it. docs/measurements/2026-08-22-drift-mechanic-spikes.md records the walk spike abandoning the seed while scoring 0.407–0.532:

step  tetherSeed   card                    seed was "public transit"
  2     0.438      yard twinkle lights
  9     0.407      croquet mallet fun
 12     0.414      lawn jenga party        ← exactly at the threshold

A cutoff that admits lawn jenga party for public transit cannot establish the invariant it is named for.

3. The harness no longer reproduces its own output. docs/measurements/2026-08-23-production-parity-and-retention.md reports surfaced minima of 0.398 and 0.377 and says npm run axis-projection reproduces them — but that harness now imports the shipped nextCard, which rejects everything below 0.414. The pre-filter measurement and the post-filter behaviour have been conflated.

Pointers

  • public/drift/position.jsSEED_TETHER_MIN, its justification comment, and isTethered
  • docs/measurements/2026-08-22-drift-mechanic-spikes.md — the counterexamples, and the correction note already appended
  • docs/measurements/2026-08-23-production-parity-and-retention.md — the retention run whose reproducibility broke
  • scripts/axis-projection-spike.ts — imports shipped nextCard; still recommends a p01 cutoff the shipped rule ignores
  • scripts/axis-power-spike.ts — the judge harness, the nearest thing to a relevance instrument
  • docs/measurements/2026-08-22-workstream-b-null-result.md — three runs showing no cheap statistic has power

Constraints

  • Do not invent a replacement threshold. The project rule is that an unmeasured threshold is a number someone made up; that is exactly how the current one arrived.
  • A committed harness must compute any number that ends up in position.js, and its raw output must be committed under docs/measurements/.
  • Separate pre-filter measurement from post-filter behaviour. A harness importing the shipped ranker cannot measure what the ranker excludes.
  • Workstream B established over three runs that no cheap client-side statistic separates a good axis from a null. An LLM judge does discriminate; using one costs an inference call on a surface whose premise is that nothing waits on an AI call. That trade is part of this decision, not separate from it.
  • Removing enforcement entirely is an acceptable outcome if it is recorded honestly.

Acceptance criteria

  • Either a committed harness computes the enforcement threshold and its raw output is committed, or SEED_TETHER_MIN is removed and the spec records that seed survival is not enforced
  • The false-accept rate is measured against a labelled set that includes the known counterexamples above
  • npm run axis-projection reproduces the numbers its measurement entry claims, at the reviewed revision
  • position.js cites the committed artifact at the constant's definition, or carries no such claim
  • npm run typecheck and npm test pass, counts reported

Out of scope

  • Changing the projection mechanic itself.
  • Re-litigating projection versus translation (settled, docs/measurements/2026-08-22-drift-mechanic-spikes.md).

Reasoning guidance

Think carefully and step-by-step. The honest answer here may be "we cannot cheaply establish this invariant, so we do not claim it" — that outcome is a success, not a failure to be worked around.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions