Define and enforce a long-run exploration yield contract #215

Product outcome

For a large, long-maintained Ink story, a marathon InkCheck run should keep converting its declared time and memory into new bugs or useful semantic territory. Finding a bug early and then retaining state until the heap watermark stops the run is good early discovery, but weak long-horizon value for a creator who granted a much larger budget.

This issue defines the missing product contract: continued search must be judged by incremental useful yield relative to retained memory and elapsed time, not by raw explored-state count alone.

InkBench evidence

The complete generated 20-minute matrix contains 120 canonical cells: 8 bug families × 5 explorers × 3 seeds. It used InkBench commit 76b1dd5 and the corrected InkCheck build at b2651f5 (CLI SHA-256 b54b01b48122151bc2105c99acbab061e1252da7f3768004d4ad037d0b0290eb).

InkCheck found all 24 planted bugs, including the strongest results on several difficult families. However:

• novelty-honeypot: 3/3 bugs found, median first discovery about 5.9 minutes, then 3/3 runs stopped on memory at a median of about 7.3 minutes.
• compound-needle: 3/3 found, median first discovery about 1 minute, then 3/3 stopped on memory at a median of about 6.5 minutes.
• rare-prefix: 3/3 found, median first discovery about 13.6 minutes; the runs then consumed nearly the full time window.
• delayed-consequence: 3/3 found, median first discovery about 15.6 minutes.

A preregistered 60-minute matrix was intentionally stopped after 54/120 durable cells because the next engineering decision was already clear and both InkCheck and InkSwarm are about to change. In the 10 completed InkCheck cells:

• all 10 planted bugs were found;
• 6 small/exhaustible cells completed normally;
• all 3 rare-prefix cells found the bug around 13.7 minutes, then stopped on memory around 33–36 minutes;
• the completed novelty-honeypot InkCheck cell found the bug around 6.0 minutes, then stopped on memory around 7.9 minutes;
• there were no adapter/runtime failures, unavailable cells, or native work-ceiling hits;
• the bounded NDJSON evidence stream stayed tiny, so output streaming was not the retained-memory driver.

The partial 60-minute run is not a comparative estimate and must not be used to claim algorithmic superiority. It does show that, on these families, the current search representation can become the binding constraint long before the granted wall-time budget.

Opportunity

Add an explicit long-run yield contract around the work already tracked in #156 and #190:

1. Report retained bytes and counts by major owner: pending frontier, deduplication, ancestry/witness data, semantic indexes, findings, checkpoints, and serialization/finalization reserve.
2. Record incremental useful yield per interval: new findings, new authored locations, choice/edge outcomes, meaningful variable transitions/combinations, and other versioned semantic signals.
3. Derive and report metrics such as useful yield per retained GiB-minute and per million transitions. Keep raw state throughput separate.
4. Put explicit byte/count ceilings on each unbounded search structure. When a ceiling binds, compact, spill, evict, or finish with an honest partial result according to a deterministic policy.
5. Make retention yield-aware: state that has not produced useful descendants should not receive unlimited memory merely because it is novel under a broad representation.
6. Support compact, resumable epochs so a long campaign can preserve its best corpus/frontier without requiring one ever-growing heap.
7. Exercise the policy on multi-bug stories so an early find cannot hide an unproductive remaining tail.

Acceptance criteria

• Long-run reports attribute retained bytes/counts to the major search structures at bounded intervals and at termination.
• The report distinguishes early discovery quality from post-discovery incremental yield.
• Identical source/config/seed/resource posture makes the same compaction, spill, eviction, and stop decisions.
• Every resource stop retains a valid partial report and replayable witnesses; no completeness claim is inferred.
• A candidate evaluation reruns the InkBench rare-prefix, novelty-honeypot, compound-needle, and false-novelty families under matched budgets, plus at least one public real story.
• Relative to the pinned build, the candidate either converts materially more of the same memory/time grant into additional findings or useful semantic territory, or documents that an earlier honest stop is the better product outcome.
• Early time-to-first-discovery and witness integrity do not regress beyond a preregistered tolerance.
• Comparisons include a simpler systematic/coverage baseline so representation cost cannot be mistaken for search competence.

Non-goals

• Claiming InkCheck is generally weaker than another explorer from this partial 60-minute matrix.
• Raising memory defaults as the primary fix.
• Treating raw state hashes, state count, or heap consumption as useful coverage by themselves.