topicSearch: rollover has no cross-cycle memory — searches re-walk the same neighbourhood, capping per-searcher discovery efficiency

[srene]

Summary

A topic search runs as a continuous rollover: topicSearch.runLoop (p2p/discover/v5_topic.go) creates a fresh topicindex.Search each cycle, re-seeds it from the routing table (tab.allNodes(), shuffled), walks until Search.IsDone(), then rolls over to a brand-new Search (logged "Topic search rollover"). Each Search instance owns its own buckets/resultBuffer/resultSeen, all wiped on rollover — no state is carried across cycles.

Consequence: every rollover re-seeds from the same table and re-walks toward the same topic neighbourhood, re-querying largely the same closest registrars and re-yielding ads already found. The consumer dedupes them as duplicates, so an individual searcher's distinct discovery climbs with diminishing returns. Aggregate coverage across all searchers is fine; per-searcher coverage efficiency is not.

Evidence — 10k nodes, no churn, all-DISC-NG (2026-06-15)

• Aggregate coverage 100% — every registrant found by some searcher (neverFound=0 for all 5 topics).
• Per-searcher recall only 17–59% (meanUniq), fullRecall 0/N.
• 0 searchers terminated (rollover is continuous; every consumer ran to the 300s timeout).
• ~28–30% of searchers found a NEW distinct registrant in the final 30s → still climbing, not plateaued.
• Per-searcher distinct count clusters at a throughput ceiling (~500–950), so recall% ≈ ceiling ÷ topic size (topic 0 with 4544 regs → 17%; topic 4 with 858 → 59%).

So the protocol surfaces everything in aggregate, but a single searcher re-treads the same ground instead of fanning out to unseen ads.

Proposed direction

A TTL-bounded "discovered" overlay on the long-lived topicSearch (not on the per-cycle Search), consulted/seeded into each new cycle:

• found-ads set → suppress duplicate yields (raises distinct/sec);
• soft deprioritization of recently-asked registrars → bias the walk toward unqueried ones.

It must be TTL-bounded so continuous re-discovery still catches newly-registered and churned-out ads (same pattern as the failure Blacklist TTL); a permanent overlay would freeze the view, which is especially bad under churn.

Caveat / measure first: registrars answer TOPICQUERY with a random sample (RandomNodes) of their ad table, and fan-out is high (~45 at 10k), so re-querying a close registrar is coupon-collector sampling, not pure waste. Whether the fix should "visit more registrars" vs "re-sample the same ones faster" depends on the ad distribution — measure duplicate-rate-over-time and distinct-registrars-queried-per-searcher before committing to a design. Hard-skipping asked registrars would lose coverage when ads concentrate on a few closest nodes.

Related: #11 (evaluation), #55 (target-distance bug also degrades walk quality), #27 / #65 (IsDone / QueryTarget).