topicindex: renew expired registrations (demote to Standby) instead of dropping — avoid rediscovery round-trip, keep tables full

[srene]

Summary

When a registration's ad reaches its TTL, Registration.Update currently deletes the attempt (removeAttempt(att, "expired")). The registrar is still a valid candidate, but to register with it again the node must first rediscover it — via a discv5 random lookup, or as a node returned during another registration/search — before it can re-register.

Impact — an optimization, not a hard cap

This is not a hard cap on fan-out. In a healthy network with periodic DHT random lookups active, the registrar is rediscovered, so dropping-on-expiry mainly adds:

• extra latency / work for the rediscovery round-trip before re-registration, and
• transient under-fill of the registration table between expiry and rediscovery.

The effect is most pronounced when periodic random lookups are not active (or sparse): the expired candidate may not be rediscovered promptly, so registration tables stay under-filled and effective fan-out drops.

Proposed improvement

On expiry, demote the attempt back to Standby (clearing ticket / wait-time / request counters) so refillAttempts can re-promote it for renewal immediately, without waiting for rediscovery. To keep the standby pool bounded, drop instead of demote only when the standby pool is already full (there are enough renewal candidates; count[Standby] stays within RegBucketStandbyLimit).

Evidence (with caveat)

In a 10k-node simnet without periodic random lookups, demote-on-expiry lifted registration visibility from ~45% to 100% and median per-registrant fan-out from 1 to ~45. That scenario amplifies the effect (no rediscovery is happening); in a network with active lookups the gain is smaller — mostly reduced rediscovery delay and steadier table occupancy.

Prototyped in PR #70 (with the gated drop-when-standby-full refinement). The PR is closed for now and may be revisited if registration-coverage or latency issues are observed at scale.

[srene]

Preferred design: reuse expired registrars only as a fallback

Rather than always demoting on expiry (and dropping only when the standby pool is full), the better approach is to prefer a fresh candidate and reuse the expired registrar only when there is nothing else to fill the freed slot. This keeps registrar selection diverse and only falls back to re-using a registrar when the bucket would otherwise be left under-served.

The signal is simply whether a fresh candidate exists to backfill the freed Registered slot (refillAttempts promotes a Standby):

case Registered:
    if att.bucket.count[Standby] > 0 {
        // A fresh candidate can take the freed slot — drop the expired one and
        // let refillAttempts register a different (more diverse) registrar.
        r.removeAttempt(att, "expired")
    } else {
        // No other candidate to register with — reuse this proven registrar
        // (demote to Standby, reset ticket/wait/reqCount) rather than go
        // under-served while waiting for rediscovery.
        heap.Remove(&r.heap, att.index)
        att.Ticket, att.totalWaitTime, att.reqCount = nil, 0, 0
        r.setAttemptState(att, Standby)
    }
    r.refillAttempts(att.bucket)

This is the inverse of the drop-when-full gate: drop in most cases, reuse only when standby is empty.

Trade-off: the expired registrar is proven (it admitted us) while a standby candidate is untried (might reject or be slow). Preferring the fresh one trades a known-good slot for diversity plus a possible failed attempt — but a dropped registrar is re-discoverable (discv5 lookups / nodes returned during registration), and the choice is self-correcting: a failed fresh candidate is deleted and the next refillAttempts (or rediscovery) tries another.

Note: after expiry count[Registered] has dropped, so there is always room for refillAttempts to promote — i.e. dropping when count[Standby] > 0 reliably results in a fresh backfill, never an empty slot.