refactor(safrole): cache ring verifier by validator-set hash, not epoch#1041
Conversation
Fork-restore (restoreWithState) called ClearVerifierCache() on every block, so the next GetVerifier had to rebuild the Bandersnatch ring verifier (the expensive KZG ring construction) from scratch. The verifier is fully determined by the validator set's bandersnatch keys (GP 0.8.0: z = O([k_b | k in gamma'_P])), so key the cache by a Blake2b hash of those keys in a small bounded map instead of by epoch. Same-validator-set forks and stable-validator epoch boundaries now hit the cache, and the restore no longer needs to clear it. Correctness across forks is guaranteed by the key: a different validator set hashes differently and misses. Eviction and clear only drop Go references (never Free the Rust verifier under a reader), preserving the existing use-after-free safety. picofuzz (tiny, median-of-3): per-block import p99 -61..90%, mean -31..53%, stdDev roughly halved; median unchanged. Correctness unchanged: conformance 1179/0, fuzzy 201/0, minifuzz (6) and picofuzz (4) all green, unit tests (hit/rebuild/eviction/clear/concurrent) pass under -race.
3 suggestion:
Only successful verifications are stored, so garbage signatures cannot pollute the cache; a miss simply falls back to normal verification. |
Closes #1040
Why
GetVerifiercached one Bandersnatch ring verifier keyed byepoch, andChainState.restoreWithState(fork-restore) calledClearVerifierCache()on every restore. The nextGetVerifierthen rebuilt the verifier viavrf.NewVerifier(the expensive KZG ring construction). In fork-heavy fuzzing this rebuild recurs on nearly every block and dominates Safrole import time.The blanket clear existed only because the epoch key was too coarse: same-epoch forks can carry different validator sets, so an epoch-only cache could return a verifier built from the wrong fork's keys.
What changed
internal/blockchain/ring_verifier.go: the cache is now a boundedmap[OpaqueHash]*vrf.Verifierkeyed byBlake2bHash(concatenated bandersnatch keys).GetVerifierdrops theepochparameter; double-checked locking; clear-all eviction at 32 entries. Clear/evict only drop Go references (never Free a verifier a reader may hold), preserving the prior finalizer-based use-after-free safety.internal/blockchain/chain_state.go:restoreWithStateno longer clears the cache — the content-hash key makes same-validator-set forks hit and different-validator-set forks miss, so clearing is unnecessary.internal/safrole/safrole.go: call site updated toGetVerifier(postGammaK).internal/blockchain/ring_verifier_test.go: new unit tests (hit / rebuild / eviction / clear / concurrent / wrong-size).The ring commitment is a pure function of the next epoch's validator bandersnatch keys (GP 0.8.0, gamma_z = O([k_b | k in gamma'_P]); RFC-0026 §6.6), so the epoch is irrelevant to the verifier's identity and is dropped from the key.
Correctness
go vetclean; unit tests pass under-race(hit / rebuild / eviction / clear / concurrent / wrong-size).test_folder(0.7.2): 1179 PASSED / 0 FAILED.Improvement
picofuzz (tiny spec, median of 3). Numbers are per-block import latency aggregated across all blocks in a suite (microseconds per block), not the suite's total wall-clock time:
Median is essentially unchanged (typical blocks never rebuilt); the gain is in the tail — periodic epoch-boundary / fork blocks that previously rebuilt the ring verifier. stdDev roughly halves (more consistent import latency). This is tiny mode (ring = 6); full mode (ring = 1023) rebuilds are far more expensive, so the effect scales up.
End-to-end whole-suite wall time (median of 3) also drops. This is a supporting signal only: most of it is fixed overhead (target container startup, per-block socket round-trips, debug logging), so it is noisier and dilutes the effect compared with the per-block numbers above.
Test plan