Skip to content

rpc/jsonrpc, db/snapshotsync: seven review follow-ups from #23322 - #23690

Merged
lupin012 merged 3 commits into
mainfrom
lupin012/prune_gates_small_fixes
Sep 1, 2026
Merged

rpc/jsonrpc, db/snapshotsync: seven review follow-ups from #23322#23690
lupin012 merged 3 commits into
mainfrom
lupin012/prune_gates_small_fixes

Conversation

@lupin012

@lupin012 lupin012 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #23322, which merged as approved with thirteen inline review notes deferred.
This closes the seven that are self-contained; the other six are grouped at the end.

Note Site Change
r3885706832 erigon_receipts.go erigon_getLogs no longer answers [] for an unknown or non-canonical block hash
r3885706060 eth_receipts.go resolveLogsRange returns the PrunedError it computes
r3885705986 eth_api.go, eth_system.go the receipts gate and eth_capabilities follow history for a sentinel retention
r3885706103 receipts_generator.go a transaction-free block is cached and answered before the exec semaphore
r3885706918 eth_api.go the Byzantium check is read once, from receipts.PostStateCalculated
r3885706948 block_reader_test.go the wall-clock assertion is replaced
r3885706992 block_reader.go, eth_api.go three comments trimmed to the invariant they carry

Three of them are worth a line:

By-hash logs. resolveLogsRange's hash branch is extracted into resolveLogsBlockHash
and erigon_getLogs goes through it, so all three by-hash endpoints answer alike. The same
function no longer discards the refusal it derives; error class and message are unchanged.

Sentinel retention. Distance.Enabled() is false for three values and the switch
special-cased two, so KeepPostMergeBlocksPruneMode produced no gate while
historyRetireCutoffs retires the RCache with history for it. Both switches now ask one
predicate, which also drops the ordering dependency. It is an unexported helper rather than a
second exported method on prune.Mode, since the existing narrow ReceiptsFollowHistory()
still drives RCache download policy — see below.

Exec semaphore. It is now taken where execution starts, immediately before PrepareEnv,
so neither a transaction-free block nor the persisted-receipts fast path queues behind real
re-executions. The per-block mutex still covers the whole function.

Tests

Five new, one changed. TestErigonGetLogsByBlockHashRequiresACanonicalBlock;
a resolveLogsRange leg on TestLogsByBlockHashNamesThePruneBoundary; a
cache_sentinel_retention row on TestReceiptsGateFollowsRetention plus
TestCapabilitiesFollowHistoryForASentinelRetention; TestGetReceiptsCachesAnEmptyBlock and
TestGetReceiptsAnswersAnEmptyBlockWithoutAnExecSlot (semaphore full and context already
cancelled, so any path reaching the semaphore reports the cancellation).
make lint and make test-short are clean.

One deviation

The flaky-assert note suggested < 2*timeout or a channel signal; this asserts
client.calls.Load() <= 2 instead — each caller reaches the backend at most once, and every
fetch it runs is bounded by that caller's own deadline. Deterministic, but it pins the timing
property less tightly; require.Less(t, time.Since(started), 2*timeout) can go back alongside
it if preferred.

Not in this PR

Each is a behaviour change wanting its own test:

  • erigon_getLatestLogs is the third by-hash log path and still hand-rolls HeaderByHash.
  • Mode.ReceiptsFollowHistory() (narrow) still drives RCache download policy in
    db/snapshotsync, so for the sentinel retention download and retirement disagree.
  • Generator.GetReceipt (singular) takes no exec slot at all, though it executes.

- erigon_getLogs resolves a block hash through the shared resolver, so an
  unknown or non-canonical hash reports "block not found" instead of []
- resolveLogsRange returns the PrunedError it computes instead of leaving the
  refusal to every caller
- the receipts gate and eth_capabilities follow history for any retention that
  is not a window of its own, which is the rule historyRetireCutoffs applies
- a transaction-free block is cached, and answered before the exec semaphore,
  which is now taken where execution starts
- the fork check is read once, from receipts.PostStateCalculated
- the FrozenBlocks wait test asserts backend calls instead of wall-clock time
- trims three comments to the invariant they carry
@lupin012
lupin012 marked this pull request as ready for review September 1, 2026 12:24
@AskAlexSharov

Copy link
Copy Markdown
Collaborator

Review pass for over-engineering only — correctness, security and perf are out of scope here. One line per finding: where, what to cut, what replaces it.

rpc/jsonrpc/check_prune_gates_test.go L1619-1638 — delete. TestCapabilitiesFollowHistoryForASentinelRetention is TestCapabilitiesFollowHistoryWhereTheCacheIsNotServed (L234) with one field changed — same setup helper, same three asserts. Add a row to pruneGatingConfigs (prune_gating_test.go L257) instead; TestCapabilitiesAgreeWithGates already pins caps-against-gate for every row there. The guard at prune_gating_test.go L290 has to widen to the new predicate for the row to pass, which is one line. The cache_sentinel_retention row in TestReceiptsGateFollowsRetention still earns its place — it pins the absolute boundary, which AgreeWithGates does not.

rpc/jsonrpc/eth_api.go L792-797 — shrink. receiptsRetiredWithHistory's amount != KeepAllReceiptsPruneMode half is dead: both call sites are the second arm of a switch whose first arm is case amount == prune.KeepAllReceiptsPruneMode. It also re-derives amount, which both callers already hold in scope. case !amount.Enabled(): inline at both sites, no helper, no doc comment.

receipts_generator_empty_block_test.go + receipts_generator_exec_sem_test.go — delete. Two new files for two tests of one code move. Fold into one test appended to post_state_calculated_test.go — same receipts package, newTestGenerator lives next door, and GetCachedReceipts is reachable from the exec-sem test, so the cache assertion costs two lines rather than a file. Drops two licence headers and two import blocks; also what CLAUDE.md asks for (append to an existing _test.go).

rpc/jsonrpc/check_prune_gates_test.go L575-583 — yagni. The resolveLogsRange leg opens its own transaction to test the function the two legs above it already route through. Nothing reaches the resolver except through one of them.

rpc/jsonrpc/eth_api.go L531-538 — shrink. holdsPreMergeBlockData's doc is still a seven-line paragraph after the trim. Keep the first sentence and "only a readable transaction of an early block settles it"; the TTL and fan-out rationale is commit-message material.

erigon_receipts_test.go L505-508, receipts_generator_empty_block_test.go L26-28 — delete. Doc comments restating names that already say it (...RequiresACanonicalBlock, ...CachesAnEmptyBlock). Keep only the second sentence of the erigon_getLogs one — the reorged-sibling case is the part the name does not carry.

net: about -95 of the 255 added lines.

@lupin012
lupin012 added this pull request to the merge queue Sep 1, 2026
Merged via the queue into main with commit 5ebf96f Sep 1, 2026
139 checks passed
@lupin012
lupin012 deleted the lupin012/prune_gates_small_fixes branch September 1, 2026 19:56
taratorio pushed a commit to nerolation/erigon that referenced this pull request Sep 2, 2026
…ontech#23725)

Follow-up to erigontech#23690, from an over-engineering review pass on it. No
behavior change.

| Site | Change |
| --- | --- |
| `eth_api.go`, `eth_system.go` | `receiptsRetiredWithHistory` inlined
as `!amount.Enabled()` — both call sites are the second arm of a switch
whose first arm already handles `KeepAllReceiptsPruneMode` |
| `check_prune_gates_test.go`, `prune_gating_test.go` |
`TestCapabilitiesFollowHistoryForASentinelRetention` → a
`pruneGatingConfigs` row; `TestCapabilitiesAgreeWithGates` pins it
against the gate more tightly |
| `receipts/` | the two new test files → one test next to the helpers it
uses, pinning the pre-semaphore answer and the cache together |
| `check_prune_gates_test.go` | the `resolveLogsRange` leg dropped —
nothing reaches the resolver except the two legs above it |
| `eth_api.go`, `erigon_receipts_test.go` | three docstrings trimmed to
what the name does not carry |

Net −110 lines. No new test: pure inlining and test consolidation,
covered by the existing suite.
pull Bot pushed a commit to Dustin4444/erigon that referenced this pull request Sep 4, 2026
…23759)

Follow-up to erigontech#23322. erigontech#23690 closed seven of its deferred review notes;
this closes the
two P3 ones it left, one per commit.

| Note | Site | Change |
| --- | --- | --- |
|
[r3880873445](erigontech#23322 (comment))
| `eth_api.go` | a log query no longer inherits the post-state
requirement of a full receipt |
|
[r3880873497](erigontech#23322 (comment))
| `prune_gating_test.go` | the fixtures persist the receipts their
retention promises |

## Logs

`checkLogsAvailable` reused the full-receipt gate, which refuses a
pre-Byzantium block
below the history cutoff: a full receipt carries a post state the cache
does not store
and only a re-execution can recompute. A log query never reads that
field — `getLogsV3`
asks `GetReceipt` for a receipt *without* a post state, which is exactly
why the
generator serves the cache. So with receipts kept beyond history, an
unfiltered
pre-Byzantium query was refused although every byte it reads was
retained.

`checkReceiptsAvailable` keeps the post-state leg; the rest moves
unchanged into
`checkReceiptSourceAvailable`. `checkLogsAvailable` composes the blocks
leg with that
one and keeps the history leg behind `usesLogIndex`, since an indexed
filter still
searches `LogAddrIdx`/`LogTopicIdx`. Filtered queries are unaffected,
and
`overlay_getLogs` gates on `checkBlockHistoryAvailable` and is
untouched.

`eth_capabilities` is unchanged on purpose: `caps.Logs` is already
`stricterRetention(receipts, history)`, so the post-state clamp was a
no-op for it.
`TestCapabilitiesAgreeWithTheLogsGatePreByzantium` pins that the
advertised boundary
does not move — the gate now reaches further back than advertised, never
short of it.

## Fixtures

They force-wrote `PersistReceipts` after `InsertChain` and never enabled
`RCacheDomain`,
so a disabled domain dropped everything execution wrote: the keep-all
cells passed by
re-executing against unpruned history. The domain is now enabled before
the chain runs,
and `requirePersistedReceipts` asserts the receipts really are on disk.

## Tests

Both commits are TDD; the fixture one first failed with `"[]" should
have 1 item(s), but
has 0`. New: `TestLogsGateSkipsThePostStateLegPreByzantium` (one block,
three answers),
`TestCapabilitiesAgreeWithTheLogsGatePreByzantium`,
`TestReceiptEndpointsCloseWhenTheCacheIsNotServed`. `go test -count=1
./rpc/jsonrpc/...`
green, `-race` green on the gate tests, `make lint` clean.

## Not in this PR

The fixture note also asked for an endpoint test with history
*physically* unavailable.
A negative control confirms the gap is real: the "endpoint answers" leg
passes even with
zero persisted receipts. Real pruning (`WithStepSize` + `WithPruneMode`)
is the right
route, but it reverses this fixture's premise of inserting without
physical pruning, so
it wants its own PR.
pull Bot pushed a commit to Dustin4444/erigon that referenced this pull request Sep 6, 2026
…#23322 (erigontech#23812)

Follow-up to erigontech#23322. erigontech#23690 closed seven of its deferred review notes
and erigontech#23759 two more; this closes the three that are about
`RemoteBlockReader.FrozenBlocks` and the mechanism behind it.

| Note | Site | Change |
| --- | --- | --- |
|
[r3885705116](erigontech#23322 (comment))
| `block_reader.go` | a slow backend no longer costs every caller a full
timeout |
|
[r3885704185](erigontech#23322 (comment))
| `block_reader.go` | the zero handed out before the backend answers is
no longer read as "no snapshots" |
|
[r3885706881](erigontech#23322 (comment))
| `block_reader.go`, `eth_api.go` | one shared TTL-cache instead of two
hand-rolled ones |

## Change

- The refresh runs on a goroutine of its own; a caller that has a value
to serve returns without waiting for the next one. A failed attempt is
stamped like a successful one, so an unreachable backend costs one
attempt per TTL instead of one per caller and per request.
- `FrozenBlocksObserved() (uint64, bool)` reports the count together
with whether the backend ever answered. `receipts.PostStateCalculated`
and the `eth_simulateV1` commitment path both read it; those are the two
sentinel readers a remote reader can reach.
- `common/concurrent.CachedValue[T]` holds what the getter and
`holdsPreMergeBlockData` each kept by hand: one TTL, one dedup, one rule
for what a failed pass leaves behind. Each site keeps its own waiting
policy — `Produce` runs on the caller's goroutine, `Go` refreshes behind
it.

## For review

**`singleflight` was tried and dropped.** `DoChan` cannot join the pass
in flight without possibly starting one of its own, and it runs the
producer on a goroutine the caller does not own. That is unsafe for the
pre-merge probe, which reads through the caller's `kv.Tx`: a caller
released by `ctx.Done()` rolls that transaction back under a live
reader. `Produce` runs the pass on the caller's goroutine instead.
`DoChan` also turns a producer panic into `go panic(e)`, which ends the
process rather than the request.

**An unobserved count now reads as "snapshots exist".** For a
pre-Byzantium block on a remote rpcdaemon whose backend has not answered
yet, `PostStateCalculated` returns false, so the stored receipt is
served — possibly without its `root` field — where before the block was
re-executed and the gate could answer `PrunedError` for receipts that
are on disk. That is the direction the note asks for, and the window is
one TTL after a failed round trip.

**Panics.** `Produce` publishes the failed pass and re-panics, so the
RPC server logs it as before; `Go` contains and logs it with the stack,
since its goroutine has no caller to recover it.

## Tests

`common/concurrent/cached_value_test.go` (11) pins the contract,
including that a caller running the pass finishes it before returning
while one that only waits is released by its own context.
`TestRemoteBlockReaderFrozenBlocks*` cover the getter,
`rpc/jsonrpc/receipts/post_state_calculated_test.go` the sentinel. The
`eth_simulateV1` leg has no dedicated test — the branch sits inside the
commitment path.

`make lint` clean; `./common/concurrent/...`,
`./db/snapshotsync/freezeblocks/...`, `./rpc/jsonrpc/...`,
`./execution/blockreplay/...`, `./cmd/rpcdaemon/...` green, with `-race`
on the concurrency tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants