perf(rpc): serve getblocktemplate from a precomputed block template

[upbqdn]

Motivation

Closes #11370. Closes #10747.

Solution

A background task keeps a block template for the current chain tip ready, and getblocktemplate
serves it instead of reading the state and the mempool, selecting transactions, and building a
coinbase transaction on the request path. Zebrad starts the task when mining.miner_address is
configured; the RPC falls back to building a template itself when no task is running.

The task publishes a coinbase-only template as soon as the chain tip changes, so miners get work
on the new tip immediately, then replaces it with one carrying mempool transactions, and refreshes
that every MEMPOOL_LONG_POLL_INTERVAL. A precomputed template can therefore be a few seconds
behind the mempool, costing the miner the fees of the transactions that arrived in the meantime.
It is never behind the chain: the RPC ignores a template whose previous block hash is not the
current tip.

That shared coinbase also removes the per-request precompute from the long-poll wait (#10747),
where each outstanding long poll ran its own shielded coinbase proof, once per 5-second iteration.
Handlers whose miner parameters were overridden after cloning — generate and generatetoaddress
— share neither the template nor the coinbase cache, so they never serve a coinbase built for
another address or another set of coinbase data.

Tests

getblocktemplate_precomputed asserts the two properties that make this safe to serve. It starts
the updater task against mock services, then stops those mocks answering: a response still
arrives, which is only possible without reading the state or the mempool. It then advances the
mock chain tip and requires the next response to extend the new tip hash — that assertion fails
if the staleness check is removed, which is the bug that would hand miners work on a chain Zebra
has already seen a block for.

getblocktemplate_long_poll_returns_submit_old_false_on_new_tip covers the rewritten long-poll
wait end to end: it keeps waiting while the template is valid, returns within the fast-path bound
after generate moves the tip, reports submit_old: false, and validates the returned template
as a block proposal under every advertised time source.

Specifications & References

Part of #11310, which also tracks the other getblocktemplate performance work discussed there.

Follow-up Work

The remaining ports named in #11310 are untouched. The per-call cost that is left is response
serialization, which is proportional to the template's transaction data.

AI Disclosure

• No AI tools were used in this PR
• AI tools were used: Claude Code, for the implementation, the tests, and this description.

PR Checklist

• The PR title follows conventional commits format: type(scope): description
• The PR follows the contribution guidelines.
• This change was discussed in an issue or with the team beforehand.
• The solution is tested.
• The documentation and changelogs are up to date.

[v12-auditor]

Note

Complete: Audit complete. V12 found nine issues worth reviewing.

Open the full results here.

Finding	Severity	Details
F-261353	🔵 Low	Updater termination is silently ignored StartCmd::start retains the block-template updater JoinHandle but omits it from the daemon's pinned ongoing-task set and main select!. If precompute::run terminates, its result is never observed and no replacement task is started. RPC handlers retain the cache, but an empty cache immediately misses and a stale cache eventually times out, causing future getblocktemplate calls to remain on the slower state, mempool, transaction-selection, root, and coinbase construction path until restart. Ordinary template-build errors are retried inside the updater, so premature termination during otherwise healthy operation requires an uncaught dependency/runtime failure; a documented normal exit occurs when the chain-tip channel closes. This is a supervision and observability defect with persistent performance impact rather than a demonstrated daemon-wide denial of service.
F-261354	🔵 Low	Time refresh discards still-valid testnet work When testnet wall-clock time crosses the minimum-difficulty boundary without a tip change, the updater refreshes the cache from a standard-difficulty template to a minimum-difficulty template with a different max_time. The fast path derives submit_old through LongPollId::submit_old, which returns false whenever that advertised maximum timestamp changes. An old share that respected the earlier max_time still carries a timestamp and standard-difficulty bits that remain consensus-valid after wall-clock time advances. The response therefore tells conforming pools to discard work that remains potentially valid. The pre-existing slow path also forces the same result when its maximum-time timer fires, while the new periodic cache refresh exposes the behavior through the fast path.
F-261355	🟡 Medium	Fee churn triggers recurring shielded proofs The new updater rebuilds a full mempool template after every completed build and five-second sleep, even when no RPC request is pending and even when the RPC listener is disabled. Each build reruns ZIP-317 selection, sums the selected fees, and reuses the real coinbase only when the exact (height, total_fee) key is cached. On a node paying a shielded miner address, a peer can relay a valid independent conventional-fee transaction between refreshes, deterministically changing the selected fee total while the template remains below block capacity and forcing another shielded coinbase proof. Once candidate transactions exceed block capacity, randomized selection can also produce distinct total fees on an unchanged mempool, while the four-entry cache evicts older nonzero-fee variants. The proof runs in spawn_blocking, which protects Tokio worker threads but still consumes host CPU for the multi-second proof duration documented by the updater.
F-261356	🟡 Medium	Tip changes detach expensive shielded proofs With a configured shielded miner receiver, run() starts seconds-long spawn_blocking coinbase proofs for future heights. When the next observed tip requires a different height, store_precomputed_coinbase calls take().filter(...), so a mismatch drops the only tracked JoinHandle without awaiting it; dropping the handle detaches the already-running blocking proof rather than cancelling it. The tip-change select! creates a second path by dropping an in-progress build_with_mempool future whose inner blocking task may already be proving a fee-bearing shielded coinbase. Zebra then starts useful proof work for the actual tip while the obsolete tasks continue untracked. Repeated rapid tip changes can therefore accumulate detached CPU-heavy proofs despite the updater retaining only one current next_coinbase handle.
F-261357	🟡 Medium	Lagging cache loops clients between template IDs The precomputed fast path treats every cached long-poll ID different from the client's ID as newer, although the cache is intentionally allowed to lag the live mempool. A client can receive a newer live-mempool template from the fallback path, immediately poll on that ID, and then be returned the older cached template it had already seen. If the client follows each returned ID, it alternates between the newer fallback ID and the older cache ID until the updater publishes a current entry. The older response also reverts the miner to an older transaction set instead of keeping the long poll pending. Requests on the old cached ID repeatedly enter the live fallback, amplifying state, mempool, and template-construction work during the oscillation.
F-261358	🔵 Low	Cache serves time-stale testnet difficulty work Cache eligibility depends only on previous_block_hash, while clock-derived time and difficulty fields can change on testnet without a tip change. Around the standard-to-minimum-difficulty transition, the updater can remain asleep or still rebuilding for up to the periodic refresh interval while the cache contains the prior standard-difficulty template. If a client ID differs from that cache entry only through mempool fields, the fast path returns the cached response immediately and computes submit_old: true, bypassing fresh chain-info calculation. The returned target is harder rather than consensus-invalid when the miner retains a timestamp inside the old range, but it is no longer the profitable current minimum-difficulty job. There is no wakeup tied to the time boundary.
F-261362	🟡 Medium	Fallback fetches delay tip invalidation indefinitely An exact cache-ID long poll awaits chain-info and full-mempool requests before it creates the future that observes best_tip_changed. Marking the tip seen preserves a pending notification but cannot interrupt either service call. If the tip changes while one of those requests is delayed or stuck, the long poll cannot serve the replacement precomputed template even if the updater has already published it. A permanently pending dependency request therefore causes the client to miss the new-tip invalidation indefinitely. This defeats the new fast-tip response specifically during the fallback fetch window.
F-261364	🟡 Medium	Block templates are returned without terminal tip freshness validation getblocktemplate can return work extending a superseded parent because neither its precomputed-cache path nor its state/mempool fallback verifies tip freshness immediately before returning. In the cached path, precomputed_block_template snapshots best_tip_hash, waits for a cache entry whose previous_block_hash matches that historical value, and returns it even if the best tip changed during the wait. The initial and long-poll callers directly return that helper result. In the fallback path, the request marks the tip as seen, obtains chain and mempool data, and may exit immediately when a mempool-derived long-poll ID differs, without consuming a concurrently pending tip update. It then constructs and returns the response from those snapshots without a terminal live-tip comparison, and can derive submit_old from the stale response ID.
F-261386	🔵 Low	Maximum-time boundary mishandles unchanged long-poll work The long-poll fallback treats the expiration of a timer scheduled to the inclusive max_time boundary as a reason to return submit_old: false, even if the refreshed template still has the same long-poll ID. Testnet remains on standard difficulty when cur_time == max_time and moves to the minimum-difficulty branch only strictly afterward, so this response can unnecessarily reset exactly equivalent work. Conversely, when a request starts with cur_time already clamped to max_time, the zero-duration timer is disabled and no persistent expiration state is encoded in the ID. With no tip or mempool change, that exact-ID long poll continues polling rather than delivering the intended maximum-time reset. These two outcomes make the behavior at the same boundary dependent on whether a timer happened to be armed before equality.

And one more auto-invalidated finding.

Analyzed four files, diff ef6325c...5216041.

[upbqdn]

The red zaino shard-5 check is a pre-existing flake in wallet.py, not a regression here.

That test asserts zallet's visible mature-coinbase count is within 5 of the tip. It fails with a different count on every run: 34 on zallet-interop-request run 33470577537 hours before this branch existed, then 52 and 44 on the two runs here. In each case the preceding assertion — getaddressbalance(miner_address) == tip * 6.25 ZEC — passes, so every coinbase paid the right address, and wallet_transparent_spend.py passes alongside it. The zebra shard-5 variant of the same shard passes here.

[upbqdn]

Pushed a third commit: the tip check now reads ReadRequest::Tip instead of latest_chain_tip.

The state's write task publishes a committed block to the read state before it updates the chain tip channel (update_latest_chain_channels()), so in that interval ChainInfo reports the new tip while latest_chain_tip still reports its parent — and validating against the channel there served a template for a chain this node had already extended. ReadRequest::Tip reads the same non-finalized state channel ChainInfo builds templates from, so the two cannot disagree. The read is skipped when no template has been published, so nodes without the updater task do not pay for it.

getblocktemplate_ignores_precomputed_template_when_tip_channel_lags_state holds that interval open — the mock chain tip never advances past the first tip while the state reports the second — and requires the response to build on the block the state committed. It fails if the tip check goes back to the channel.

The first commit's changelog entry is reworded to match: a call validates the tip against the state rather than skipping state reads entirely.

[conradoplg]

Some Claude findings, haven't fully verified them:

---

1. Long polling degenerates into an unthrottled busy loop (high)

zebra-rpc/src/methods.rs:2566 and :2723 serve templates from the precompute cache; zebra-rpc/src/methods.rs:2576-2657 builds them from a fresh state+mempool read. Long-poll "has anything changed?" is now
evaluated against two independent sources that never agree, so it oscillates:

• Client long-polls with id S (from the fallback path). Top-of-function check: cache holds P, P != S → returns P immediately.
• Client long-polls with P. Top check: P == P → None (methods.rs:1075) → falls into the loop → fresh mempool read yields S'; S' != P → breaks and returns immediately.
• Repeat, with no wait on either side.

The long poll ID is (tip_height, tip_hash, max_time, mempool tx checksum). max_time is median_time_past + 90min (zebra-state/src/service/read/difficulty.rs:230), constant per tip, so the only differing
component is the mempool set — and the precomputed snapshot is up to MEMPOOL_LONG_POLL_INTERVAL = 5s old by construction (precompute.rs:252). So every mempool arrival opens a spin window that lasts until the
next 5-second refresh. Each spin iteration costs a Tip read, a ChainInfo read, a full mempool::Request::FullTransactions, ZIP-317 selection, and Merkle/auth root computation — plus a fresh shielded-coinbase
proof each time the fee total changes (the fallback's fee total won't match any entry the updater cached).

Before this branch both sides read the same fresh mempool, so the second poll converged and slept 5s. This is a new regression.

Concrete consequence for the internal miner (zebrad/src/components/miner.rs:282-320), which is exactly this long-polling client: it alternates between two templates with different Merkle roots, so
template_sender.send_if_modified reports a change every iteration, cancel_fn (miner.rs:439) cancels the equihash solver every iteration, and it mines nothing while spinning and logging "mining with an updated
block template" at full speed.

Fix direction: pick one change detector. When template_cache is live, implement the long-poll wait on the watch channel itself (receiver.changed(), alongside tip-change and max-time), instead of falling
through to a path that rebuilds from the state and mempool.

2. The updater task is unsupervised (medium)

zebrad/src/commands/start.rs:490 spawns the task, and :866-868 aborts it at shutdown — but unlike every other ongoing task it is never added to the select! at :748-846, so its handle is never joined. If run()
panics or returns early (precompute.rs:213, :249 return silently when the tip channel closes), nothing logs it and nothing restarts it.

The failure is silent but not harmless: TemplateCache::is_empty() (precompute.rs:76) stays false because the last template is still there, so every subsequent getblocktemplate call still enters wait_for_tip,
and after the next tip change it burns the full NEW_TIP_TIMEOUT of 1 second (precompute.rs:49) before falling back — permanently, on every request. Either put the handle in the supervision select! like the
other ongoing tasks, or have run() clear the cache on exit so is_empty() short-circuits.

3. Unconditional background shielded proving (medium)

run() refreshes every 5 seconds forever (precompute.rs:252) whenever mining.miner_address is set. Each refresh calls new_internal, which misses the coinbase cache whenever the mempool fee total moved and
re-runs the Sapling/Orchard proof (get_block_template.rs:333-341) — the code's own comments put that at "seconds". That is close to a permanently busy core.

It runs regardless of whether anyone is mining: spawn_block_template_updater is called at start.rs:490 before the config.rpc.listen_addr.is_some() check at :493, so a node with a configured miner address and
no RPC listener and no internal miner still proves continuously. Previously this work was strictly on demand. Consider gating the spawn on an RPC listener or the internal miner being enabled, and/or backing
off the refresh when no getblocktemplate call has arrived recently.

4. Mempool failures are masked (low)

If the updater's mempool build keeps failing, build() returns Err, gets logged at debug (precompute.rs:216), and the cache keeps the last template. The RPC serves that instead of surfacing the error the
fallback path would have returned. Staleness is bounded by the inter-block interval (the next tip change makes wait_for_tip time out and fall back), so this is minor — but a repeated failure deserves more than
debug!, since it silently costs miners the fees of everything in the mempool.

[upbqdn]

Thanks — 1 was real and the worst of it. All four addressed in 28b8f1f, 0808c8c, 1bffc8a.

1. Long polling degenerates into an unthrottled busy loop. Confirmed, and your fix direction is what I took: long polling now waits on the cache it is served from (published template, tip change, or max_time), and never falls through to a path that derives a competing ID. Worth noting the answers were also going backwards — the client got handed the older mempool snapshot each time it polled with the fresher ID.

Regression test getblocktemplate_long_poll_waits_for_a_new_template asserts a long poll on the current template does not return. The fixture has to make the two ID sources disagree to exercise this: the cache holds a template built from an empty mempool while a fresh read sees a transaction. My first attempt kept the mempool empty throughout, both sources agreed, and the test passed against the unfixed code — so the fixture is the load-bearing part. With the fall-through restored it fails in 1.2 s.

2. The updater task is unsupervised. Confirmed; its handle is now in the supervision select! with the other ongoing tasks.

3. Unconditional background shielded proving. Two halves. The gating half is fixed: the spawn now requires an RPC listener or the internal miner, so a node with a miner address and neither no longer proves anything. The refresh half is fixed in the stacked #11374, which replaces the five second timer with mempool change notifications plus a thirty second backstop — an idle node rebuilds 6x less often, and those rebuilds hit the coinbase cache because the fee total is unchanged, so they do not re-prove.

4. Mempool failures are masked. Now warns when a failing spell starts and logs the recovery, staying at debug! for the repeats since the retry delay is one second.

The zaino shard-5 check is the pre-existing wallet.py scan-lag flake, unrelated.

[upbqdn]

Force-pushed: the long-poll fix had a bug of its own, and writing a sharper test for it is what found it.

My first version cloned the chain tip receiver inside the wait loop, so best_tip_changed() reported the tip it was created with as a change and returned immediately — the call parked in name only and spun. The receiver is now cloned once and marked seen before the loop.

The test asserts the absence of that too. "Did not return" is satisfied by a spin as well as by a park, so it now also measures the work: it samples the state-tip read count across two windows and requires no growth while waiting. Against the re-cloned receiver that assertion fails with 37,129 tip reads in 250 ms, each of which would have carried a mempool selection and a coinbase build in production. Against the original fall-through it fails as before.

New hashes: 0a630a6 (long polling), 7bc3645 (supervision and gating), 756e51b (warn on failing builds).

[upbqdn]

Force-pushed once more, 42bd34b. Two things beyond the four findings:

The race in the fix itself. The cache subscription was taken at the point of waiting, after the call had already read the template, so a template published in between was marked seen and skipped. The other wake conditions are a chain tip change and max_time, neither of which fires for a mempool-only change, so a long poll would have sat on a template it had already been told about until the updater's next refresh. A TemplateChanges subscription is now taken before the read and held across the wait. Two unit tests pin it: the old shape fails them with Elapsed.

Changelog. The zebra-rpc and zebrad entries now describe the net state rather than the first draft: long polling waits on the precomputed template, and Zebra only keeps one ready when the RPC server or the internal miner can ask for it.

Also correcting something I wrote earlier in this thread: I said each of the 37,129 spinning iterations would carry a mempool selection and a coinbase build. It would not — the spin stays inside the cache path, so each iteration is a state tip read and an ID comparison. The busy loop was real, its cost was state traffic.