fix(rpc)!: bound getaddressutxos by the requested height range and limit - #11317
Merged
mergify[bot] merged 2 commits intoAug 25, 2026
Merged
Conversation
`ReadRequest::UtxosByAddresses` becomes a struct variant carrying a `height_range` and an optional `max_entries`, and both are pushed down to the finalized address index scan. The index is keyed by `(AddressLocation, OutputLocation)`, so a start height is a seek and a limit is a `take`. Two parts are easy to get wrong, and the new tests cover them. The non-finalized chain can spend UTXOs that the finalized query returned, so a limited finalized scan over-fetches by the number of UTXOs the chain spends for those addresses. Without that, a full page of results can come back short, and a client paging by count reads a short page as the end of the address' UTXOs. Truncation happens after the finalized and non-finalized sets are merged, not while they are chained. Chaining two sorted maps is not globally sorted, and entries in the overlap window would otherwise be counted twice. No caller sets either bound yet: `get_address_utxos` passes the full height range and no limit, so this commit changes no behavior.
LarryRuane
force-pushed
the
fix/getaddressutxos-pushdown
branch
from
August 21, 2026 06:26
4e72794 to
4229b64
Compare
The `GetAddressUtxos` gRPC method accepted both arguments and applied them, but only to the finished reply: the node read every UTXO held by the named addresses first, then discarded whatever was not asked for. The `getaddressutxos` JSON-RPC underneath had no such arguments at all, so no caller could ask for less work. Pass both bounds into the state query, and drop the reply-side filter in the gRPC method. The cost is now set by what the caller asked for rather than by the size of the addresses' UTXO sets. That is the unremediated part of GHSA-x4m7-3gpp-xc36, and item 1 of ZcashFoundation#11239. A start height above the chain tip is clamped rather than rejected, because a start height above the chain already selects nothing, which is what the request means. Because the JSON-RPC previously ignored both fields, a client that already sends them now receives fewer entries than before for the same request.
LarryRuane
force-pushed
the
fix/getaddressutxos-pushdown
branch
from
August 21, 2026 15:25
4229b64 to
b772175
Compare
conradoplg
self-requested a review
August 24, 2026 21:47
Contributor
|
Thank you for this PR, we'll review shortly |
conradoplg
approved these changes
Aug 25, 2026
Contributor
Merge Queue Status
This pull request spent 36 minutes 49 seconds in the queue, including 35 minutes 40 seconds running CI. Required conditions to merge
|
47 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Zebra serves
GetAddressUtxosover its built-inCompactTxStreamergRPC server, added in #10953. That method takes astartHeightand amaxEntries, and it honors both, but only by filtering the finished reply: the node reads every unspent output the named addresses hold, and then discards whatever the caller did not ask for. The answers are correct, but the work is not bounded by the request. A caller asking for ten entries from an address holding thousands pays for all of them.The
getaddressutxosJSON-RPC underneath has no such arguments, which is why the gRPC layer had to filter afterwards, and why no other caller can ask for less work at all. Adding the two arguments there lets the bounds reach the index scan, which is the only place they can actually save anything.The same shape appears one layer further out. lightwalletd, a server that serves Zcash chain data to light wallets, offers its clients the same method and answers it with this RPC; as of zcash/lightwalletd#592 it forwards both arguments, and a node without this change silently drops them as unknown JSON keys.
Bounding the scan is worth doing on its own, but it is also a fix: this is the unremediated part of GHSA-x4m7-3gpp-xc36, and item 1 of #11239. The worst shape is a single high-UTXO address with a start height at the chain tip, which is a tiny request and an all-but-empty reply on top of a full index scan, leaving no reply bandwidth with which to throttle the caller.
Measured on the built-in gRPC server against a synced mainnet node, on
t1cArVf7BgN3zPx6UHGy87ZAurTA4AQejj7, holding ~6,704 UTXOs:maxEntries=10startHeight=3450000The address is in active use, and the "after" column was measured first, so one output arrived at it between the two runs. That single output is the whole of the difference between 6,704 and 6,705, and between 7 entries and 8: it landed above height 3,450,000, so it is counted in both rows. Nothing is dropped by the change.
The entries returned are otherwise the same before and after; only the work behind them changes. The improvement is understated here, because a trivial
GetLightdInfocall through the samegrpcurlharness costs 15ms, so the 17ms row is about 2ms of server time against roughly 91ms before.Solution
ReadRequest::UtxosByAddressesbecomes a struct variant carrying aheight_rangeand an optionalmax_entries, and both are pushed down to the finalized index scan. The RocksDB address index is keyed by(AddressLocation, OutputLocation), so a start height is a seek and a limit is atake.Two parts are subtle and are what the new tests cover.
The non-finalized chain can spend UTXOs that the finalized query returned, so a limited finalized scan over-fetches by the number of UTXOs the chain spends for those addresses. Without that, a full page of results can come back short, and a client paging by count reads a short page as the end of the address' UTXOs.
Truncation happens after the finalized and non-finalized sets are merged, not while they are chained. Chaining two sorted maps is not globally sorted, and entries in the overlap window would otherwise be counted twice.
A start height above the chain tip is clamped rather than rejected, because a start height above the chain already selects nothing, which is what the request means.
Tests
Unit tests in
zebra-statecover the height-range filter on non-finalized UTXOs, that a limited result keeps the lowest entries in chain order, and that over-fetching keeps a limited page full when the chain spends an entry the finalized query returned.zebra-rpcgainsrpc_getaddressutxos_limits, covering each argument alone, both together, and clamping at and aboveu32::MAX.Mutation testing drove the shape of those tests: truncating inside the merge, dropping the over-fetch, and an off-by-one in the limit all initially survived. Two mutants still escape and are called out under Follow-up Work.
Manually tested on both interfaces, on mainnet and on regtest. Every comparison is against
mainbuilt from source with the same release profile, takes the median after three warm-up passes, and reports entry counts so any drift is visible. JSON-RPC rows usecurlover 15 runs; gRPC rows usegrpcurlover 11, and carry that client's ~15ms startup floor.JSON-RPC
getaddressutxos, mainnet, same address as above, 6,704 entries in both runs. Here the returned entries change, because the arguments did not previously exist:maxEntries=10maxEntries=100startHeight=3450000Results were verified, not just timed:
maxEntries=Nreturns exactly the first N entries of the unbounded result,startHeight=Hreturns exactly the unbounded result filtered toheight >= H, and results stay ascending by height.Regtest, reproducible. A regtest node mines to a fixed transparent address, so
generate 10000accrues exactly one coinbase UTXO per block totmJymvcUCn1ctbghvTJpXBwHiMEB8P6wxNV, and nothing ever spends them:maxEntries=10startHeight=9000The unbounded rows improve because the scan replaces a per-entry
zs_next_key_value_fromseek with a single bounded range iterator. That gain is smaller on regtest than on mainnet, because a 41MB database has a shallow, fully cached LSM tree, which is the case most favorable to the old per-entry seek.Also tested end to end through lightwalletd. Against a released 6.3.0 node every query returned the address' full UTXO set, confirming a node without this change ignores the fields rather than erroring; against this branch the same requests came back bounded.
Specifications & References
service.protodocuments the reply as sorted by height, which is the order this code returns and assertsFollow-up Work
Two mutants survive, both needing fixtures this PR does not add: the
finalized_scan_limitcall site needs a populated non-finalized chain, and the range end bound is never exercised because the RPC always passesHeight::MAX.A backend must implement both arguments or neither. Honoring
maxEntrieswhile ignoringstartHeightwould truncate before the range filter runs and silently return fewer entries than exist, which a caller has no way to detect.Paging by
startHeightcannot express a resume point when one height holds more entries than the page size. That is a protocol-level gap, tracked in zcash/lightwallet-protocol#33, and is independent of this change.AI Disclosure
PR Checklist