db/state: make COLLATE/MERGE/COMPRESS_WORKERS reach the aggregator - #23615
Open
awskii wants to merge 5 commits into
Open
db/state: make COLLATE/MERGE/COMPRESS_WORKERS reach the aggregator#23615awskii wants to merge 5 commits into
awskii wants to merge 5 commits into
Conversation
…ld or merge pins the config
…ruction, not only via Preset*
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
COLLATE_WORKERS,MERGE_WORKERSandCOMPRESS_WORKERSnever reach the aggregator on a node that restarts with state files on disk, and nothing reports the request was dropped.setCollateAndBuild,setMergeandtrySetreturn without writing whileeditLocks > 0, whichbuildFilesand the merge loop hold back to back from startup — so thePreset*call at the head of the execution stage lands in the gap. The construction defaultworkersCfg{merge: 1, collateAndBuild: 1}ignoresdbg.*, soPreset*was the only path from env to aggregator. A fresh datadir hides it: nothing is in flight at the firstPreset*. Seen on a mainnet archive node as2/1/1syncing from zero, then1/1/1for the whole run after a restart at block 8.1M.Changes
trySetqueues a request that arrives while pinned; the lastunlockEditingruns the queue. A request still cannot change config under a running build or merge.dbg.MergeWorkers/dbg.CollateWorkersinstead of hardcoded1, 1.RegisterDomain/RegisterIIstampCOMPRESS_WORKERS, the first point a domain's compressor config exists.Notes
Not an argument for higher defaults. On a 16-core / 125 GB box with snapshots 18x RAM,
2/1/1to4/2/2cost 23% end-to-end (55.8 to 42.8 blk/s), major faults 2626 to 5359/s, iowait 1.9% to 17.3%: collate/merge streaming evicts the commitment trie's branch pages, and trie descent is serial, so each fault is dead wall-clock. The knob now works in both directions.