Skip to content

db/state: make COLLATE/MERGE/COMPRESS_WORKERS reach the aggregator - #23615

Open
awskii wants to merge 5 commits into
mainfrom
awskii/agg-worker-preset-deferred
Open

db/state: make COLLATE/MERGE/COMPRESS_WORKERS reach the aggregator#23615
awskii wants to merge 5 commits into
mainfrom
awskii/agg-worker-preset-deferred

Conversation

@awskii

@awskii awskii commented Aug 27, 2026

Copy link
Copy Markdown
Member

COLLATE_WORKERS, MERGE_WORKERS and COMPRESS_WORKERS never reach the aggregator on a node that restarts with state files on disk, and nothing reports the request was dropped.

setCollateAndBuild, setMerge and trySet return without writing while editLocks > 0, which buildFiles and the merge loop hold back to back from startup — so the Preset* call at the head of the execution stage lands in the gap. The construction default workersCfg{merge: 1, collateAndBuild: 1} ignores dbg.*, so Preset* was the only path from env to aggregator. A fresh datadir hides it: nothing is in flight at the first Preset*. Seen on a mainnet archive node as 2/1/1 syncing from zero, then 1/1/1 for the whole run after a restart at block 8.1M.

Changes

  • trySet queues a request that arrives while pinned; the last unlockEditing runs the queue. A request still cannot change config under a running build or merge.
  • Construct from dbg.MergeWorkers / dbg.CollateWorkers instead of hardcoded 1, 1.
  • RegisterDomain / RegisterII stamp COMPRESS_WORKERS, the first point a domain's compressor config exists.

Notes

Not an argument for higher defaults. On a 16-core / 125 GB box with snapshots 18x RAM, 2/1/1 to 4/2/2 cost 23% end-to-end (55.8 to 42.8 blk/s), major faults 2626 to 5359/s, iowait 1.9% to 17.3%: collate/merge streaming evicts the commitment trie's branch pages, and trie descent is serial, so each fault is dead wall-clock. The knob now works in both directions.

@awskii
awskii requested a lite review from Copilot August 27, 2026 03:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@awskii
awskii requested a lite review from Copilot August 27, 2026 03:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants