Repository navigation
store: read with libc pread by default on macOS - #1058
Merged
reardencode merged 1 commit intoOct 17, 2026
Merged
Conversation
macOS has no io_uring. Its only completion session is a worker pool that hands every read to a pool thread through one shared lock and wakes two threads per read. During IBD most reads are small and often cached, so the handoff cost more than the read. Signet to 326,785 on a 10-core Mac, scripts skipped, all indexes: - warm: pool 639/661 s, pread 539/528 s (about 18% faster) - cold (resume from 250,875 after a cache flush): pool 355/351 s, pread 275/281 s (about 21% faster) - system CPU drops about four times; same tip hash every run RBITCOIN_IO=pool still opens the pool session. Linux and Windows defaults are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
|
Very interesting results. I wonder if we'll need to go deeper on this to get good performance later in a mainnet sync when the reads for past transactions are widespread. At the very least, we're overusing the pooled reads. Since I don't have a Mac, I'll take this without digging deeper, but if you do a mainnet sync and it gets surprisingly slow on IO later on, some IO may need to be moved to a pooled IO engine to keep the disk busy. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
macOS has no io_uring. Its only completion session is a worker pool: each read takes one shared lock, wakes a pool thread, then wakes the caller. During IBD most reads are small and often cached, so that handoff costs more than the read. This makes libc
preadthe macOS default.Evidence
Signet IBD to 326,785 from one local peer, scripts skipped, all indexes on, 10-core Mac. Same binary, only
RBITCOIN_IOchanges. Same tip hash in every run.pool(old default)pread(new default)unset_io_env_opens_a_session_except_on_darwinfails on master ("Darwin defaults to libc pread").After: passes. fmt, clippy, deny, ast-grep, os-smoke, and the os-smoke self-test pass locally. Workspace tests pass except three that also fail on master on this Mac.
ci-os-smoke.shnow pinsbulk_io::tests::unset_io_env_opens_a_sessionon every OS; the olddefault_kind_follows_ostest now only exists on Linux and Windows, so only the Windows smoke lists it.Merge Danger
Door: two-way
Revert, or set
RBITCOIN_IO=pool, to get the old macOS reads back. No schema, on-disk, durability, or consensus change.Blast Radius: macOS
Every store reader on macOS (IBD, Electrum, Esplora, RPC) uses pread. Linux and Windows are unchanged. Not measured: heavy serving load on macOS.
🤖 Generated with Claude Code