IPC support for Bitcoin Core v31 using libmultiprocess#48
Closed
Sjors wants to merge 10 commits into
Closed
Conversation
git-subtree-dir: src/ipc/libmultiprocess git-subtree-split: 51f3d1a
Vendor Bitcoin Core's contrib/devtools/git-subtree-check.sh as tests/lint/git-subtree-check.sh and run it on every push/PR via a new 'subtree-check' CI job. The script walks the merge history of a given subtree path and verifies that the current tree matches what was last committed via the subtree merge, catching accidental in-place edits to the vendored libmultiprocess copy. The check is wired in here (rather than in the main build matrix) because it needs fetch-depth: 0 to walk the full history.
Mirror Bitcoin Core's util/strencodings.h API as header-only helpers
under src/util/, shared between the upcoming IPC layer and any future
non-IPC callers without dragging in interfaces/ headers:
- HexStr(span) / TryParseHex(string_view): forward, low-index-first
lowercase hex; TryParseHex returns std::nullopt on odd length or
any non-hex character.
- HashHexStr(span) / TryParseHashHex(string_view): the byte-reversed
variants used to display Bitcoin hashes (which are stored
little-endian internally but rendered big-endian over RPC). The
upcoming uint256 stand-in delegates to these instead of carrying
its own GetHex/FromHex implementation.
Add the minimum scaffolding to open an IPC connection to a v31.0
bitcoin-node:
- WITH_IPC CMake option (ON by default; forced OFF on Windows
pending bitcoin-core/libmultiprocess#231) that pulls in the
vendored libmultiprocess subtree and Cap'n Proto via
find_package(CapnProto 1.0 REQUIRED).
- A wire-compatible Init capnp schema that preserves upstream's
interface ID (@0xf2c5cfa319406aa6) so this client can connect
to an unmodified v31.0 bitcoin-node IPC socket. Only Init.construct
is declared; factory methods are added in follow-up commits as
bitcoin-tui starts to use them.
- IpcClient: opens the unix socket and creates the Init proxy on a
background EventLoop thread. Connection errors (missing socket,
refused connection, oversized path) propagate as runtime_error.
- Two unit tests covering the bogus-path and oversized-path failure
modes. The successful-connect path will be exercised in the next
commit once there is an in-process Init server to talk to.
Grow the bare Init plumbing introduced in the previous commit with the
Mining capability, plus a one-shot version probe in IpcClient's
constructor that fails fast against pre-v31.0 servers:
- capnp Init schema gains makeEcho, makeMiningOld2 and makeMining at
ordinals 1, 2 and 3 to match v31.0's bitcoin-node wire layout.
- capnp Mining + common schemas, with a wire-compatible BlockRef and
a uint256 stand-in (which uses util::HexStr / util::TryParseHex
from the strencodings commit for hex conversion).
- IpcClient constructor now eagerly calls init->makeMining(). v30.x
has makeMining at ordinal 2 (we call ordinal 3) and faults at the
wire level with 'Method not implemented; methodId = 3'; v31.0+
accepts the call. Round-tripping makeMining alone is enough — no
follow-up probe on the returned Mining capability is needed.
- PathServer test helper now takes any interfaces::Init, plus a new
mock_ipc::InitV30 that throws from makeMining() to simulate the
wire-level fault. The C++ mock can't model the real schema-level
'method not implemented' directly (both v30 and v31 mocks share
the same Init class), so InitV30 stands in for that response.
- Adds a test that connects to InitV30 via PathServer and verifies
IpcClient surfaces a friendly 'older than v31.0' runtime_error.
Now that the version probe established the server speaks v31.0 wire,
expose the Mining methods bitcoin-tui actually calls:
- IpcClient::wait_tip_changed(current_tip_hex, timeout) wraps
Mining.waitTipChanged. v31.0's wire format has no 'hasResult'
companion field, so the server signals 'no result / interrupted'
by returning a null BlockRef; we surface that as std::nullopt.
- IpcClient::interrupt() calls Mining.interrupt() and is
thread-safe so the poll loop can wake the waiting thread on
shutdown. Best-effort: any exception is swallowed.
- tests/mock/ipc/mining_v31.h: configurable v31.0-style Mining mock
(set_tip, last_current_tip, interrupt with epoch-based wakeup).
- test_ipc gains four new cases against this mock: connect + read
tip, null hash maps to nullopt, interrupt unblocks an in-flight
wait_tip_changed within a tight wall-clock bound, and the
current_tip_hex argument is passed through to the server
end-to-end (catches a regression where the field is dropped).
Use the typed Mining proxy added in the previous commit to gate the
poll loop on real tip-change notifications instead of just sleeping
for --refresh seconds. JSON-RPC over HTTP still drives every actual
data fetch -- this only changes when the loop runs.
- --ipcconnect <path>: opt out with 'off', or pass an explicit
socket path; otherwise auto-detect <datadir>/<net>/node.sock and
use it when present.
- The IpcClient is constructed before the poll thread starts so the
main thread can call interrupt() during shutdown to unblock any
in-flight waitTipChanged.
- Footer gains an 'IPC' badge while the typed transport is active
(driven by AppState::ipc_connected, plumbed through state.hpp).
- Errors from waitTipChanged are logged in --debug mode and treated
as 'tip didn't change', followed by a 1s back-off so a transiently
misbehaving server doesn't melt the CPU.
Document the v31.0 IPC transport in the README:
- Requirements bullet now states Cap'n Proto >= 1.0 is required by
default, with -DWITH_IPC=OFF as an explicit opt-out, and notes
that Windows is forced off pending bitcoin-core/libmultiprocess#231.
- Build section shows both the default invocation and the IPC-off
variant.
- New 'IPC transport' section explains the --ipcconnect flag,
points at the IPC badge in the footer, and links to bitcoin/
bitcoin#32297 for the v32 JSON-RPC-over-IPC interface (makeRpc),
which bitcoin-tui will adopt once v32 ships.
Install capnproto + libcapnp-dev (apt) / capnp (brew) on the Linux and macOS jobs and pass -DWITH_IPC=ON explicitly. CMakeLists.txt already requires Cap'n Proto when WITH_IPC is on, so the configure step now fails loudly on those jobs if the install step ever breaks rather than silently skipping IPC and the test_ipc suite. The Windows job stays at -DWITH_IPC=OFF, with an inline comment linking bitcoin-core/libmultiprocess#231 (the pending Windows IPC support PR) so the reason is visible in the workflow file itself.
Owner
On the roadmap there is a plan to rewrite the main tabs to LUA. :) So it's just a question of when not if. |
Author
|
@janb84 nice, please ping me here when that's ready. Can you give CI a run? |
Author
|
ok that broke a bunch of things :-) Going to close this for now, until IPC has a killer feature. Or at least until after LUA. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A more narrow alternative to #47 which works against Bitcoin Core v31.
This is probably too much plumbing for the tiny benefit of faster tip updates (via the Mining interface).
It's worth revisiting after:
It's also worth contrasting to the alternative approach of using LUA capnproto bindings instead of libmultiprocess. But that requires further refactoring in this project, since most of the tabs are still implemented in c++.