Conversation
illuzen
commented
Jan 26, 2026
- use QUIC instead of HTTP (no polling, just bidirectional push)
- simplify protocol to two message types
n13
left a comment
There was a problem hiding this comment.
Minor comment on if-else above - not a blocker though
Gemini Review:
Review of PR #363: QUIC miner
Summary
This PR replaces the HTTP-based external miner protocol with a QUIC-based protocol. This is a significant architectural improvement that enables:
- Lower Latency: Server-push for new jobs (no polling).
- Efficiency: Persistent connections and bidirectional streams.
- Simplicity: Reduced protocol complexity (only 2 main message types).
The changes involve:
- Node Side: A new
MinerServerimplementation usingquinn(QUIC). - Protocol: Definition of
MinerMessagetypes inminer-api. - Integration: Updates to
node/src/service.rsto use the new server. - Cleanup: Removal of the old
external_miner_client.rs.
Protocol Design
The move to QUIC is well-justified for a mining protocol where latency is critical. The "push" model for jobs ensures miners start working on new blocks immediately.
Message Types
The protocol is clean, but there is a slight discrepancy between EXTERNAL_MINER_PROTOCOL.md and the implementation:
- Documentation: Lists only
NewJobandJobResult. - Implementation: Includes a
Readymessage sent by the miner immediately after connection.- Reason: To establish the bidirectional stream (QUIC streams are lazily initialized).
- Recommendation: Update
EXTERNAL_MINER_PROTOCOL.mdto document theReadymessage so independent miner implementers are aware of it.
Code Review
node/src/miner_server.rs
The implementation of the QUIC server is solid.
- Concurrency: Uses
Arc<RwLock<HashMap>>for miner management, which is appropriate. - Resource Management: Miners are correctly registered and unregistered upon connection/disconnection.
- Broadcasting:
broadcast_jobiterates through all connected miners. This is efficient for the expected number of miners. - Security:
- Enforces
MAX_MESSAGE_SIZE(16MB) to prevent OOM attacks. - Uses self-signed certificates. This is acceptable for the current stage, but the documentation correctly advises using proper CAs or pinning for production.
- Enforces
node/src/service.rs
The integration logic in mining_loop is robust.
- Verification: Crucially,
worker_handle.verify_seal(&seal)is called before submitting the block. This prevents invalid work from clogging the import queue. - Stale Work: The logic handles stale results gracefully by checking
job_id. - Cancellation: The
cancellation_tokenis checked in the wait loop, ensuring the node shuts down cleanly.
miner-api/src/lib.rs
- The data structures are well-defined using
serde. MiningResultincludeselapsed_timeandhash_count, which are useful for metrics.
Suggestions
- Update Documentation: Add the
Readymessage to the protocol specification. - Metrics: Consider exposing the number of connected miners as a Prometheus metric (currently only logged).
- Configuration: The
miner_listen_portis optional. Ensure the default behavior (when not set) is clearly communicated (i.e., external mining disabled).
Conclusion
This is a high-quality PR that significantly improves the mining infrastructure. The code is clean, safe, and follows Rust idioms.
Verdict: ✅ Approve (with minor documentation update)
|
Took suggestions except for adding miner count as it requires an additional internal server. |
* feat: qp-header for Planck release (#338) * no circuit padding hasher for block header * *use custom hasher for header that encodes the pre-image in a felt aligned manner. * *bespoke header hasher * *patch bug with hash header fall back * *replace custom poseidon header hasher on generic header with a fork of header that has a custom hasher that overrides default on the header trait. * *rmv commented out impl of prior hash method * Update primitives/header/src/lib.rs Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com> * fixed tests * Use inherent struct method * Update Cargo.toml --------- Co-authored-by: Ethan <tylercemer@gmail.com> Co-authored-by: illuzen <illuzen@users.noreply.github.com> Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com> * burn high-security fee instead of sending to treasury (#357) * Continuous Mining (#358) * new block trigger * dedupe logic, remove unnecessary field * simplify again * fmt * clippy * fmt * feat: Vesting and MerkleAirdrop removed (#360) * feat: Merkle Airdrop - removed * feat: Vesting pallet - removed * fix: Clippy for header * Enable wormhole addresses in genesis (#359) * generate transfer proofs for genesis endowment * fmt * fix balances tests * fmt * add recover funds call (#361) * add recover funds call * add unit tests * fix up remaining tests * cargo fmt * fix benchmarks, update weights * fix merge error * feat: Custom Mutisig Pallet (#352) * feat: Merkle Airdrop - removed * feat: Vesting pallet - removed * poc: First multisig version * fix: Taplo * fix: Execution for expired & address simplified fallback * draft: Historical proposals - paginaged endpoint * draft: Historical proposals - from events only * ref: Events renamed + Deposits logic simplified * feat: GracePeriod param removed * fix: Reentrancy * feat: History cleaning redesigned * fix: Expiry - additional validation * feat: Proposal nonce * feat: Dynamic weights * feat: Multisig deposit fee * feat: MaxExpiry param * feat: Fees to Treasury * feat: History removable only by signers * fix: Weights * feat: Fees burned * feat: Filibuster protection * feat: Proposals auto cleaning * feat: Proposal id - nonce instead of hash * feat: Calls - production whitelist * feat: Remove call whitelisting * fix: Test fix after balances pallet update * fix: Review cleaning * fix: Multisig - auto-cleaning expanded (#364) * fix: Multisig - auto-cleaning expanded * fix: Weights related to storage size * QUIC miner (#363) * quic implementation * refactor for readability * simplify * miner initiates, multiple miners supported * simplify loop further * short job counters * simplify new_full * gracefully handle invalid seals * remove unused and log misbehaving miners * emoji * fmt * taplo * improve readability, logs, documentation * feat: High-Security Integration for Multisig Pallet (#368) * feat: Multisig + HS integrated * feat: Weights update * fix: Benchmarks + README * feat: HS trait defined in primitives * fix: Taplo * fix: Sell order tracking * feat: Deterministic address + simplified cleaning * fix: Deposits for multisig * fix: Dynamic weight + benchmarks refactor * feat: Execute - separated * feat: Multisig - benchamarks refactor * fix: Benchmarks - corrected HS multisig cost * feat: Dynamic cleaning methods * feat: Approve dissolve - two variants * feat: Approval with negative weight * Switch to new plonky2 (#367) * No pad hasher header (#327) * no circuit padding hasher for block header * *use custom hasher for header that encodes the pre-image in a felt aligned manner. * *bespoke header hasher * *patch bug with hash header fall back * *replace custom poseidon header hasher on generic header with a fork of header that has a custom hasher that overrides default on the header trait. * *rmv commented out impl of prior hash method * Update primitives/header/src/lib.rs Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com> * fixed tests * Use inherent struct method * Update Cargo.toml --------- Co-authored-by: Ethan <tylercemer@gmail.com> Co-authored-by: illuzen <illuzen@users.noreply.github.com> * Verify header in the wormhole proof (#295) * Use canonical balances pallet and add support for assets in wormhole (#333) * Use canonical balances pallet, add assets support to wormhole * Ignore old tests * Remove tests * Override native asset id * Use poseidon hasher * Use poseidon storage hasher * Passing wormhole proof tests * Update binaries * Update binaries * Update zk-circuits crates * Use crates.io dep versions * Use `ToFelts` trait in the wormhole pallet (#347) * Apply ToFelts changes to wormhole * Fix checks * Passing tests * Revert unit test line * Rename explicit AccountId * Add ValidateUnsigned impl to wormhole (#353) * feat: aggregated proof verification in wormhole (#351) * Aggregated proofs verification wormhole * clippy * check block hash in agg proof * feat: quantized funding amounts (#354) * feat/quantized_wormhole_funding_amount * *fix formatting * *rollback zk enabled circuit artfiact builds at runtime. * fmt --------- Co-authored-by: illuzen <illuzen@users.noreply.github.com> * Enforce miner wormhole address (#344) * feat: qp-header for Planck release (#338) * no circuit padding hasher for block header * *use custom hasher for header that encodes the pre-image in a felt aligned manner. * *bespoke header hasher * *patch bug with hash header fall back * *replace custom poseidon header hasher on generic header with a fork of header that has a custom hasher that overrides default on the header trait. * *rmv commented out impl of prior hash method * Update primitives/header/src/lib.rs Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com> * fixed tests * Use inherent struct method * Update Cargo.toml --------- Co-authored-by: Ethan <tylercemer@gmail.com> Co-authored-by: illuzen <illuzen@users.noreply.github.com> Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com> * Exponentially decaying token rewards (#340) * exponentially decaying token rewards * script to simulate emissions * clean up constants and switch python script to rust test * log if we hit max supply somehow * convert rewards_address to rewards_preimage to enforce wormhole address usage * better documentation * change arg name * Exponentially decaying token rewards (#340) * exponentially decaying token rewards * script to simulate emissions * clean up constants and switch python script to rust test * log if we hit max supply somehow * convert rewards_address to rewards_preimage to enforce wormhole address usage * better documentation * change arg name * address style comments --------- Co-authored-by: Cezary Olborski <cezary.olborski@gmail.com> Co-authored-by: Ethan <tylercemer@gmail.com> Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com> * update qp-poseidon version * made transfer count per-recipient * feat: enable wormhole verifier tests (#356) * bring back wormhole transfer proof generation tests * fmt --------- Co-authored-by: illuzen <illuzen@users.noreply.github.com> * remove painful test, we sent it to quantus-cli * burn half the volume fee * fmt * use new plonky2-verifier crate * fix: Remove no_random feature and patch plonky2 crates to use local versions - Remove no_random feature from qp-wormhole-verifier and qp-zk-circuits-common dependencies - Add patches to use local qp-plonky2 and qp-plonky2-field with fixed rand feature handling - These changes ensure consistent feature resolution across all workspace members * lock * new agg logic * better logging of agg proof failure modes * refresh bins * only one proof verified event necessary * update to latest rusty crystals * no minimum, no single proof verification * lock * handle new derivation rules * fmt * put wormhole transfer minimum back in, remove unused single-proof files * better lock * remove local plonky2 references * fix build.rs bin validation * remove unused functions * format * no more local deps * missing dev accounts * clippy --------- Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com> Co-authored-by: Ethan <tylercemer@gmail.com> Co-authored-by: Cezary Olborski <cezary.olborski@gmail.com> * generate TransferProofs on batch transfer * clippy + zk versions * feat: Treasury config pallet (#372) * feat: Treasury config pallet * fix: Taplo --------- Co-authored-by: Ethan <tylercemer@gmail.com> Co-authored-by: illuzen <illuzen@users.noreply.github.com> Co-authored-by: Dastan <88332432+dastansam@users.noreply.github.com> Co-authored-by: Nikolaus Heger <nheger@gmail.com>