eth/consensus : implement eccpow consensus engine#10
Open
mmingyeomm wants to merge 3199 commits intocryptoecc:worldlandfrom
Open
eth/consensus : implement eccpow consensus engine#10mmingyeomm wants to merge 3199 commits intocryptoecc:worldlandfrom
mmingyeomm wants to merge 3199 commits intocryptoecc:worldlandfrom
Conversation
…on (#33301) This change fixes a stall in the legacy blob sidecar conversion pipeline where tasks that arrived during an active batch could remain unprocessed indefinitely after that batch completed, unless a new external event arrived. The root cause was that the loop did not restart processing in the case <-done: branch even when txTasks had accumulated work, relying instead on a future event to retrigger the scheduler. This behavior is inconsistent with the billy task pipeline, which immediately chains to the next task via runNextBillyTask() without requiring an external trigger. The fix adds a symmetric restart path in `case <-done`: that checks `len(txTasks) > 0`, clones the accumulated tasks, clears the queue, and launches a new run with a fresh done and interrupt. This preserves batching semantics, prevents indefinite blocking of callers of convert(), and remains safe during shutdown since the quit path still interrupts and awaits the active batch. No public interfaces or logging were changed.
… to 0.18.1 in /cmd/keeper (#33256) Bumps [github.com/consensys/gnark-crypto](https://github.com/consensys/gnark-crypto) from 0.18.0 to 0.18.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/consensys/gnark-crypto/releases">github.com/consensys/gnark-crypto's releases</a>.</em></p> <blockquote> <h2>v0.18.1</h2> <p><strong>Full Changelog</strong>: <a href="https://github.com/Consensys/gnark-crypto/compare/v0.18.0...v0.18.1">https://github.com/Consensys/gnark-crypto/compare/v0.18.0...v0.18.1</a></p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/Consensys/gnark-crypto/blob/master/CHANGELOG.md">github.com/consensys/gnark-crypto's changelog</a>.</em></p> <blockquote> <h2>[v0.18.1] - 2025-10-28</h2> <h3>Docs</h3> <ul> <li>add CHANGELOG for 0.18.1</li> </ul> <h3>Perf</h3> <ul> <li>limit memory allocation during Vector deserialization (<a href="https://redirect.github.com/Consensys/gnark-crypto/issues/759">#759</a>)</li> </ul> <p><!-- raw HTML omitted --><!-- raw HTML omitted --></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/Consensys/gnark-crypto/commit/fb04e95c3b2fca34f2cc773d3e8c1f930ede271a"><code>fb04e95</code></a> docs: add CHANGELOG for 0.18.1</li> <li><a href="https://github.com/Consensys/gnark-crypto/commit/0a4d04ae621bc19b6b40c6c7b8fa91f08b434533"><code>0a4d04a</code></a> perf: limit memory allocation during Vector deserialization (<a href="https://redirect.github.com/consensys/gnark-crypto/issues/759">#759</a>)</li> <li>See full diff in <a href="https://github.com/consensys/gnark-crypto/compare/v0.18.0...v0.18.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/ethereum/go-ethereum/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Fix for #33060 which I can't directly fix in the branch.
Fixes error messages to print the actual blob gas value instead of the pointer address by dereferencing `ExcessBlobGas`, `BlobGasUsed` and `ParentBeaconRoot`.
No matter what value of P2P.DiscoveryV4 or DiscoveryV5 is set in config file, it will be overwritten by the CLI flag, even if the flag is not set. This fixes it to apply the flag only if set.
Enable blocktest to read filenames from stdin when no path argument is provided, matching the existing statetest behavior. This allows efficient batch processing of blockchain tests. Usage: - Single file: evm blocktest <path> - Batch mode: find tests/ -name "*.json" | evm blocktest --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de> Co-authored-by: Felix Lange <fjl@twurst.com>
…3282) This was found because other clients are failing RPC tests generated by Geth. Nethermind and Besu return the correct error code, -32602, in this situation.
This adds checks into getPayload to ensure the correct version is called for the fork which applies to the payload. --------- Co-authored-by: jsvisa <delweng@gmail.com>
Fixes a test/lint regression introduced by #32754
Recently in #31630 we removed support for overriding the network id in preset networks. While this feature is niche, it is useful for shadow forks. This PR proposes we add the functionality back, but in a simpler way. Instead of checking whether the flag is set in each branch of the network switch statement, simply apply the network flag after the switch statement is complete. This retains the following behavior: 1. Auto network id based on chain id still works, because `IsSet` only returns true if the flag is _actually_ set. Not if it just has a default set. 2. The preset networks will set their network id directly and only if the network id flag is set is it overridden. This, combined with the override genesis flag is what allows the shadow forks. 3. Setting the network id to the same network id that the preset _would have_ set causes no issues and simply emits the `WARN` that the flag is being set explicitly. I don't think people explicitly set the network id flag often. ``` WARN [10-22|09:36:15.052] Setting network id with flag id=10 ``` --------- Co-authored-by: Felix Lange <fjl@twurst.com>
We still default to legacy txes for methods like eth_sendTransaction, eth_signTransaction. We can default to 0x2 and if someone would like to stay on legacy they can do so by setting the `gasPrice` field. cc @deffrian
This PR introduces a new debug feature, logging the slow blocks with detailed performance statistics, such as state read, EVM execution and so on. Notably, the detailed performance statistics of slow blocks won't be logged during the sync to not overwhelm users. Specifically, the statistics are only logged if there is a single block processed. Example output ``` ########## SLOW BLOCK ######### Block: 23537063 (0xa7f878611c2dd27f245fc41107d12ebcf06b4e289f1d6acf44d49a169554ee09) txs: 248, mgasps: 202.99 EVM execution: 63.295ms Validation: 1.130ms Account read: 6.634ms(648) Storage read: 17.391ms(1434) State hash: 6.722ms DB commit: 3.260ms Block write: 1.954ms Total: 99.094ms State read cache: account (hit: 622, miss: 26), storage (hit: 1325, miss: 109) ############################## ```
…33338) geth --nodiscover=false may result in ctx.IsSet(NoDiscoverFlag.Name) is true, but cfg. NoDiscovery should be false, not true.
This introduces two main changes to Pebble's configuration: (a) Remove the Bloom filter at Level 6 The Bloom filter is never used at the bottom-most level, so keeping it serves no purpose. Removing it saves storage without affecting read performance. (b) Re-enable read-sampling compaction Read-sampling compaction was previously disabled in the hash-based scheme because all data was identified by hashes and basically no data overwrite. Read sampling compaction makes no sense. After switching to the path-based scheme, data overwrites are much more common, making read-sampling compaction beneficial and reasonable to re-enable.
This PR exposes the state size statistics to the metrics, making them easier to demonstrate. Note that the contract code included in the metrics is not de-duplicated, so the reported size will appear larger than the actual storage footprint.
This is to benchmark how much the internal parts of GetBlobsV2 take. This is not an RPC-level benchmark, so JSON-RPC overhead is not included. Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Should fix decoding JSON blocks in the Fulu fork. This diff was missing from #33349.
This improves the error code for cases where invalid query parameters are submitted to `eth_getLogs`. I also improved the error message that is emitted when querying into the future.
…33280) The original condition `deleted && !logPrinted || time.Since(...)` was incorrectly grouping due to operator precedence, causing logs to print every 10 seconds even when no deletion was happening (deleted=false). According to SafeDeleteRange documentation, the 'deleted' parameter is "true if entries have actually been deleted already". The logging should only happen when deletion is active. Fixed by adding parentheses: `deleted && (!logPrinted || time.Since(...))`Now logs print only when items are being deleted AND either it's the first log or 10+ seconds have passed since the last one.
This PR changes the Pebble configurations as below: - increase the MemTableStopWritesThreshold for handling temporary spike - decrease the L0CompactionConcurrency and CompactionDebtConcurrency to scale up compaction readily
The random-port retry loop in addAnyPortMapping shadowed the err variable, causing the function to return (0, nil) when all attempts failed. This change removes the shadowing and preserves the last error across both the fixed-port and random-port retries, ensuring failures are reported to callers correctly.
…33331) XORBytes was added to package crypto/subtle in Go 1.20, and it's faster than our bitutil.XORBytes. There is only one use of this function across go-ethereum so we can simply deprecate the custom implementation. --------- Co-authored-by: Felix Lange <fjl@twurst.com>
The EIP says to increment PC by 2 _instead of_ the standard increment by 1. The opcode handlers added in #33095 result in incrementing PC by 3, because they ignored the increment already present in `interpreter.go`. Does this need to be better specified in the EIP? I've added a [new test case](ethereum/EIPs#10859) for it anyway. Found by @0xriptide.
This moves the tracking of the current syncmode into the downloader, fixing an issue where the syncmode being requested through the engine API could go out-of-sync with the actual mode being performed by downloader. Fixes #32629 --------- Co-authored-by: Felix Lange <fjl@twurst.com>
Fixes an issue where HashFolder skipped the root directory upon hitting the first file in the excludes list. This happened because the walk function returned SkipDir even for regular files.
Return the Amsterdam instruction set from `LookupInstructionSet` when `IsAmsterdam` is true, so Amsterdam rules no longer fall through to the Osaka jump table. --------- Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
For bal-devnet-3 we need to update the EIP-8024 implementation to the latest spec changes: ethereum/EIPs#11306 > Note: I deleted tests not specified in the EIP bc maintaining them through EIP changes is too error prone.
Pebble maintains a batch pool to recycle the batch object. Unfortunately batch object must be explicitly returned via `batch.Close` function. This PR extends the batch interface by adding the close function and also invoke batch.Close in some critical code paths. Memory allocation must be measured before merging this change. What's more, it's an open question that whether we should apply batch.Close as much as possible in every invocation.
Implements https://eips.ethereum.org/EIPS/eip-7778 --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Mainnet was already overriding --cache to 4096. This PR just makes this the default.
…33927) The BatchSpanProcessor queue size was incorrectly set to DefaultMaxExportBatchSize (512) instead of DefaultMaxQueueSize (2048). I noticed the issue on bloatnet when analyzing the block building traces. During a particular run, the miner was including 1000 transactions in a single block. When telemetry is enabled, the miner creates a span for each transaction added to the block. With the queue capped at 512, spans were silently dropped when production outpaced the span export, resulting in incomplete traces with orphaned spans. While this doesn't eliminate the possibility of drops under extreme load, using the correct default restores the 4x buffer between queue capacity and export batch size that the SDK was designed around.
Fixes a regression in #33593 where a block gas limit > gasCap resulted in more execution than the gas cap.
Eth currently has a flaky test, related to the tx fetcher. The issue seems to happen when Unsubscribe is called while sub is nil. It seems that chain.Stop() may be invoked before the loop starts in some tests, but the exact cause is still under investigation through repeated runs. I think this change will at least prevent the error.
The computation of `MAIN_STORAGE_OFFSET` was incorrect, causing the last byte of the stem to be dropped. This means that there would be a collision in the hash computation (at the preimage level, not a hash collision of course) if two keys were only differing at byte 31.
…le tree (#33961) This is an optimization that existed for verkle and the MPT, but that got dropped during the rebase. Mark the nodes that were modified as needing recomputation, and skip the hash computation if this is not needed. Otherwise, the whole tree is hashed, which kills performance.
`GenerateChain` commits trie nodes asynchronously, and it can happen that some nodes aren't making it to the db in time for `GenerateChain` to open it and find the data it is looking for.
Add nil checks to prevent potential panics when keystore backend is unavailable in the Clef signer API.
Reduce allocations in calculation of tx cost. --------- Co-authored-by: weixie.cui <weixie.cui@okg.com> Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
Updates go-eth-kzg to https://github.com/crate-crypto/go-eth-kzg/releases/tag/v1.5.0 Significantly reduces the allocations in VerifyCellProofBatch which is around ~5% of all allocations on my node --------- Co-authored-by: Guillaume Ballet <3272758+gballet@users.noreply.github.com>
I observed failing tests in Hive `engine-withdrawals`: - https://hive.ethpandaops.io/#/test/generic/1772351960-ad3e3e460605c670efe1b4f4178eb422?testnumber=146 - https://hive.ethpandaops.io/#/test/generic/1772351960-ad3e3e460605c670efe1b4f4178eb422?testnumber=147 ```shell DEBUG (Withdrawals Fork on Block 2): NextPayloadID before getPayloadV2: id=0x01487547e54e8abe version=1 >> engine_getPayloadV2("0x01487547e54e8abe") << error: {"code":-38005,"message":"Unsupported fork"} FAIL: Expected no error on EngineGetPayloadV2: error=Unsupported fork ``` The same failure pattern occurred for Block 3. Per Shanghai engine_getPayloadV2 spec, pre-Shanghai payloads should be accepted via V2 and returned as ExecutionPayloadV1: - executionPayload: ExecutionPayloadV1 | ExecutionPayloadV2 - ExecutionPayloadV1 MUST be returned if payload timestamp < Shanghai timestamp - ExecutionPayloadV2 MUST be returned if payload timestamp >= Shanghai timestamp Reference: - https://github.com/ethereum/execution-apis/blob/main/src/engine/shanghai.md#engine_getpayloadv2 Current implementation only allows GetPayloadV2 on the Shanghai fork window (`[]forks.Fork{forks.Shanghai}`), so pre-Shanghai payloads are rejected with Unsupported fork. If my interpretation of the spec is incorrect, please let me know and I can adjust accordingly. --------- Co-authored-by: muzry.li <muzry.li1@ambergroup.io>
This PR fixes a regression introduced in https://github.com/ethereum/go-ethereum/pull/33836/changes Before PR 33836, running mainnet would automatically bump the cache size to 4GB and trigger a cache re-calculation, specifically setting the key-value database cache to 2GB. After PR 33836, this logic was removed, and the cache value is no longer recomputed if no command line flags are specified. The default key-value database cache is 512MB. This PR bumps the default key-value database cache size alongside the default cache size for other components (such as snapshot) accordingly.
We got a report for a bug in the tracing journal which has the responsibility to emit events for all state that must be reverted. The edge case is as follows: on CREATE operations the nonce is incremented. When a create frame reverts, the nonce increment associated with it does **not** revert. This works fine on master. Now one step further: if the parent frame reverts tho, the nonce **should** revert and there is the bug.
This PR contains two changes: Firstly, the finalized header will be resolved from local chain if it's not recently announced via the `engine_newPayload`. What's more importantly is, in the downloader, originally there are two code paths to push forward the pivot point block, one in the beacon header fetcher (`fetchHeaders`), and another one is in the snap content processer (`processSnapSyncContent`). Usually if there are new blocks and local pivot block becomes stale, it will firstly be detected by the `fetchHeaders`. `processSnapSyncContent` is fully driven by the beacon headers and will only detect the stale pivot block after synchronizing the corresponding chain segment. I think the detection here is redundant and useless.
…3984) The `--remove.chain` flag incorrectly described itself as selecting "state data" for removal, which could mislead operators into removing the wrong data category. This corrects the description to accurately reflect that the flag targets chain data (block bodies and receipts).
This PR allows users to prune their nodes up to the Prague fork. It indirectly depends on #32157 and can't really be merged before eraE files are widely available for download. The `--history.chain` flag becomes mandatory for `prune-history` command. Here I've listed all the edge cases that can happen and how we behave: ## prune-history Behavior | From | To | Result | |-------------|--------------|--------------------------| | full | postmerge | ✅ prunes | | full | postprague | ✅ prunes | | postmerge | postprague | ✅ prunes further | | postprague | postmerge | ❌ can't unprune | | any | all | ❌ use import-history | ## Node Startup Behavior | DB State | Flag | Result | |-------------|--------------|----------------------------------------------------------------| | fresh | postprague | ✅ syncs from Prague | | full | postprague | ❌ "run prune-history first" | | postmerge | postprague | ❌ "run prune-history first" | | postprague | postmerge | ❌ "can't unprune, use import-history or fix flag" | | pruned | all | ✅ accepts known prune points |
This PR adds a cmd tool fetchpayload which connects to a node and gets all the information in order to create a serialized payload that can then be passed to the zkvm.
`TestSubscribePendingTxHashes` hangs indefinitely because pending tx events are permanently missed due to a race condition in `NewPendingTransactions` (and `NewHeads`). Both handlers called their event subscription functions (`SubscribePendingTxs`, `SubscribeNewHeads`) inside goroutines, so the RPC handler returned the subscription ID to the client before the filter was installed in the event loop. When the client then sent a transaction, the event fired but no filter existed to catch it — the event was silently lost. - Move `SubscribePendingTxs` and `SubscribeNewHeads` calls out of goroutines so filters are installed synchronously before the RPC response is sent, matching the pattern already used by `Logs` and `TransactionReceipts` <!-- START COPILOT CODING AGENT TIPS --> --- 💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey). --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: s1na <1591639+s1na@users.noreply.github.com>
Implement EIP7954, This PR raises the maximum contract code size to 32KiB and initcode size to 64KiB , following https://eips.ethereum.org/EIPS/eip-7954 --------- Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
Binary tree hashing is quite slow, owing to many factors. One of them is the GC pressure that is the consequence of allocating many hashers, as a binary tree has 4x the size of an MPT. This PR introduces an optimization that already exists for the MPT: keep a pool of hashers, in order to reduce the amount of allocations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implements eccpow consensus engine for Worldland Network