diff --git a/.claude/skills/erigon-ci/SKILL.md b/.claude/skills/erigon-ci/SKILL.md index e570d9a7fdc..2835d4dfeda 100644 --- a/.claude/skills/erigon-ci/SKILL.md +++ b/.claude/skills/erigon-ci/SKILL.md @@ -22,7 +22,7 @@ Each test group has its own dedicated skill for drill-down on failures. Use thos | unit | `erigon-test-unit` | `make test-short` | ~5 min | Pre-push gate | | all | `erigon-test-all` | `GOGC=80 make test-all` | ~30 min | Before PR review | | race | `erigon-test-race` | `make test-all-race` | ~60 min | Concurrency changes | -| eest-spec | *(inline)* | `make eest-spec--[-{sequential,parallel}]` | varies | EEST state/blockchain/engine-x changes (most shards split into `-sequential` / `-parallel` pairs that pin `ERIGON_EXEC3_PARALLEL`; see `tools/eest-spec-shards.yml`) | +| eest-spec | *(inline)* | `make eest-spec-` | varies | EEST state/blockchain/engine-x changes (paired shards pin `ERIGON_COMMITMENT_PARALLEL`; execution remains parallel; see `tools/eest-spec-shards.yml`) | | caplin spec | *(inline)* | `cd cl/spectest && make tests && make mainnet` | ~15 min | CL/consensus changes | | hive | `erigon-test-hive` | `make test-hive` | ~20 min | EL/CL interop changes | | rpc | `erigon-test-rpc` | *(requires synced DB)* | ~10 min | RPC API changes | diff --git a/.claude/skills/erigon-implement-eip/SKILL.md b/.claude/skills/erigon-implement-eip/SKILL.md index cd5b3dc0900..9c257121b3d 100644 --- a/.claude/skills/erigon-implement-eip/SKILL.md +++ b/.claude/skills/erigon-implement-eip/SKILL.md @@ -115,16 +115,13 @@ Run local tests using the `/erigon-test-all` skill. Analyse and fix any failures The most important tests when implementing a new EIP for the EL are the EEST spec test shards, exercised by the `cmd/evm` runners (`statetest`, `blocktest`, `enginextest`) via the Makefile targets: -- `make eest-spec-statetests-stable` / `…-devnet` — state-tests against the stable/devnet EEST fixtures -- `make eest-spec-blocktests-stable-sequential` / `…-devnet` — blockchain-tests against the stable/devnet EEST fixtures. The devnet shard always runs under `ERIGON_EXEC3_PARALLEL=true` (the in-development hardfork requires it); the `…-sequential` shard pins `ERIGON_EXEC3_PARALLEL=false`. -- `make eest-spec-blocktests-stable-parallel` — same fixtures as `…-stable-sequential` but with `ERIGON_EXEC3_PARALLEL=true`; useful for catching parallel-only regressions on stable fixtures. -- `make eest-spec-enginextests-stable-sequential` — engine-x tests against the stable EEST fixtures with `ERIGON_EXEC3_PARALLEL=false`. -- `make eest-spec-enginextests-stable-parallel` — same fixtures as `…-stable-sequential` but with `ERIGON_EXEC3_PARALLEL=true`; useful for catching parallel-only regressions on engine-x stable fixtures. -- `make eest-spec-enginextests-devnet` — engine-x tests against the devnet EEST fixtures with `ERIGON_EXEC3_PARALLEL=true`. -- `make eest-spec-enginextests-benchmark-{1m,5m,10m,30m,60m,100m,150m}-{sequential,parallel}` — engine-x tests against the per-gas-target benchmark fixtures, with `--time` per-test stats. Each gas target has a `-sequential` (`ERIGON_EXEC3_PARALLEL=false`) and `-parallel` (`ERIGON_EXEC3_PARALLEL=true`) variant. -- `make eest-spec-blocktests-stable-race-{pre-cancun,cancun,prague,osaka}-{sequential,parallel}` and `make eest-spec-blocktests-devnet-race-amsterdam` — race-detector variants split by fork. Each stable-race sub-shard has a `-sequential` / `-parallel` pair; the `-parallel` siblings exercise parallel exec3 under the race detector. The `blocktests-devnet-race-amsterdam` shard is always parallel (matches the non-race devnet behaviour). - -The shard list / failure budgets / `exec3-parallel` flags are defined in `tools/eest-spec-shards.yml` (single source of truth shared with the CI workflow and the local runner script). See `EEST_SPEC_SHARDS` / `EEST_SPEC_RACE_SHARDS` in the root `Makefile` for the partition into non-race vs race targets. +- `make eest-spec-statetests-{stable,devnet}-{sequential,parallel}` — state tests against the stable/devnet EEST fixtures in both commitment modes. +- `make eest-spec-blocktests-{stable,devnet}-{sequential,parallel}` — blockchain tests against the stable/devnet EEST fixtures in both commitment modes. +- `make eest-spec-enginextests-{stable,devnet}-{sequential,parallel}` — engine-x tests against the stable/devnet EEST fixtures in both commitment modes. +- `make eest-spec-enginextests-benchmark-{1m,5m,10m,30m,60m,100m,150m}-{sequential,parallel}` — engine-x tests against the per-gas-target benchmark fixtures, with `--time` per-test stats and both commitment modes. +- `make eest-spec-blocktests-stable-race-{pre-cancun,cancun,prague,osaka}-{sequential,parallel}` and `make eest-spec-blocktests-devnet-race-amsterdam-{sequential,parallel}` — race-detector variants split by fork and commitment mode. + +The shard list, failure budgets, and `commitment-parallel` flags are defined in `tools/eest-spec-shards.yml` (single source of truth shared with the CI workflow and the local runner script). Execution remains parallel in every paired shard. See `EEST_SPEC_SHARDS` / `EEST_SPEC_RACE_SHARDS` in the root `Makefile` for the partition into non-race vs race targets. **Pitfall: stale `evm` / `evm.race` binary.** When iterating on an EIP implementation, always invoke shards via `make eest-spec-` rather than `bash tools/run-eest-spec-test.sh ` — the make target lists `evm` (or `evm.race`) as a prereq and `go build` is cache-aware, so a fresh binary is built before each run. The script invoked directly **bypasses** that rebuild, so the runners exercise whatever `build/bin/evm{,.race}` happens to be on disk against current fixtures — silently inflating failures (e.g. devnet shards "regressing" by thousands of tests) or hiding regressions when comparing budgets before/after a change. diff --git a/.claude/skills/erigon-test-all/SKILL.md b/.claude/skills/erigon-test-all/SKILL.md index 00ec2c66e84..ed11436457d 100644 --- a/.claude/skills/erigon-test-all/SKILL.md +++ b/.claude/skills/erigon-test-all/SKILL.md @@ -14,32 +14,40 @@ Runs the complete test suite with 60-minute timeout and coverage output. Takes ~ To exercise the EEST suites locally, see `erigon-eest-spec` (or run a specific shard directly): ```bash -make eest-spec-statetests-stable # state tests vs eest_stable fixtures -make eest-spec-blocktests-stable-sequential # blockchain tests vs eest_stable fixtures (ERIGON_EXEC3_PARALLEL=false) -make eest-spec-blocktests-stable-parallel # same, but with ERIGON_EXEC3_PARALLEL=true -make eest-spec-enginextests-stable-sequential # engine-x tests vs eest_stable (ERIGON_EXEC3_PARALLEL=false) -make eest-spec-enginextests-stable-parallel # same, but with ERIGON_EXEC3_PARALLEL=true -make eest-spec-statetests-devnet # …vs eest_devnet fixtures -make eest-spec-blocktests-devnet # devnet blocktests (always parallel exec3) -make eest-spec-enginextests-devnet # devnet engine-x tests (always parallel exec3) -make eest-spec-statetests-legacy # pinned legacy Cancun state-test archive +make eest-spec-statetests-stable-{sequential,parallel} + # state tests vs eest_stable fixtures +make eest-spec-blocktests-stable-sequential # blockchain tests vs eest_stable fixtures (serial commitment) +make eest-spec-blocktests-stable-parallel # same, but with parallel commitment +make eest-spec-enginextests-stable-sequential # engine-x tests vs eest_stable (serial commitment) +make eest-spec-enginextests-stable-parallel # same, but with parallel commitment +make eest-spec-enginextests-devnet-{sequential,parallel} + # devnet engine-x tests in both commitment modes +make eest-spec-statetests-devnet-{sequential,parallel} + # state tests vs eest_devnet fixtures +make eest-spec-blocktests-devnet-{sequential,parallel} + # devnet blocktests in both commitment modes +make eest-spec-statetests-legacy-{sequential,parallel} + # pinned legacy Cancun state-test archive make eest-spec-rlptests-legacy-race # complete pinned legacy RLP suite make eest-spec-transactiontests-legacy-race # complete pinned legacy transaction suite make eest-spec-difficultytests-legacy-race # complete pinned legacy difficulty suite make eest-spec-blocktests-legacy-consensus-sequential # Hive consensus fixture selection; - # -parallel and -race variants too + # -parallel and + # -race-{sequential,parallel} variants too make eest-spec-blocktests-legacy-constantinople-sequential # Hive legacy fixture selection; # -parallel plus three race partitions: # ...-race-constantinople, # ...-race-constantinople-fix, and - # ...-race-other-forks + # ...-race-other-forks; each race + # partition has sequential/parallel variants make eest-spec-blocktests-legacy-cancun-sequential # Hive legacy-cancun selection; # -parallel plus six race partitions: # ...-race-{berlin,shanghai,cancun, - # london,paris,other-forks} + # london,paris,other-forks}; each race + # partition has sequential/parallel variants make eest-spec-enginextests-benchmark-1m-sequential # engine-x benchmark fixtures @ 1M gas target # (with per-test --time stats); @@ -48,13 +56,14 @@ make eest-spec-enginextests-benchmark-1m-sequential make eest-spec-blocktests-stable-race-cancun-sequential # race-detector variant, sharded per fork: # -pre-cancun/-cancun/-prague/-osaka, plus - # eest-spec-blocktests-devnet-race-amsterdam. - # Each stable-race sub-shard has a + # eest-spec-blocktests-devnet-race-amsterdam- + # {sequential,parallel}. Each stable-race + # and devnet-race sub-shard has a # "-sequential" / "-parallel" pair # (e.g. ...-race-cancun-{sequential,parallel}) ``` -The shard list / failure budgets / `exec3-parallel` flags live in `tools/eest-spec-shards.yml` (single source of truth for both this workflow and `tools/run-eest-spec-test.sh`). See `EEST_SPEC_SHARDS` / `EEST_SPEC_RACE_SHARDS` in the root `Makefile` for the partition into race vs non-race targets. +The shard list / failure budgets / `commitment-parallel` flags live in `tools/eest-spec-shards.yml` (single source of truth for both this workflow and `tools/run-eest-spec-test.sh`). See `EEST_SPEC_SHARDS` / `EEST_SPEC_RACE_SHARDS` in the root `Makefile` for the partition into race vs non-race targets. **Pitfall: stale `evm` / `evm.race` binary.** Always invoke shards via `make eest-spec-` — the Makefile lists `evm` (or `evm.race`) as a prereq and `go build` is cache-aware, so a stale binary gets rebuilt automatically. Calling `bash tools/run-eest-spec-test.sh ` directly **bypasses** the rebuild and silently exercises whatever `build/bin/evm{,.race}` happens to be on disk against current fixtures, inflating failures or hiding regressions. After pulling code, switching branches, or any time you suspect the binary is older than HEAD: `rm -f build/bin/evm build/bin/evm.race && make evm evm.race` before re-running. diff --git a/.claude/skills/erigon-test-race/SKILL.md b/.claude/skills/erigon-test-race/SKILL.md index 44ad24f492f..5acf47a2d21 100644 --- a/.claude/skills/erigon-test-race/SKILL.md +++ b/.claude/skills/erigon-test-race/SKILL.md @@ -16,13 +16,13 @@ Use the dedicated EEST race shards: ```bash make eest-spec-{rlptests,transactiontests,difficultytests}-legacy-race make eest-spec-blocktests-stable-race-{pre-cancun,cancun,prague,osaka}-{sequential,parallel} -make eest-spec-blocktests-devnet-race-amsterdam -make eest-spec-blocktests-legacy-consensus-race -make eest-spec-blocktests-legacy-constantinople-race-{constantinople,constantinople-fix,other-forks} -make eest-spec-blocktests-legacy-cancun-race-{berlin,shanghai,cancun,london,paris,other-forks} +make eest-spec-blocktests-devnet-race-amsterdam-{sequential,parallel} +make eest-spec-blocktests-legacy-consensus-race-{sequential,parallel} +make eest-spec-blocktests-legacy-constantinople-race-{constantinople,constantinople-fix,other-forks}-{sequential,parallel} +make eest-spec-blocktests-legacy-cancun-race-{berlin,shanghai,cancun,london,paris,other-forks}-{sequential,parallel} ``` -These targets build a race-instrumented `evm.race` binary automatically (see `EEST_SPEC_RACE_SHARDS` in the root `Makefile`). The stable `-sequential` / `-parallel` pairs pin both execution modes; the devnet and legacy race shards pin parallel execution to match their fixture topology. For the consensus spec suite or other Go packages, pass `GOFLAGS='-race'` or invoke `go test -race` against the relevant package directly. +These targets build a race-instrumented `evm.race` binary automatically (see `EEST_SPEC_RACE_SHARDS` in the root `Makefile`). The stable, devnet, and legacy blocktest `-sequential` / `-parallel` pairs pin both commitment modes; execution remains parallel in every pair. For the consensus spec suite or other Go packages, pass `GOFLAGS='-race'` or invoke `go test -race` against the relevant package directly. **Pitfall: stale `evm.race` binary.** `make eest-spec-` lists `evm.race` as a prereq and `go build` is cache-aware, so a stale binary gets rebuilt. Calling `bash tools/run-eest-spec-test.sh ` directly with `EVM_BIN=build/bin/evm.race` **bypasses** the rebuild and silently runs an old race-instrumented binary against current fixtures — race reports against code that no longer exists, missed races against code that does. After pulling or switching branches: `rm -f build/bin/evm.race && make evm.race` before re-running. diff --git a/.github/workflows/qa-rpc-integration-tests-latest.yml b/.github/workflows/qa-rpc-integration-tests-latest.yml index 1a8f7f8b352..af93eca6764 100644 --- a/.github/workflows/qa-rpc-integration-tests-latest.yml +++ b/.github/workflows/qa-rpc-integration-tests-latest.yml @@ -26,7 +26,7 @@ permissions: jobs: mainnet-rpc-integ-tests-latest: - name: mainnet-rpc-integ-tests-latest (${{ matrix.exec_mode }}) + name: mainnet-rpc-integ-tests-latest (${{ matrix.commitment_mode }}) concurrency: group: >- ${{ @@ -45,7 +45,7 @@ jobs: # just one after the other (~2× wall-clock). max-parallel: 1 matrix: - exec_mode: + commitment_mode: - serial - parallel env: @@ -56,10 +56,10 @@ jobs: ERIGON_ASSERT: true RPC_PAST_TEST_DIR: /opt/rpc-past-tests CHAIN: mainnet - # Toggle dbg.Exec3Parallel from CI without code changes. envLookup - # in common/dbg/dbg_env.go auto-prepends ERIGON_, so this maps to - # the EXEC3_PARALLEL flag declared in common/dbg/experiments.go. - ERIGON_EXEC3_PARALLEL: ${{ matrix.exec_mode == 'parallel' && 'true' || 'false' }} + # Toggle statecfg.ExperimentalParallelCommitment from CI without code + # changes. envLookup in common/dbg/dbg_env.go auto-prepends ERIGON_, so + # this maps to the COMMITMENT_PARALLEL flag read in db/state/statecfg. + ERIGON_COMMITMENT_PARALLEL: ${{ matrix.commitment_mode == 'parallel' && 'true' || 'false' }} steps: - name: Check out repository @@ -128,7 +128,7 @@ jobs: if: failure() && steps.preparing_step.outcome == 'failure' uses: actions/upload-artifact@v7 with: - name: preparing-step-logs-${{ matrix.exec_mode }} + name: preparing-step-logs-${{ matrix.commitment_mode }} path: ${{ env.ERIGON_REFERENCE_DATA_DIR }}/logs/ - name: Pause the Erigon instance dedicated to db maintenance @@ -185,7 +185,7 @@ jobs: if: failure() && steps.pre_test_step.outcome == 'failure' uses: actions/upload-artifact@v7 with: - name: pre-test-logs-${{ matrix.exec_mode }} + name: pre-test-logs-${{ matrix.commitment_mode }} path: ${{ env.ERIGON_TESTBED_DATA_DIR }}/logs/ - name: Run RPC Integration Tests @@ -257,7 +257,7 @@ jobs: if: always() && steps.test_step.outputs.test_executed == 'true' uses: actions/upload-artifact@v7 with: - name: test-results-${{ matrix.exec_mode }} + name: test-results-${{ matrix.commitment_mode }} path: | ${{ env.TEST_RESULT_DIR }} ${{ env.ERIGON_TESTBED_DATA_DIR }}/logs/ @@ -277,7 +277,7 @@ jobs: --repo erigon \ --commit $(git rev-parse HEAD) \ --branch "$GITHUB_REF_NAME" \ - --test_name rpc-integration-tests-latest${{ matrix.exec_mode == 'parallel' && '-parallel' || '' }} \ + --test_name rpc-integration-tests-latest-commitment-${{ matrix.commitment_mode }} \ --chain $CHAIN \ --runner "$RUNNER_NAME" \ --db_version $db_version \ diff --git a/.github/workflows/qa-rpc-performance-comparison-tests.yml b/.github/workflows/qa-rpc-performance-comparison-tests.yml index 85c2a309337..957b7d9ac23 100644 --- a/.github/workflows/qa-rpc-performance-comparison-tests.yml +++ b/.github/workflows/qa-rpc-performance-comparison-tests.yml @@ -8,8 +8,8 @@ on: type: boolean required: false default: false - exec_mode: - description: 'Erigon execution mode ("default" leaves ERIGON_EXEC3_PARALLEL unset)' + commitment_mode: + description: 'Erigon commitment mode ("default" leaves ERIGON_COMMITMENT_PARALLEL unset)' type: choice required: false default: default @@ -203,22 +203,22 @@ jobs: id: erigon_running_step working-directory: ${{ github.workspace }}/build/bin env: - EXEC_MODE: ${{ github.event.inputs.exec_mode }} + COMMITMENT_MODE: ${{ github.event.inputs.commitment_mode }} run: | set +e # Disable exit on error echo "Starting Erigon..." - # Only set ERIGON_EXEC3_PARALLEL on an explicit choice; otherwise ensure it's + # Only set ERIGON_COMMITMENT_PARALLEL on an explicit choice; otherwise ensure it's # unset so erigon uses its built-in default (and doesn't inherit runner env). - if [ "$EXEC_MODE" = "parallel" ]; then - export ERIGON_EXEC3_PARALLEL=true - echo "Set ERIGON_EXEC3_PARALLEL=true (parallel mode)" - elif [ "$EXEC_MODE" = "serial" ]; then - export ERIGON_EXEC3_PARALLEL=false - echo "Set ERIGON_EXEC3_PARALLEL=false (serial mode)" + if [ "$COMMITMENT_MODE" = "parallel" ]; then + export ERIGON_COMMITMENT_PARALLEL=true + echo "Set ERIGON_COMMITMENT_PARALLEL=true (parallel mode)" + elif [ "$COMMITMENT_MODE" = "serial" ]; then + export ERIGON_COMMITMENT_PARALLEL=false + echo "Set ERIGON_COMMITMENT_PARALLEL=false (serial mode)" else - unset ERIGON_EXEC3_PARALLEL - echo "Leaving ERIGON_EXEC3_PARALLEL unset (default behavior)" + unset ERIGON_COMMITMENT_PARALLEL + echo "Leaving ERIGON_COMMITMENT_PARALLEL unset (default behavior)" fi ./erigon --prune.mode=minimal --datadir $ERIGON_TESTBED_DATA_DIR --http.api admin,debug,eth,parity,erigon,trace,web3,txpool,ots,net --ws > erigon.log 2>&1 & @@ -389,7 +389,7 @@ jobs: --repo $client \ --branch $branch_name \ --commit $commit_hash \ - --test_name rpc-performance-test-latest${{ (matrix.client == 'erigon' && github.event.inputs.exec_mode == 'parallel' && '-parallel') || (matrix.client == 'erigon' && github.event.inputs.exec_mode == 'serial' && '-serial') || '' }}-$method \ + --test_name rpc-performance-test-latest${{ (matrix.client == 'erigon' && github.event.inputs.commitment_mode == 'parallel' && '-commitment-parallel') || (matrix.client == 'erigon' && github.event.inputs.commitment_mode == 'serial' && '-commitment-serial') || '' }}-$method \ --chain $CHAIN \ --runner "$RUNNER_NAME" \ --db_version $db_version \ @@ -465,7 +465,7 @@ jobs: if: (matrix.client == 'erigon' || needs.setup.outputs.run_geth == 'true') && steps.test_step.outputs.test_executed == 'true' uses: actions/upload-artifact@v7 with: - name: test-results-${{ env.CHAIN }}-${{ matrix.client }}${{ (matrix.client == 'erigon' && github.event.inputs.exec_mode == 'parallel' && '-parallel') || (matrix.client == 'erigon' && github.event.inputs.exec_mode == 'serial' && '-serial') || '' }} + name: test-results-${{ env.CHAIN }}-${{ matrix.client }}${{ (matrix.client == 'erigon' && github.event.inputs.commitment_mode == 'parallel' && '-commitment-parallel') || (matrix.client == 'erigon' && github.event.inputs.commitment_mode == 'serial' && '-commitment-serial') || '' }} path: ${{ env.past_test_dir }} - name: Stop Erigon @@ -485,7 +485,7 @@ jobs: if: matrix.client == 'erigon' && steps.test_step.outputs.test_executed == 'true' uses: actions/upload-artifact@v7 with: - name: erigon-logs-${{ env.CHAIN }}${{ (github.event.inputs.exec_mode == 'parallel' && '-parallel') || (github.event.inputs.exec_mode == 'serial' && '-serial') || '' }} + name: erigon-logs-${{ env.CHAIN }}${{ (github.event.inputs.commitment_mode == 'parallel' && '-parallel') || (github.event.inputs.commitment_mode == 'serial' && '-serial') || '' }} path: ${{ github.workspace }}/build/bin/erigon.log - name: Delete Erigon Testbed Data Directory diff --git a/.github/workflows/qa-stage-exec.yml b/.github/workflows/qa-stage-exec.yml index 382d986698f..0a2e6c71654 100644 --- a/.github/workflows/qa-stage-exec.yml +++ b/.github/workflows/qa-stage-exec.yml @@ -20,48 +20,48 @@ permissions: jobs: stage-exec-test: - name: stage-exec-test (${{ matrix.mode_name }}, ${{ matrix.exec_mode }}) + name: stage-exec-test (${{ matrix.mode_name }}, ${{ matrix.commitment_mode }}) runs-on: [self-hosted, qa, Ethereum, tip-tracking] strategy: fail-fast: false matrix: - # Each (mode_name, exec_mode) pair runs separately. The testbed + # Each (mode_name, commitment_mode) pair runs separately. The testbed # data dir already disambiguates by mode_name; we extend it with - # exec_mode so serial+parallel entries don't clobber each other. + # commitment_mode so serial+parallel entries don't clobber each other. include: - mode_name: resume-nonchaintip extra_flags: "" test_name: stage_exec_resume_nonchaintip - exec_mode: serial + commitment_mode: serial - mode_name: resume-nonchaintip extra_flags: "" test_name: stage_exec_resume_nonchaintip - exec_mode: parallel + commitment_mode: parallel - mode_name: from-0 extra_flags: "--rm-state-all" test_name: stage_exec_from_0 - exec_mode: serial + commitment_mode: serial - mode_name: from-0 extra_flags: "--rm-state-all" test_name: stage_exec_from_0 - exec_mode: parallel + commitment_mode: parallel - mode_name: chaintip extra_flags: "--chaintip" test_name: stage_exec_resume_chaintip - exec_mode: serial + commitment_mode: serial - mode_name: chaintip extra_flags: "--chaintip" test_name: stage_exec_resume_chaintip - exec_mode: parallel + commitment_mode: parallel env: - ERIGON_TESTBED_DATA_DIR: /opt/erigon-testbed/datadir-${{ matrix.mode_name }}-${{ matrix.exec_mode }} + ERIGON_TESTBED_DATA_DIR: /opt/erigon-testbed/datadir-${{ matrix.mode_name }}-${{ matrix.commitment_mode }} ERIGON_QA_PATH: /home/qarunner/erigon-qa TIMEOUT_SECONDS: 360 CHAIN: mainnet - # Toggle dbg.Exec3Parallel from CI without code changes. envLookup - # in common/dbg/dbg_env.go auto-prepends ERIGON_, so this maps to - # the EXEC3_PARALLEL flag declared in common/dbg/experiments.go. - ERIGON_EXEC3_PARALLEL: ${{ matrix.exec_mode == 'parallel' && 'true' || 'false' }} + # Toggle statecfg.ExperimentalParallelCommitment from CI without code + # changes. envLookup in common/dbg/dbg_env.go auto-prepends ERIGON_, so + # this maps to the COMMITMENT_PARALLEL flag read in db/state/statecfg. + ERIGON_COMMITMENT_PARALLEL: ${{ matrix.commitment_mode == 'parallel' && 'true' || 'false' }} steps: - name: Check out repository @@ -119,7 +119,7 @@ jobs: python3 $ERIGON_QA_PATH/test_system/qa-tests/stage-exec/run_and_check_stage_exec.py \ ${{ github.workspace }}/build/bin $ERIGON_TESTBED_DATA_DIR $TIMEOUT_SECONDS $CHAIN \ - --result-file ${{ github.workspace }}/result-$CHAIN-${{ matrix.mode_name }}-${{ matrix.exec_mode }}.json \ + --result-file ${{ github.workspace }}/result-$CHAIN-${{ matrix.mode_name }}-${{ matrix.commitment_mode }}.json \ ${{ matrix.extra_flags }} test_exit_status=$? @@ -138,9 +138,9 @@ jobs: if: ${{ always() && steps.test_step.outputs.test_executed == 'true' }} env: TEST_RESULT: ${{ steps.test_step.outputs.TEST_RESULT }} - TEST_NAME: ${{ matrix.test_name }} + TEST_NAME: ${{ matrix.test_name }}-commitment-${{ matrix.commitment_mode }} MODE_NAME: ${{ matrix.mode_name }} - EXEC_MODE: ${{ matrix.exec_mode }} + COMMITMENT_MODE: ${{ matrix.commitment_mode }} run: | db_version=$(python3 $ERIGON_QA_PATH/test_system/qa-tests/uploads/prod_info.py $ERIGON_REFERENCE_DATA_DIR/../production.ini production erigon_repo_commit) if [ -z "$db_version" ]; then @@ -156,20 +156,20 @@ jobs: --runner "$RUNNER_NAME" \ --db_version $db_version \ --outcome $TEST_RESULT \ - --result_file "$GITHUB_WORKSPACE/result-$CHAIN-$MODE_NAME-$EXEC_MODE.json" + --result_file "$GITHUB_WORKSPACE/result-$CHAIN-$MODE_NAME-$COMMITMENT_MODE.json" - name: Upload test results if: ${{ always() && steps.test_step.outputs.test_executed == 'true' }} uses: actions/upload-artifact@v7 with: - name: test-results-${{ matrix.mode_name }}-${{ matrix.exec_mode }} - path: ${{ github.workspace }}/result-${{ env.CHAIN }}-${{ matrix.mode_name }}-${{ matrix.exec_mode }}.json + name: test-results-${{ matrix.mode_name }}-${{ matrix.commitment_mode }} + path: ${{ github.workspace }}/result-${{ env.CHAIN }}-${{ matrix.mode_name }}-${{ matrix.commitment_mode }}.json - name: Upload erigon logs if: ${{ always() && steps.test_step.outputs.test_executed == 'true' }} uses: actions/upload-artifact@v7 with: - name: erigon-logs-${{ matrix.mode_name }}-${{ matrix.exec_mode }} + name: erigon-logs-${{ matrix.mode_name }}-${{ matrix.commitment_mode }} path: ${{ env.ERIGON_TESTBED_DATA_DIR }}/logs/ - name: Delete Erigon Testbed Data Directory diff --git a/.github/workflows/qa-txpool-performance-test.yml b/.github/workflows/qa-txpool-performance-test.yml index a03fc203c6c..a45575e9670 100644 --- a/.github/workflows/qa-txpool-performance-test.yml +++ b/.github/workflows/qa-txpool-performance-test.yml @@ -21,23 +21,23 @@ permissions: jobs: tx_pool_assertoor_test: - name: tx_pool_assertoor_test (${{ matrix.exec_mode }}) + name: tx_pool_assertoor_test (${{ matrix.commitment_mode }}) runs-on: [self-hosted, qa, X64, long-running] strategy: fail-fast: false # Self-hosted long-running pool is small; matrix entries serialize. max-parallel: 1 matrix: - exec_mode: + commitment_mode: - serial - parallel env: ERIGON_QA_PATH: /home/qarunner/erigon-qa - ENCLAVE_NAME: "kurtosis-run-${{ github.run_id }}-${{ matrix.exec_mode }}" - # Toggle dbg.Exec3Parallel from CI without code changes. envLookup - # in common/dbg/dbg_env.go auto-prepends ERIGON_, so this maps to - # the EXEC3_PARALLEL flag declared in common/dbg/experiments.go. - ERIGON_EXEC3_PARALLEL: ${{ matrix.exec_mode == 'parallel' && 'true' || 'false' }} + ENCLAVE_NAME: "kurtosis-run-${{ github.run_id }}-${{ matrix.commitment_mode }}" + # Toggle statecfg.ExperimentalParallelCommitment from CI without code + # changes. envLookup in common/dbg/dbg_env.go auto-prepends ERIGON_, so + # this maps to the COMMITMENT_PARALLEL flag read in db/state/statecfg. + ERIGON_COMMITMENT_PARALLEL: ${{ matrix.commitment_mode == 'parallel' && 'true' || 'false' }} steps: - name: Fast checkout git repository @@ -55,14 +55,14 @@ jobs: run: | docker build -t test/erigon:current-base . - - name: Bake exec_mode env into the runtime image - # Add ERIGON_EXEC3_PARALLEL as an ENV layer on top of the base image - # so the matrix entries differ in exec mode. ENV layers are cheap and - # don't invalidate earlier layer caches. + - name: Bake commitment_mode env into the runtime image + # Add ERIGON_COMMITMENT_PARALLEL as an ENV layer on top of the base + # image so the matrix entries differ in commitment mode. ENV layers are + # cheap and don't invalidate earlier layer caches. run: | docker build -t test/erigon:current - <> clients/erigon/Dockerfile + echo "ENV ERIGON_COMMITMENT_PARALLEL=${ERIGON_COMMITMENT_PARALLEL}" >> clients/erigon/Dockerfile erigon_extra_flags="${{ matrix.erigon-extra-flags }}" if [ -n "$erigon_extra_flags" ]; then echo "Patching erigon.sh with extra flags: $erigon_extra_flags" @@ -354,9 +399,9 @@ jobs: if: always() uses: actions/upload-artifact@v7 with: - # exec_mode in the artifact name keeps the two matrix entries from + # commitment_mode in the artifact name keeps the two matrix entries from # clobbering each other's logs on the same artifact key. - name: hive-workspace-log-${{ matrix.shard }}-${{ matrix.exec_mode }} + name: hive-workspace-log-${{ matrix.shard }}-${{ matrix.commitment_mode }} path: hive/workspace/logs if-no-files-found: ignore - name: Test Results diff --git a/.github/workflows/test-hive.yml b/.github/workflows/test-hive.yml index 0e69a18266a..7f32d638bd8 100644 --- a/.github/workflows/test-hive.yml +++ b/.github/workflows/test-hive.yml @@ -16,7 +16,7 @@ permissions: jobs: test-hive: - name: test-hive (${{ matrix.sim }}, ${{ matrix.sim-limit }}, ${{ matrix.exec_mode }}) + name: test-hive (${{ matrix.sim }}, ${{ matrix.sim-limit }}, ${{ matrix.commitment_mode }}) if: >- ${{ !github.event.pull_request.number || (!github.event.pull_request.draft @@ -31,10 +31,11 @@ jobs: # picture across every shard. fail-fast: ${{ github.event_name == 'merge_group' }} matrix: - # Each (sim, sim-limit) pair is run twice — once with serial exec - # (ERIGON_EXEC3_PARALLEL=false) and once with parallel — so engine-API - # / wire-protocol divergence between the two paths is caught on the - # PR. Matrix entries spawn separate `hive` group runners and run + # Each (sim, sim-limit) pair is run twice — once with serial commitment + # (ERIGON_COMMITMENT_PARALLEL=false) and once with parallel — so + # engine-API / wire-protocol divergence between the two paths is caught + # on the PR. Execution is parallel in both legs (default). Matrix + # entries spawn separate `hive` group runners and run # concurrently — wall-clock unchanged, runner-minutes doubled. # `sim` is the simulator path passed to `hive --sim`. Most simulators # live under simulators/ethereum/, but a few (e.g. devp2p) are top-level. @@ -42,53 +43,53 @@ jobs: - sim: ethereum/engine sim-limit: exchange-capabilities|auth max-allowed-failures: 0 - exec_mode: serial + commitment_mode: serial - sim: ethereum/engine sim-limit: exchange-capabilities|auth max-allowed-failures: 0 - exec_mode: parallel + commitment_mode: parallel - sim: ethereum/engine sim-limit: withdrawals max-allowed-failures: 0 - exec_mode: serial + commitment_mode: serial - sim: ethereum/engine sim-limit: withdrawals max-allowed-failures: 0 - exec_mode: parallel + commitment_mode: parallel - sim: ethereum/engine sim-limit: cancun max-allowed-failures: 0 - exec_mode: serial + commitment_mode: serial - sim: ethereum/engine sim-limit: cancun max-allowed-failures: 0 - exec_mode: parallel + commitment_mode: parallel - sim: ethereum/engine sim-limit: api max-allowed-failures: 0 - exec_mode: serial + commitment_mode: serial - sim: ethereum/engine sim-limit: api max-allowed-failures: 0 - exec_mode: parallel + commitment_mode: parallel - sim: ethereum/rpc-compat sim-limit: ".*" max-allowed-failures: 0 - exec_mode: serial + commitment_mode: serial - sim: ethereum/rpc-compat sim-limit: ".*" max-allowed-failures: 0 - exec_mode: parallel + commitment_mode: parallel - sim: devp2p sim-limit: eth max-allowed-failures: 0 - exec_mode: serial + commitment_mode: serial # discv5 exercises peer discovery, not the EL exec path, so it runs in - # just one exec mode — duplicating it in the serial leg adds no signal. + # just one commitment mode — duplicating it in the serial leg adds no signal. - sim: devp2p sim-limit: eth|discv5 max-allowed-failures: 0 - exec_mode: parallel + commitment_mode: parallel steps: - name: Checkout Erigon uses: actions/checkout@v7 @@ -156,10 +157,10 @@ jobs: - name: Get dependencies and build hive env: EXECUTION_APIS_REF: ${{ steps.hive-version.outputs.execution_apis_ref }} - # Toggle dbg.Exec3Parallel inside the hive erigon container. - # We bake this as an ENV directive into the client Dockerfile so - # every erigon instance hive launches inherits it. - ERIGON_EXEC3_PARALLEL: ${{ matrix.exec_mode == 'parallel' && 'true' || 'false' }} + # Toggle statecfg.ExperimentalParallelCommitment inside the hive + # erigon container. We bake this as an ENV directive into the client + # Dockerfile so every erigon instance hive launches inherits it. + ERIGON_COMMITMENT_PARALLEL: ${{ matrix.commitment_mode == 'parallel' && 'true' || 'false' }} run: | cd hive retry() { @@ -182,10 +183,10 @@ jobs: echo "ERROR: failed to repoint hive's erigon client Dockerfile at hive/erigon:cilocal" exit 1 fi - # Inject ERIGON_EXEC3_PARALLEL into the runtime image so the + # Inject ERIGON_COMMITMENT_PARALLEL into the runtime image so the # erigon process inside hive picks it up. Append as the last layer # so it doesn't invalidate earlier build caches. - echo "ENV ERIGON_EXEC3_PARALLEL=${ERIGON_EXEC3_PARALLEL}" >> clients/erigon/Dockerfile + echo "ENV ERIGON_COMMITMENT_PARALLEL=${ERIGON_COMMITMENT_PARALLEL}" >> clients/erigon/Dockerfile # Pin the execution-apis ref used by the rpc-compat simulator so that # upstream test additions don't break CI unexpectedly. # SECURITY: value comes from hive-versions.json which fork PRs can modify; @@ -261,13 +262,13 @@ jobs: - name: Compute artifact name id: artifact-name env: - RAW_NAME: hive-workspace-log-${{ matrix.sim }}-${{ matrix.sim-limit }}-${{ matrix.exec_mode }} + RAW_NAME: hive-workspace-log-${{ matrix.sim }}-${{ matrix.sim-limit }}-${{ matrix.commitment_mode }} run: echo "name=${RAW_NAME//[^A-Za-z0-9._-]/_}" >> "$GITHUB_OUTPUT" - name: Upload output log uses: actions/upload-artifact@v7 with: - # exec_mode in the artifact name keeps the two matrix entries from + # commitment_mode in the artifact name keeps the two matrix entries from # clobbering each other's logs on the same artifact key. name: ${{ steps.artifact-name.outputs.name }} path: hive/workspace/logs diff --git a/.github/workflows/test-kurtosis-assertoor.yml b/.github/workflows/test-kurtosis-assertoor.yml index a6320256901..a759807b009 100644 --- a/.github/workflows/test-kurtosis-assertoor.yml +++ b/.github/workflows/test-kurtosis-assertoor.yml @@ -312,7 +312,7 @@ jobs: key: docker-buildkit-${{ env.BUILDKIT_IMAGE }} assertoor_test: - name: assertoor_${{ matrix.suite }}_${{ matrix.exec_mode }}_test + name: assertoor_${{ matrix.suite }}_${{ matrix.commitment_mode }}_test needs: build-erigon-image # On cache-warming runs build-erigon-image alone warms the kurtosis image # cache; there's nothing for the matrix to do (the test step is skipped). @@ -336,32 +336,33 @@ jobs: # pass but pathological hangs (e.g. a node that silently stops proposing) # fail in reasonable time instead of burning hours. # - # Each suite runs twice — once with serial exec - # (ERIGON_EXEC3_PARALLEL=false) and once with parallel — so divergence - # is caught on the PR. Matrix entries spawn separate hosted runners - # and run concurrently — wall-clock unchanged, runner-minutes doubled. + # Each suite runs twice — once with serial commitment + # (ERIGON_COMMITMENT_PARALLEL=false) and once with parallel — so + # divergence is caught on the PR. Execution is parallel in both legs + # (default). Matrix entries spawn separate hosted runners and run + # concurrently — wall-clock unchanged, runner-minutes doubled. # All entries share the single erigon image built by build-erigon-image. include: - suite: regular package_args: .github/workflows/kurtosis/regular-assertoor.io ethereum_package_branch: "5.0.1" test_timeout_minutes: 50 - exec_mode: serial + commitment_mode: serial - suite: regular package_args: .github/workflows/kurtosis/regular-assertoor.io ethereum_package_branch: "5.0.1" test_timeout_minutes: 50 - exec_mode: parallel + commitment_mode: parallel - suite: pectra package_args: .github/workflows/kurtosis/pectra.io ethereum_package_branch: "5.0.1" test_timeout_minutes: 45 - exec_mode: serial + commitment_mode: serial - suite: pectra package_args: .github/workflows/kurtosis/pectra.io ethereum_package_branch: "5.0.1" test_timeout_minutes: 45 - exec_mode: parallel + commitment_mode: parallel - suite: glamsterdam package_args: .github/workflows/kurtosis/glamsterdam.io # Pinned to 6.1.0 rather than main: commit 835dd9b on main introduced GpuConfig, @@ -369,19 +370,27 @@ jobs: # Unpin to main once CI is upgraded to Kurtosis 1.18.1. ethereum_package_branch: "6.1.0" test_timeout_minutes: 20 - exec_mode: parallel + commitment_mode: serial + - suite: glamsterdam + package_args: .github/workflows/kurtosis/glamsterdam.io + # Pinned to 6.1.0 rather than main: commit 835dd9b on main introduced GpuConfig, + # a Starlark built-in that requires Kurtosis CLI >=1.18.1, breaking Starlark eval. + # Unpin to main once CI is upgraded to Kurtosis 1.18.1. + ethereum_package_branch: "6.1.0" + test_timeout_minutes: 20 + commitment_mode: parallel - suite: caplin-minimal package_args: .github/workflows/kurtosis/caplin-minimal-assertoor.io ethereum_package_url: "github.com/erigontech/ethereum-package" ethereum_package_branch: "erigontech/fix-caplin-launcher" test_timeout_minutes: 20 - exec_mode: serial + commitment_mode: serial - suite: caplin-minimal package_args: .github/workflows/kurtosis/caplin-minimal-assertoor.io ethereum_package_url: "github.com/erigontech/ethereum-package" ethereum_package_branch: "erigontech/fix-caplin-launcher" test_timeout_minutes: 20 - exec_mode: parallel + commitment_mode: parallel steps: - name: Fast checkout git repository @@ -516,16 +525,16 @@ jobs: - name: Load erigon base image into daemon run: docker load -i "${RUNNER_TEMP}/erigon-base-image.tar" - - name: Bake exec_mode env into the runtime image + - name: Bake commitment_mode env into the runtime image # Kurtosis launches `test/erigon:current` as the EL participant. - # Add ERIGON_EXEC3_PARALLEL as an ENV layer on top of the shared - # base image so the matrix entries differ only in exec mode. + # Add ERIGON_COMMITMENT_PARALLEL as an ENV layer on top of the shared + # base image so the matrix entries differ only in commitment mode. env: - ERIGON_EXEC3_PARALLEL: ${{ matrix.exec_mode == 'parallel' && 'true' || 'false' }} + ERIGON_COMMITMENT_PARALLEL: ${{ matrix.commitment_mode == 'parallel' && 'true' || 'false' }} run: | docker build -t test/erigon:current - <. + +package main + +import ( + "testing" + + "github.com/stretchr/testify/require" + + "github.com/erigontech/erigon/db/kv/temporal/temporaltest" + "github.com/erigontech/erigon/db/state/statecfg" + "github.com/erigontech/erigon/execution/commitment" +) + +func TestNewStateTestSharedDomainsUsesSelectedCommitment(t *testing.T) { + originalParallel := statecfg.ExperimentalParallelCommitment + originalStreaming := statecfg.ExperimentalStreamingCommitment + t.Cleanup(func() { + statecfg.ExperimentalParallelCommitment = originalParallel + statecfg.ExperimentalStreamingCommitment = originalStreaming + }) + + for _, tc := range []struct { + name string + parallel bool + variant commitment.TrieVariant + }{ + {name: "serial", variant: commitment.VariantHexPatriciaTrie}, + {name: "parallel", parallel: true, variant: commitment.VariantParallelHexPatricia}, + } { + t.Run(tc.name, func(t *testing.T) { + statecfg.ExperimentalParallelCommitment = tc.parallel + statecfg.ExperimentalStreamingCommitment = false + + db, tx := temporaltest.NewTestTx(t) + sd, err := newStateTestSharedDomains(db, tx) + require.NoError(t, err) + t.Cleanup(sd.Close) + + require.Equal(t, tc.variant, sd.GetCommitmentCtx().Trie().Variant()) + }) + } +} diff --git a/db/state/aggregator_bench_test.go b/db/state/aggregator_bench_test.go index 418f9eb98d5..789b9391091 100644 --- a/db/state/aggregator_bench_test.go +++ b/db/state/aggregator_bench_test.go @@ -70,6 +70,8 @@ func BenchmarkAggregator_Processing(b *testing.B) { domains, err := execctx.NewSharedDomains(ctx, tx, log.New()) require.NoError(b, err) defer domains.Close() + domains.EnableParaTrieDB(db) + require.Equal(b, execctx.PickTrieVariant(), domains.GetCommitmentCtx().Trie().Variant()) b.ReportAllocs() diff --git a/db/test/domain_shared_bench_test.go b/db/test/domain_shared_bench_test.go index 13c3f17e86c..1d355c9cea2 100644 --- a/db/test/domain_shared_bench_test.go +++ b/db/test/domain_shared_bench_test.go @@ -60,6 +60,15 @@ func testDbAndAggregatorBench(b *testing.B, aggStep uint64) (kv.TemporalRwDB, *s return db, db.(state.HasAgg).Agg().(*state.Aggregator) } +func newSharedDomainsBench(b *testing.B, db kv.TemporalRoDB, tx kv.TemporalTx) *execctx.SharedDomains { + b.Helper() + domains, err := execctx.NewSharedDomains(b.Context(), tx, log.New()) + require.NoError(b, err) + domains.EnableParaTrieDB(db) + require.Equal(b, execctx.PickTrieVariant(), domains.GetCommitmentCtx().Trie().Variant()) + return domains +} + func composite(k, k2 []byte) []byte { return append(bytes.Clone(k), k2...) } @@ -73,8 +82,7 @@ func Benchmark_SharedDomains_GetLatest(t *testing.B) { require.NoError(t, err) defer rwTx.Rollback() - domains, err := execctx.NewSharedDomains(t.Context(), rwTx, log.New()) - require.NoError(t, err) + domains := newSharedDomainsBench(t, db, rwTx) defer domains.Close() maxTx := stepSize * 258 @@ -156,8 +164,7 @@ func BenchmarkSharedDomains_ComputeCommitment(b *testing.B) { require.NoError(b, err) defer rwTx.Rollback() - domains, err := execctx.NewSharedDomains(b.Context(), rwTx, log.New()) - require.NoError(b, err) + domains := newSharedDomainsBench(b, db, rwTx) defer domains.Close() maxTx := stepSize * 4 @@ -338,8 +345,7 @@ func BenchmarkPruneSmallBatches(b *testing.B) { require.NoError(b, err) defer rwTx.Rollback() - domains, err := execctx.NewSharedDomains(ctx, rwTx, log.New()) - require.NoError(b, err) + domains := newSharedDomainsBench(b, db, rwTx) usedKeys := make(map[string]struct{}, keysCount*maxTx) for txNum := uint64(1); txNum <= maxTx; txNum++ { diff --git a/db/test/lifecycle_bench_test.go b/db/test/lifecycle_bench_test.go index 20f27f1002c..e58cd76ffa9 100644 --- a/db/test/lifecycle_bench_test.go +++ b/db/test/lifecycle_bench_test.go @@ -26,7 +26,6 @@ import ( "github.com/stretchr/testify/require" "github.com/erigontech/erigon/common/length" - "github.com/erigontech/erigon/common/log/v3" "github.com/erigontech/erigon/db/kv" "github.com/erigontech/erigon/db/state" "github.com/erigontech/erigon/db/state/execctx" @@ -171,8 +170,7 @@ func runLifecycle(b *testing.B, cfg lifecycleConfig) (*lifecycleTimings, kv.Temp require.NoError(b, err) defer rwTx.Rollback() - domains, err := execctx.NewSharedDomains(ctx, rwTx, log.New()) - require.NoError(b, err) + domains := newSharedDomainsBench(b, db, rwTx) defer domains.Close() rnd := newRnd(42) @@ -338,8 +336,7 @@ func BenchmarkLifecycle_PhaseIsolation(b *testing.B) { require.NoError(b, err) defer rwTx.Rollback() - domains, err := execctx.NewSharedDomains(ctx, rwTx, log.New()) - require.NoError(b, err) + domains := newSharedDomainsBench(b, db, rwTx) defer domains.Close() rnd := newRnd(42) @@ -371,8 +368,7 @@ func BenchmarkLifecycle_PhaseIsolation(b *testing.B) { require.NoError(b, err) defer rwTx.Rollback() - domains, err := execctx.NewSharedDomains(ctx, rwTx, log.New()) - require.NoError(b, err) + domains := newSharedDomainsBench(b, db, rwTx) defer domains.Close() rnd := newRnd(42) @@ -413,8 +409,7 @@ func BenchmarkLifecycle_PhaseIsolation(b *testing.B) { require.NoError(b, err) defer rwTx.Rollback() - domains, err := execctx.NewSharedDomains(ctx, rwTx, log.New()) - require.NoError(b, err) + domains := newSharedDomainsBench(b, db, rwTx) defer domains.Close() txNum := initAccounts(b, domains, rwTx, keyGen) @@ -446,8 +441,7 @@ func BenchmarkLifecycle_PhaseIsolation(b *testing.B) { require.NoError(b, err) defer rwTx.Rollback() - domains, err := execctx.NewSharedDomains(ctx, rwTx, log.New()) - require.NoError(b, err) + domains := newSharedDomainsBench(b, db, rwTx) defer domains.Close() rnd := newRnd(42) diff --git a/tools/eest-spec-shards.yml b/tools/eest-spec-shards.yml index ff53bbb87ea..9c56d43fefa 100644 --- a/tools/eest-spec-shards.yml +++ b/tools/eest-spec-shards.yml @@ -18,7 +18,11 @@ # explaining why and a tracking issue. # expected-tests (optional) — exact result count. Used by legacy shards # to guard the coverage moved out of Go tests. -# exec3-parallel (optional, default false) — pins ERIGON_EXEC3_PARALLEL. +# commitment-parallel (required) — pins ERIGON_COMMITMENT_PARALLEL. +# Execution is parallel in every shard +# (dbg.Exec3Parallel defaults true); the +# -sequential/-parallel suffix selects +# serial vs parallel commitment. # no-ramdisk (optional, default false) — true keeps the shard's # datadirs on disk instead of a RAM-backed # tmpfs. The benchmark shards use a few @@ -35,45 +39,70 @@ # -sequential/-parallel variants of a fork # carry the same regex. -- shard: statetests-stable +- shard: statetests-stable-sequential workers: 12 max-allowed-failures: 0 -- shard: statetests-devnet + commitment-parallel: false +- shard: statetests-stable-parallel workers: 12 max-allowed-failures: 0 -- shard: statetests-legacy + commitment-parallel: true +- shard: statetests-devnet-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false +- shard: statetests-devnet-parallel + workers: 12 + max-allowed-failures: 0 + commitment-parallel: true +- shard: statetests-legacy-sequential + workers: 12 + max-allowed-failures: 0 + commitment-parallel: false + expected-tests: 108593 +- shard: statetests-legacy-parallel + workers: 12 + max-allowed-failures: 0 + commitment-parallel: true expected-tests: 108593 - shard: rlptests-legacy-race workers: 12 max-allowed-failures: 0 + commitment-parallel: false expected-tests: 55 no-ramdisk: true - shard: transactiontests-legacy-race workers: 12 max-allowed-failures: 0 + commitment-parallel: false expected-tests: 212 no-ramdisk: true - shard: difficultytests-legacy-race workers: 12 max-allowed-failures: 0 + commitment-parallel: false expected-tests: 18598 no-ramdisk: true - shard: blocktests-stable-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false - shard: blocktests-stable-parallel workers: 12 max-allowed-failures: 0 - exec3-parallel: true -- shard: blocktests-devnet + commitment-parallel: true +- shard: blocktests-devnet-sequential workers: 12 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: false +- shard: blocktests-devnet-parallel + workers: 12 + max-allowed-failures: 0 + commitment-parallel: true - shard: blocktests-legacy-consensus-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false expected-tests: 1142 exclude: - '/\.meta/' @@ -81,19 +110,27 @@ workers: 12 max-allowed-failures: 0 expected-tests: 1142 - exec3-parallel: true + commitment-parallel: true exclude: - '/\.meta/' -- shard: blocktests-legacy-consensus-race +- shard: blocktests-legacy-consensus-race-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false expected-tests: 1142 - exec3-parallel: true + exclude: + - '/\.meta/' +- shard: blocktests-legacy-consensus-race-parallel + workers: 12 + max-allowed-failures: 0 + expected-tests: 1142 + commitment-parallel: true exclude: - '/\.meta/' - shard: blocktests-legacy-constantinople-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false expected-tests: 32615 exclude: - '/\.meta/' @@ -101,38 +138,74 @@ workers: 12 max-allowed-failures: 0 expected-tests: 32615 - exec3-parallel: true + commitment-parallel: true + exclude: + - '/\.meta/' +- shard: blocktests-legacy-constantinople-race-constantinople-sequential + workers: 12 + # PoW fork-choice and total-difficulty support: https://github.com/erigontech/erigon/issues/22061 + max-allowed-failures: 5 + commitment-parallel: false + expected-tests: 10807 + run: '_Constantinople$' exclude: - '/\.meta/' -- shard: blocktests-legacy-constantinople-race-constantinople +- shard: blocktests-legacy-constantinople-race-constantinople-parallel workers: 12 max-allowed-failures: 0 expected-tests: 10807 - exec3-parallel: true + commitment-parallel: true run: '_Constantinople$' exclude: - '/\.meta/' -- shard: blocktests-legacy-constantinople-race-constantinople-fix +- shard: blocktests-legacy-constantinople-race-constantinople-fix-sequential + workers: 12 + # PoW fork-choice and total-difficulty support: https://github.com/erigontech/erigon/issues/22061 + max-allowed-failures: 5 + commitment-parallel: false + expected-tests: 10802 + run: '_ConstantinopleFix$' + exclude: + - '/\.meta/' +- shard: blocktests-legacy-constantinople-race-constantinople-fix-parallel workers: 12 max-allowed-failures: 0 expected-tests: 10802 - exec3-parallel: true + commitment-parallel: true run: '_ConstantinopleFix$' exclude: - '/\.meta/' -- shard: blocktests-legacy-constantinople-race-byzantium +- shard: blocktests-legacy-constantinople-race-byzantium-sequential + workers: 8 + max-allowed-failures: 0 + commitment-parallel: false + expected-tests: 5000 + run: '_Byzantium$' + exclude: + - '/\.meta/' +- shard: blocktests-legacy-constantinople-race-byzantium-parallel workers: 8 max-allowed-failures: 0 expected-tests: 5000 - exec3-parallel: true + commitment-parallel: true run: '_Byzantium$' exclude: - '/\.meta/' -- shard: blocktests-legacy-constantinople-race-other-forks +- shard: blocktests-legacy-constantinople-race-other-forks-sequential + workers: 12 + # PoW fork-choice and total-difficulty support: https://github.com/erigontech/erigon/issues/22061 + max-allowed-failures: 14 + commitment-parallel: false + expected-tests: 6006 + exclude: + - '/\.meta/' + - '::.*_Constantinople(Fix)?$' + - '::.*_Byzantium$' +- shard: blocktests-legacy-constantinople-race-other-forks-parallel workers: 12 max-allowed-failures: 0 expected-tests: 6006 - exec3-parallel: true + commitment-parallel: true exclude: - '/\.meta/' - '::.*_Constantinople(Fix)?$' @@ -140,6 +213,7 @@ - shard: blocktests-legacy-cancun-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false expected-tests: 111983 exclude: - '/\.meta/' @@ -147,175 +221,249 @@ workers: 12 max-allowed-failures: 0 expected-tests: 111983 - exec3-parallel: true + commitment-parallel: true + exclude: + - '/\.meta/' +- shard: blocktests-legacy-cancun-race-berlin-sequential + workers: 12 + # PoW fork-choice and total-difficulty support: https://github.com/erigontech/erigon/issues/22061 + max-allowed-failures: 7 + commitment-parallel: false + expected-tests: 14026 + run: '(_Berlin$|fork_Berlin-)' exclude: - '/\.meta/' -- shard: blocktests-legacy-cancun-race-berlin +- shard: blocktests-legacy-cancun-race-berlin-parallel workers: 12 max-allowed-failures: 0 expected-tests: 14026 - exec3-parallel: true + commitment-parallel: true run: '(_Berlin$|fork_Berlin-)' exclude: - '/\.meta/' -- shard: blocktests-legacy-cancun-race-shanghai +- shard: blocktests-legacy-cancun-race-shanghai-sequential + workers: 12 + max-allowed-failures: 0 + commitment-parallel: false + expected-tests: 20689 + run: '(_Shanghai$|fork_Shanghai-)' + exclude: + - '/\.meta/' +- shard: blocktests-legacy-cancun-race-shanghai-parallel workers: 12 max-allowed-failures: 0 expected-tests: 20689 - exec3-parallel: true + commitment-parallel: true run: '(_Shanghai$|fork_Shanghai-)' exclude: - '/\.meta/' -- shard: blocktests-legacy-cancun-race-cancun +- shard: blocktests-legacy-cancun-race-cancun-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false expected-tests: 21849 - exec3-parallel: true run: '(_Cancun$|fork_Cancun-)' exclude: - '/\.meta/' -- shard: blocktests-legacy-cancun-race-london +- shard: blocktests-legacy-cancun-race-cancun-parallel workers: 12 max-allowed-failures: 0 + expected-tests: 21849 + commitment-parallel: true + run: '(_Cancun$|fork_Cancun-)' + exclude: + - '/\.meta/' +- shard: blocktests-legacy-cancun-race-london-sequential + workers: 12 + # PoW fork-choice and total-difficulty support: https://github.com/erigontech/erigon/issues/22061 + max-allowed-failures: 7 + commitment-parallel: false expected-tests: 20337 - exec3-parallel: true run: '(_London$|fork_London-)' exclude: - '/\.meta/' -- shard: blocktests-legacy-cancun-race-paris +- shard: blocktests-legacy-cancun-race-london-parallel workers: 12 max-allowed-failures: 0 + expected-tests: 20337 + commitment-parallel: true + run: '(_London$|fork_London-)' + exclude: + - '/\.meta/' +- shard: blocktests-legacy-cancun-race-paris-sequential + workers: 12 + max-allowed-failures: 0 + commitment-parallel: false expected-tests: 20369 - exec3-parallel: true run: '(_Paris$|fork_Paris-)' exclude: - '/\.meta/' -- shard: blocktests-legacy-cancun-race-other-forks +- shard: blocktests-legacy-cancun-race-paris-parallel + workers: 12 + max-allowed-failures: 0 + expected-tests: 20369 + commitment-parallel: true + run: '(_Paris$|fork_Paris-)' + exclude: + - '/\.meta/' +- shard: blocktests-legacy-cancun-race-other-forks-sequential + workers: 12 + # PoW fork-choice and total-difficulty support: https://github.com/erigontech/erigon/issues/22061 + max-allowed-failures: 12 + commitment-parallel: false + expected-tests: 14713 + exclude: + - '/\.meta/' + - '::.*(_(Berlin|Shanghai|Cancun|London|Paris)$|fork_(Berlin|Shanghai|Cancun|London|Paris)-)' +- shard: blocktests-legacy-cancun-race-other-forks-parallel workers: 12 max-allowed-failures: 0 expected-tests: 14713 - exec3-parallel: true + commitment-parallel: true exclude: - '/\.meta/' - '::.*(_(Berlin|Shanghai|Cancun|London|Paris)$|fork_(Berlin|Shanghai|Cancun|London|Paris)-)' - shard: enginextests-stable-sequential workers: 8 max-allowed-failures: 0 + commitment-parallel: false - shard: enginextests-stable-parallel workers: 8 max-allowed-failures: 0 - exec3-parallel: true -- shard: enginextests-devnet + commitment-parallel: true +- shard: enginextests-devnet-sequential workers: 8 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: false +- shard: enginextests-devnet-parallel + workers: 8 + max-allowed-failures: 0 + commitment-parallel: true - shard: enginextests-benchmark-1m-sequential workers: 1 max-allowed-failures: 0 + commitment-parallel: false no-ramdisk: true - shard: enginextests-benchmark-1m-parallel workers: 1 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true no-ramdisk: true - shard: enginextests-benchmark-5m-sequential workers: 1 max-allowed-failures: 0 + commitment-parallel: false no-ramdisk: true - shard: enginextests-benchmark-5m-parallel workers: 1 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true no-ramdisk: true - shard: enginextests-benchmark-10m-sequential workers: 1 max-allowed-failures: 0 + commitment-parallel: false no-ramdisk: true - shard: enginextests-benchmark-10m-parallel workers: 1 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true no-ramdisk: true - shard: enginextests-benchmark-30m-sequential workers: 1 max-allowed-failures: 0 + commitment-parallel: false no-ramdisk: true - shard: enginextests-benchmark-30m-parallel workers: 1 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true no-ramdisk: true - shard: enginextests-benchmark-60m-sequential workers: 1 max-allowed-failures: 0 + commitment-parallel: false no-ramdisk: true - shard: enginextests-benchmark-60m-parallel workers: 1 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true no-ramdisk: true - shard: enginextests-benchmark-100m-sequential workers: 1 max-allowed-failures: 0 + commitment-parallel: false no-ramdisk: true - shard: enginextests-benchmark-100m-parallel workers: 1 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true no-ramdisk: true - shard: enginextests-benchmark-150m-sequential workers: 1 max-allowed-failures: 0 + commitment-parallel: false no-ramdisk: true - shard: enginextests-benchmark-150m-parallel workers: 1 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true no-ramdisk: true - shard: zkevm-witness workers: 8 max-allowed-failures: 0 + commitment-parallel: false - shard: blocktests-stable-race-pre-cancun-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false run: 'fork_(Frontier|Homestead|TangerineWhistle|SpuriousDragon|Byzantium|ConstantinopleFix|Istanbul|Berlin|London|Paris|Shanghai)' - shard: blocktests-stable-race-cancun-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false run: 'fork_Cancun' - shard: blocktests-stable-race-prague-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false run: 'fork_Prague' - shard: blocktests-stable-race-osaka-sequential workers: 12 max-allowed-failures: 0 + commitment-parallel: false run: 'fork_(Osaka|BPO)' - shard: blocktests-stable-race-pre-cancun-parallel workers: 12 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true run: 'fork_(Frontier|Homestead|TangerineWhistle|SpuriousDragon|Byzantium|ConstantinopleFix|Istanbul|Berlin|London|Paris|Shanghai)' - shard: blocktests-stable-race-cancun-parallel workers: 12 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true run: 'fork_Cancun' - shard: blocktests-stable-race-prague-parallel workers: 12 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true run: 'fork_Prague' - shard: blocktests-stable-race-osaka-parallel workers: 12 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true run: 'fork_(Osaka|BPO)' -- shard: blocktests-devnet-race-amsterdam +- shard: blocktests-devnet-race-amsterdam-sequential + workers: 12 + max-allowed-failures: 0 + commitment-parallel: false + run: 'fork_Amsterdam' +- shard: blocktests-devnet-race-amsterdam-parallel workers: 12 max-allowed-failures: 0 - exec3-parallel: true + commitment-parallel: true run: 'fork_Amsterdam' - shard: zkevm-witness-race workers: 8 max-allowed-failures: 0 + commitment-parallel: false diff --git a/tools/run-eest-spec-test.sh b/tools/run-eest-spec-test.sh index e672d1cb506..ef19280fa22 100755 --- a/tools/run-eest-spec-test.sh +++ b/tools/run-eest-spec-test.sh @@ -5,28 +5,31 @@ # # Where is one of: # -# statetests-stable state tests vs. eest_stable -# statetests-devnet state tests vs. eest_devnet -# statetests-legacy complete legacy Cancun state tests +# statetests-stable-{sequential,parallel} state tests vs. eest_stable +# statetests-devnet-{sequential,parallel} state tests vs. eest_devnet +# statetests-legacy-{sequential,parallel} complete legacy Cancun state tests # rlptests-legacy-race complete legacy RLP tests # transactiontests-legacy-race complete legacy transaction tests # difficultytests-legacy-race complete legacy difficulty tests # blocktests-stable-sequential blockchain tests vs. eest_stable -# blocktests-devnet blockchain tests vs. eest_devnet -# blocktests-legacy-consensus-{sequential,parallel,race} +# blocktests-devnet-{sequential,parallel} blockchain tests vs. eest_devnet +# blocktests-legacy-consensus-{sequential,parallel} +# Hive consensus suite +# blocktests-legacy-consensus-race-{sequential,parallel} +# race-detector variants of the # Hive consensus suite # blocktests-legacy-constantinople-{sequential,parallel} # Hive legacy suite -# blocktests-legacy-constantinople-race-{constantinople,constantinople-fix,other-forks} -# race-detector partition of the -# Hive legacy suite +# blocktests-legacy-constantinople-race-* +# three fork partitions, each with +# sequential/parallel commitment # blocktests-legacy-cancun-{sequential,parallel} # Hive legacy-cancun suite -# blocktests-legacy-cancun-race-{berlin,shanghai,cancun,london,paris,other-forks} -# race-detector partition of the -# Hive legacy-cancun suite +# blocktests-legacy-cancun-race-* +# six fork partitions, each with +# sequential/parallel commitment # enginextests-stable-{sequential,parallel} engine-x tests vs. eest_stable -# enginextests-devnet engine-x tests vs. eest_devnet +# enginextests-devnet-{sequential,parallel} engine-x tests vs. eest_devnet # enginextests-benchmark-{1m,5m,10m,30m,60m,100m,150m}-sequential # engine-x benchmark fixtures per # gas-target subdir; each value @@ -42,7 +45,8 @@ # export EVM_BIN to the race-built # binary; otherwise -race # detection doesn't fire. -# blocktests-devnet-race-amsterdam race-detector variant filtered +# blocktests-devnet-race-amsterdam-{sequential,parallel} +# race-detector variants filtered # to the Amsterdam fork only. # zkevm-witness zkevm execution-witness conformance # (eest_zkevm corpus) via the zkevmtest @@ -50,13 +54,15 @@ # (canonical) per block. # zkevm-witness-race race-detector variant of the above over # the whole corpus (run via evm.race). -# *-parallel variants run with -# ERIGON_EXEC3_PARALLEL=true; -# dedicated devnet/race shards may -# also opt in via the manifest. -# Every shard pins the mode so a -# runtime default change cannot -# redefine its coverage. +# *-parallel any of the above with "-parallel" +# appended runs with +# ERIGON_COMMITMENT_PARALLEL=true. Every +# other shard runs with +# ERIGON_COMMITMENT_PARALLEL=false so +# the runtime default in +# statecfg.ExperimentalParallelCommitment +# can flip without redefining the shards. +# Execution is parallel in every shard. # # Each shard maps to one cmd/evm subcommand running with --jsonout. Pass/fail # is decided here (not by the binary, which always exits 0): the shard fails @@ -90,7 +96,7 @@ case "$shard" in *-stable*) fixture_sets=(eest_stable) ;; *-devnet*) fixture_sets=(eest_devnet) ;; *-benchmark*) fixture_sets=(eest_benchmark) ;; - statetests-legacy) fixture_sets=(legacy_cancun) ;; + statetests-legacy-*) fixture_sets=(legacy_cancun) ;; rlptests-legacy-* | \ transactiontests-legacy-* | \ difficultytests-legacy-*) fixture_sets=(legacy_tests) ;; @@ -107,27 +113,32 @@ fixture_base() { } base=$(fixture_base "${fixture_sets[0]}") -# Resolve workers + failure budget + exec3-parallel + the optional race --run -# regex from the single-source manifest. This script, the test-eest-spec.yml -# load-matrix job, and the coverage guard all read tools/eest-spec-shards.yml, -# so adding a shard / tweaking a budget / changing a fork filter is a one-file -# edit. yq converts YAML→JSON so it can be queried with jq. +# Resolve workers + failure budget + commitment-parallel + the optional race +# --run regex from the single-source manifest. This script, the test-eest-spec.yml +# load-matrix job, and the coverage guard all read tools/eest-spec-shards.yml, so +# adding a shard / tweaking a budget / changing a fork filter is a one-file edit. +# yq converts YAML→JSON so it can be queried with jq. manifest=tools/eest-spec-shards.yml -shard_row=$(yq -o=json '.' "$manifest" | jq -r --arg s "$shard" '.[] | select(.shard == $s) | "\(.workers)\t\(."max-allowed-failures")\t\(."exec3-parallel" // false)\t\(."no-ramdisk" // false)\t\(."expected-tests" // 0)\t\(.run // "")"') +shard_row=$(yq -o=json '.' "$manifest" | jq -r --arg s "$shard" ' + .[] | select(.shard == $s) + | if (."commitment-parallel" | type) != "boolean" + then error("shard \($s) must define boolean commitment-parallel") + else "\(.workers)\t\(."max-allowed-failures")\t\(."commitment-parallel")\t\(."no-ramdisk" // false)\t\(."expected-tests" // 0)\t\(.run // "")" + end') if [[ -z "$shard_row" ]]; then echo "shard $shard not found in $manifest" >&2 exit 2 fi -IFS=$'\t' read -r default_workers default_max exec3_parallel shard_no_ramdisk expected_tests run_regex <<<"$shard_row" -# Always set ERIGON_EXEC3_PARALLEL explicitly (true or false) so the shard's -# behaviour is pinned to the manifest, independent of whatever dbg.Exec3Parallel -# defaults to at runtime. If the default flips, the shards still run the mode -# they were defined for. -export ERIGON_EXEC3_PARALLEL="$exec3_parallel" +IFS=$'\t' read -r default_workers default_max commitment_parallel shard_no_ramdisk expected_tests run_regex <<<"$shard_row" +# Always set ERIGON_COMMITMENT_PARALLEL explicitly (true or false) so the shard's +# commitment mode is pinned to the manifest, independent of whatever +# statecfg.ExperimentalParallelCommitment defaults to at runtime. Execution is +# parallel in every shard (dbg.Exec3Parallel defaults true). +export ERIGON_COMMITMENT_PARALLEL="$commitment_parallel" # Strip "-parallel" / "-sequential" suffix for case-arm routing — both variants # share the same fixture path / regex as the parent shard; only the -# ERIGON_EXEC3_PARALLEL env var differs. +# ERIGON_COMMITMENT_PARALLEL env var differs. shard_route="${shard%-parallel}" shard_route="${shard_route%-sequential}" @@ -251,7 +262,7 @@ fi # code 66: the Go race runtime's "data race detected" signal, emitted even when # the run completes and the JSON parses clean, so it must be checked explicitly. # The grep filter strips any init-time log lines (e.g. dbg.envLookup's -# "[WARN] [env]" message when ERIGON_EXEC3_PARALLEL is set fires before cmd/evm +# "[WARN] [env]" message when ERIGON_COMMITMENT_PARALLEL is set fires before cmd/evm # sets the log handler, and the default log handler writes to stdout) so jq # sees only JSON. for path in "${paths[@]}"; do