feat: add load-test payload worker for base-load-test integration#171
feat: add load-test payload worker for base-load-test integration#171
Conversation
Add a new payload worker type 'load-test' that integrates the Rust base-load-test binary as an external transaction generator. The worker follows the same proxy-based pattern as the tx-fuzz integration: - New LoadTestPayloadWorker spawns the binary with a generated YAML config pointing to a proxy server that captures transactions - CLI flag --load-test-bin (env: BASE_BENCH_LOAD_TEST_BIN) for binary path - Config interface extended with LoadTestBinary() method - Factory routes 'load-test' payload type to the new worker - Generated config supports configurable sender_count, target_gps, and duration via YAML payload params
The proxy server used rlp.DecodeBytes to decode captured transactions, which only handles legacy transactions. Typed transactions (EIP-2718, e.g. EIP-1559 type 2) have a type prefix byte before the RLP payload that causes rlp.DecodeBytes to fail with 'typed transaction too short'. Switch to Transaction.UnmarshalBinary() which handles both legacy and typed transaction formats. Also fix load-test worker config to include required fields for calldata (max_size) and precompile (target) transaction types.
🟡 Heimdall Review Status
|
- sender_count and transactions passed from benchmark YAML payload params - target_gps computed from gas_limit / block_time (no hardcoding) - seed randomized using crypto/rand - duration set to 99999s (process killed on Stop) - Stop() kills the load-test process and reaps it - transactions YAML passed through as raw yaml.Node to support the full Rust config schema (transfer, calldata, precompile, erc20, etc.)
DebugResponse unconditionally tried gzip decompression on every proxied response, logging at Error level when it failed. Most responses are plain JSON (not gzip-compressed), causing hundreds of spurious error lines per second. Fall back to logging the raw body at Debug level when the response is not gzip-encoded.
This reverts commit 01d084d.
- Add load-test.yaml workflow: triggers on PR and workflow_dispatch, builds binaries via _build-binaries.yaml, runs load-test benchmark, builds report, uploads output and report artifacts (always, even on failure) - Update _build-binaries.yaml to build and upload base-load-test artifact alongside base-reth-node and builder - Update build-base-reth-node.sh to also build base-load-test binary - Add configs/examples/load-test.yml benchmark config (10 senders, 70% transfer / 20% calldata / 10% precompile, 10 blocks, 1B gas)
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| name: Run load test benchmark | ||
| runs-on: ubuntu-latest | ||
| needs: [build-binaries] | ||
| steps: | ||
| - name: Harden the runner (Audit all outbound calls) | ||
| uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1 | ||
| with: | ||
| egress-policy: audit | ||
|
|
||
| - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 | ||
|
|
||
| - name: Set up Go | ||
| uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0 | ||
|
|
||
| - name: Download base-reth-node | ||
| uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8 | ||
| with: | ||
| name: base-reth-node | ||
| path: ${{ runner.temp }}/bin/ | ||
|
|
||
| - name: Download builder | ||
| uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8 | ||
| with: | ||
| name: builder | ||
| path: ${{ runner.temp }}/bin/ | ||
|
|
||
| - name: Download base-load-test | ||
| uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8 | ||
| with: | ||
| name: base-load-test | ||
| path: ${{ runner.temp }}/bin/ | ||
|
|
||
| - name: Make binaries executable | ||
| run: chmod +x ${{ runner.temp }}/bin/* | ||
|
|
||
| - name: Run load test benchmark | ||
| run: | | ||
| mkdir -p ${{ runner.temp }}/data-dir | ||
| mkdir -p ${{ runner.temp }}/output | ||
|
|
||
| go run benchmark/cmd/main.go \ | ||
| --log.level info \ | ||
| run \ | ||
| --config configs/examples/load-test.yml \ | ||
| --root-dir ${{ runner.temp }}/data-dir \ | ||
| --output-dir ${{ runner.temp }}/output \ | ||
| --builder-bin ${{ runner.temp }}/bin/builder \ | ||
| --base-reth-node-bin ${{ runner.temp }}/bin/base-reth-node \ | ||
| --load-test-bin ${{ runner.temp }}/bin/base-load-test | ||
|
|
||
| - name: Setup Node.js | ||
| if: always() | ||
| uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0 | ||
| with: | ||
| node-version: "20" | ||
|
|
||
| - name: Build Report | ||
| if: always() | ||
| run: | | ||
| cp -r ${{ runner.temp }}/output/ ./output/ || true | ||
| pushd report | ||
| npm install | ||
| npm run build | ||
| popd | ||
|
|
||
| - name: Upload Output | ||
| if: always() | ||
| uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0 | ||
| with: | ||
| name: load-test-output | ||
| path: ${{ runner.temp }}/output/ | ||
| retention-days: 7 | ||
|
|
||
| - name: Upload Report | ||
| if: always() | ||
| uses: actions/upload-artifact@6f51ac03b9356f520e9adb1b1b7802705f340c2b # v4.5.0 | ||
| with: | ||
| name: load-test-report | ||
| path: report/dist/ | ||
| retention-days: 7 |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 14 hours ago
In general, the fix is to add an explicit permissions block so that the GITHUB_TOKEN used by this workflow is restricted to the least privilege it needs. For a typical test/build workflow that only checks out code and works with artifacts, contents: read is usually sufficient. This matches CodeQL’s suggested minimal starting point and is unlikely to break existing behavior.
The single best fix here, without changing functionality, is to add a root-level permissions block just after the on: section in .github/workflows/load-test.yaml. This will apply to both jobs (build-binaries and load-test) unless they define their own permissions. Based on the visible steps, the workflow only needs read access to repository contents to allow actions/checkout to function, and no write scopes are required. Thus we can safely set:
permissions:
contents: readNo additional imports, methods, or definitions are needed; this is purely a YAML configuration change within the workflow file.
| @@ -4,6 +4,9 @@ | ||
| pull_request: | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} | ||
| cancel-in-progress: true |
Both examples.yaml and load-test.yaml now use the same base repo SHA (38de4e038) which has the base-load-test binary target. The shared _build-binaries.yaml and build script build all three binaries.
setup-rust-toolchain runs cargo metadata to compute a cache key, but the benchmark repo has no Cargo.toml (it's Go). This causes exit code 101. Disable the built-in cache since we use our own actions/cache step for the built binaries.
The short SHA was not fetchable because git clone only fetches the default branch. Use the branch name instead and add a fetch of the specific ref before checkout to handle both branch names and SHAs.
Update versions.env to point at feature/load-test-benchmark and update the default in _build-binaries.yaml. Remove explicit version overrides from load-test.yaml and examples.yaml.
| name: Build binaries | ||
| uses: ./.github/workflows/_build-binaries.yaml | ||
|
|
||
| load-test: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 14 hours ago
In general, the fix is to add an explicit permissions: block to the workflow (at the top level or per job) that grants only the minimal scopes required. Because this workflow only reads the repository and uses artifacts, it can safely use contents: read and no other write scopes. Defining permissions at the workflow root applies to all jobs that do not override it.
The best fix here is to add a root-level permissions: block just after the on: block, setting contents: read. This documents that the workflow only needs read access to the repo contents and ensures the GITHUB_TOKEN will not unexpectedly have broader privileges if organization or repository defaults change. No functional behavior of the workflow changes, since none of the steps require write access to repository contents or other resources.
Concretely:
- Edit
.github/workflows/load-test.yaml. - After line 5 (
workflow_dispatch:) and before theconcurrency:block, insert:
permissions:
contents: readNo imports or additional methods are needed because this is purely a YAML configuration change for the workflow.
| @@ -4,6 +4,9 @@ | ||
| pull_request: | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }} | ||
| cancel-in-progress: true |
cargo can't find the bin target without -p base-load-tests because it's in a different workspace package than the default.
2 participants
Summary
Adds a new
load-testpayload worker type that integrates the Rustbase-load-testbinary as an external transaction generator for benchmarks.LoadTestPayloadWorkerinrunner/payload/loadtest/— follows the same proxy-based pattern astx-fuzz, generating a temp YAML config and spawning the binary--load-test-bin/BASE_BENCH_LOAD_TEST_BINfor binary pathConfiginterface extended withLoadTestBinary()methodload-testpayload type to the new workerproxy.goswitched fromrlp.DecodeBytestotx.UnmarshalBinary()to support typed (EIP-2718) transactions — the previous code only handled legacy transactionsTesting
Tested locally end-to-end: benchmark harness successfully started the load-test binary, captured transactions via the proxy, and built blocks with them.
numSuccess=1 numFailure=0.Companion PR
feature/load-test-benchmark(adds binary target + workflow)