Skip to content

perf: record only the top of the stack for non-verbose raw traces - #1544

Merged
Wodann merged 1 commit into
feat/tracing-back-compatfrom
perf/traces-top-of-stack-fork
Jul 7, 2026
Merged

Wodann merged 1 commit into
feat/tracing-back-compatfrom
perf/traces-top-of-stack-fork

Conversation

@nebasuke

@nebasuke nebasuke commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

On top of #1301 (feat/tracing-back-compat) to resolve the benchmark regression of up to 2.45x reported there. Supersedes #1536: this directly uses StackSnapshotType::Top (paradigmxyz/revm-inspectors#468) from a fork branch that backports it onto the 0.39.0 release EDR already pins.

Note there is an expected performance regression compared to not producing traces, see below. This holds only for Hardhat 2.

Benchmark suite Current: 1379aaa Previous: 46b2d01 Ratio
All Scenarios 245310.748094 ms 214303.369506 ms 1.14
neptune-mutual-blue-protocol_8db6480 25903.104518 ms 22167.746617 ms 1.17
openzeppelin-contracts_0a5fba7a 11628.837737000002 ms 9068.634355999999 ms 1.28
rocketpool_6a9dbfd8 14754.833111 ms 13019.832804000001 ms 1.13
safe-contracts_914d0f8 730.062472 ms 657.7373329999999 ms 1.11
seaport_585b2ef8 5946.402571 ms 4910.134137999999 ms 1.21
synthetix_9a3a109f 181693.82570000002 ms 160506.680113 ms 1.13
uniswap-v3-core_d8b1c63 4653.681985 ms 3972.6041450000002 ms 1.17

Claude summary

Problem

To expose the top of the stack through the non-verbose Response::traces() output, #1301 sets StackSnapshotType::Full, which clones the entire EVM stack (up to 1024 words) on every step of every transaction. The cost is paid unconditionally — even when the arena is discarded (includeCallTraces defaults to IncludeTraces::None, the Hardhat 3 default). The consumer only ever reads step.stack.last() in non-verbose mode, and no released revm-inspectors (0.33–0.41) offers a top-of-stack-only snapshot type.

Approach

Use the real thing: StackSnapshotType::Top from paradigmxyz/revm-inspectors#468 records a single-element (or empty) snapshot per step — one U256 copy instead of a full stack clone — into the same step.stack field, so .last() consumers are unchanged. When includeCallTraces is None, snapshots are skipped entirely: the stack-trace and internal-OOG paths read pcs and statuses, never step stacks.

Since upstream #468 targets the revm-41 line and EDR pins revm-inspectors 0.39.0 / revm 38, the dependency temporarily points at nebasuke/revm-inspectors branch feat/top-of-stack-snapshot-0.39 — the upstream PR commit cherry-picked onto the 0.39.0 release commit (applied cleanly, full test suite green including the two new stack snapshot tests). Once #468 is released and EDR moves to its revm major, the dependency reverts to crates.io with a version bump and this PR's diff shrinks to the config gating.

Resulting cost per configuration:

Configuration Before After
includeCallTraces: None (Hardhat 3 default) full stack clone per step zero — identical to main
includeCallTraces: All, non-verbose (Hardhat 2) full stack clone per step one U256 copy per step (the historical baseline)
verbose full stack clone per step (all()) unchanged

Changes

  • Cargo.toml / Cargo.lock: revm-inspectors from the fork branch, pinned in the lockfile to the backport commit. Git sources are skipped by cargo-cooldown-check; nothing else re-resolved.
  • crates/edr_provider/src/observability.rs: non-verbose tracing config selects StackSnapshotType::Top, or StackSnapshotType::None when traces can't be included in responses.
  • crates/edr_napi/test/provider.ts: nested-call-frames top-of-stack integration test carried over from perf: capture only the top of the stack for non-verbose raw traces #1536; step semantics (Some([]) for an empty stack, Some([top]) otherwise) are additionally pinned down by the fork's own tests/it/stack.rs.

Testing

  • Fork backport branch: full cargo test green (30 integration tests + test_top_stack_snapshots + test_top_stack_snapshots_match_the_top_of_full_snapshots).
  • EDR: cargo check --no-default-features (workspace), cargo test --all-features -p edr_solidity -p edr_provider (remote-fork tests requiring ALCHEMY_URL excluded locally), and the napi mocha tests matching "top of the stack" against a rebuilt binding.

Replaces the StackSnapshotType::Full snapshots introduced for
Response::traces back-compat, which clone the entire EVM stack (up to
1024 words) on every step, with the new StackSnapshotType::Top from
paradigmxyz/revm-inspectors#468: a single-element (or empty) snapshot
per step, matching what non-verbose consumers read via
step.stack.last(). When includeCallTraces is None (the Hardhat 3
default), snapshots are skipped entirely: the stack-trace and
internal-OOG paths read pcs and statuses, never step stacks.

Until revm-inspectors#468 is released and EDR is on its revm major,
revm-inspectors is consumed from a fork branch that backports the
upstream PR commit onto the 0.39.0 release
(nebasuke/revm-inspectors@feat/top-of-stack-snapshot-0.39); the
dependency then reverts to crates.io with a version bump.

The integration test is carried over from the superseded hand-rolled
capture approach (#1536).
@changeset-bot

changeset-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 1379aaa

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 6, 2026 18:04 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 6, 2026 18:07 — with GitHub Actions Inactive
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 6, 2026 18:07 — with GitHub Actions Failure
@codecov

codecov Bot commented Jul 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.10%. Comparing base (a8ef4d2) to head (1379aaa).

Additional details and impacted files
@@                    Coverage Diff                    @@
##           feat/tracing-back-compat    #1544   +/-   ##
=========================================================
  Coverage                     80.10%   80.10%           
=========================================================
  Files                           445      445           
  Lines                         76436    76439    +3     
  Branches                      76436    76439    +3     
=========================================================
+ Hits                          61228    61233    +5     
+ Misses                        13086    13085    -1     
+ Partials                       2122     2121    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nebasuke
nebasuke marked this pull request as ready for review July 6, 2026 19:16
@nebasuke
nebasuke requested a review from Wodann July 6, 2026 19:18

@Wodann Wodann left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code itself looks good. Thank you for this fix!

I did a local profiling run to analyse the remaining performance regression. The fix in this PR covers the worst offenders:

  • memory increase to store full stacks (can cause OOM)
  • performance regression due to increase mem copy & free load

With Claude's help, I analysed the performance (samples collected with perf) of EDR versions with:

  • includeCallTraces: All
    • prior to fix
    • after your fix
  • includeCallTraces: None

The seaport scenario illustrates the regression and fix well:

mechanism baseline regressed fixed
TracingInspector::start_step incl. snapshot allocs 0.6s 2.6s 0.7s
Arena free when napi Response drops ~0 1.2s 0.5s
mimalloc page management churn ~0 1.0s 0.1s
wall time 5.51s 12.67s 7.47s

It also shows that even the baseline is spending a lot of time in TracingInspector::start_step. I'll continue investigating differences compared to TraceCollector and address those on the existing PR.

@Wodann
Wodann merged commit 41931b5 into feat/tracing-back-compat Jul 7, 2026
59 of 60 checks passed
@Wodann
Wodann deleted the perf/traces-top-of-stack-fork branch July 7, 2026 13:42
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Oct 6, 2026

This branch had an error being deployed

1 failed deployment
github-action-benchmark — 1379aaa1 Deployed Jul 6, 2026 by nebasuke via Run JS scenario runner benchmark for Hardhat Node style workload #4075
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants