Skip to content

perf: capture only the top of the stack for non-verbose raw traces - #1536

Closed
nebasuke wants to merge 2 commits into
feat/tracing-back-compatfrom
perf/traces-top-of-stack
Closed

nebasuke wants to merge 2 commits into
feat/tracing-back-compatfrom
perf/traces-top-of-stack

Conversation

@nebasuke

@nebasuke nebasuke commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

On top of #1301 (feat/tracing-back-compat) to resolve the benchmark regression of up to 2.45x reported there.

PR filed on paradigmxyz/revm-inspectors#468 to add a StackSnapshotType::Top. We will need to upgrade to revm 41 when it does merge.

Claude summary

Problem

To expose the top of the stack through the non-verbose Response::traces() output, #1301 sets StackSnapshotType::Full, which clones the entire EVM stack on every step of every transaction. The cost is paid unconditionally — even when the arena is discarded (includeCallTraces defaults to IncludeTraces::None, the Hardhat 3 default). The consumer only ever reads step.stack.last() in non-verbose mode, and no released revm-inspectors (0.33–0.41) offers a top-of-stack-only snapshot type.

Approach

Record exactly what the output needs, the way the pre-unification TraceCollector did (Stack::Top):

  • SolidityTracingInspector::step captures interp.stack.last().copied() — one U256 copy per step into a flat vector, no per-step allocation.
  • At collect()/take() time, the captured values are written into the arena's steps as single-element stack snapshots, following each node's ordering depth-first (i.e. execution order, which matches capture order one-to-one).
  • The napi conversion, Response marshalling, and the JS-visible TracingStep.stack semantics are unchanged: non-verbose still reads step.stack.last().

Resulting cost per configuration:

Configuration Before After
includeCallTraces: None (Hardhat 3 default) full stack clone per step zero — identical to main
includeCallTraces: All, non-verbose (Hardhat 2) full stack clone per step one U256 copy per step (the historical baseline)
verbose full stack clone per step (all()) unchanged

Anticipated questions

  • Why not upstream a top-of-stack StackSnapshotType? Now filed as feat: add top-of-stack snapshot type paradigmxyz/revm-inspectors#468 (StackSnapshotType::Top, drop-in for the mechanism here: same step.stack field, same pre-op timing, .last() consumers unchanged). If accepted, it lands on the revm-41 line, so we can only adopt it after our next revm major bump — at which point the capture, write_stack_tops, and the ordering DFS in this PR get deleted in favor of set_stack_snapshots(StackSnapshotType::Top), with the tests kept as the contract. Until then, this PR is the fix that works on revm 38 today, and it stands alone if the upstream PR stalls.
  • Why gate capture on includeCallTraces != None? Arenas are still recorded in that mode (for stack-trace generation and internal-OOG detection), but nothing on those paths reads step.stack, so capturing tops would be pure waste on the default path.
  • The .expects in the ordering DFS: the indices come from the arena's own ordering/children invariants; a violation means a corrupt arena, where a loud failure beats silently emitting misaligned traces. The two debug_asserts guard the capture/step count alignment in both directions; in release builds a mismatch degrades to steps without a stack (rendered as an empty stack), never wrong values.

Testing

  • New unit tests for the ordering DFS: execution-order assignment across interleaved steps/logs/child calls, and preservation of already-recorded (verbose) snapshots when no tops were captured.
  • New edr_napi test with a nested CALL across two contracts, asserting per-step tops in both frames — including the callee frame starting empty and the caller's post-CALL step seeing the success flag. This fails loudly on any cross-frame misalignment or off-by-one.
  • The existing feat: add backwards compatibility for Response::traces #1301 contract tests ("top of the stack by default", verbose full stacks, memory, debug_traceTransaction/debug_traceCall) pass unchanged; debug_traceTransaction structLogs are unaffected as they come from the separate DebugInspector.
  • The edr-benchmark job fails on this PR, and that is expected. The base branch's hardhat 2.28.6 patch enables includeCallTraces: All at provider creation, and the JS scenario benchmark instantiates its provider through that patched hardhat — so benchmarks on this stack now measure the traces-on configuration against a stored traces-off baseline. The +14–28% alert is the cost of All with top-of-stack capture (one U256 copy per step plus single-element stack snapshots at collect time), i.e. the historical HH2 baseline — down from up to 2.45x with full-stack snapshots, which is the regression this PR fixes. The alert is accepted deliberately. perf: make raw trace collection demand-driven #1541 (stacked on this PR) makes the remaining cost lazy and lets Hardhat 2 only enable trace collection while the vm trace bridge has listeners, which returns the benchmark to the None path. (The fluctuating single-suite alerts in the Solidity test runner job are unrelated noise: that path doesn't go through the hardhat provider.)

@changeset-bot

changeset-bot Bot commented Jul 3, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: b5184e1

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 3, 2026 21:06 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 3, 2026 21:08 — with GitHub Actions Inactive
@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 3, 2026 21:08 — with GitHub Actions Inactive
@codecov

codecov Bot commented Jul 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.58156% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.13%. Comparing base (a8ef4d2) to head (b5184e1).

Files with missing lines Patch % Lines
crates/edr_solidity/src/tracing.rs 98.54% 2 Missing ⚠️
Additional details and impacted files
@@                     Coverage Diff                      @@
##           feat/tracing-back-compat    #1536      +/-   ##
============================================================
+ Coverage                     80.10%   80.13%   +0.03%     
============================================================
  Files                           445      445              
  Lines                         76436    76572     +136     
  Branches                      76436    76572     +136     
============================================================
+ Hits                          61228    61363     +135     
- Misses                        13086    13087       +1     
  Partials                       2122     2122              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nebasuke
nebasuke marked this pull request as ready for review July 3, 2026 23:09
@nebasuke
nebasuke requested review from a team and Copilot July 3, 2026 23:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Reduces non-verbose raw-tracing overhead by capturing only the top-of-stack value per EVM step (instead of cloning full stack snapshots), while preserving the JS-facing TracingStep.stack.last() semantics and keeping verbose tracing behavior unchanged.

Changes:

  • Added optional per-step “top-of-stack” capture to SolidityTracingInspector and post-processing to write those values into arena steps in execution order.
  • Removed StackSnapshotType::Full usage in non-verbose tracing configuration and introduced a lightweight top-of-stack capture gate.
  • Added unit tests for execution-order assignment across nested frames and a NAPI integration test covering nested calls.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
crates/edr_solidity/src/tracing.rs Captures per-step stack tops and writes them into trace steps in execution order; adds unit tests.
crates/edr_provider/src/observability.rs Stops recording full stack snapshots in non-verbose mode; enables lightweight top-of-stack capture when traces are included.
crates/edr_napi/test/provider.ts Adds an integration test asserting correct per-step top-of-stack across nested call frames.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/edr_solidity/src/tracing.rs Resolved
Comment thread crates/edr_solidity/src/tracing.rs Resolved
Comment thread crates/edr_napi/test/provider.ts Resolved
@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 5, 2026 20:52 — with GitHub Actions Inactive
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 5, 2026 20:55 — with GitHub Actions Failure
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 5, 2026 20:55 — with GitHub Actions Failure
@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 5, 2026 22:56 — with GitHub Actions Inactive
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 5, 2026 22:58 — with GitHub Actions Failure
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 5, 2026 22:58 — with GitHub Actions Failure
@nebasuke
nebasuke marked this pull request as draft July 6, 2026 08:01
@nebasuke
nebasuke force-pushed the perf/traces-top-of-stack branch from 79caff6 to 16dcddd Compare July 6, 2026 08:24
@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 6, 2026 08:24 — with GitHub Actions Inactive
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 6, 2026 08:27 — with GitHub Actions Failure
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 6, 2026 08:27 — with GitHub Actions Failure
@nebasuke
nebasuke marked this pull request as ready for review July 6, 2026 10:32
@nebasuke
nebasuke force-pushed the feat/tracing-back-compat branch from 9acaaac to a8ef4d2 Compare July 6, 2026 11:12
@nebasuke
nebasuke force-pushed the perf/traces-top-of-stack branch from 16dcddd to f25666b Compare July 6, 2026 11:12
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 6, 2026 11:12 — with GitHub Actions Error
nebasuke added 2 commits July 6, 2026 11:40
Recording `StackSnapshotType::Full` clones the entire EVM stack on
every step of every transaction, causing up to a 2.45x slowdown in the
provider benchmarks - paid even when the arena is discarded
(`include_call_traces` defaults to `None`). The non-verbose `traces()`
output only ever reads the top of the stack at step start, so record
exactly that, the way the pre-unification `TraceCollector` did.

`SolidityTracingInspector` now captures `interp.stack.last()` in its
`step` hook (one `U256` copy, no per-step allocation) into a flat,
execution-ordered vector, and writes the values into the collected
arena's steps by following each node's `ordering` depth-first. The
napi conversion is unchanged: it still reads `step.stack.last()`.

Capture is skipped when raw traces cannot reach a response
(`include_call_traces == None`) or when verbose tracing records full
snapshots anyway, so Hardhat 3's default path has zero overhead.
@nebasuke
nebasuke force-pushed the perf/traces-top-of-stack branch from f25666b to b5184e1 Compare July 6, 2026 11:48
@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 6, 2026 11:49 — with GitHub Actions Inactive
@nebasuke
nebasuke had a problem deploying to github-action-benchmark July 6, 2026 12:19 — with GitHub Actions Failure
@nebasuke
nebasuke temporarily deployed to github-action-benchmark July 6, 2026 12:19 — with GitHub Actions Inactive
@Wodann
Wodann requested review from Wodann and removed request for a team July 6, 2026 15:20
@Wodann

Wodann commented Jul 8, 2026

Copy link
Copy Markdown
Member

Superseded by #1544

@Wodann Wodann closed this Jul 8, 2026
@github-actions github-actions Bot locked as resolved and limited conversation to collaborators Oct 7, 2026

This branch was previously deployed

1 inactive deployment
github-action-benchmark — b5184e12 Deployed Jul 6, 2026 by nebasuke via Run JS Solidity test runner benchmark #4069
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants