[codex] Harden public release readiness - #8
Merged
Conversation
OSS-prep work (extracted from beef-evm): - Add extensible revert decoder, examples, benches, and integration tests - Add fixtures (MockERC20) and CI workflow updates - README and crate-level documentation polish P0 pre-release fixes: P0.1 Remove leaked `amms` type from the public API and drop the dependency - Introduce a clean, dependency-free public `TickInfo` struct (liquidity_gross/liquidity_net/initialized) in src/cache/tick_snapshot.rs and re-export it from `cache`. - Replace `amms::amms::uniswap_v3::Info` at all five public sites (inject_v3_ticks / inject_v3_ticks_with_base, V3PoolTickSnapshot::to_ticks and from_pool_data) with `TickInfo`. - Remove the `amms = "0.7.4"` dependency from Cargo.toml. Pure type swap; tick storage-key math and injection logic unchanged. P0.2 Fix two silent footguns - set_block no longer lets the EVM block context silently diverge from the pinned block: for a concrete BlockId::Number it now updates block_number (NUMBER opcode) in lockstep. basefee caveat documented; repin_to_block doc clarified. - The rpc_call and storage_batch_fetcher closures no longer panic on a current-thread runtime (or with no runtime). They guard with Handle::try_current() + RuntimeFlavor and degrade to typed errors instead (rpc_call -> Err; batch fetcher -> Err per request). Documented the multi-thread tokio runtime requirement on EvmCache::new/with_cache. Verification: cargo fmt --check, clippy -D warnings, full test suite (101 lib + 11 cache_state + 2 storage_keys + doctests), rustdoc -D warnings, and example/bench builds all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ilder
- errors: derive SimError with thiserror; add a first-class `Halt { reason,
gas_used }` variant instead of folding halts into `Other`. Keep
`SimulationErrorKind` as a deprecated alias. Add `is_revert`/`is_halt`/
`as_revert` accessors.
- tx env: add `TxConfig` (value/gas_limit/gas_price/nonce/access_list) and
`call_raw_with`; enable revm `optional_balance_check` + `disable_balance_check`
so value-bearing simulation works without funding the caller.
- block env: populate coinbase/prevrandao/gas_limit from the fetched header on
both the cache and overlay (snapshot) paths; add setters.
- benches: `benches/simulation.rs` covers the real hot paths.
- builder: `EvmCache::builder(provider)` fluent constructor.
- docs/ROADMAP.md: vision, target architecture, three pillars, phased plan.
Verified: fmt, clippy --all-targets -D warnings, cargo test (118 + doctests),
RUSTDOCFLAGS=-D warnings cargo doc.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a `[features]` table (`default = ["protocols"]`). Gate the protocol-specific surface so the generic engine builds without it: - `storage_keys` (V2/V3/Pancake/Slipstream slot layouts + tick-key math) and the `tick_snapshot` module + their re-exports. - The `tick_snapshot_cache` field, its load/save, and the `inject_v2_pool_metadata` / `inject_v3_*` methods + `CacheConfig::tick_snapshot_cache_path`. The library compiles and lints cleanly with `--no-default-features`; CI now runs `cargo clippy --lib --no-default-features -- -D warnings`. README + ROADMAP updated. This surface is slated to move into `evm-amm-state`. Note: in-crate unit tests for the tick math still assume the default feature, so `cargo test --no-default-features` is a documented follow-up. Verified: fmt, clippy --all-targets -D warnings (default), clippy --lib --no-default-features -D warnings, cargo test, RUSTDOCFLAGS=-D warnings cargo doc. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The phase-2-spec.md build contract and the detailed Phase 2 design section in ROADMAP.md (decisions locked). The ROADMAP status flips to Done at the end of the phase. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rams Make the observation tracker take an explicit `now: u64` (clock units) on `observe` and `should_refetch`, and move the hardcoded thresholds into a new `freshness::FreshnessParams` (block-oriented defaults, plus `for_wall_clock` helper). Drops the internal `unix_now` so the tracker is driven by a configurable clock. Updates the in-module tests to the new signatures and adds clock-recording / max-reuse coverage. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the classification layer (`Validity` + `FreshnessRegistry` with slot-account-default resolution and `is_volatile`), the configurable clock (`FreshnessClock` + `BlockClock`/`WallClock`), and the `FreshnessPolicy` trait with `AlwaysVerify`/`NeverVerify`/`ObservationDriven` built-ins. All generic core (no `protocols` dep). Unit tests cover resolution order, the ValidThrough boundary, clock sharing, and each policy's select. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the EvmCache freshness primitives: - `verify_slots` re-fetches via the batch fetcher, compares to cached values, injects the changed ones, and returns `Vec<SlotChange>`. - `purge_account` drops an account's info + storage from both the CacheDB overlay and the BlockchainDb accounts/storage maps. - `set_storage_batch_fetcher` (test/extensibility seam) and a `cached_storage_value` read helper. Add `SlotChange` to freshness, `EvmSnapshot::storage_value` and `EvmOverlay::override_slot` accessors for the background validator, plus a stub/failing fetcher helper in tests/common and the step-3 integration tests in tests/freshness.rs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the optimistic verify-and-rerun loop: - `SimRequest`, `Validation` (Confirmed/Corrected/Unverified), `SpeculativeSim` (optimistic results + deferred-validation JoinHandle, abort-on-drop), and the generic `FreshnessController<P, C>`. - `run` drains pending corrections, snapshots, runs optimistic sims capturing per-sim volatile read sets, asks the policy which predicted candidates to verify, then spawns a Send-only background validator and returns immediately. - The validator re-fetches (policy set ∪ actual read sets), compares to the snapshot, observes into the shared tracker, and on a mismatch queues corrections + re-runs only the affected sims (overlay override_slot) → Corrected; fetcher error → Unverified. The full-loop integration tests cover the match/mismatch/unverified paths, pending flow-back, selective re-run, NeverVerify reconcile, a pinned slot, ValidThrough boundary, and BlockClock vs WallClock. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Re-export the key freshness types from the crate root and add the module to the crate-level doc. - Add the offline `examples/freshness_optimistic.rs` demonstrating a `Corrected` validation via a stub fetcher, and list it in the README example table. - Flip the ROADMAP Phase 2 status to Done and note what landed. The `freshness` module-level doctest (registry + policy, no network) is the runnable doc example. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
C1: add `pub output: Bytes` to CallSimulationResult, populated in all four
constructors (two simulate methods in cache/mod.rs, the overlay simulate
path, and result_to_sim). Maps Success/Revert payloads, empty on Halt, so
a corrected view-call's new return value is observable.
C2: remove the inert `params` field + `with_params` from FreshnessController
(params belong to ObservationDriven); update phase-2-spec §7.
C3: FreshnessClock::advance default no-op; BlockClock::advance sets the block;
on_new_block advances the clock then notifies the policy, so ValidThrough
aging and reuse-window progress flow through the natural API.
C4: poison-tolerant freshness mutex locks (unwrap_or_else into_inner).
D1: ROADMAP verify_slots bullet no longer claims it observes the tracker.
D2: spec notes the new output field and the corrected on_new_block behavior.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rewrites the freshness tests so each would fail on the corresponding regression (verified by mutation), plus two small enabling production changes. Production: - FreshnessController::rerun_count() + an Arc<AtomicUsize> threaded into the validator, incremented once per re-executed sim. Makes selective re-run observable (skip vs identical re-run). - The spawned validator yields once before any work so an abort-on-drop can deterministically cancel it before it touches the tracker or queues a correction (run_validator is otherwise fully synchronous). Tests (tests/freshness.rs unless noted): - T1: a balanceOf VIEW call corrected success→different-success; asserts the output return data differs (new balance) with both runs succeeding — not keyed off logs. - T2: run_mismatch test now asserts rerun_count() == 1, so removing the `intersects` filter (which would re-run both sims) fails the test. - T3: real Drop-abort test (drop with no await; assert pending stays 0 and the fetcher was never reached) + fixed into_optimistic abort test to use a correction-queuing fetcher and assert pending stays 0. - T4: missing-slot-as-zero through the controller → Corrected with old=ZERO. - T5: pending drain alters the SECOND run's RESULT (transfer reverts after drain). - T6: panicking fetcher → Unverified (JoinError); no-fetcher cache → Unverified with the "no storage batch fetcher available" reason. - T7: probabilistic should_refetch unit tests (slot_observations.rs) covering change_rate≈0.15 at now==last_checked, ≈0.01 reuse-then-refetch, and a cycle_interval>1 scaling case; plus an ObservationDriven end-to-end controller test seeding the tracker via with_tracker. - T8: on_new_block ages a ValidThrough slot into volatile via the natural API. - T9: assert optimistic/corrected token_deltas.is_empty() (documented stub). Adds tracking_fetcher/panicking_fetcher helpers to tests/common. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Benchmarks the optimistic sims + background slot validation on a swap-shaped sim (MockERC20 transfer reads/writes a balance slot) with correct vs stale snapshots. Two groups, fully offline via stub fetchers: - phase2_cpu: freshness-layer overhead (optimistic run, confirmed/corrected full cycle) with a zero-latency fetcher. - phase2_latency_50ms: latency hiding vs a naive fetch-then-simulate baseline, using a stub fetcher with a 50ms simulated RPC round-trip — optimistic result in ~9us vs ~55ms for the naive path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Snapshot of the current working tree. Headline change: address the seven
Phase 2 review findings, each with a reproduction test added to the freshness
suite (25 -> 33 tests):
- F1: heal corrections through the CacheDB overlay (inject_storage_batch_fresh)
so a validated correction is not shadowed by a stale overlay slot; route the
controller drain and verify_slots through it.
- F2: iterate the background validator to a fixed point so a correction that
flips control flow onto a new volatile slot verifies that slot too (bounded by
MAX_VALIDATION_ROUNDS); rerun_count still counts distinct affected sims once.
- F3: thread SimRequest.tx (value / gas limit / gas price / nonce / access list)
through a new EvmOverlay::call_raw_with_access_list_with; add
SimRequest::with_value / with_gas_limit / with_gas_price.
- F4: cooperative, best-effort cancellation for the validator (a cancel flag is
checked before fetching, before observing, and before queuing corrections) and
honest abort rustdoc.
- F5: block-aware StorageBatchFetchFn (now takes Option<BlockId>); the controller
captures the cache's pinned block at run() so the deferred validator fetches at
the snapshot's block, immune to a concurrent set_block re-pin.
- F6: SimStatus on CallSimulationResult (Success / Revert / Halt { reason });
CallSimulationResult is now #[non_exhaustive]. The optimistic example branches
on status instead of inferring success from logs.
- F7: gate protocol-only tests behind the `protocols` feature (file-level on
tests/storage_keys.rs; protocol-free unit tests relocated to a core_tests
module) so `cargo test --no-default-features` builds and runs.
Green: cargo test (default and --no-default-features), clippy --all-targets
--all-features -D warnings, fmt --check, RUSTDOCFLAGS=-Dwarnings cargo doc,
cargo check --examples (both feature sets), cargo bench --no-run.
This commit also captures other in-progress working-tree changes (README,
additional examples/tests, and assorted engine/OSS-prep edits) committed in the
same snapshot at the author's request.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…remediation
Implements the Pillar B.1 state-update layer: a generic `StateUpdate` vocabulary
the (Phase 4) event decoder will emit, applied through one unified write-through
path, with a structured `StateDiff` output. Existing writers (`inject_*`,
`purge_*`, `override_account_code*`) and the freshness pending-drain are refolded
onto that path.
Vocabulary & apply:
- `StateUpdate::{Slot, SlotDelta, Account, Purge, BalanceDelta}` + constructors
(`slot`, `slot_delta`, `balance`, `balance_delta`, `nonce`, `code`, `account`,
`purge`); `AccountPatch` (partial balance/nonce/code); `PurgeScope`.
- `EvmCache::apply_update` / `apply_updates` -> `StateDiff { slots, accounts,
purged, skipped, skipped_balances }` with `is_empty`/`len` (changes-only),
`has_skipped`/`skipped_len`/`is_fully_applied`, and `merge`.
Relative read-modify-write (cold-aware):
- `SlotDelta::{Add,Sub}` (saturating) for storage; `BalanceDelta` for native
balance; closure escape hatches `modify_slot` / `modify_account_balance`.
- A delta against a value the cache does not hold (cold) is NOT applied — it is
surfaced in `skipped` / `skipped_balances` so the caller can seed the truth,
never corrupting an unknown base. This powers event-driven balance tracking
(index a Transfer -> `[Sub on from, Add on to]`).
Audit remediation (5-lens adversarial audit; see docs/phase-3-spec.md §16):
- FIX (HIGH, silent corruption): `cached_storage_value` is now `account_state`-
aware — a slot absent from a `StorageCleared`/`NotExisting` overlay account
reads as ZERO (mirroring the EVM SLOAD / `CacheDB::storage_ref`) instead of
returning a shadowed backend value. Pre-fix, a relative update computed against
a base the EVM never sees. Pinned by an SLOAD-validated reproducer.
- FIX (no-op Account patch no longer materializes a backend account).
- `serde` on the whole vocabulary + `freshness::SlotChange`; `#[non_exhaustive]`
on `StateDiff` + `AccountPatch` (leaf record types kept exhaustive so callers
can still build them for equality assertions).
- Perf: batched single-lock fast-path for runs of `Slot`/`SlotDelta` writes in
`apply_updates` (one backend storage write-guard per run; dropped before
`Account`/`BalanceDelta`/`Purge`), plus elimination of the `SlotDelta`
double-read. Validated byte-for-byte against the sequential fold.
Tests/docs/benches: tests/state_update.rs (49) covering every variant, the
cold-skip guarantee, write-through layering, the batched==sequential equivalence
net, serde round-trip, and the protocols Decision-2 write-through pins; the
freshness/snapshot seeds were corrected to be EVM-visible (overlay-resident)
since a backend-only seed on a StorageCleared account is invisible post-fix.
benches/state_update.rs, examples/state_update_apply.rs, CHANGELOG, ROADMAP,
KNOWN_ISSUES, README updated. Full suite green (250 tests + 31 doctests),
clippy default + --no-default-features, fmt, RUSTDOCFLAGS=-D warnings doc, and
cargo bench --no-run all clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…account-info paths An adversarial review of the §16 fixes found the §16.0 `cached_storage_value` fix had not been propagated to two sibling read paths, leaving the same silent-corruption class open elsewhere. This completes it. create_snapshot (HIGH — was: snapshot/overlay/validator read a shadowed slot where the live cache now reads ZERO): - `EvmSnapshot` gains a `storage_cleared: HashSet<Address>` set. `create_snapshot` captures a `StorageCleared`/`NotExisting` overlay account's storage as ONLY its overlay slots (shadowed backend slots dropped) and records the address in the set. - `EvmSnapshot::storage_value` and `EvmOverlay::storage` honor the set: a slot absent from a cleared account reads ZERO and does NOT fall through to the backend / `ext_db` — mirroring `cached_storage_value` and the live EVM SLOAD. This also realigns the background validator's `old` (snapshot value) with the synchronous `verify_slots` (`cached_storage_value`), which had diverged. loaded_account_info (MED — account-axis analog): - Mirror revm `DbAccount::info()`: a `NotExisting` overlay account is absent to the EVM (returns `None`) and does not fall through to the backend, so a `BalanceDelta` / partial `Account` patch skips rather than computing against a stale `info` the EVM never sees. write_account_info_through (LOW — layer hygiene): - Normalize a `ZERO` `code_hash` to `KECCAK_EMPTY` before the backend write so both layers store an identical hash (matching revm's `insert_contract`, which the overlay write already applies). Tests: - New regressions: `snapshot_mirrors_live_read_for_cleared_account` (snapshot/overlay read ZERO for a cleared account's shadowed slot), `balance_delta_on_notexisting_overlay_account_is_skipped`, `account_patch_normalizes_zero_code_hash_across_layers`. - Freshness seed corrections: the optimistic-loop tests (incl. the `cache_with_balance` helper) seeded balances backend-only on the StorageCleared MockERC20 fixture — invisible to the EVM/snapshot post-fix. Reseeded overlay-resident (EVM-visible) so they exercise a realistic, EVM-consistent state; assertions unchanged. These tests had been passing only because the pre-fix snapshot read the shadowed value (the bug this commit closes). Full suite green (253 tests + 31 doctests), clippy default + --no-default-features, fmt, RUSTDOCFLAGS=-D warnings doc, and cargo bench --no-run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…is account_state Addresses a fresh Phase 2 freshness review (4 findings) and the round-2 adversarial pass on the snapshot fixes (1 HIGH + LOWs). Unifying theme: the validator must never return a trusted verdict on incomplete verification, and account_state- awareness must hold on EVERY read/flatten path (storage AND account/basic). Validator trust contract (src/freshness.rs): - Fixed-point round cap now returns `Validation::Unverified` (was a best-effort, trusted `Corrected`) and queues no corrections — the results had not reached a verified fixed point. (P1) - A corrected re-run that fails to execute (host/transact `Err`, not a revert/halt) returns `Unverified` instead of silently keeping the stale optimistic result. (P2) - New `collect_fetch_results` requires the batch fetcher to return EVERY requested slot; an omitted slot → `Unverified` rather than defaulting to zero (a custom fetcher dropping a slot could otherwise cause a false confirm/correct). (P2) Account-axis account_state (round-2 HIGH + LOWs): - `create_snapshot` / `EvmOverlay::basic`: a `NotExisting` overlay account is now excluded from the snapshot `accounts`/`code_by_hash` and recorded in a new `EvmSnapshot.accounts_not_existing` set; `basic` returns `None` for it (no ext_db fall-through), mirroring revm `DbAccount::info()` and `loaded_account_info`. Pre-fix the snapshot/parallel/validator path saw a phantom existing account. - `target_account_info` (deploy path): `NotExisting` overlay account treated as a missing target rather than returning stale info. - `loaded_account_info`: normalizes a `ZERO` code_hash to `KECCAK_EMPTY` at load, so a patch's `old_code_hash` matches what is written (self-consistent diff). - `modify_account_balance` doc corrected (a `NotExisting` account is also cold). Access-list checkpoint (src/cache/overlay.rs + mod.rs): - `call_raw_with_access_list` / `call_raw_with_access_list_with` now `checkpoint_revert` on every path (success and host error) via a `match` instead of `?`-before-revert. Resolves KNOWN_ISSUES #9. Tests (regressions, red before / green after): - `snapshot_basic_returns_none_for_notexisting_account` (account-axis HIGH). - `run_unverified_when_fixed_point_round_cap_exceeded` (P1; drives a generated 12-deep chained-SLOAD contract past the 8-round cap, asserts Unverified + pending_len()==0). - `run_unverified_when_fetcher_omits_requested_slot` (P2; omitting fetcher). Not separately unit-tested (covered by the code change; not cleanly triggerable offline): the corrected-rerun host-error path and the deploy-path `target_account_info` guard. Docs: PurgeScope StorageCleared/NotExisting refetch caveat; KNOWN_ISSUES #9 marked resolved; CHANGELOG. Full suite green (256 tests + 31 doctests), clippy default + --no-default-features, fmt, RUSTDOCFLAGS=-D warnings doc, cargo bench --no-run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Build contract for the reader half of Pillar B: an EventDecoder trait + StateView, a DecoderRegistry, an ERC-20 Transfer decoder, a UniswapV3 Swap/Mint/Burn adapter, and the EventPipeline (ingest_logs / reorg_to / reconcile) that drives reactive cache updates. Adds the cold-aware StateUpdate::SlotMasked vocabulary variant so a pure decoder can express a partial update to a packed storage word (V3 slot0) without clobbering bits it does not own. Decisions locked with the user 2026-06-16 (SlotMasked; full Swap+Mint/Burn V3 coverage; purge-and-resync reorgs; sampled correct+alarm reconciliation). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Authored before implementation, per the phased workflow — these define correctness for Pillar B.2 and will validate the deliverable: - tests/state_update.rs: SlotMasked cold-aware masked-write tests (sets only masked bits / no-op / cold skip+surface via skipped_masks / both-layer write-through / full-mask-vs-cold / serde round-trip). - tests/event_pipeline.rs: DecoderRegistry dispatch (address-scoped + global); ERC-20 Transfer -> Sub/Add SlotDeltas (mint/burn zero-address legs, per-token slot override, ingest conserves balances via real SLOAD, cold skip); pipeline reorg_to purge-and-resync + ReorgConfig scope; reconcile correct+alarm / match / no-fetcher error; UniswapV3 adapter (Swap preserves slot0 unlocked/observation bits + absolute liquidity + cold-skip; Mint gross/net signs + initialize + bitmap flip + in/out-of-range global liquidity; Burn uninitialize+clear via same-block sequencing; cold tick skip; unregistered pool no-op). Verified the alloy event-ABI plumbing (sol! Swap/Mint/Burn construction + encode_log_data + decode_log round-trip) in isolation before committing. Tests are RED until the Phase 4 surface lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add StateUpdate::SlotMasked (new(old & !mask) | (value & mask)), the slot_masked constructor, the SkippedMask leaf record, and the StateDiff.skipped_masks field (+ merge / has_skipped / skipped_len). Apply arm in cache/mod.rs mirrors the SlotDelta arm: cold-aware RMW via write_slot_through. Re-export SkippedMask. Makes the tests/state_update.rs SlotMasked block green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- events/mod.rs (generic core): StateView, EventDecoder, DecoderRegistry, EventPipeline (ingest_logs log-by-log, reorg_to purge-and-resync, reconcile correct+alarm, derived_slots), BlockDigest, ReconcileReport, ReorgConfig, and the async LogSource/drive convenience. - events/erc20.rs (generic core): Erc20TransferDecoder -> Sub/Add balance SlotDeltas, skipping the zero-address leg; reuses parse_transfer. - events/uniswap_v3.rs (protocols): UniswapV3Decoder/UniswapV3Layout. Swap -> masked slot0 (preserves unlocked/observation bits) + absolute liquidity; Mint/Burn -> per-tick gross/net, initialized flag, tickBitmap bit, global liquidity (in-range), all cold-aware via StateView with a mask==MAX,value==0 could-not-compute marker. - cache/mod.rs: impl StateView for EvmCache; refactor verify_slots into verify_slots_inner (+fetched_ok count) and add reconcile_slots, which errors on a total fetch failure (honest-freshness) so the pipeline's reconcile surfaces an unverifiable re-read rather than a false all-clear. - lib.rs: pub mod events + re-exports. - tests/event_pipeline.rs: #[allow(dead_code)] on the unused tick_word test helper (no assertion/behaviour change) so clippy --all-targets -D warnings stays clean. All 24 event_pipeline tests + existing suites green on both feature configs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the Phase 4 deliverable on top of the sub-agent's src/ implementation: - examples/reactive_cache.rs (offline): register an ERC-20 + UniswapV3 decoder, ingest a block of logs (a Transfer + a Swap), show the BlockDigest, the preserved slot0 `unlocked` bit, a reconcile drift alarm+correction, and a reorg purge. Feature-gated (UniswapV3 adapter) with a fallback main. - benches/event_pipeline.rs (offline, registered in Cargo.toml): per-event decode cost (ERC-20 Transfer ~435ns, V3 Swap ~68ns, V3 Mint ~1.0us), ingest_logs decode+apply throughput (linear, ~112ns/log to 1000), reorg_to purge cost (~378us/1000 addrs). - CHANGELOG: Phase 4 `### Added` (event pipeline + adapters + SlotMasked). - ROADMAP: Phase 4 row -> Done + a "Landed on ..." section. - KNOWN_ISSUES: refreshed the Pillar B status (reader+writer halves done, live WS transport not) and recorded the §6.4 V3 fee-growth/oracle maintenance gap. - README: reactive_cache + event_pipeline rows. - tests/event_pipeline.rs: removed an unused tick_word helper (mine; the sub-agent had `#[allow(dead_code)]`-silenced it) + fmt. Independently verified green on both feature configs: fmt; clippy --all-targets (default) + --lib --no-default-features; cargo test (321 passed = 289 tests + 32 doctests) + --no-default-features (269 = 241 + 28); RUSTDOCFLAGS=-D warnings doc; cargo bench --no-run. The example runs offline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Validates the event → state pipeline against real EVM execution: run a swap in a ground-truth revm instance and replay ONLY its emitted logs into a twin cache, then assert the token balances and the packed pool slot0 (price/tick) match bit-for-bit. - fixtures/EventGroundTruthPool.sol + test_v3_pool_creation.hex: a faithful UniswapV3-pool stand-in whose slot0 is a Solidity struct with the identical field widths to UniswapV3Pool.Slot0 — so the *compiler* does the real bit-packing and our StateUpdate::SlotMasked is the thing under test. Its swap does real ERC-20 transfers (canonical Transfer logs) + a compiler-masked slot0 update + the canonical Swap event. - tests/event_ground_truth.rs (protocols-gated): deploy two MockERC20 tokens + the pool into a ground-truth cache (deterministic CREATE addresses), seed liquidity, execute a real swap (capture its 2 Transfer + 1 Swap logs), build the identical pre-swap state in a twin cache, feed only the logs through EventPipeline, and assert balances + slot0 + liquidity equal the ground truth. Explicitly checks the slot0 unlocked + observation-index bits survive the masked update. Result: the event-derived state reproduces the ground-truth EVM execution exactly (swapper/pool balances of both tokens, packed slot0, liquidity). Full suite green both feature configs (322 default incl. the new test, 269 --no-default-features); fmt, clippy --all-targets + --lib --no-default-features, doc, bench --no-run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 4: event pipeline + adapters (Pillar B.2)
docs/phase-5-spec.md is the build contract for the memoized-immutable-base snapshot design + overlay buffer/instance reuse, with locked decisions D1-D5. tests/cow_snapshot.rs is the red gate: a differential-equivalence property (create_snapshot must be read-indistinguishable from a retained create_snapshot_deep_clone reference after every mutation kind) plus the overlay reset()/buffer-reuse contract. Red until the implementation lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the O(total state) deep-clone create_snapshot with a two-tier copy-on-write snapshot: the cold layer-2 BlockchainDb index is flattened once into an immutable Arc<BaseState> (per-account storage shared by Arc), memoized across snapshots and rebuilt copy-on-write only for changed addresses; each snapshot folds just the hot layer-1 CacheDB delta over a cheap Arc::clone. Reads stay O(1) and lock-free (no persistent-map dep, D1); EvmSnapshot stays Send+Sync. create_snapshot is now &mut self (D5); create_snapshot_deep_clone is retained as the A/B baseline and the differential read-equivalence reference (D3). Every controlled layer-2 write marks the base dirty; an O(accounts) length-scan catches the append-only lazy-fetch growth. Overlay reuse (D4): EvmOverlay::reset() recycles an overlay across sims, and the 64KB shared-memory buffer is reused across calls via a Send-preserving take/reclaim (plain Vec field, method-local Rc) instead of re-allocating per build. Indicative: create_snapshot ~30-60x faster than the deep clone on the cold-index sweep; reset()-recycled fan-out beats fresh-overlay. Overseer review + adversarial-panel remediation: - mark_base_dirty in override_account_code_with_missing_target (D2 uniformity). - invalidate_snapshot_base() public re-honest hook + rustdoc warnings on blockchain_db()/backend() + a load-bearing-invariant note in refresh_base, for the one residual edge (an out-of-band same-length layer-2 overwrite through the public escape hatches); pinned by a guard test and recorded in KNOWN_ISSUES. Tests: 328 (default) / 275 (--no-default-features), incl. the cow_snapshot differential gate (COW == deep-clone after every mutation kind) and overlay-reuse contract. fmt + clippy (both configs) + doc + bench --no-run clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The per-context EVM working-memory buffer was hardcoded to 64 KB in two places
(EvmCache + EvmOverlay), tuned for a state-heavy upstream workload. Make it a
first-class knob:
- `SharedMemoryCapacity { Fixed(usize), Auto }`, default `Fixed(64_000)`,
configured via `EvmCacheBuilder::shared_memory_capacity`. `Fixed` pins the size
(general users running wide fan-outs of small sims can lower it to cut
per-overlay memory); `Auto` sizes from the chain state loaded at build time
(e.g. a bincode state file) — `loaded_slots * 16`, clamped to a 64 KB floor /
4 MiB ceiling.
- Resolution happens in the new `with_cache_capacity` constructor (the builder's
worker; `with_cache`/`new`/`from_backend` keep their signatures, defaulting to
Fixed(64_000)). `Auto` reads the post-load layer-2 slot count, so it captures
the maintain-list filter and any source, not just the raw file.
- The resolved size is stored on EvmCache, exposed via
`EvmCache::shared_memory_capacity()`, raised by `reserve_shared_memory`, and
copied onto every EvmSnapshot so snapshot-backed EvmOverlays pre-allocate the
same amount (overlay gains a `buffer_capacity` field; the hardcoded overlay
constant is removed).
Tests: a `resolve` heuristic unit test (floor/linear/ceiling, both feature
configs) and `tests/shared_memory_capacity.rs` end-to-end over the builder
(default, Fixed, Auto-with-no-state floor, and Auto sizing 10k loaded slots →
160_000). Full suite 335 (default) / 282 (--no-default-features); fmt + clippy
(both configs) + doc + bench --no-run clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`dropping_speculative_sim_aborts_before_queueing_correction` assumed the SpeculativeSim's drop-abort would win a race against the spawned multi-thread validator's first poll, which fails intermittently under full-suite parallel load. Replace the racy "called" atomic flag with a `Gate` (Mutex + Condvar): the fetcher blocks until the test releases the gate, and the test releases it only *after* `drop(sim)` sets the cancel flag. So the validator's fetch — and thus its post-fetch, correction-queuing checkpoint — can only complete once cancellation is already observable, regardless of scheduler interleaving. Drops the over-strict "fetcher never reached" assertion (the product guarantees a cancel seen at a checkpoint suppresses side effects, not that an in-flight fetch is skipped) and keeps the real invariants: no correction queued, no re-run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses three PR-review findings on the copy-on-write snapshot: - P2 (correctness): refresh_base's Case-4 partial rebuild cloned the previous code_by_hash and only added refreshed dirty-account codes, so a purged or recoded account left a stale hash. A direct EvmOverlay::code_by_hash(old_hash) then returned removed bytecode while create_snapshot_deep_clone (which rebuilds the index from current accounts) returned none — a read-equivalence violation and a slow memory leak. Fix: rebuild the index from the refreshed accounts via a shared `code_index` helper used by both build_base_full and the Case-4 path, so the two stay in lockstep; handles shared hashes (a hash survives iff some present account still carries it) and prunes unreferenced ones. - P3 (coverage): the differential gate now also compares code_by_hash for each probed account's code hash, and a new regression test (cow_code_index_matches_deep_clone_after_base_account_recoded) warms the base with bytecode, recodes the account, dirties it via a controlled per-address write (Case-4 partial rebuild), and asserts the old hash no longer resolves. Verified red against the pre-fix code. - P3 (docs): create_snapshot rustdoc no longer claims it "merges both layers into a single flat HashMap"; it now describes the memoized layer-2 base + layer-1 overlay fold and the &mut self receiver. Tests 329 (default) / 276 (--no-default-features); fmt + clippy (both configs) + doc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deflake the drop-abort freshness test with a deterministic gate
Configurable EVM shared-memory pre-allocation (SharedMemoryCapacity)
Address known issues
Phase 5: copy-on-write snapshots (Pillar A) + overlay reuse
Phase 3: state-update primitives (Pillar B.1) + relative read-modify-write
Phase 2: freshness core + optimistic verify-and-rerun loop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation
Notes