Repository navigation
op-reth,op-devstack: behavioral tests for the proofs CLI subcommands - #23437
einar-oplabs wants to merge 7 commits into
Conversation
Wiz Scan Summary
To detect these findings earlier in the dev lifecycle, try the Wiz Code extension for VS Code, JetBrains, or Visual Studio. |
| logger := n.p.Logger().New("name", n.name, "chain", n.chainID, "running", n.sub != nil, "subcommand", subcommand) | ||
| logger.Info("Running op-reth proofs subcommand", "cmd", n.execPath, "args", args) | ||
| started := time.Now() | ||
| output, err := exec.CommandContext(n.p.Ctx(), n.execPath, args...).CombinedOutput() |
There was a problem hiding this comment.
OS Command Injection (CWE-78)
More Details
OS command injection is a critical vulnerability that allows an attacker to execute arbitrary commands on the system. This can lead to a full system compromise, as the attacker can potentially gain complete control over the application and the underlying system.
The vulnerability arises when user input is used to construct commands or command arguments that are then executed by the application. This can occur when user-supplied data is passed directly to functions that execute OS commands, such as exec.Command, exec.CommandContext, syscall.ForkExec, or syscall.StartProcess.
If an attacker can inject malicious code into these commands, they can potentially execute any command on the system, including installing malware, stealing data, or causing a denial of service. This vulnerability can have severe consequences, including data breaches, system compromise, and unauthorized access to sensitive information.
To avoid this vulnerability, user input should never be used directly in constructing commands or command arguments. Instead, the application should use a hardcoded set of arguments and validate any user input to ensure it does not contain malicious code.
| Attribute | Value |
|---|---|
| Impact | |
| Likelihood |
Remediation
OS command injection is a critical vulnerability that allows an attacker to execute arbitrary commands on the system. This can lead to a full system compromise, as the attacker can potentially gain complete control over the application and the underlying system. The vulnerability arises when user input is used to construct commands or command arguments that are then executed by the application.
To fix this issue, user input should never be used directly in constructing commands or command arguments. Instead, the application should use a hardcoded set of arguments and validate any user input to ensure it does not contain malicious code. Additionally, it is recommended to use language-specific facilities for executing commands, which provide better input validation and sanitization.
Code examples
// VULNERABLE CODE - User input is directly used in constructing the command
userInput := getInput() // Assume this comes from user input
cmd := exec.Command("/bin/sh", "-c", userInput)
err := cmd.Run()// SECURE CODE - Command and arguments are hardcoded, and user input is validated
userInput := getInput() // Assume this comes from user input
if !isValidInput(userInput) {
// Handle invalid input
return
}
cmd := exec.Command("/bin/sh", "-c", "echo", userInput)
err := cmd.Run()Additional recommendations
- Follow the principle of least privilege and run the application with the minimum required permissions.
- Use input validation and sanitization techniques to ensure that user input does not contain malicious code.
- Implement secure coding practices and follow security standards such as OWASP Top 10 and CWE.
- Consider using language-specific facilities for executing commands, such as os/exec in Go, which provide better input validation and sanitization.
- Regularly update and patch the application and its dependencies to address known vulnerabilities.
- Implement logging and monitoring mechanisms to detect and respond to potential security incidents.
Rule ID: WS-I011-GO-00026
To ignore this finding as an exception, reply to this conversation with #wiz_ignore reason
If you'd like to ignore this finding in all future scans, add an exception in the .wiz file (learn more) or create an Ignore Rule (learn more).
To get more details on how to remediate this issue using AI, reply to this conversation with #wiz remediate
22e19a8 to
a0d5d60
Compare
|
This pr has been automatically marked as stale and will be closed in 5 days if no updates |
`op-reth proofs unwind --target T` documented removing every block after T but passed T to the store's `unwind_history`, which removes T too, so the documented recovery for ethereum-optimism/protocol-team#464 left the store one block short of reth. The command now keeps the target as the store's latest block, allows a target equal to the earliest block, and refuses a target below it instead of exiting 0. `op-reth proofs prune` rejects a batch size of 0 and exits non-zero when the pruner fails, through the new `OpProofStoragePruner::try_run`; `PrunerError::BlockNotFound` names the block. New integration test target `rust/op-reth/crates/cli/tests/proofs_cli/` seeds a real datadir through the CLI's own `EnvironmentArgs` and drives `Cli` in-process for init, unwind (every target edge, a pruned body, a store ahead of reth), prune, backfill and snapshot init|drop. Devstack: `stack.ProofsHistoryUnwind`, `sysgo.OpReth.UnwindProofsHistory` and DSL `L2ELNode.UnwindProofsHistory` for a stopped node, with an e2e smoke test in `rust/op-reth/tests/proofs/core` that unwinds a stopped follower, restarts it, and requires the store to resume right above the target before sweeping the window against the sequencer. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 697689c) (cherry picked from commit 7baa99d5b53a1ba8f2b91044d29fb2755be7a70b)
`proofs unwind` refusals are a typed `UnwindError`, with the doc and message for a target below the earliest block pointing at a backfill rather than at a re-init; `proofs prune` treats an empty storage as nothing to prune, like unwind; `OpProofStoragePruner::with_batch_size` raises a zero batch to one and `PrunerError` renders through thiserror with the block it could not find; `strum` is only needed by the trie crate's metrics. `alloy-eips/serde` is propagated for zepter. The CLI tests assert on the typed errors, verify the storage's state root at the new tip after an unwind and at the new earliest block after a prune through `snapshot init`, and pin prune and backfill on an empty storage. The pruner tests cover a block reth does not hold and the batch size clamp. The public historical-proofs tutorial now says the unwind target stays. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 6f87a18) (cherry picked from commit 1fbdf0e48824f018c9ca90f58ad5a74e4d5af282)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 35c09c3) (cherry picked from commit 40ac8fe7664e72aea0b376ca9338a82855120be3)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit 2d9becf) (cherry picked from commit 25eee69e7412a21d1951f02c618d4bc2b6612bc0)
…runer behavior `prune_fails_when_the_new_earliest_block_is_not_in_reth` expects the pruner to target `latest - window` and fail on a block reth does not hold. The reth 2.6 pin's pruner (#23358) caps the target at reth's last block, prunes up to it and succeeds. No consistent datadir satisfies both: `last_block_number` and `block_hash` read the same static-file headers segment, so a block at or below the last block number always has a hash. The scenario now lives in one fixture helper with a test per expectation: the failing one stays active on this pin, and the capped one is ignored naming #23358, which also makes the failing one obsolete. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> (cherry picked from commit a0d5d60) (cherry picked from commit 9acfb95d21d749000055317e22978d9e36d81b55)
The pruner resolves the new earliest block through the chain reader that also reads the Finish checkpoint, so the block-not-in-reth test mocks that reader, and the bump's pruner tests take the renamed try_run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r has Since #23358 the pruner caps the new earliest block at reth's Finish checkpoint instead of failing on a block reth does not hold, so the capped case runs and the failing case goes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
a0d5d60 to
3328f77
Compare
Why
No
op-reth proofssubcommand had a behavioral test, only the CLI snapshot.proofs unwind --target Tdocumented removing every block after T but passed T to the store'sunwind_history, which removes T as well, so the documented recovery for ethereum-optimism/protocol-team#464 (proofs unwind --target <reth tip>) left the store one block short of reth.What
op-reth proofs unwindnow matches its documentation. The store'sunwind_history(to)removesto.block.number..=latestand keeps(to.block.number - 1, to.parent)as the tip (the trait doc now states that contract); the command passestarget + 1with the target's hash as the parent, so the target stays and becomes the store's latest block. A target equal to the earliest block is allowed and leaves[earliest, earliest]. A target below the earliest block is refused with a non-zero exit (it was a silent exit 0 with "Nothing to unwind" while the store kept history above the target); the message points at a backfill past the target, or at a re-init when reth no longer holds those blocks. A target at or above the latest block stays a no-op, and so does an empty store. The refusals are a typedUnwindError; the pruned-body refusal is unchanged.op-reth proofs prunerejects--proofs-history.prune-batch-size=0at parse time and exits non-zero when the pruner fails:OpProofStoragePruner::runlogs and swallows errors, so the command reported success for a prune it did not do; the command now calls the newtry_run. An empty store is nothing to prune, as it is nothing to unwind. In the trie crate,with_batch_size(0)is raised to one (the pruner could not advance with it), andPrunerErrorrenders through thiserror, soBlockNotFoundnames the block instead of printing the variant name.Behavior changes an operator can see
proofs unwind --target Tleaves the store atT; it left it atT - 1.proofs unwindwith a target below the store's earliest block exits non-zero; it exited 0.proofs pruneexits non-zero when the pruner fails and rejects a batch size of 0; it exited 0 in both cases. On an empty store it exits 0 instead of failing withNoBlocksFound.The first is the fix the #464 recovery depends on; the others turn silent failures into exits a runbook can check. None of them changes a flag, so this is not marked
op-reth!:; say so if you want theBREAKING CHANGEparagraph for the release notes.Rust:
rust/op-reth/crates/cli/tests/proofs_cli/(40 tests)Every test seeds a real datadir through the CLI's own
EnvironmentArgs(genesis from a JSON file with every OP fork active, storage v2) with a short chain committed the way reth's persistence commits blocks (append_blocks_with_statepluswrite_trie_updates; every block changes an EOA balance and a contract storage slot, so every block has account and storage changesets), then drivesCliin-process the way the binary does and asserts onget_proof_windowand the snapshot anchor read from the MDBX store. Refusals are asserted on the typed errors (UnwindError,PrunerError,SnapshotError,BackfillError,clap::Error). After every unwind and prune the store's state at the new tip or earliest block is checked against reth's header by building the snapshot there (SnapshotInitJobvalidates the state root), so a store that moved its pointers without its state would fail.init--proofs-history.skip-backfillleaves[tip, tip]; idempotent; a second init only extends the backfill (a wider window extends, a narrower one never prunes)unwind(store[3, 8])prune(store[2, 8])BlockNotFound(9)and leaves the window; an empty store is a no-opbackfill(store[8, 8])NoBlocksFoundsnapshot init|drop(store[4, 8]without a snapshot)e2e:
rust/op-reth/tests/proofs/core/unwind_test.goTestProofsFollowerServesTheWindowAfterItsStoreIsUnwoundAndRestarted(threshold 7, index path, 20-block window): the follower is stopped cleanly,op-reth proofs unwind --target <head - 5>runs against its datadir, op-reth restarts with its CL still down and must re-index the datadir up to its tip (the ExEx's startupsync_to), every proof of the window is swept against the sequencer, then the CL comes back, traffic runs ten more blocks and the sweep repeats over the live window.Seams
stack.ProofsHistoryUnwind{Target},stack.ProofsHistoryUnwinder, andProofsHistoryInitReportrenamed toProofsHistoryCommandReportsince both subcommands report the same thing.sysgo.OpReth.UnwindProofsHistory(sharesrunProofsHistorySubcommandwith the init), forwarded by the preset frontend; DSLL2ELNode.UnwindProofsHistory(target)requires a stopped node and fails the test on error.OpProofStoragePruner::try_runreturns the pruner's result instead of logging it; the trie crate'sstrumdependency is now optional undermetrics, its only user.Follow-ups, not in this PR
OpProofsProviderRw::unwind_historytakes aBlockWithParentof the first removed block but no implementation reads its hash; the CLI passesB256::ZEROthere (reth does not hold that block when the store is ahead). Narrowing the parameter to the new tip'sBlockNumHashwould remove the sentinel. The in-memory implementation also lacks the MDBX guards (no-op above latest, error at or below earliest).docs/public-docs/node-operators/tutorials/reth-historical-proofs.mdxnow says the unwind target stays and a target below the earliest block fails; the generated CLI reference is rebuilt at release time.Matrix
Storage version v2 only (the CLI's
--storage.v2default; v1 datadirs are legacy). Unwind targets as listed above. The bump-only init guards (lagging trie, moved tip) do not exist on develop and are not tested here.Local verification
Rust, from
rust/(debug op-reth test profile, shared target dir):Finding while stabilizing the target: with every test seeding its own datadir in parallel threads of one process, one or two tests per run failed in
EnvironmentArgs::initorMdbxProofsStorage::newwithCannot allocate memory (12)(3 of 4 runs of the compiled binary). reth and the proofs storage each map an 8 TB MDBX geometry, and a dozen of them exhaust the process's address space. CI's nextest runs every test in its own process, butcargo testdoes not, so the fixture holds a process-wide lock for the lifetime of each seeded datadir; the target then passed three runs in a row.Go:
e2e (
rust/op-reth/tests/proofs/core,TestProofsFollowerServesTheWindowAfterItsStoreIsUnwoundAndRestarted, run with the recipe's env vars by hand against a debug op-reth built from this branch, sincejust test-e2e-sysgohard-codes a release binary):The test took head 24 and target 19 while the store already served [5, 25];
proofs unwind --target=19took 0.52 s and removed 20..25; the restarted follower's datadir tip was 25, the store's first re-executed block was 20 and it was back at [5, 25] ten seconds after the start with the CL still down; the first sweep covered [5, 25] and the second, after ten more blocks of traffic, [35, 52], every block for the four accounts,pruned_during_sweep=0.Not run locally: the op-geth validator variant (
OP_DEVSTACK_PROOF_VALIDATOR_EL=op-gethskips the test) and a kona-node CL (the preset runs op-node).Notes for the #464 and Notion recovery text
The recovery
proofs unwind --target <reth tip>now does what the text says: the store ends at reth's tip. The text does not need to change; the previous behavior left the store at tip - 1. A target below the store's earliest block now fails instead of exiting 0, which the runbook should mention if it ever lists it.Stacked on #23434, #23429, #23428, #23421 and #23418.
Closes #23393
🤖 Generated with Claude Code
CI follow-up: the prune-ahead-of-reth case is split by pruner behavior
prune_fails_when_the_new_earliest_block_is_not_in_rethis pin-specific: the pruner of the reth 2.6 bump with #23358 caps the target at reth's last block, so there the prune succeeds at block 8 instead of failing on block 9. No consistent datadir makesblock_hash(target)Nonewhilelast_block_number() >= target(both read the static-file headers segment on both pins), so one failing case cannot hold on both. The scenario now lives inseeded_storage_ahead_of_rethwith a test per expectation: the failing one stays active on develop, andprune_stops_at_reths_last_block_when_the_storage_is_ahead_of_rethis added#[ignore]d naming #23358; when this PR lands on the bump, drop the#[ignore]from the capped test and delete the failing one, which pins a failure #23358 removes (no pin sniffing). Verified withcargo test -p reth-optimism-cli --test proofs_cli prune: on develop the active test passes and the ignored one fails under--ignored; on the bump evaluation tree the reverse (the ignored one passes under--ignored, the active one fails).