Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,12 @@ graph TD

## Extension Points

Future contributors can extend the debugger in several ways:
Contributors can extend the debugger in the following ways:

- **New Commands**: Add new subcommands to the `cli` module to support different debugging workflows.
- **Custom Inspectors**: Create new inspectors in the `inspector` module to track specific resources or state types (e.g., event logs).
- **UI Enhancements**: Improve the TUI in the `ui` module to provide better visualization of complex data structures or execution paths.
- **Backend Adapters**: (Future) Extend `ContractExecutor` to support connecting to remote RPC nodes for live debugging.
- **Backend Adapters**: Extend `ContractExecutor` to support connecting to remote RPC nodes for live debugging.
- **Plugin System**: Plugins are loaded from TOML manifests (`plugin.toml`) and can register new CLI subcommands, intercept execution hooks, and publish custom output. See [`docs/plugin-api.md`](docs/plugin-api.md) and [`docs/architecture-plugins.md`](docs/architecture-plugins.md) for the full plugin lifecycle and API surface.
- **Remote Debug Server**: The `server` / `remote` command pair exposes the debugger over a line-delimited JSON wire protocol (TCP), supporting token authentication and native TLS. New transports or protocol extensions can be added in `src/server/`. See [`docs/architecture-remote.md`](docs/architecture-remote.md) for the design.
- **Batch Executor**: The `BatchExecutor` in `src/batch.rs` runs multiple argument sets in parallel using Rayon. It can be extended to support new result aggregation strategies, custom assertion formats, or streaming output. See [`docs/design/batch-execution-design.md`](docs/design/batch-execution-design.md) for the canonical design reference.
12 changes: 6 additions & 6 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,12 @@ For precise control, use `{"type": "...", "value": ...}`:

Filter large storage outputs by key pattern using `--storage-filter`:

| Pattern | Type | Matches |
|------------------|--------|----------------------------------------|
| `balance:*` | Prefix | Keys starting with `balance:` |
| `re:^user_\d+$` | Regex | Keys matching the regex |
| `total_supply` | Exact | Only the key `total_supply` |

```bash
# Prefix match: keys starting with "balance:"
soroban-debug run --contract token.wasm --function mint \
Expand Down Expand Up @@ -213,12 +219,6 @@ An exported trace includes versioning, metadata, and full execution state:
}
```

| Pattern | Type | Matches |
|------------------|--------|----------------------------------------|
| `balance:*` | Prefix | Keys starting with `balance:` |
| `re:^user_\d+$` | Regex | Keys matching the regex |
| `total_supply` | Exact | Only the key `total_supply` |

### Interactive Command

Start an interactive debugging session:
Expand Down
2 changes: 2 additions & 0 deletions docs/batch-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ The batch execution feature allows you to run the same contract function with mu

See the [Scenario Cookbook](scenario-cookbook.md) for examples of complex test patterns that can be used in batch execution.

For the architectural rationale, parallelism model, and extension points, see the [Batch Execution Design](design/batch-execution-design.md) document.

## Usage

```bash
Expand Down
195 changes: 195 additions & 0 deletions docs/design/batch-execution-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
# Batch Execution Design

> **Canonical design reference.** For user-facing usage instructions see
> [`docs/batch-execution.md`](../batch-execution.md). For the implementation
> summary produced during the initial feature work see
> [`BATCH_EXECUTION_SUMMARY.md`](../../BATCH_EXECUTION_SUMMARY.md).

## Goals

1. Allow users to run the same contract function against many argument sets in a single invocation.
2. Execute those argument sets in parallel so wall-clock time scales with CPU cores, not input count.
3. Aggregate individual results into a deterministic, human- and machine-readable summary.
4. Keep batch mode a composable layer on top of the existing single-run execution path rather than a separate code path.

## Non-Goals

- Interactive / step-through debugging of individual batch items (use single-run mode for that).
- Ordered / sequential execution (ordering is not guaranteed between items; each item is independent).
- Distributed execution across multiple hosts.

---

## Architecture

### Components

```
CLI (--batch-args <file>)
BatchExecutor (src/batch.rs)
├─ load_batch_file() – deserialise JSON array of BatchItems
├─ execute_batch() – par_iter() over items → Vec<BatchResult>
│ │
│ └─ execute_single() – delegates to ContractExecutor (single-run path)
├─ summarize() – fold Vec<BatchResult> → BatchSummary
└─ display_results() – render to stdout (text or JSON)
```

### Key Types

| Type | Location | Purpose |
|------|----------|---------|
| `BatchItem` | `src/batch.rs` | Deserialised entry from the JSON file: `args`, optional `expected`, optional `label` |
| `BatchResult` | `src/batch.rs` | Outcome of a single execution: result value, pass/fail status, duration, error |
| `BatchSummary` | `src/batch.rs` | Aggregate counts (total, passed, failed, errors) and total duration |
| `BatchExecutor` | `src/batch.rs` | Orchestrates the above; receives a shared reference to `ContractExecutor` |

---

## Parallelism Model

Batch execution uses **Rayon's work-stealing thread pool** (`par_iter()`).

```
items: Vec<BatchItem>
▼ rayon::par_iter()
┌──────────────────────────────────────┐
│ Thread 0 │ Thread 1 │ Thread N │ (Rayon pool, N = logical CPU cores)
│ item[0] │ item[1] │ item[k] │
│ ↓ │ ↓ │ ↓ │
│ exec_single│ exec_single│ exec_single│
└──────────────────────────────────────┘
▼ collect()
results: Vec<BatchResult> (order matches input order after collect)
```

**Thread safety constraints:**

- `ContractExecutor` is instantiated fresh per item inside `execute_single`. Each invocation gets its own `soroban-env-host` `Host` instance, so there is no shared mutable state between parallel executions.
- The JSON input file is loaded once on the main thread and shared as an immutable slice.
- Result collection via `collect()` on a `ParallelIterator` is deterministic in output order (Rayon preserves index order for `collect`).

---

## Result Aggregation

After all executions complete, `summarize()` produces a `BatchSummary`:

```rust
BatchSummary {
total: usize, // number of BatchItems run
passed: usize, // items where actual result == expected (or no expected set)
failed: usize, // items where actual result != expected
errors: usize, // items that panicked or returned an execution error
duration: Duration,// wall-clock time from start of par_iter to end of collect
}
```

### Pass/Fail Logic

| Condition | Classification |
|-----------|----------------|
| No `expected` field set and execution succeeded | **Passed** |
| `expected` field set and `actual == expected` (string comparison) | **Passed** |
| `expected` field set and `actual != expected` | **Failed** |
| Execution returned an error or panicked | **Error** |

---

## Input File Format

```json
[
{
"args": "[\"Alice\", \"Bob\", 100]",
"expected": "void",
"label": "transfer 100 units"
},
{
"args": "[\"Charlie\", \"Dave\", 0]",
"label": "zero-value transfer (no assertion)"
}
]
```

- The top-level value **must** be a JSON array.
- `args` is a **JSON string** (not a nested array) containing the argument list exactly as it would be passed to `--args`.
- `expected` is an optional string compared against the serialised return value.
- `label` is optional and used only for human-readable output.

---

## CLI Integration

```
soroban-debug run \
--contract path/to/contract.wasm \
--function transfer \
--batch-args path/to/batch.json
```

When `--batch-args` is provided:

1. The `run` command delegates to `BatchExecutor::load_batch_file()`.
2. `execute_batch()` is called instead of the normal single-run path.
3. Results are printed to stdout; exit code is `0` if all items pass, non-zero otherwise.

`--batch-args` is mutually exclusive with `--args` in the CLI argument parser.

---

## VS Code Extension

The extension exposes batch mode via the `batchArgs` field in `launch.json`:

```json
{
"name": "Soroban: Batch Test",
"type": "soroban",
"request": "launch",
"contractPath": "${workspaceFolder}/target/wasm32-unknown-unknown/release/contract.wasm",
"entrypoint": "transfer",
"batchArgs": "${workspaceFolder}/tests/batch_inputs.json"
}
```

When `batchArgs` is set the adapter passes `--batch-args` to the spawned `soroban-debug server` process. Breakpoints and stepping are skipped in batch mode; use single-run mode to debug individual failing cases.

---

## Performance Characteristics

| Batch size | Approximate speed-up vs sequential |
|-----------|--------------------------------------|
| 10 items | ~10× |
| 100 items | ~50× (plateau depends on core count) |

Speed-up is roughly linear up to the Rayon thread-pool size (default: logical CPU count) then plateaus. Contract execution time dominates; scheduling overhead is negligible for all practical batch sizes.

---

## Extension Points

| What to extend | Where |
|---------------|-------|
| New result aggregation strategies (e.g., percentiles, histogram) | `BatchExecutor::summarize()` in `src/batch.rs` |
| Custom assertion formats (e.g., regex match, numeric tolerance) | `BatchItem` struct + `execute_single()` comparison logic |
| Streaming / incremental output | `BatchExecutor::display_results()` — replace `collect` + render with a channel-based sink |
| Different parallelism back-ends (e.g., async, remote dispatch) | Replace `par_iter()` in `execute_batch()` with the desired scheduler |

---

## Related Documents

- [`docs/batch-execution.md`](../batch-execution.md) — user guide with CLI examples and output format
- [`docs/batch-result-buckets.md`](../batch-result-buckets.md) — result classification details
- [`BATCH_EXECUTION_SUMMARY.md`](../../BATCH_EXECUTION_SUMMARY.md) — implementation summary from the original feature PR
- [`ARCHITECTURE.md`](../../ARCHITECTURE.md) — system-level overview including the Batch Executor extension point
38 changes: 38 additions & 0 deletions docs/remote-debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,44 @@ soroban-debug remote \
--args '["user1", 100]'
```

### Connect from VS Code (Attach)

You can also use the VS Code extension to attach to the remote debugger server. Add a configuration of type `"soroban"` and `"request": "attach"` to your `.vscode/launch.json`.

Below is a complete snippet matching the TLS client setup above:

```json
{
"version": "0.2.0",
"configurations": [
{
"name": "Soroban: Attach to Remote Debugger",
"type": "soroban",
"request": "attach",
"host": "localhost",
"port": 9229,
"contractPath": "${workspaceFolder}/contract.wasm",
"entrypoint": "increment",
"args": ["user1", 100],
"token": "your-token-here",
"tlsCert": "${workspaceFolder}/path/to/client-cert.pem",
"tlsKey": "${workspaceFolder}/path/to/client-key.pem"
}
]
}
```

#### Configuration Properties

- **host**: The hostname or IP address of the remote debug server (defaults to `127.0.0.1`).
- **port**: The TCP port the server is listening on.
- **contractPath**: Path to the contract WASM compiled locally.
- **entrypoint**: The function name to execute.
- **args**: JSON array of arguments for the entrypoint.
- **token**: The authentication token required by the server.
- **tlsCert**: Optional path to your client TLS certificate file.
- **tlsKey**: Optional path to your client TLS private key file.

### Timeouts and Retries (network instability)

Remote sessions often run across CI, containers, or flaky links. The remote client supports deterministic timeouts and controlled retries for **idempotent** operations.
Expand Down
Loading