Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -305,11 +305,12 @@ leave you answering one call blind.

One question, one answer, then the process ends. `sandbx tui` takes the same
flags and draws that turn on a screen instead of streaming it, where ctrl-c or
escape ends it mid-flight — `--session` keeping the rounds it finished but not
the one it stopped in, and a running `bash` command killed, where any other tool
call finishes first. When the turn ends the screen goes and the transcript is
printed in its place, so the answer stays in your scrollback. It needs a
terminal on stdout, so it refuses a
escape ends it mid-flight — `--session` keeping the rounds it finished, and the
one it stopped in if it was running tools, with the output of the call it
stopped on withheld, and a running `bash` command killed and a call that only
reads left running, where a `write` or `edit` finishes first. When the turn
ends the screen goes and the transcript is printed in its place, so the answer
stays in your scrollback. It needs a terminal on stdout, so it refuses a
piped run. `--approve call` asks on the screen, in rows under the transcript,
and one key answers: `y`, `n` or `a`, escape or ctrl-c refusing the call and
ending the turn ([#225](https://github.com/danczw/sandbx/issues/225)):
Expand All @@ -335,7 +336,7 @@ resuming one another user can write is refused — [SECURITY.md](SECURITY.md).
|---|---|
| `0` | the model finished its answer |
| `2` | a bound cut the turn short, named on stderr — `--max-tokens` or `--max-rounds`. Out of rounds, one more request answers the turn with no tool, so stdout usually holds a summary a blank line below what arrived before the cap; `--no-wrap-up` skips it, leaving nothing at all if the model opened with a tool call ([#204](https://github.com/danczw/sandbx/pull/204)). Under `tui` it also means you interrupted the turn |
| `3` | the terminal went away, so the turn ended there and nobody was watching it. Under `--approve call` that is the terminal it asks on: that call and the ones behind it were refused and no further request was sent. What the turn did before is on stdout and in `--session`. Under `tui` it is the screen or the keyboard hanging up mid-turn, which ends the turn where it stood: `--session` keeps the rounds it finished and loses the one it stopped in, so a hangup before the first round finishes stores nothing, the prompt included ([#271](https://github.com/danczw/sandbx/issues/271)). In the wrap-up round after `--max-rounds` it loses only the wrap-up. The audit trail of what it touched is released to stderr rather than dropped unwritten, so `2>` a file to keep it: where stderr is the terminal that died, the write fails and it goes with it |
| `3` | the terminal went away, so the turn ended there and nobody was watching it. Under `--approve call` that is the terminal it asks on: that call and the ones behind it were refused and no further request was sent. What the turn did before is on stdout and in `--session`. Under `tui` it is the screen or the keyboard hanging up mid-turn, which ends the turn where it stood: `--session` keeps the rounds it finished and the one it stopped in if it was running tools, and loses it if the model was still writing, so a hangup before the first round finishes streaming stores nothing, the prompt included ([#271](https://github.com/danczw/sandbx/issues/271)). In the wrap-up round after `--max-rounds` it loses only the wrap-up. The audit trail of what it touched is released to stderr rather than dropped unwritten, so `2>` a file to keep it: where stderr is the terminal that died, the write fails and it goes with it |
| `64` | sandbx would not take the command line, so no turn was started and nothing was spawned. The complaint is on stderr, as clap wrote it. `EX_USAGE`, and the same code under every subcommand |
| anything else | it failed before or during the turn, with the reason on stderr |

Expand Down
52 changes: 32 additions & 20 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,15 +329,21 @@ architecture the filter gates on.
Whatever a tool read is in it: a token in a config, a `.env` under
`--allow-read`, a key a `bash` printed. No encryption, no redaction, no expiry,
no deletion. A turn stopped on `tui`'s screen stores the prompt and every
round it finished, under a stop naming an interrupt or a hung-up terminal, and
loses the round it stopped in whole: whatever it streamed, including the part
of the answer you read, and every tool call it asked for. The calls it had
already run did run, and one still running is killed if it is a `bash` and
finishes if it is not, but the transcript records the effect of neither. A
stop before the first round finishes stores
nothing, the prompt included, and leaves no trace in the session that a call
ran ([#271](https://github.com/danczw/sandbx/issues/271)). Where a round was
stored, resuming says so on stderr; the model is told nothing of it. What the
round it finished, under a stop naming an interrupt or a hung-up terminal. A
stop while the model is still writing loses that round whole: whatever it
streamed, including the part of the answer you read, and every tool call it
asked for. A stop while its tools run keeps the round with every call
answered: a call that had settled when the stop was raised keeps its output;
one still running then, or settling as it was raised, is killed if it is a
`bash`, left running if it only reads and finishes if it writes, and is
stored with its output withheld, since you stopped before reading it and a
resume would send it to the provider; one behind it is stored as not run.
What a withheld call did still stands, and a killed or abandoned one is
stored as cancelled with anything it had done possibly remaining. A stop
before the first round finishes streaming stores nothing, the prompt
included ([#271](https://github.com/danczw/sandbx/issues/271)). Where a round
was stored, resuming says so on stderr, and the model sees those results and
nothing more of the stop. What the
turn showed is printed back to the terminal once the screen is gone, and the
call a stop stood on is named there as it ended, run or cancelled, but never
with its output. Where the terminal is
Expand Down Expand Up @@ -374,17 +380,23 @@ architecture the filter gates on.
instead — a cap on files visited and bytes read for `grep` and `find`, the
single file or directory for the rest
([`ToolLimits`](crates/sandbx-tools/src/tool_limits.rs)) — on a thread that
cannot be cancelled, so one read on a stalled filesystem hangs indefinitely.
Nothing else is capped: no CPU bound, no memory bound, no limit on processes
spawned. A fork bomb is unbounded while the call lasts; bounded is that it does
not outlive it.
- **Only a spawned command can be interrupted.** A stop on `tui` (ctrl-c,
escape, or a hung-up terminal) kills a running `bash` command with the kill a
timeout makes, and starts no call behind it. The six in-process tools cannot
be cancelled: one already running finishes, so a `write` under way lands, and
the stop waits for it — for as long as a stalled filesystem takes. Nothing a
killed command had already done is undone
([#26](https://github.com/danczw/sandbx/issues/26)).
cannot be cancelled, so one read on a stalled filesystem hangs indefinitely,
holding that thread until the process exits; a stop stops waiting for it,
below. Nothing else is capped: no CPU bound, no memory bound, no limit on
processes spawned. A fork bomb is unbounded while the call lasts; bounded is
that it does not outlive it.
- **A `write` or `edit` cannot be interrupted, and a read is abandoned, not
stopped.** A stop on `tui` (ctrl-c, escape, or a hung-up terminal) kills a
running `bash` command with the kill a timeout makes, and stops waiting for a
running `read`, `ls`, `grep` or `find`: the call is left on its blocking
thread with its result unread, a `grep` also stopping before its next file
(not during the directory walk before it). It starts no call behind either. A
`write` or `edit` already running finishes, so a `write` under way lands, and
the stop waits for it. An abandoned read goes on until it finishes or the
process exits, so a stop is not a promise that nothing is still reading: its
access was recorded in the audit trail before it began, and a stalled one
holds a blocking thread until exit. Nothing a killed command had already done
is undone ([#26](https://github.com/danczw/sandbx/issues/26)).
- **A bound on the harness's own loop is not a containment claim.** A turn is
bounded — `TurnLimits::max_rounds` caps the requests one turn may make of the
model, `TurnLimits::stream_timeout` how long one of them may spend streaming
Expand Down
18 changes: 11 additions & 7 deletions context/decision-approval-gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@ that ask it and implement it.
In `answer_calls`, between resolving the name and `spawn_blocking`.

A blocking task cannot be cancelled (#26): dropping the `JoinHandle` leaves the
task running to completion, and only a `bash` has a kill to reach for besides.
task running to completion. A stop abandons that handle only for a call that
reads, which leaves nothing behind it; a `write` or `edit` is waited for, and
that is the case this position guards.
So a gate consulted *concurrently* with the call — `select!` on a decision and a
`JoinHandle` — would answer "denied" about a `write` that had already landed.
Before the spawn is the only position where a refusal means anything.
Expand Down Expand Up @@ -201,10 +203,12 @@ case without `--approve call`; with it, a hangup lands on a question and is the

It also keeps less. The abort is a `TurnStop`, so the turn returns and its
messages, usage and `withheld` are stored. A `tui` stop ends the turn through
`run_turn_until`, which keeps the rounds before it and drops the one it lands
in, so the round the operator was watching is stored nowhere, for a hangup as
for an interrupt (#271). A tool call already running is killed if it is a `bash`
and finishes if it is not (#26), on both surfaces alike.
`run_turn_until`, which keeps the rounds before it, and the one it lands in if
that was running tools, every call answered and the one under way with its
output withheld. A round the model was still writing is stored nowhere, for a
hangup as for an interrupt (#271). A tool call already running is killed if it
is a `bash`, left running if it only reads, and finishes if it writes (#26), on
both surfaces alike.

## Deny by default, and the honest claim

Expand Down Expand Up @@ -364,8 +368,8 @@ descriptor no task feeds, and nothing else is in flight during `approve`, since
- **A stop given earlier outranks an `a`.** Before the blanket is read, the gate
asks whether the turn was interrupted or the terminal has gone, and refuses or
aborts at once. An `a` was the operator's word about a tool, not a way past a
stop they gave since: a covered call would otherwise run, and an in-process
call cannot be cancelled (#26).
stop they gave since: a covered call would otherwise run, and a `write` or
`edit` cannot be cancelled (#26).
- **Refusal outranks an answer already pressed.** An interrupt beats a queued
`y`, and a hangup beats both. Each leaves the call refused, the direction a
doubtful reading must lean.
Expand Down
2 changes: 1 addition & 1 deletion crates/sandbx-cli/src/agent_gate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ fn report(call: Settled<'_>) -> String {
ToolError::TimedOut { after, .. } => {
format!("{head} — timed out after {after:?} and was killed")
}
ToolError::Cancelled { .. } => format!("{head} — cancelled and killed"),
ToolError::Cancelled { .. } => format!("{head} — cancelled"),
},
}
}
Expand Down
8 changes: 4 additions & 4 deletions crates/sandbx-cli/src/agent_gate_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -359,17 +359,17 @@ fn a_policy_refusal_is_not_reported_as_a_call_that_ran() {
);
}

/// The operator who pressed ctrl-C reads this line, so it must say the command was killed
/// and not that it ran or that a limit was hit.
/// The operator who pressed ctrl-C reads this line, so it must say the call was cancelled
/// and not that it ran or that a limit was hit. Not "killed": a `grep` is stopped, not killed.
#[test]
fn a_cancelled_call_is_reported_as_killed() {
fn a_cancelled_call_is_reported_as_cancelled() {
let error = ToolError::Cancelled {
subject: "run `sleep 80`".to_string(),
};

assert_eq!(
reported_bash("sleep 80", Outcome::Errored(&error)),
"bash sleep 80 — cancelled and killed"
"bash sleep 80 — cancelled"
);
}

Expand Down
9 changes: 5 additions & 4 deletions crates/sandbx-cli/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,11 @@ pub enum Command {
/// screen goes, and what it showed is printed in its place.
///
/// Interrupting costs two things, which sandbx prints again once the screen goes:
/// `--session` keeps the rounds the turn finished but not the one it stopped
/// in, and a tool already running is dealt with first: a `bash` command is
/// killed, any other tool call finishes, and the process waits for it before
/// it exits. A turn interrupted while it is being summarised after
/// `--session` keeps the rounds the turn finished, and the one it stopped in
/// if it was running tools, and a tool already running is dealt with first:
/// a `bash` command is killed, a call that only reads is left running, and a
/// `write` or `edit` finishes, the process waiting for it before it exits.
/// A turn interrupted while it is being summarised after
/// `--max-rounds` loses only the summary.
///
/// `--show-thinking` reads differently here: it draws the reasoning in the
Expand Down
12 changes: 9 additions & 3 deletions crates/sandbx-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -122,13 +122,19 @@ fn report(result: Result<i32, impl Error>, failure: u8) -> ExitCode {
fn block_on(
future: impl Future<Output = Result<i32, sandbx_cli::AgentError>>,
) -> Result<i32, sandbx_cli::AgentError> {
tokio::runtime::Builder::new_current_thread()
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
// Named at the one call site that can produce it rather than by a blanket
// `From`, which would label any later io error as this one.
.map_err(sandbx_cli::AgentError::Runtime)?
.block_on(future)
.map_err(sandbx_cli::AgentError::Runtime)?;
let result = runtime.block_on(future);

// Dropping a runtime waits for every blocking task, and a read-only tool call a stop
// abandoned may be parked in one for as long as a stalled filesystem takes (#26). Only
// those are left by now, every other call having been awaited.
runtime.shutdown_background();
result
}

#[cfg(test)]
Expand Down
19 changes: 19 additions & 0 deletions crates/sandbx-cli/src/main_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,3 +49,22 @@ fn an_argv_sandbx_accepts_is_not_a_usage_error() {
Cli::try_parse_from(["sandbx", "sandbox-run", "--", "true"])
.expect("a well-formed sandbox-run");
}

/// A read abandoned on a stop is still parked on its blocking thread, and dropping the
/// runtime waits for every such thread. The process must not (#26).
#[test]
fn the_runtime_does_not_wait_for_a_blocking_task_left_running() {
let started = std::time::Instant::now();

let result = super::block_on(async {
tokio::task::spawn_blocking(|| std::thread::sleep(std::time::Duration::from_secs(3)));
Ok(0)
});

assert!(matches!(result, Ok(0)));
assert!(
started.elapsed() < std::time::Duration::from_secs(1),
"the runtime waited {:?} for a task nobody awaits",
started.elapsed()
);
}
17 changes: 10 additions & 7 deletions crates/sandbx-cli/src/tui.rs
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,9 @@ impl Tui {
matches!(self.run.approve, Approve::Call).then(|| Modal::new(&pane, keys.answers()));

// Inside the loop rather than raced against it, so the rounds before the stop come
// back to be stored. The stop kills a running `bash` and waits for an in-process
// tool, so the call it stood on settles, and is drawn, before the turn returns (#26).
// back to be stored. The stop kills a running `bash`, leaves a read-only call running
// and waits for a `write` or `edit`, so the call it stood on settles, and is drawn,
// before the turn returns (#26).
let mut seen = None;
let first = run_turn_until(
&mut open,
Expand Down Expand Up @@ -336,7 +337,7 @@ fn held(stderr: Device) -> Option<sandbx_logging::Held> {
/// What the turn's futures came to, before any of it is accounted for.
enum Driven {
/// Stopped before the turn had finished, which keeps the rounds before the one it
/// stopped in.
/// stopped in, and that one too if it was running tools.
Stopped(Stopped, TurnOutcome),
/// Stopped in the wrap-up round of a turn that ran out after `usize` rounds, which is
/// kept as the cap left it.
Expand Down Expand Up @@ -404,7 +405,7 @@ fn stored(stopped: Stopped) -> StoredStop {

/// The code a stop exits with, and its line. `wrapping` is whether it found the turn in
/// its wrap-up round, which keeps every round before it, where any other stop loses the
/// round it lands in.
/// round it lands in if the model was still writing it.
///
/// A hangup has its own line and not the interrupt's: the operator did not stop this turn,
/// and telling them they did is the one reading of a hangup that is certainly wrong (#264).
Expand All @@ -427,9 +428,11 @@ fn stop(stopped: Stopped, wrapping: bool) -> Ending {
)
} else {
format!(
"{how}. Under `--session` the rounds it finished are kept and the one it stopped \
in is lost. A `bash` command already running is killed; any other tool call \
already running finishes first, and what it did stays (#26)"
"{how}. Under `--session` the rounds it finished are kept, and the one it stopped \
in is kept if it was running tools and is lost if the model was still writing. A \
`bash` command already running is killed, and a call that only reads is left \
running and not waited for; a `write` or `edit` already running finishes first, \
and what it did stays (#26)"
)
};

Expand Down
2 changes: 1 addition & 1 deletion crates/sandbx-cli/src/tui_modal.rs
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ impl<S: Shows, R: Replies> Operator for Modal<S, R> {
fn ask(&mut self, call: ToolCall<'_>) -> ApprovalDecision {
// Before the `a` that covers a tool: that was the operator's word about the tool,
// not a way past a stop they gave since, or a terminal that has gone. A covered
// call would otherwise run, and an in-process call cannot be cancelled (#26).
// call would otherwise run, and a `write` or `edit` cannot be cancelled (#26).
match self.replies.stopped() {
Some(Reply::Interrupted) => return deny(INTERRUPTED),
Some(Reply::Closed) => return abort(CLOSED),
Expand Down
15 changes: 14 additions & 1 deletion crates/sandbx-cli/src/tui_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -371,15 +371,28 @@ fn every_stop_keeps_the_rounds_it_finished() {
let (_, stop) = turn.expect("the finished rounds are kept");
assert_eq!(stop, stored, "{stopped:?}");
assert_eq!(mid_turn, code, "{stopped:?}");
// Both stops are named: one in the stream loses its round, one in a tool call keeps it.
assert!(
said[0].contains("the one it stopped in is lost"),
said[0].contains("is lost if the model was still writing"),
"{said:?}"
);
assert!(
said[0].contains("is kept if it was running tools"),
"{said:?}"
);
// What the operator is told about a call under way: the mechanism's whole claim.
assert!(
said[0].contains("`bash` command already running is killed"),
"{said:?}"
);
assert!(
said[0].contains("a call that only reads is left running and not waited for"),
"{said:?}"
);
assert!(
said[0].contains("a `write` or `edit` already running finishes first"),
"{said:?}"
);
}
}

Expand Down
Loading
Loading