[WSLC] Stop state-aware exec failures from corrupting the output they report on

docs/isolation-session/state-aware-rust.md

@@ -190,10 +190,10 @@ wrong-nesting cases are rejected as `malformed_request`.
 **Config (none).** Exec uses only the cross-cutting `process` block on the
 top-level wire envelope (`commandLine`, `cwd`, `env`, `timeout`).
 
-**Output.** Stdout is the agent process's live-streamed output (the SDK
-discriminates this from a JSON envelope by exit code + stdout-parseability;
-the dispatcher never emits a JSON envelope on stdout for exec on success).
-The wxc-exec process exit code is the agent process's exit code.
+**Output.** Stdout is the agent process's live-streamed output; the dispatcher
+never emits a JSON envelope on stdout for a non-dry-run exec, so a failure's
+`{error}` envelope goes to stderr instead. The wxc-exec process exit code is the
+agent process's exit code.
 
 ### Stop
 

docs/state-aware-lifecycle/mxc-state-aware-sandbox-api.md

@@ -765,15 +765,18 @@ the executor CLI represents the same outcomes through stdout, stderr, and its ex
 | Phase / outcome | stdout | stderr |
 |---|---|---|
 | Non-exec (provision, start, stop, deprovision), success or failure | Single JSON envelope (`{result}` or `{error}`) | Buffered MXC diagnostics and warnings, when produced; may be empty |
+| Exec dry-run, success or failure | Single JSON envelope (`{result}` or `{error}`) | Buffered MXC diagnostics and warnings, when produced; may be empty |
 | Exec, dispatch succeeded | Script's stdout | Script's stderr plus buffered MXC diagnostics and warnings, when produced |
-| Exec, dispatch failed | Single JSON envelope (`{error}`) | Buffered MXC diagnostics and warnings, when produced; may be empty |
+| Exec, dispatch failed | Script's stdout, up to the point of failure | Single JSON envelope (`{error}`), after the script's stderr and any buffered MXC diagnostics |
 
-`stdout` is authoritative: for non-exec phases it carries exactly one envelope; for exec
-it carries either the script's output (success) or exactly one envelope (failure).
-`stderr` is informational. MXC routes its diagnostic logger output to `stderr` in
-state-aware mode so `stdout` remains parseable without sentinels. (One-shot dispatch
-keeps its existing `stdout` logger behaviour — the stricter routing applies to
-state-aware only.)
+`stdout` is authoritative: for non-exec phases and exec dry-run it carries exactly one
+envelope; for a non-dry-run exec it carries the script's output and nothing else. An exec
+can fail after the script has already streamed output, so its error envelope goes to
+`stderr` — appending it to `stdout` would leave neither the script's output nor the
+envelope parseable. `stderr` is otherwise informational. MXC routes its diagnostic logger
+output to `stderr` in state-aware mode so `stdout` remains parseable without sentinels.
+(One-shot dispatch keeps its existing `stdout` logger behaviour — the stricter routing
+applies to state-aware only.)
 
 When `--operation` is present, the executor selects the lifecycle contract before
 reading or decoding the configuration source. Every failure from that point onward —
@@ -821,18 +824,22 @@ type NonExecResponseEnvelope<TResult> = { result: TResult } | { error: ErrorEnve
 
 **Distinguishing exec dispatch-failure from script execution:**
 
-The SDK uses exit code plus stdout content:
-
-- `exitCode == 0`: the script ran and exited successfully. SDK constructs
-  `{stdout, stderr, exitCode}` from PTY / pipe events.
-- `exitCode != 0` AND stdout's entire content parses as a complete `{error: {...}}`
-  envelope: dispatch failed before the script ran; SDK surfaces the typed error.
-- `exitCode != 0` AND stdout does NOT parse as an envelope: the script ran and exited
+Which stream carries the typed error depends on whether the script could already
+have written output, so the SDK discriminates differently per phase:
+
+- Non-exec phases and exec **dry-run**: stdout is exactly one envelope. `{error}`
+  is the typed failure; `{result}` is success.
+- Non-dry-run **exec**, `exitCode == 0`: the script ran and exited successfully.
+  SDK constructs `{stdout, stderr, exitCode}` from PTY / pipe events.
+- Non-dry-run **exec**, `exitCode != 0` AND the final non-empty line of stderr
+  parses as a complete `{error: {...}}` envelope: dispatch failed; SDK surfaces
+  the typed error.
+- Non-dry-run **exec**, `exitCode != 0` otherwise: the script ran and exited
   non-zero. SDK constructs `{stdout, stderr, exitCode}`.
 
-Because MXC diagnostic output is routed to `stderr` in state-aware mode, this
-stdout-based discrimination has no false positives or negatives — the content is always
-either pure envelope or pure script output.
+For a non-dry-run exec, stdout is always pure script output, so it is never
+parsed for an envelope. The executor writes its error envelope on its own line,
+because a script's stderr may not end in a newline.
 
 `code` and `message` are always present. `code` is the machine-readable category a
 consumer branches on; `message` is the human-readable description, and for a failure

docs/wsl/wslc-state-aware.md

@@ -83,10 +83,12 @@ as `--sandbox-id`; the dispatcher derives the backend from the `wslc:` prefix (l
 For attached execution, a **successful** `exec` relays the script's raw stdout/stderr live from
 daemon frames and exits with the script's own exit code — it does **not** wrap the result in an
 envelope. Piped execution writes those same live frames into separate anonymous stdout/stderr
-pipes returned to the in-process SDK caller; no stdin pipe is returned. Callers discriminate
-attached dispatch failures via the exit code + whether stdout parses as an envelope. A timeout
-on the attached relay surfaces as a backend error because that path cannot return a typed timeout;
-the piped path reports it through its wait result.
+pipes returned to the in-process SDK caller; no stdin pipe is returned. An attached dispatch
+**failure** writes its `{error}` envelope to stderr, because the script's output may already own
+stdout by the time the failure is known — so stdout carries the script's output either way, and a
+caller reads the typed error from stderr. A timeout on the attached relay surfaces as a backend
+error because that path cannot return a typed timeout; the piped path reports it through its wait
+result.
 
 ### exec admission, cancellation, and failure containment
 
@@ -105,8 +107,14 @@ without creating the process.
 
 Live output uses bounded queues in both the daemon and the in-process native
 pipe bridge. If a caller does not drain stdout/stderr quickly enough, excess
-output is dropped and completion becomes an explicit backend error reporting
-truncation; incomplete output is never reported as successful.
+output is dropped and the run is reported as truncated; incomplete output is
+never reported as successful. A run that would otherwise have exited cleanly
+becomes an explicit backend error carrying the process's exit code in
+`details.exitCode`. A run that timed out or was cancelled keeps that outcome —
+truncation is reported alongside it rather than replacing it. For an in-process
+piped caller that pairing is only available on a clean exit: `ExecOutcome` has no
+field for a modifier, so a piped run that both timed out and truncated reports
+only the timeout.
 
 If process termination cannot be positively confirmed after creation, the
 container is quarantined and cannot be started or used for another exec. The

src/backends/wslc/common/src/daemon_client.rs

@@ -138,6 +138,14 @@ pub struct ExecResult {
 /// Terminal outcome of a streaming daemon exec.
 pub type DaemonExecOutcome = ExecTerminal;
 
+/// How a streaming exec finished, and whether the daemon had to drop live
+/// output on the way there.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct DaemonExecCompletion {
+    pub outcome: DaemonExecOutcome,
+    pub truncated: bool,
+}
+
 /// An exec request that the daemon has admitted and whose pipe is now carrying
 /// only the [`StreamFrame`] data phase.
 #[derive(Debug)]
@@ -150,27 +158,53 @@ impl AdmittedExec {
     pub(crate) fn read_to_completion(
         mut self,
         mut on_output: impl FnMut(OutStream, &[u8]),
-    ) -> DaemonResult<DaemonExecOutcome> {
+    ) -> DaemonResult<DaemonExecCompletion> {
+        let mut truncated = false;
         loop {
-            match read_frame::<StreamFrame>(&mut self.pipe)? {
-                StreamFrame::Stdout { data } => on_output(OutStream::Stdout, &data),
-                StreamFrame::Stderr { data } => on_output(OutStream::Stderr, &data),
-                StreamFrame::Exit { code } => return Ok(DaemonExecOutcome::Exited(code)),
-                StreamFrame::TimedOut => return Ok(DaemonExecOutcome::TimedOut),
-                StreamFrame::Cancelled => return Ok(DaemonExecOutcome::Cancelled),
+            let outcome = match read_frame::<StreamFrame>(&mut self.pipe)? {
+                StreamFrame::Stdout { data } => {
+                    on_output(OutStream::Stdout, &data);
+                    continue;
+                }
+                StreamFrame::Stderr { data } => {
+                    on_output(OutStream::Stderr, &data);
+                    continue;
+                }
+                StreamFrame::Truncated => {
+                    truncated = true;
+                    continue;
+                }
+                StreamFrame::Exit { code } => DaemonExecOutcome::Exited(code),
+                StreamFrame::TimedOut => DaemonExecOutcome::TimedOut,
+                StreamFrame::Cancelled => DaemonExecOutcome::Cancelled,
                 StreamFrame::Error { message } => {
-                    return Err(DaemonError::transport(format!("exec failed: {message}")))
+                    return Err(DaemonError::transport(format!(
+                        "exec failed: {message}{}",
+                        truncation_suffix(truncated)
+                    )))
                 }
                 StreamFrame::Stdin { .. } => {
                     return Err(DaemonError::transport(
                         "protocol error: daemon sent a Stdin frame to the client",
                     ))
                 }
-            }
+            };
+            return Ok(DaemonExecCompletion { outcome, truncated });
         }
     }
 }
 
+/// Suffix naming lost output, for an error whose own classification the caller
+/// still needs.
+pub(crate) fn truncation_suffix(truncated: bool) -> &'static str {
+    if truncated {
+        " (live output was truncated: it was not drained fast enough and a bounded output queue \
+         overflowed)"
+    } else {
+        ""
+    }
+}
+
 /// A typed failure from a daemon call. `Daemon` carries the daemon's stable
 /// [`ErrKind`] token so the state-aware backend can map it onto the matching
 /// `MxcError` code (e.g. `NotProvisioned` / `NotStarted`) without string
@@ -339,17 +373,31 @@ impl DaemonClient {
     pub fn exec(&self, config: ExecConfig) -> DaemonResult<ExecResult> {
         let mut stdout = Vec::new();
         let mut stderr = Vec::new();
-        let outcome = self.exec_streaming(config, |stream, data| match stream {
+        let completion = self.exec_streaming(config, |stream, data| match stream {
             OutStream::Stdout => stdout.extend_from_slice(data),
             OutStream::Stderr => stderr.extend_from_slice(data),
         })?;
-        let exit_code = match outcome {
+        let exit_code = match completion.outcome {
             DaemonExecOutcome::Exited(code) => code,
-            DaemonExecOutcome::TimedOut => return Err(DaemonError::transport("exec timed out")),
+            DaemonExecOutcome::TimedOut => {
+                return Err(DaemonError::transport(format!(
+                    "exec timed out{}",
+                    truncation_suffix(completion.truncated)
+                )))
+            }
             DaemonExecOutcome::Cancelled => {
-                return Err(DaemonError::transport("exec was cancelled"))
+                return Err(DaemonError::transport(format!(
+                    "exec was cancelled{}",
+                    truncation_suffix(completion.truncated)
+                )))
             }
         };
+        if completion.truncated {
+            return Err(DaemonError::transport(format!(
+                "exec output was truncated before it could be captured (the process exited \
+                 {exit_code})"
+            )));
+        }
         Ok(ExecResult {
             exit_code,
             stdout,
@@ -369,7 +417,7 @@ impl DaemonClient {
         &self,
         config: ExecConfig,
         on_output: impl FnMut(OutStream, &[u8]),
-    ) -> DaemonResult<DaemonExecOutcome> {
+    ) -> DaemonResult<DaemonExecCompletion> {
         self.admit_exec(config)?.read_to_completion(on_output)
     }
 

src/backends/wslc/common/src/daemon_protocol.rs

@@ -50,7 +50,7 @@ pub const MAX_EXEC_ID_BYTES: usize = 128;
 /// A new optional field counts: these structs do not deny unknown fields, so a
 /// daemon predating one drops it and acts on a request it only partly
 /// understood.
-pub const PROTOCOL_VERSION: u32 = 5;
+pub const PROTOCOL_VERSION: u32 = 6;
 
 fn deserialize_exec_id<'de, D>(deserializer: D) -> Result<String, D::Error>
 where
@@ -310,6 +310,8 @@ pub enum StreamFrame {
     /// Daemon→client: terminal frame; an explicit cancellation request was
     /// accepted and the process was confirmed terminated.
     Cancelled,
+    /// Daemon→client: live output was dropped; the terminal frame still follows.
+    Truncated,
     /// Daemon→client: terminal frame; the exec failed before or during the run.
     Error { message: String },
 }
@@ -555,6 +557,7 @@ mod tests {
         roundtrip(StreamFrame::Exit { code: 42 });
         roundtrip(StreamFrame::TimedOut);
         roundtrip(StreamFrame::Cancelled);
+        roundtrip(StreamFrame::Truncated);
         roundtrip(StreamFrame::Error {
             message: "spawn failed".to_string(),
         });

src/backends/wslc/common/src/state_aware.rs

@@ -33,7 +33,7 @@ use wxc_common::state_aware_backend::{
 use wxc_common::validator::validate_state_aware_network_policy_support;
 
 use crate::container_steps::OutStream;
-use crate::daemon_client::{DaemonClient, DaemonError, DaemonExecOutcome};
+use crate::daemon_client::{truncation_suffix, DaemonClient, DaemonError, DaemonExecOutcome};
 use crate::daemon_protocol::{
     DeprovisionConfig, ErrKind, ExecConfig, NetworkMode, ProvisionConfig, StartConfig, StopConfig,
     VolumeMount,
@@ -251,7 +251,7 @@ fn exec_relayed(client: DaemonClient, config: ExecConfig) -> Result<ExecHandle,
     // Best-effort: a failed local write must not mask the container's exit code.
     let stdout = std::io::stdout();
     let stderr = std::io::stderr();
-    let exit_code = {
+    let completion = {
         let mut out = stdout.lock();
         let mut err = stderr.lock();
         let result = client.exec_streaming(config, |stream, bytes| match stream {
@@ -276,17 +276,22 @@ fn exec_relayed(client: DaemonClient, config: ExecConfig) -> Result<ExecHandle,
         result.map_err(map_daemon_error)?
     };
 
-    let exit_code = match exit_code {
+    let exit_code = match completion.outcome {
+        DaemonExecOutcome::Exited(code) if completion.truncated => {
+            return Err(truncated_run_error(code))
+        }
         DaemonExecOutcome::Exited(code) => code,
         DaemonExecOutcome::TimedOut => {
             return Err(MxcError::backend_error(format!(
-                "WSLc exec timed out after {timeout_ms}ms"
+                "WSLc exec timed out after {timeout_ms}ms{}",
+                truncation_suffix(completion.truncated)
             )))
         }
         DaemonExecOutcome::Cancelled => {
-            return Err(MxcError::backend_error(
-                "WSLc relayed exec was cancelled unexpectedly",
-            ))
+            return Err(MxcError::backend_error(format!(
+                "WSLc relayed exec was cancelled unexpectedly{}",
+                truncation_suffix(completion.truncated)
+            )))
         }
     };
 
@@ -349,25 +354,24 @@ fn exec_piped(
                         OutStream::Stderr => stderr_writer.write(bytes),
                     })
                 })
-                .map(|outcome| match outcome {
-                    DaemonExecOutcome::Exited(code) => ExecOutcome::Exited(code),
-                    DaemonExecOutcome::TimedOut => ExecOutcome::TimedOut,
-                    // `SandboxProcess::kill` is a request followed by reaping;
-                    // there is no distinct cancelled variant in `ExecOutcome`.
-                    DaemonExecOutcome::Cancelled => ExecOutcome::Exited(-1),
-                })
-                .map_err(map_daemon_error);
-            let result = match result {
-                Ok(ExecOutcome::Exited(_))
-                    if stdout_overflow.has_overflowed() || stderr_overflow.has_overflowed() =>
-                {
-                    Err(MxcError::backend_error(
-                        "WSLc live output was truncated because the caller did not drain the \
-                         synthesized stdout/stderr pipes fast enough",
-                    ))
-                }
-                other => other,
-            };
+                .map_err(map_daemon_error)
+                .and_then(|completion| {
+                    // Either end of the bridge can drop output: the daemon's
+                    // queue on the way out, or these pipes if the caller is slow.
+                    let truncated = completion.truncated
+                        || stdout_overflow.has_overflowed()
+                        || stderr_overflow.has_overflowed();
+                    match completion.outcome {
+                        DaemonExecOutcome::Exited(code) if truncated => {
+                            Err(truncated_run_error(code))
+                        }
+                        DaemonExecOutcome::Exited(code) => Ok(ExecOutcome::Exited(code)),
+                        DaemonExecOutcome::TimedOut => Ok(ExecOutcome::TimedOut),
+                        // `SandboxProcess::kill` is a request followed by reaping;
+                        // there is no distinct cancelled variant in `ExecOutcome`.
+                        DaemonExecOutcome::Cancelled => Ok(ExecOutcome::Exited(-1)),
+                    }
+                });
             stdout_writer.close();
             stderr_writer.close();
             let _ = done_tx.send(result);
@@ -458,6 +462,18 @@ fn connect_daemon() -> Result<DaemonClient, MxcError> {
     })
 }
 
+/// The process finished, but the caller's copy of its output is incomplete.
+///
+/// Reported as a failure so a short capture is never mistaken for a whole one,
+/// with `details.exitCode` keeping the code the process produced.
+fn truncated_run_error(exit_code: i32) -> MxcError {
+    MxcError::backend_error(
+        "WSLc live output was truncated: it was not drained fast enough and a bounded output \
+         queue overflowed",
+    )
+    .with_details(serde_json::json!({ "exitCode": exit_code }))
+}
+
 /// Map a typed [`DaemonError`] onto the matching wire-format [`MxcError`] code.
 fn map_daemon_error(err: DaemonError) -> MxcError {
     match err {
@@ -611,6 +627,27 @@ mod tests {
         NetworkIngressPolicy, ProxyAddress, ProxyConfig,
     };
 
+    /// The exit code is unrecoverable once dropped, so failing the call must not
+    /// discard it.
+    #[test]
+    fn a_truncated_run_reports_the_exit_code_it_produced() {
+        let error = truncated_run_error(3);
+        assert_eq!(
+            error.code,
+            wxc_common::mxc_error::MxcErrorCode::BackendError
+        );
+        assert_eq!(
+            error.details.as_ref().and_then(|d| d.get("exitCode")),
+            Some(&serde_json::json!(3))
+        );
+    }
+
+    #[test]
+    fn an_untruncated_run_adds_no_suffix() {
+        assert_eq!(truncation_suffix(false), "");
+        assert!(truncation_suffix(true).contains("truncated"));
+    }
+
     #[test]
     fn backend_key_matches_wire_format() {
         assert_eq!(