[rybosome]

Executive summary

mic run now captures what application code writes to sys.stdout/sys.stderr during a run, so it no longer bleeds into the live progress stream or the stdout summary/--json manifest:

• Default: hidden.
• --output DIR: recorded to DIR/output.jsonl (new mic-output-v1 format). Each record carries the evaluation, source, row, trial, and scorer that wrote it.
• --show-output: shown on stderr above the status line, each line prefixed with where it came from (row 2 · trial 1 · recall │ …). It can't be combined with -q.

run()/arun() gain capture_output: bool = False, so library callers keep their streams unless they opt in. Version bumped to 0.6.0.

Why

The streaming TUI wrapped sys.stdout/sys.stderr only so that prints wouldn't land on the status line. Application output still flooded the progress display, and prints to stdout could corrupt mic run --json | jq. There was no way to keep the stream clean, and no way to keep the output for later debugging.

Design and implementation

• Attribution (src/mic/_runtime/capture.py): a ContextVar[Origin] is set around each trial in the engine worker and narrowed around each scorer in run_case.
  • Async tasks, child tasks, and asyncio.to_thread inherit the trial.
  • CallbackPool.invoke now runs sync callbacks under contextvars.copy_context(). run_in_executor doesn't carry context the way to_thread does; without this, all sync task output would be untagged.
  • Writes with no origin (sources, sinks, bare threads) are run-level.
• Routing: while a capturing run is active, one OutputCapture replaces sys.stdout/sys.stderr with a TextIOBase stream.
  • Partial lines are buffered per (origin, stream). Concurrent trials therefore can't splice into each other's lines, which the old single shared buffer allowed.
  • Complete text is recorded and/or echoed; held text flushes at the end of each trial or scorer phase, or at 4 KiB.
  • The original streams are restored in finally, including on cancellation.
  • After close, kept references pass through to the original stream.
• Recording: one combined, capture-ordered JSONL file rather than per-case files or fields on trial events.
  • This keeps memory bounded, keeps events.jsonl unchanged, and preserves output from trials that hang or time out (late writes keep their trial attribution).
  • Output from before the recorder acquires the directory is held and written first.
  • Recording stops at 64 MiB with one truncated record.
  • output.jsonl is closed before run.json is published.
  • A failed write disables recording and adds an output run failure (exit 1) without losing results.
• Display: the CLI passes an internal echo callback from the live display.
  • Lines are prefixed from progress state, keep only the text after \r (progress bars), and replace control characters, so they can't move the cursor.
  • Progress lines that are already due are written first, so the header stays first.
  • Errors in the echo are dropped, like observer errors.
• Concurrency guard: only one run per process can capture at a time; a second capturing run raises ConfigurationError. Trials from a concurrent run that doesn't capture still write to the real streams, because their origin carries a different run_id.
• examples/chatty.py demonstrates all three modes.

Verification and confidence

Risks exercised:

• Attribution under concurrency: sync tasks with trials=3 and concurrency=4; async tasks with child tasks and to_thread; bare threads counted as run-level; a barrier that forces four trials to interleave partial lines.
• Data integrity: for each origin, concatenated text equals what it wrote; schema validation of every record; the truncation cap.
• Failure paths: a failed output.jsonl write gives exit 1 with results kept and no exception text; late output after a timeout is still attributed; echo errors never fail the run.
• Resource ownership: streams are restored after success and after cancellation; overlapping capturing runs are refused; pass-through for other runs.
• Stream behaviour: isatty() is false, fileno() raises UnsupportedOperation, bytes raise TypeError, and close() is a no-op.
• Display: prefix formatting and sanitization; the status line is erased before echoed output; ordering in plain mode.
• CLI end to end: output hidden by default with a clean --json stdout and the expected output.jsonl; --show-output prefixes; -q --show-output exits 2.
• Library default: mic.run() without capture_output still writes to stdout.

Results:

• uv run python scripts/verify.py: tests, coverage gates, strict and authoring typing, lint, and format all pass.
• report_javascript and report_interactions could not run locally because Node isn't installed. This PR doesn't touch the report; CI covers them.
• uv build plus scripts/verify_packaging.py: pass, at 0.6.0.
• A manual run under a pseudo-terminal with --show-output showed prefixed output above a correctly redrawn status line, with stdout holding only the summary.

Compatibility and risks

• Changes in default behaviour (CLI): application prints during mic run no longer appear unless you pass --show-output; with --show-output, both streams appear on stderr. The release notes should say this.
• Public API: an addition only (capture_output=False on run/arun). The internal progress() now yields (observer, echo).
• Format: new mic-output-v1 / output.jsonl. mic-run-v4 and mic-event-v1 are unchanged.
• Privacy: output.jsonl holds application output word for word, with no redaction. It is documented in artifacts and reporting docs, and is never sent to sinks or included in HTML reports.
• Limits (documented): only Python-level writes are captured. File-descriptor writes (subprocesses, C extensions) and logging handlers created before the run bypass capture. While capturing, fileno() is unsupported.
• No new dependencies.