[rybosome]

Executive summary

mic run now shows progress while it streams instead of printing nothing until the end. In a terminal, each case prints one line when its last trial finishes, and a status line at the bottom is redrawn in place. Outside a terminal (CI, redirects), Mic prints plain lines with no escape codes. -v shows every case with its score means, and -q turns progress off. The stdout summary and --json output are unchanged.

mic run 126050df · necessity · 3 trials/case · concurrency 8 · timeout 30s
  ✓✓✓    5.8s  row 2
  ✓✗✓   30.4s  row 6 (refund-edge) · trial 2 task_failed (TimeoutError)
  ✓✓!    7.1s  row 9 · trial 3 scoring_failed in recall (KeyError)
⠼ 14 cases · 38 trials done · 4/8 running · 1 failed · 0.7/s · 0:52

After Ctrl-C, the status line becomes cancelling · waiting for 2 running trials to return (rows 2, 3). Cases that were cancelled or never run are then listed, followed by mic: interrupted.

Why

A multi-minute run looked hung and then printed everything at once. This is option D from the design discussion: one line per case in scrollback plus a sticky status footer. The design recommendations were accepted: no running score means in the footer, sys.stdout/sys.stderr wrapped while the footer is shown, and summary formatting changes deferred to a separate PR.

Design and implementation

• src/mic/reporters/live.py uses only the standard library and has three parts:
  • ProgressState implements the internal RunObserver from Add internal run progress observer hooks to the engine #30. It turns notifications into finished lines and status text. It holds a case only until that case's last admitted trial finishes, so memory is bounded by concurrency, not dataset size.
  • LiveDisplay redraws from a daemon thread about 10 times a second, so elapsed time and the slow-trial callout stay current even if a task blocks the event loop.
    • Each redraw erases the footer, prints pending lines, then draws the footer.
    • The footer is cut to the terminal width (wide characters counted as two columns) so it never wraps.
    • While it's shown, sys.stdout (if it's a terminal) and sys.stderr are wrapped: the footer is erased before application output, and incomplete lines are held until their newline (or 4 KiB).
  • PlainDisplay prints new lines every 0.5s, a status line every 30s, and a status line when the run starts cancelling or finishing.
• How the mode is chosen:
  • Live if stderr is a terminal, TERM isn't dumb, and ANSI support is available. On Windows this is enabled through SetConsoleMode, falling back to plain if that fails.
  • Color only in live mode, and only when NO_COLOR is unset.
  • ASCII glyphs when stderr's encoding can't represent the Unicode ones.
• What's displayed:
  • Case labels are dataset content, so control characters are replaced with ?. A label can't move the cursor or inject escape codes.
  • Failures show the status, the scorer, and the exception type only, the same as recorded evidence.
• CLI (cli.py): run now calls the private execute(..., observer=...) through the existing run_sync helper, inside a with progress(...) block. Closing that block clears the footer before the summary, Evidence …, a config error, or mic: interrupted prints. -q and -v are mutually exclusive.
• Docs: a new "Progress output" section in docs/cli.md covers behavior, the non-terminal mode, flags, NO_COLOR, the ASCII fallback, cancellation, and the logging-handler limitation. The README run example now shows the progress lines.

Verification and confidence

Risk	Evidence
Case lines wrong or misordered	Pure ProgressState tests check exact strings for: trials finishing out of order; grouped failures with scorer names; several evaluations and sources; verbose means with missing scores; quiet mode showing only problems; rejected records; cases flushed at close that were cancelled or never run.
Status line misleading	Exact status text for each phase: starting, reading, running against --limit, the slow-trial callout past both thresholds, cancelling (with and without running trials), finishing sinks.
Footer wraps, or labels inject escapes	fit handles wide characters; a label containing \x1b[2J\n is printed safely; the footer stays within the requested width.
Application output corrupts the terminal	A LiveDisplay test on a fake terminal: application output appears whole after an erase, unterminated output is completed on close, sys.stderr is restored, and no status line is left behind.
A display failure affects the run	A stream that raises on write stops the display without raising. With ProgressState hooks forced to raise, the run still completes normally and the error text isn't shown.
stdout or --json contaminated	A POSIX pty test with stderr on a pseudo-terminal and stdout piped: stdout parses as JSON, the terminal shows the erase sequence, header and case lines, and nothing is left drawn after the final erase.
Non-terminal output	A subprocess test: progress goes to stderr with no escape codes, the failure line names RuntimeError but not the exception message, and stdout is unchanged. -q leaves stderr empty, and -q -v exits with code 2.

• By hand, using a pty harness and scratch evals (a slow eval with failures and print()s in task code, and a two-evaluation suite), I checked:
  • the live redraw;
  • prints from task code landing whole;
  • -v;
  • a 40-column terminal (the footer truncates);
  • suite marks and labels;
  • Ctrl-C (the "waiting for N running trials (rows …)" footer, then the remaining cases, then mic: interrupted, exit 130).
• Automated checks:
  • scripts/verify.py: 778 tests, 0 failures. Strict and authoring typing, lint, format and the coverage gates all pass.
  • scripts/verify_packaging.py passes.
  • live.py has 93% in-process coverage. The uncovered lines are the Windows console path and terminal-detection fallbacks; the live path also runs in the pty subprocess test, whose coverage isn't counted.

Compatibility and risks

• CLI: adds -q/--quiet and -v/--verbose to run, and prints progress on stderr. Scripts that require an empty stderr from mic run will now see progress lines; they can pass -q. The stdout summary, --json, exit codes and evidence are unchanged.
• No changes to the Python API, artifact formats or dependencies. The display uses private _runtime modules from Add internal run progress observer hooks to the engine #30.
• Known limitation (documented): a logging handler created before the run holds the original stderr and bypasses the wrapper, so a log line can share a line with the footer until the next redraw (within 100ms).