Skip to content

terminal_capture_screen returns empty screen on Windows ConPTY #361

Description

@Bahtya

Summary

terminal_capture_screen consistently returns empty/blank screen content when using cmd.exe sessions on Windows via ConPTY. The tool executes without errors and the agent can create sessions, send input, and read raw output, but the screen model shows no visible content.

Steps to Reproduce

  1. Start kestrel gateway with WebSocket channel on Windows
  2. Connect via WebSocket and send a message like:

    "Create a terminal session (shell: cmd.exe, 80x24). Send 'echo HELLO_TUI_TEST'. Wait 3 seconds with terminal_read_output. Then use terminal_capture_screen."

  3. Observe agent response — terminal_read_output returns raw output text, but terminal_capture_screen returns empty lines.

Expected Behavior

capture_screen should show the cmd.exe banner and command output (e.g., HELLO_TUI_TEST) in the screen snapshot.

Actual Behavior

Agent consistently reports "The screen appears empty" after capture_screen. This happens across multiple test scenarios (basic echo, screen change detection, resize). Tested with kestrel v0.10.4 on Windows 11.

Test Evidence

End-to-end WS test results (3 test cases, all show empty screen):

T1_basic_shell: Agent created ts-1, sent echo HELLO_TUI_TEST, read_output drained buffer successfully, 
  but capture_screen returned empty. Agent retried with wait_for_screen_change — still empty.

T2_screen_change: capture_screen showed "The screen appears empty — cmd.exe may still be initializing."

T3_resize: 3 parallel sessions, all showed empty screens despite multiple retries.

Root Cause Hypothesis

The pump_output thread feeds PTY output to both decoded_buffer (for read_output) and emulator (for capture_screen). Since read_output works but capture_screen doesn't, the issue is likely:

  1. ConPTY output encoding: Windows ConPTY may emit output in a form that the VT parser (vte crate) consumes without updating the screen model cells — e.g., the output may use VT sequences that our TerminalScreen::apply() doesn't handle, leaving cells at default (empty).
  2. Race condition: pump_output acquires the emulator lock, feeds bytes, then releases. But the VT parser in TerminalEmulatorHandle batches ops internally (self.ops Vec). If capture_screen is called before flush_parser() processes the batched ops, the screen model may not reflect recent output.
  3. cmd.exe banner sequences: Windows cmd.exe initial banner may use uncommon VT sequences (e.g., Windows-specific console APIs translated by ConPTY) that the vte parser skips, leaving the screen in an inconsistent state.

Key Files

  • crates/kestrel-tools/src/builtins/terminal/session.rs:542-621pump_output thread
  • crates/kestrel-tools/src/builtins/terminal/session.rs:393-405capture_screen() method
  • crates/kestrel-tools/src/builtins/terminal/emulator.rs — VT parser and emulator handle
  • crates/kestrel-tools/src/builtins/terminal/screen.rs:680-695snapshot() method
  • crates/kestrel-tools/src/builtins/terminal/screen.rs:334-347row_to_string() method

Environment

  • Windows 11 (Build 22631)
  • kestrel v0.10.4
  • ConPTY (portable-pty crate)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions