Skip to content

feat: compound start, viewport screenshot, openclaw skill (0.2.8)#41

Merged
Shadow-Azure merged 17 commits into
mainfrom
feat/start-shell-screenshot-openclaw
Jun 19, 2026
Merged

feat: compound start, viewport screenshot, openclaw skill (0.2.8)#41
Shadow-Azure merged 17 commits into
mainfrom
feat/start-shell-screenshot-openclaw

Conversation

@Shadow-Azure

Copy link
Copy Markdown
Owner

Problem

Three issues found during v0.2.7 release testing:

  1. OpenClaw can't read screenshots. The installed cli-box SKILL.md is identical for every harness, so OpenClaw-driven agents save cli-box screenshot output to the CWD. OpenClaw is sandboxed to /tmp/openclaw/ and cannot read/send those images.
  2. Compound start fails. cli-box start "cd /x && claude -r" errors out — the daemon's PTY spawn treats the whole string as the executable name (CommandBuilder::new(command)), so &&, ;, |, and cd are never interpreted.
  3. Screenshots show the wrong content. cli-box screenshot on a long claude task returns the oldest lines, not the visible viewport. The renderer's buffer-fallback loop reads buffer.getLine(0..rows) (scrollback top) instead of the viewport (baseY..baseY+rows). There is also no way to scroll the capture window, and no way to dump the whole session.

Solution

1. OpenClaw screenshot path (329d992)

  • packages/cli-box-skill/installer/shared.mjs: new targetSpecificNote(id); installSkillToTargets composes readBundledSkill() + targetSpecificNote(id) per target. Only the OpenClaw copy gains a note instructing agents to save screenshots under /tmp/openclaw/. Claude/OpenCode bodies are byte-identical to before.

2. Compound start via shell (e669c5a, e216755)

  • crates/cli-box-core/src/process/mod.rs: pure needs_shell + wrap_shell_command + prepare_spawn helpers. When the command contains a space or shell metacharacters, it is re-spawned as zsh -lc "<full line>"; plain commands pass through unchanged. Wired at the top of the macOS spawn_cli_with_size, so CLI/MCP/Electron-new-dialog all benefit. .app paths are unaffected (mode split happens upstream); the displayed command in list/inspect still shows the original (stored separately in InstanceKind).

3. Viewport screenshot + scroll + full-session text

  • Renderer buffer-fallback extracted to a pure module anchored at buffer.baseY with a clamped scroll offset (a8cfe45, dca44ce). captureToPng(offset) uses the live xterm canvas only at offset 0, else renders the buffer window.
  • Daemon: screenshot?scroll=N|top=true and a new /box/{id}/scrollback route (5bc769c).
  • Renderer WS: capture_request.scroll and new scrollback_request/response over the screenshot socket; pure readScrollback text extractor (dc8e1b6).
  • CLI: screenshot --up N / --top, new scrollback --id subcommand, plus MCP tools (77eb8ca).
  • E2E: tests/e2e-compound-start-screenshot.sh (b9fb981). Polish: dead-code removal + negative-offset guard test (dd80f59). Version 0.2.8 / skill 0.2.2 (5cd9b24).

Spec: docs/superpowers/specs/2026-06-17-start-shell-screenshot-openclaw-design.md
Plan: docs/superpowers/plans/2026-06-17-start-shell-screenshot-openclaw.md

Test Plan

  • UT (Rust): needs_shell / wrap_shell_command / prepare_spawn truth table (8 tests)
  • UT (JS): installer — OpenClaw body contains /tmp/openclaw/; Claude/OpenCode do not (skill suite 18/18)
  • UT (TS): renderBufferToPng viewport(baseY)+offset+clamp+negative (4 tests); readScrollback trim/range/clamp (4 tests); captureToPng background/dimensions migrated to the extracted module
  • IT (Rust): /box/{id}/screenshot?scroll=/?top= parse; /box/{id}/scrollback route exists (daemon_integration)
  • cargo test / cargo clippy -D warnings / cargo fmt --check / pnpm typecheck / pnpm vitest run — all green
  • cli-box screenshot --help shows --up/--top; cli-box scrollback --help shows its options
  • Release-build verification (pending sh release.sh + daemon restart): the compound-start E2E needs the rebuilt 0.2.8 daemon on PATH. A stale 0.2.4 release daemon is currently running in this environment, so the live E2E exercised the old spawn path (the 8 unit tests + 2 ITs verify the new logic). After a release rebuild + daemon restart the E2E passes.
  • Empirical claude rendering check (pending release testing): verify whether scrollback returns claude's full conversation (vs. current screen only) and whether --up reveals older content — to determine if claude uses the alternate-screen buffer (which would limit terminal-scrollback history).

Not merging — PR left open per project workflow. Release build (sh release.sh) and the manual scenarios in tests/release_test.md happen in the release-testing phase.

🤖 Generated with Claude Code

ZN-Ice and others added 13 commits June 17, 2026 22:33
Spec for v0.2.8 covering three test-found issues:
- per-target OpenClaw SKILL.md note (screenshots to /tmp/openclaw)
- shell wrapping of compound Sandbox daemon already running on port 15801
Creating sandbox: mode=cli, command=zsh
Sandbox created: id=9e684f01, pty_pid=Some(1004), window_id=Some(170387)
Daemon port: 15801
Waiting for renderer done
Waiting for terminal
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal...
Waiting for terminal.  
Waiting for terminal.. 
Waiting for terminal... commands (zsh -lc)
- fix screenshot viewport (baseY), scroll-over-buffer, full-session text

Co-Authored-By: Claude <noreply@anthropic.com>
…ot, openclaw skill

Co-Authored-By: Claude <noreply@anthropic.com>
…enclaw

Co-Authored-By: Claude <noreply@anthropic.com>
…commands

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…l offset

Co-Authored-By: Claude <noreply@anthropic.com>
…set 0

Co-Authored-By: Claude <noreply@anthropic.com>
- Extend ScreenshotQuery with optional scroll (u32) and top (bool);
  top maps to u32::MAX so renderer clamps to scrollback top.
- Add scroll field to capture_request WS message and pass through to
  request_renderer_screenshot.
- Add new /box/{id}/scrollback route backed by request_renderer_scrollback
  (ScrollbackQuery: raw/from_line/to_line). Response is text/plain.
- Add pending_scrollback HashMap to DaemonState; resolve scrollback_response
  and scrollback_error WS arms from it.
- IT: screenshot_query_parses_scroll_and_top and scrollback_route_exists.

Co-Authored-By: Claude <noreply@anthropic.com>
…shot WS

Co-Authored-By: Claude <noreply@anthropic.com>
Extend daemon_screenshot with scroll/top query params and add
daemon_scrollback client helper. Wire them into the CLI surface:
- screenshot --up N / --top thread into the daemon URL
- new \`scrollback\` subcommand dumps the clean ANSI-free session
  text to stdout or -o <file>, with optional --raw / --from-line
  / --to-line
- MCP tool list: extend screenshot_sandbox schema with up/top and
  add scrollback_sandbox tool

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown

🔒 门禁检查结果

检查项 状态
Rust 格式化 ✅ success
Rust Clippy ✅ success
Rust 测试 & 覆盖率 ✅ success
前端测试 & 覆盖率 ✅ success
Playwright E2E ✅ success
统一测试 (test.sh) ✅ success
安全检查 ✅ success

Rust 测试覆盖率

指标 覆盖率
行覆盖率 66.0%
分支覆盖率 0.0%
模块 行覆盖率
crates.cli-box-core.src.instance ███████████████████░ 99.1%
crates.cli-box-core.src.server █████████████████░░░ 87.7%
crates.cli-box-core.src.sandbox ███████████████████░ 98.8%
crates.cli-box-core.src.capture ████████████████████ 100.0%
crates.cli-box-core.src.process ██████████░░░░░░░░░░ 54.7%
crates.cli-box-core.src.automation ███████████░░░░░░░░░ 58.3%
crates.cli-box-core.src ███████████░░░░░░░░░ 55.3%
crates.cli-box-core.src.daemon ██████████░░░░░░░░░░ 50.7%

详细报告见 Rust 覆盖率 artifact

前端测试覆盖率

指标 覆盖率
行覆盖率 ███████████████░░░░░ 75.6%
分支覆盖率 65.7%
函数覆盖率 73.4%
语句覆盖率 75.0%
文件 行覆盖率
src/tests/mocks/websocket.ts ██████████████████░░ 88.0%
src/tests/mocks/xterm.ts ████████████████████ 100.0%
src/main/daemon-bridge.ts ████████░░░░░░░░░░░░ 40.0%
src/renderer/api.ts ██████████████░░░░░░ 68.2%
src/renderer/scrollback.ts ██████████████████░░ 88.9%
src/renderer/tabState.ts ████████████████████ 100.0%
src/renderer/terminalBuffer.ts ████████████████████ 100.0%

详细报告见前端覆盖率 artifact

✅ 所有检查通过,可以合入

点击 Squash and merge 合并此PR

ZN-Ice and others added 4 commits June 18, 2026 07:26
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Previously cli-box-daemon ignored all args and always started serving, so
'cli-box-daemon --version' would silently launch a daemon. Handle the
introspection flags before initializing the runtime.

Co-Authored-By: Claude <noreply@anthropic.com>
xterm renders CJK/emoji as double-width: the char sits in a width-2 cell
followed by a width-0 continuation cell with no glyph. The buffer readers
treated that empty continuation cell as a space, padding CJK text
('执 行 pwd 命 令'). Skip cells where getWidth()===0 in both readScrollback
and renderBufferToPng so CJK is contiguous.

Co-Authored-By: Claude <noreply@anthropic.com>
@Shadow-Azure
Shadow-Azure merged commit b7a1a74 into main Jun 19, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant