Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Apps/CLI/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Warm ScreenCaptureKit ownership validation off the main actor before Bridge socket/capability publication, with explicit publication and daemon-readiness reserves beyond the bounded scan.
- Claim and generation-check the host's ScreenCaptureKit lease before trying the concurrent engine first for background Bridge full-screen automatic capture, preserving legacy fallback after modern failure and automatic fallback on every claim failure or competing owner.
- Prevent agent-spawned exec children from retaining the global ScreenCaptureKit transaction lock after an interrupted capture owner exits.
- Return exit status 2 when `verify` cannot evaluate state because its underlying tool fails.
- Report background text, editable special keys, and clears with their actual AXValue, event, or composite delivery; count only real key events as key presses; preserve the planned receiver literal after escape processing; and require protocol 1.36 before AX-capable remote type requests.
- Revalidate exact-window focused elements and the application's internal key window before typing, reject parent targets with attached sheets while preserving independently identified exact sheet targets, confirm clear-plus-literal text only from a generation-bound value change after bounded event settlement, keep pixel-focus setup confirmation separate from its typing leaf, and stop reporting no-change, missing, or dispatched-but-unverified outcomes as typed characters.
- Require explicit standalone CLI foreground consent for application focus/switch and Dock visibility changes, and reject contradictory app-switch selectors before runtime discovery.
Expand Down
5 changes: 3 additions & 2 deletions Apps/CLI/Sources/PeekabooCLI/Commands/AI/VerifyCommand.swift
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,10 @@ struct VerifyCommand: ErrorHandlingCommand, OutputFormattable, RuntimeBackedComm
tool: tool.name,
response: response,
jsonOutput: self.jsonOutput,
logger: self.outputLogger
logger: self.outputLogger,
errorExitCode: ExitCode(2)
)
return
throw ExitCode(2)
}

let screenshotPath = try self.saveScreenshot(from: response)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,8 @@ enum MCPToolCommandOutput {
tool: String,
response: ToolResponse,
jsonOutput: Bool,
logger: Logger
logger: Logger,
errorExitCode: ExitCode = ExitCode(1)
) throws {
let payload = self.payload(tool: tool, response: response)
if jsonOutput {
Expand All @@ -123,7 +124,7 @@ enum MCPToolCommandOutput {
}

if response.isError {
throw ExitCode(1)
throw errorExitCode
}
}

Expand Down
39 changes: 39 additions & 0 deletions Apps/CLI/Tests/CoreCLITests/MCPToolCommandOutputTests.swift
Original file line number Diff line number Diff line change
@@ -1,10 +1,49 @@
import Commander
import MCP
import PeekabooFoundation
import TachikomaMCP
import Testing
@testable import PeekabooCLI

struct MCPToolCommandOutputTests {
@Test(arguments: [false, true])
func `Tool errors use exit status one by default`(jsonOutput: Bool) {
let exitCode = #expect(throws: ExitCode.self) {
try MCPToolCommandOutput.output(
tool: "fixture",
response: .error("fixture failure"),
jsonOutput: jsonOutput,
logger: .shared
)
}
#expect(exitCode == ExitCode(1))
}

@Test(arguments: [false, true])
func `Tool errors honor an explicit exit status`(jsonOutput: Bool) {
let exitCode = #expect(throws: ExitCode.self) {
try MCPToolCommandOutput.output(
tool: "fixture",
response: .error("fixture failure"),
jsonOutput: jsonOutput,
logger: .shared,
errorExitCode: ExitCode(2)
)
}
#expect(exitCode == ExitCode(2))
}

@Test(arguments: [false, true])
func `Successful tool output does not throw`(jsonOutput: Bool) throws {
try MCPToolCommandOutput.output(
tool: "fixture",
response: .text("fixture success"),
jsonOutput: jsonOutput,
logger: .shared,
errorExitCode: ExitCode(2)
)
}

@Test
func `Browser CLI envelope projects canonical failure and exact target metadata`() throws {
let outcome = DesktopActionOutcome.indeterminate(
Expand Down
74 changes: 74 additions & 0 deletions Apps/CLI/Tests/CoreCLITests/VerifyCommandTests.swift
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
import Commander
import Foundation
import PeekabooAgentRuntime
import PeekabooAutomationKit
import PeekabooBridge
import PeekabooCore
import Testing
@testable import PeekabooCLI

Expand Down Expand Up @@ -47,4 +53,72 @@ struct VerifyCommandTests {
}
#expect(category == .vision)
}

@Test(arguments: [false, true])
func `tool failure exits with unknown error status`(jsonOutput: Bool) async throws {
let directory = FileManager.default.temporaryDirectory
.appendingPathComponent("verify-preflight-\(UUID().uuidString)")
defer { try? FileManager.default.removeItem(at: directory) }
var command = try VerifyCommand.parse([
"--app", "Fixture", "--window-exists", "--screenshot", directory.appendingPathComponent("unused.png").path,
])
let runtime = CommandRuntime(
configuration: .init(verbose: false, jsonOutput: jsonOutput, logLevel: nil),
services: VerifyPreflightServices(directory: directory),
toolCapturePreflightRefusal: MCPToolCapturePreflightRefusal(message: "fixture capture refusal"),
interactionMutationTracker: InteractionMutationTracker(
desktopMutationWatermarkStore: DesktopMutationWatermarkStore(directoryURL: directory)
)
)

let exitCode = await #expect(throws: ExitCode.self) {
try await command.run(using: runtime)
}
#expect(exitCode == ExitCode(2))
#expect(!FileManager.default.fileExists(atPath: directory.appendingPathComponent("unused.png").path))
}
}

@MainActor
private final class VerifyPreflightServices: PeekabooServiceProviding {
let automation: any UIAutomationServiceProtocol = MockAutomationService()
let windows: any WindowManagementServiceProtocol = MockWindowService(result: [])
let menu: any MenuServiceProtocol = MockMenuService(barItems: [])
let dock: any DockServiceProtocol = MockDockService(items: [])
let snapshots: any SnapshotManagerProtocol = InMemorySnapshotManager()
let permissions = PermissionsService()
let screens: any ScreenServiceProtocol = ScreenService()
let clipboard: any ClipboardServiceProtocol = ClipboardService()
let agent: (any AgentServiceProtocol)? = nil
let screenCapture: any ScreenCaptureServiceProtocol
let applications: any ApplicationServiceProtocol
let dialogs: any DialogServiceProtocol
let browser: any BrowserMCPClientProviding

init(directory: URL) {
// These adapters are inert until called; preflight must refuse before any Bridge request.
let client = PeekabooBridgeClient(socketPath: directory.appendingPathComponent("absent.sock").path)
self.screenCapture = RemoteScreenCaptureService(client: client)
self.applications = RemoteApplicationService(client: client)
self.dialogs = RemoteDialogService(client: client)
self.browser = RemoteBrowserMCPClient(client: client)
}

var configuration: PeekabooCore.ConfigurationManager {
fatalError("Verification preflight must not load shared configuration")
}

var audioInput: AudioInputService {
fatalError("Verification preflight must not initialize AI providers")
}

var logging: any LoggingServiceProtocol {
fatalError("Verification preflight uses only the CLI logger")
}

var files: any FileServiceProtocol {
fatalError("Verification preflight must not access files")
}

func ensureVisualizerConnection() {}
}
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
- Warm ScreenCaptureKit ownership validation off the main actor before Bridge socket/capability publication, with explicit publication and daemon-readiness reserves beyond the bounded scan.
- Claim and generation-check the host's ScreenCaptureKit lease before trying the concurrent engine first for background Bridge full-screen automatic capture, preserving legacy fallback after modern failure and automatic fallback on every claim failure or competing owner.
- Prevent agent-spawned exec children from retaining the global ScreenCaptureKit transaction lock after an interrupted capture owner exits.
- Return exit status 2 when `verify` cannot evaluate state because its underlying tool fails.
- Report background text, editable special keys, and clears with their actual AXValue, event, or composite delivery; count only real key events as key presses; preserve the planned receiver literal after escape processing; and require protocol 1.36 before AX-capable remote type requests.
- Revalidate exact-window focused elements and the application's internal key window before typing, reject parent targets with attached sheets while preserving independently identified exact sheet targets, confirm clear-plus-literal text only from a generation-bound value change after bounded event settlement, keep pixel-focus setup confirmation separate from its typing leaf, and stop reporting no-change, missing, or dispatched-but-unverified outcomes as typed characters.
- Require process-generation receipts for process-scoped `action` and `set-value` snapshots, revalidate them before dispatch, and preserve their canonical target metadata through MCP and signed Bridge results.
Expand Down
4 changes: 3 additions & 1 deletion docs/commands/verify.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ read_when:

`peekaboo verify` polls fresh native window and accessibility state until every requested predicate is stable or the timeout expires. It is the deterministic replacement for sleep-based polling: the command never focuses, clicks, types, or treats an incomplete observation as success.

Results are ternary. `satisfied` exits 0, `unsatisfied` exits 1, and `unknown` exits 2. JSON output includes every predicate result and an `unknown_reason` field; it is `null` when the result is not unknown.
Results are ternary. `satisfied` exits 0, `unsatisfied` exits 1, and `unknown` exits 2. Evaluated results in JSON output include every predicate result and an `unknown_reason` field; it is `null` when the result is not unknown.

Tool failures that prevent evaluation also exit 2. These failures use the standard error envelope in JSON mode, without predicate results or an `unknown_reason` field.

## Key options

Expand Down
Loading