feat(mcp): include resolved element bounds in tap_on and run_flow responses

[sinano1107]

Summary

• Add CommandResult and FlowResult data classes to Orchestra so that element metadata (bounds, center) captured during command execution is propagated back to MCP tool callers.
• Orchestra.runFlow() now returns FlowResult (success + commandResults) instead of a bare Boolean. Internal command handlers are unchanged; an accumulator captures element info from tapOnElement and swipeCommand without touching the 40+ other handlers.
• tap_on response now includes "bounds": [x1,y1,x2,y2] and "center": [cx,cy]
• run_flow response now includes a "command_results" array with per-command bounds / start_point / end_point when available

Motivation

When building automation that records tap/swipe coordinates (e.g. for video overlay generation), the resolved element bounds are needed but currently discarded inside Orchestra. This change surfaces them through the existing MCP tool responses without altering Orchestra's internal command execution logic.

Changes

File	Change
Orchestra.kt	CommandResult/FlowResult data classes, _commandResults accumulator, runFlow() returns FlowResult
CommandResultSerializer.kt	Shared addBoundsTo() extension for JSON serialization
TapOnTool.kt	Include bounds/center in response
RunFlowTool.kt	Include command_results array in response
RunFlowFilesTool.kt	Use flowResult.success
MaestroCommandRunner.kt	.success accessor (backward-compatible)
TestSuiteInteractor.kt	.success accessor (backward-compatible)

Test plan

• maestro-orchestra:test passes
• maestro-cli:test passes
• Manual test: tap_on returns correct bounds/center on iOS simulator (Settings app)
• Manual test: run_flow with tap + swipe returns command_results with tap bounds

🤖 Generated with Claude Code

[sinano1107]

Code review

Found 1 issue:

1. The message field in RunFlowTool and RunFlowFilesTool is hardcoded to "Flow executed successfully" even when flowResult.success is false. This PR correctly wires up flowResult.success for the success field, but leaves the message static — so an MCP consumer will receive a contradictory response (success: false, message: "Flow executed successfully") whenever a flow fails due to a failing assertion or command error.

https://github.com/mobile-dev-inc/maestro/blob/0360f7e864833cc530d2dd4a1e240ea7e0a40422/maestro-cli/src/main/java/maestro/cli/mcp/tools/RunFlowTool.kt#L120-L126

Same issue in RunFlowFilesTool:

https://github.com/mobile-dev-inc/maestro/blob/0360f7e864833cc530d2dd4a1e240ea7e0a40422/maestro-cli/src/main/java/maestro/cli/mcp/tools/RunFlowFilesTool.kt#L102-L108

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[sinano1107]

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

[Fishbowler]

Can you elaborate on the value and motivation for this change?

[sinano1107]

I'm building a system that records iOS Simulator interactions (via Maestro MCP) and composites tap/swipe indicators onto the recorded video using Remotion. To place the indicators accurately, I need the resolved element coordinates from each tap_on and swipe command.

Currently, Maestro resolves the element internally but discards the coordinates before returning the MCP response. The only alternative is calling inspect_view_hierarchy before each tap and re-matching the element, but this introduces race conditions (UI may change between hierarchy fetch and tap) and doubles the session overhead per interaction.

This change surfaces coordinates that Maestro already computes, with no impact on existing behavior.

[Fishbowler]

Oooh, neat idea ♥️

I think there might be other applications folks find for this too.

Just need to check that this doesn't "cross-pollenate" too heavily. Will review properly once I'm back at my desk on Tuesday 👍