From f76a57f1bbd92624f1f0cd3d2d59f203d39c2ea2 Mon Sep 17 00:00:00 2001 From: Gabe Debes Date: Fri, 23 Oct 2026 13:03:57 -0700 Subject: [PATCH] Add direct iOS controls to the Simulator skill Co-authored-by: Codex --- plugins/expo/.claude-plugin/plugin.json | 2 +- plugins/expo/plugin.json | 2 +- plugins/expo/skills/eas-simulator/SKILL.md | 15 +++-- .../eas-simulator/references/controllers.md | 6 +- .../references/remote-controls.md | 56 +++++++++++++++++++ .../eas-simulator/references/run-your-app.md | 14 +++++ .../references/troubleshooting.md | 2 +- 7 files changed, 85 insertions(+), 12 deletions(-) create mode 100644 plugins/expo/skills/eas-simulator/references/remote-controls.md diff --git a/plugins/expo/.claude-plugin/plugin.json b/plugins/expo/.claude-plugin/plugin.json index 7c5e5b6f..fb372968 100644 --- a/plugins/expo/.claude-plugin/plugin.json +++ b/plugins/expo/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "expo", - "version": "1.14.0", + "version": "1.14.1", "description": "Official Expo skills for building, deploying, upgrading, and debugging Expo apps", "author": { "name": "Expo Team", diff --git a/plugins/expo/plugin.json b/plugins/expo/plugin.json index 321e1110..f0396a4d 100644 --- a/plugins/expo/plugin.json +++ b/plugins/expo/plugin.json @@ -1,7 +1,7 @@ { "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", "name": "expo", - "version": "1.14.0", + "version": "1.14.1", "description": "Official Expo skills for building, deploying, upgrading, and debugging Expo and React Native apps.", "author": { "name": "Expo Team", diff --git a/plugins/expo/skills/eas-simulator/SKILL.md b/plugins/expo/skills/eas-simulator/SKILL.md index b1a22611..0ef06264 100644 --- a/plugins/expo/skills/eas-simulator/SKILL.md +++ b/plugins/expo/skills/eas-simulator/SKILL.md @@ -33,7 +33,7 @@ if the user set a limit you would go past, or if the work grows beyond what they - **Run every `eas` command via `npx --yes eas-cli@latest …`** — guarantees a CLI new enough to have `simulator:*` (a global `eas` is often too old), and `--yes` skips npx's prompt. (Bare `eas` is fine if `eas --version` is current.) - **Authenticated.** Interactive machine → `npx --yes eas-cli@latest login`. **Cloud sandbox / CI / headless agent has no browser login — set `EXPO_TOKEN`** (expo.dev → Account → Access Tokens) in the env instead. Verify either way with `npx --yes eas-cli@latest whoami`. - Run from an Expo **project directory.** A fresh app needs one-time setup: `npx --yes eas-cli@latest init` to create/link the project (when there's no `projectId`), and **set `ios.bundleIdentifier`** in app config if it's missing — a fresh `create-expo-app` often has none, and `prebuild`/`eas build` need it (they prompt or fail without it; e.g. `dev..`). Read current config with `npx expo config --json` (it may live in `app.config.js`). The first Mode-C run is slow (native build); later runs reuse it. -- A controller to drive the device. This skill uses **agent-device** (open source, MIT), run on demand via `npx agent-device@latest` — nothing globally installed. **Appium** and **argent** are alternative automation interfaces; `web-preview-only` has no automation interface. See [references/controllers.md](./references/controllers.md). +- A controller to drive the device. This skill uses **agent-device** (open source, MIT), run on demand via `npx agent-device@latest` — nothing globally installed. **Appium** and **argent** are alternative automation interfaces. On iOS, `simulator:control` also runs actions exposed by the preview host, including in `web-preview-only` sessions; that type has no separate automation daemon. See [references/controllers.md](./references/controllers.md). - **`.env.eas-simulator`** is written/managed by eas-cli (not this skill): it holds the session id (`EAS_SIMULATOR_SESSION_ID`) + the daemon URL/**token**, so `get`/`stop`/`exec` default to that session (usually **omit `--id`**; pass `--id ` to target another). It carries a **token → keep it gitignored** (eas-cli marks it "do not commit" but may not add the ignore rule, and a fresh app's `.gitignore` won't cover it — add `.env.eas-simulator` if missing). - **The command blocks assume a POSIX shell** (bash/zsh) — `printf`, `lsof`, `$(seq …)` loops won't run in cmd/PowerShell. On Windows, run them in WSL or Git Bash, or translate as you go (the `eas-cli`/`agent-device` invocations themselves are cross-platform). @@ -41,7 +41,8 @@ if the user set a limit you would go past, or if the work grows beyond what they - `--max-duration-minutes N` is the hard automatic-stop deadline. Any plan can set it, up to the plan limit (40 minutes on Free, 115 on paid plans). The timer starts when the session is ready. - `--max-idle-time-minutes N` stops a session after that many inactive minutes. Omitted means **no idle timeout**: the session runs until its maximum duration or an explicit stop. -- **Controller activity resets the idle timer** (`agent-device`, `argent`, and Appium commands, which are recorded as session events with producer `appium`). **Browser-preview activity does not reset it.** For a user-driven browser preview, rely on the maximum duration, not idle time, to bound the session. +- **Controller activity resets the idle timer** (`agent-device`, `argent`, and Appium commands, recorded as session events with producer `appium`). On iOS automation sessions using the current worker path, serve-sim preview input and host actions also reset it; simply watching the video does not. Android previews and older workers do not report this activity. Use the maximum duration as the hard lifetime bound; omit the idle timeout when the requested preview must remain available without input. +- `npx --yes eas-cli@latest account:usage` shows EAS Simulator minutes for the selected account on current CLIs. ## Check availability first @@ -61,7 +62,7 @@ If it's **not** available, don't call `simulator:start` (it will fail). Instead, ## The core loop (always the same) -A session is: **start → (install your app) → drive → stop.** `eas-cli` owns the *session*; the device *verbs* (open/tap/screenshot) come from the controller, which `npx --yes eas-cli@latest simulator:exec` runs for you with the session's connection env loaded. +A session is: **start → (install your app) → drive → stop.** `eas-cli` owns the *session*; the controller supplies the interactive device verbs, and `npx --yes eas-cli@latest simulator:exec` runs it with the session's connection env loaded. For iOS host actions such as screenshots, clipboard, and settings, `simulator:control` can also act directly; see [remote controls](./references/remote-controls.md). ```bash # 1. Start a session (boots the remote sim + agent-device daemon; writes .env.eas-simulator). @@ -86,11 +87,11 @@ npx --yes eas-cli@latest simulator:stop printf '# managed by eas-cli\n' > .env.eas-simulator ``` -To **watch** it live, hand the user the `webPreviewUrl` that `start` prints (in `--json`, `remoteConfig.webPreviewUrl`; for `web-preview-only` sessions, `remoteConfig.previewUrl`). All current session types include a browser preview, on iOS and Android; `agent-device`, `appium`, and `argent` also provide automation, while `web-preview-only` provides no automation interface. On Android, the expo.dev session page does not show the preview inline yet: **Open preview** opens it in a new tab. Android support is in development, so some iOS features may not work there. **This URL is for the *user's* browser — you cannot open it for them, and it must never touch the sim:** +To **watch** it live, hand the user the `webPreviewUrl` that `start` prints (in `--json`, `remoteConfig.webPreviewUrl`; for `web-preview-only` sessions, `remoteConfig.previewUrl`). All current session types include a browser preview, on iOS and Android; `agent-device`, `appium`, and `argent` also provide automation, while `web-preview-only` has no separate automation daemon but supports the iOS host actions exposed through `simulator:control`. On Android, the expo.dev session page does not show the preview inline yet: **Open preview** opens it in a new tab. Android support is in development, so some iOS features may not work there. **This URL is for the *user's* browser — you cannot open it for them, and it must never touch the sim:** - **"Open it here" (Cursor/VS Code)** → print the URL on its own line and tell the user to open Simple Browser (`Cmd/Ctrl+Shift+P` → "Simple Browser: Show") and paste it. Then **stop**: do not shell out to a system browser or a Cursor/VS Code URL handler, and do not ask "did a tab appear?" — you can't confirm it, the handoff is done. - **Never `open` the `webPreviewUrl` on the sim.** It's a browser preview, not a deep link and not an `agent-device open` argument; routing it to the device renders a browser-in-a-browser (a real past failure). - **Headless agent** (no display) → just return the URL as the deliverable. -- **Keeping it alive for the user to drive** → use `--max-duration-minutes N` (up to the plan limit). Browser-preview activity does not reset `--max-idle-time-minutes`, so idle timeout is not a reliable lifetime bound for this case. Tell the user when the session expires, using the CLI's reported duration or expiry. Keep it running for the requested preview; stop sessions created for one-shot tasks when the task finishes. +- **Keeping it alive for the user to drive** → use `--max-duration-minutes N` (up to the plan limit). Merely viewing the video does not reset the idle timer; input reporting also varies by platform and worker. Follow [Session lifetime](#session-lifetime) when choosing an idle timeout. Tell the user when the session expires, using the CLI's reported duration or expiry. Keep it running for the requested preview; stop sessions created for one-shot tasks when the task finishes. `start` also prints a job-run URL. @@ -135,6 +136,7 @@ copy of the CLI surface. Keep non-obvious behavioral guidance from this skill— |---|---| | `npx --yes eas-cli@latest simulator:availability [--json] [--non-interactive]` | Check access without creating a session. | | `npx --yes eas-cli@latest simulator:start --platform ios\|android --name "" [flags]` | Create a session; boot the sim + selected interface; write `.env.eas-simulator` by default; print the preview + job-run URLs. **Always pass `--name`**. `--json` does not suppress the dotenv; use `--out-config-type env` when no file should be written (it prints only the controller `export` lines, without `EAS_SIMULATOR_SESSION_ID`, so pass `--id` to `get`/`stop`/`events`). | +| `npx --yes eas-cli@latest simulator:control [action] [values…] --id ` | Discover and run actions on a live iOS preview host (CLI ≥ 25.2.0). Use per-action `--help`; `--output` saves returned files locally. See [remote controls](./references/remote-controls.md). | | `npx --yes eas-cli@latest simulator:exec [args…]` | Load `.env.eas-simulator`, then run `` with that env. The bridge to the controller. | | `npx --yes eas-cli@latest simulator:get [--id ] [--json] [--non-interactive]` | Session status + connection details, including the session name. **Use this to confirm readiness** (see *Operating principles*). | | `npx --yes eas-cli@latest simulator:list [filters] [--limit N] [--after ] [--json]` | List and paginate project sessions; filter by status, type, platform, name prefix, and tags. | @@ -204,7 +206,7 @@ The non-obvious mental model worth internalizing. Specific error→fix lookups ( - **build fits intent** — a **release build can't live-reload**; if live edits are wanted and a release build is installed, **install the dev build, don't reconnect**. If current code isn't rendering after your **first** connect, stop poking live state: **reset to baseline** (stop session → clear dotenv → kill your Metro) and redo the mode **once**; a second failure → stop and report. Never restart Metro in place, reconnect more than once, rebuild the native client to fix a JS/connection problem, or surface a preview URL while state is unknown. (A daemon drop — `ERR_NGROK_3200` / `Remote daemon is unavailable` — is the same: reset, don't retry.) -2. **`exec` is a wrapper, not a driver.** `simulator:exec` loads `.env.eas-simulator` and spawns the command you pass; the device verbs come from the controller (`npx agent-device@latest`). There is no `simulator:tap`. +2. **`exec` is a wrapper, not a driver.** `simulator:exec` loads `.env.eas-simulator` and spawns the command you pass; the device verbs come from the controller (`npx agent-device@latest`). There is no `simulator:tap`; `simulator:control` is a separate iOS host-action client. 3. **Act immediately; don't park an idle session.** Sessions are short-lived — install and drive right after `start`. Leaving one idle drops the tunnel/daemon (→ reset, per #1). 4. **Stop sessions you created on completion or failure and reset the dotenv.** `--non-interactive` does not stop a session when your task ends. For a requested live preview, follow the duration guidance above. Poll the existing session during a slow boot; starting another creates an extra session and overwrites the dotenv's session id. 5. **Screenshot only the correct, fresh build.** Mode C only after the dev client connects to Metro; A/B only from a build matching current source — reusing a pre-existing build is the #1 "my edits don't show" cause (see the build caveat above). (`9:41` in the status bar is the sim default, not staleness.) @@ -222,6 +224,7 @@ printf '# managed by eas-cli\n' > .env.eas-simulator # clear the stale session ## References - [references/run-your-app.md](./references/run-your-app.md) — full command sequences for modes A, B, and C (read before running a mode). +- [references/remote-controls.md](./references/remote-controls.md) — direct iOS actions, local file transfers, and embedding the preview. - [references/controllers.md](./references/controllers.md) — agent-device verb reference and the `argent` alternative. - [references/logs-and-crashes.md](./references/logs-and-crashes.md) — device logs and crash reports from an iOS session (read when the app crashes or misbehaves). - [references/troubleshooting.md](./references/troubleshooting.md) — concrete errors and fixes. diff --git a/plugins/expo/skills/eas-simulator/references/controllers.md b/plugins/expo/skills/eas-simulator/references/controllers.md index f81a238d..dfc9b259 100644 --- a/plugins/expo/skills/eas-simulator/references/controllers.md +++ b/plugins/expo/skills/eas-simulator/references/controllers.md @@ -1,13 +1,13 @@ # Controllers: agent-device, Appium, and argent -`eas-cli` has no device verbs — it manages the *session*. Automation commands come from the interface selected by `simulator:start --type`: +`eas-cli` manages the session and exposes iOS preview-host actions through [`simulator:control`](./remote-controls.md). For the interactive controller workflow, select an interface with `simulator:start --type`: - `agent-device` (Callstack, MIT) — used throughout this skill; runs on demand via `npx agent-device@latest`, nothing installed globally. - `appium` — exposes `APPIUM_URL` and `APPIUM_CAPS` for an Appium client. - `argent` (Software Mansion) — a capable alternative controller; check its license for your use. -- `web-preview-only` — browser preview with no programmatic control. +- `web-preview-only` — browser preview without a separate automation daemon; iOS host actions remain available through `simulator:control` when the host supports them. -All four types include a web preview. Before setting `--max-idle-time-minutes`, follow [Session lifetime](../SKILL.md#session-lifetime); browser-preview activity does not reset the timer. +All four types include a web preview. Before setting `--max-idle-time-minutes`, follow [Session lifetime](../SKILL.md#session-lifetime); iOS preview input can reset the timer on current workers, but watching the video does not; Android and older workers differ. ## Appium diff --git a/plugins/expo/skills/eas-simulator/references/remote-controls.md b/plugins/expo/skills/eas-simulator/references/remote-controls.md new file mode 100644 index 00000000..793104f7 --- /dev/null +++ b/plugins/expo/skills/eas-simulator/references/remote-controls.md @@ -0,0 +1,56 @@ +# Direct iOS controls and preview embedding + +## Discover actions on the selected session + +**eas-cli 25.2.0+** provides `simulator:control` for a running iOS session whose +preview host exposes a token and an action catalog (serve-sim 0.10.0+). It uses +the authenticated session query, so it does not need `--public-link` or a token +copied from JSON output. Android sessions are not supported by this command. + +Run from the Expo project directory and pass the session id explicitly. Discover +what that host supports before invoking an action; upgrading the local CLI does +not upgrade an existing session's host: + +```bash +npx --yes eas-cli@latest simulator:control --id +npx --yes eas-cli@latest simulator:control screenshot.capture --help --id +npx --yes eas-cli@latest simulator:control screenshot.capture --output ./shot.png --id +``` + +Use `--json` on the discovery command for parameter schemas, or per-action +`--help` for usage. Actions accept positional values or `name=value`; put `--` +before values starting with `--`. Available actions can cover clipboard, +appearance, permissions, input, app lifecycle, app files, and device controls. +Choose only actions needed for the user's task. Keep using the selected +controller for snapshots and element-reference navigation. + +## Files and screenshots + +`--output` is a **local** destination; action path arguments refer to the remote +host unless the action declares an upload. Screenshots get a generated filename +when no output path is given. On serve-sim 0.10.4+, captures follow the active +display; use the action's optional `display` parameter only with a known simctl +screen ID from that device, not a guessed panel index. + +For installation, `app.install` accepts a file, not a local `.app` directory. +Use an installable simulator `.ipa` and consult that session's action help. +Upload-enabled actions stage an existing local file automatically. A nonexistent +absolute path is treated as a remote path; confirm which machine owns a path +before passing it. Downloads and clipboard output can contain app data; keep +those files and output private and inspect the saved image before reporting it. + +## Embed or call the preview from another page + +With **eas-cli 25.1.1+**, `simulator:start` supports two separate repeatable flags: + +- `--frame-ancestor https://app.example.com` lets that page embed the preview in + an iframe. Use the direct preview URL printed for embedding, not the expo.dev + page URL. +- `--cors-origin https://app.example.com` lets that page call the preview API, + for example through `@expo/hub-client`. It is not needed just for an iframe. + +Use the specific origins needed for the requested integration. The printed direct +URL carries the session token; a page that receives it can control the simulator +and read app data and logs. The allowlist does not replace token authentication. +Keep that URL out of source control and public pages. For ordinary viewing, use +the browser preview link rather than adding embedding or CORS grants. diff --git a/plugins/expo/skills/eas-simulator/references/run-your-app.md b/plugins/expo/skills/eas-simulator/references/run-your-app.md index a3a36237..1d91feb7 100644 --- a/plugins/expo/skills/eas-simulator/references/run-your-app.md +++ b/plugins/expo/skills/eas-simulator/references/run-your-app.md @@ -67,6 +67,20 @@ npx --yes eas-cli@latest simulator:exec npx agent-device@latest open --- +## Android system image + +With **eas-cli 25.3.0+**, `--system-image` selects the Android SDK system-image +package; `--device` still selects the AVD hardware profile. It does not select an +iOS runtime. Omit the image flag to use the runner's default, and verify that a +requested package is supported by the remote runner. Quote the package string +because its semicolons are shell separators: + +```bash +npx --yes eas-cli@latest simulator:start --platform android --device pixel_7 \ + --system-image 'system-images;android-35-ext15;google_apis_playstore;x86_64' \ + --non-interactive --name "Android compatibility check" +``` + ## Mode A — Local release build (embedded JS, no Metro) A Release build bundles the JS into the binary, so it renders without Metro. Good for a quick "run my current code on a cloud device" when a Mac toolchain is available. diff --git a/plugins/expo/skills/eas-simulator/references/troubleshooting.md b/plugins/expo/skills/eas-simulator/references/troubleshooting.md index 0f4f5d33..f276de62 100644 --- a/plugins/expo/skills/eas-simulator/references/troubleshooting.md +++ b/plugins/expo/skills/eas-simulator/references/troubleshooting.md @@ -12,7 +12,7 @@ Concrete errors seen while validating this flow, and the fix. | `simulator:start`/`build`: no linked project / missing `projectId` | A fresh `create-expo-app` isn't linked to EAS | `npx --yes eas-cli@latest init` to create/link it (writes `extra.eas.projectId`). | | `prebuild`/`eas build` prompts for or fails on a missing **iOS bundle identifier** | A fresh app often has no `ios.bundleIdentifier` | Set it in app config (e.g. `dev..`); confirm via `npx expo config --json` (may live in `app.config.js`). | | `--max-duration-minutes` rejected | The value is above the plan limit (40 minutes on Free, 115 on paid plans); inspect the CLI error | Pass a value within the plan limit, or omit the flag to use the default. | -| Browser-preview session stops despite ongoing interaction | Browser-preview activity does not reset `--max-idle-time-minutes`; only controller activity (`agent-device`, `argent`, Appium) does | Use the maximum duration as the lifetime bound for user-driven previews. Set it with `--max-duration-minutes`, and omit `--max-idle-time-minutes` unless controller inactivity is the intended stop condition. | +| Browser-preview session stops while being watched | Viewing video is not input; Android previews and older workers do not report preview activity to the idle timer | Current iOS automation sessions count serve-sim input and host actions. Use `--max-duration-minutes` as the hard deadline, and omit `--max-idle-time-minutes` when the preview must stay available without input. | | `simulator:start` fails with `not enabled for this account` / not-allowlisted | EAS Simulator is limited-access and isn't enabled for this account | Don't retry. Confirm with `simulator:availability`, then hand off gracefully — tell the user and fall back to a local sim / EAS Build (see SKILL.md *Check availability first*). | | `start` keeps "Waiting for … session to be ready" but it never returns | `start`'s readiness poll can miss a session that's actually live | Don't rely on it — poll `npx --yes eas-cli@latest simulator:get --id --json` for `status: IN_PROGRESS` + a populated `remoteConfig`. | | `ERR_NGROK_3200` / endpoint offline; `Remote daemon is unavailable` | The session's tunnel/daemon dropped — left idle and timed out, or the VM was torn down | A drop invalidates the **whole** session (installed app, `@e` refs, Metro). **Don't retry the failed verb** — start a fresh session, reset the dotenv, and re-run install→open→drive from the top, acting immediately. |