Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Apps/CLI/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## Unreleased

- Refresh the canonical agent skill with current host selection, permission, background input, coordinate, and result-verification guidance; document linked distribution and correct snapshot-relative click coordinates in CLI help and docs.

## 4.3.4 - 2026-09-11

**Highlights:** Restore provider-compatible MCP tools and frontmost daemon captures.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ extension ClickCommand: CommanderSignatureProviding {
),
.commandOption(
"at",
help: "x,y — target-relative when --app/--window-* given; global otherwise " +
"(use --global for explicit global)",
help: "x,y — background: relative to the snapshot window; foreground: target-relative " +
"with target flags, global without them. --global overrides either mode",
long: "at"
),
.commandOption(
Expand Down Expand Up @@ -115,7 +115,8 @@ extension ClickCommand: CommanderSignatureProviding {
),
.commandFlag(
"global",
help: "Treat --at as global screen coordinates even with target options",
help: "Treat --at as global screen coordinates instead of snapshot-window or " +
"target-relative coordinates",
long: "global"
),
],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,12 @@ struct ClickCommand: ActionOutputFormattable, ErrorHandlingCommand, OutputFormat
@OptionGroup var target: InteractionTargetOptions

@Option(
help: "x,y — target-relative when --app/--window-* given; global otherwise (use --global for explicit global)"
help: "x,y — background: relative to the snapshot window; foreground: target-relative with " +
"target flags, global without them. --global overrides either mode"
)
var at: String?

@Flag(help: "Treat --at as global screen coordinates even when target options are supplied")
@Flag(help: "Treat --at as global screen coordinates instead of snapshot-window or target-relative coordinates")
var global = false

@Option(help: "Maximum time to wait for an element (bare values are milliseconds)")
Expand Down
70 changes: 44 additions & 26 deletions docs/agent-skill.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,72 @@
---
summary: 'Install and maintain the thin Peekaboo CLI agent skill.'
summary: 'Install and maintain the canonical Peekaboo agent skill.'
read_when:
- 'setting up Peekaboo with AI agents'
- 'updating the peekaboo skill'
- 'updating or distributing the peekaboo skill'
---

# Agent Skill for Peekaboo

The `peekaboo` skill teaches agents when and how to use Peekaboo for macOS automation, screenshots, native accessibility inspection, native app and browser chrome, browser-page tooling, and repo validation. It intentionally stays thin: agents should use live CLI help, `peekaboo learn`, `peekaboo tools`, and canonical docs instead of a copied command reference that can drift.
The [Peekaboo skill](../skills/peekaboo/SKILL.md) teaches agents to observe macOS UI, select an exact target, act in the background where supported, and verify the result. It covers native apps and browser chrome, routes page content to browser tooling, and explains execution-host permissions, snapshot ownership, coordinates, and unverified input outcomes. Ordinary automation uses the installed CLI; source builds belong to Peekaboo development.

## Install
## Prerequisites

Install the signed CLI and check permissions on the execution host:

```bash
brew install openclaw/tap/peekaboo
peekaboo --version
peekaboo bridge status --verbose --json
peekaboo permissions status --all-sources --json
```

See the [installation guide](install.md) for the GUI app and release archives. The CLI and app are separate executables; keep archive-supplied compatibility libraries with the CLI. Current matching releases reduce capability drift, while the actual Bridge contract is negotiated by protocol and operation capability. Grant permissions to the host performing the operation, not merely to the calling terminal. The skill includes an explicit GUI-socket recipe when app-held grants are needed.

## Install from a maintained checkout

Copy the skill directory into your agent's skills folder:
Run from the Peekaboo repository root. Link the canonical directory into the agent's skill folder so updating this checkout updates its guidance:

```bash
PEEKABOO_REPO="$(pwd -P)"

# Codex
mkdir -p ~/.codex/skills
ln -s "$PEEKABOO_REPO/skills/peekaboo" ~/.codex/skills/peekaboo

# Claude Code
mkdir -p ~/.claude/skills
cp -r skills/peekaboo ~/.claude/skills/
ln -s "$PEEKABOO_REPO/skills/peekaboo" ~/.claude/skills/peekaboo

# OpenClaw
mkdir -p ~/.openclaw/skills
cp -r skills/peekaboo ~/.openclaw/skills/
ln -s "$PEEKABOO_REPO/skills/peekaboo" ~/.openclaw/skills/peekaboo
```

Restart the agent after installing or updating the skill.
Use only the entry for the agent being configured. If that destination already exists, inspect and preserve local content before replacing it; do not nest a new link inside an existing directory. Verify the installed `peekaboo/SKILL.md` resolves to this checkout. The skill uses canonical web links for references, so documentation links remain useful through symlinks in another repository or agent directory. Start a new agent session after installing or updating, according to the host agent's skill-loading behavior.

## Prerequisites
## Distribution ownership

Install Peekaboo and grant macOS permissions:
`skills/peekaboo/SKILL.md` in this repository is the canonical owner of Peekaboo command behavior and automation workflow guidance. A managed instruction repository can link to a sibling Peekaboo checkout instead of maintaining a second copy; its sync must update and verify the target checkout before exposing the link. Preserve host-specific binary paths, permission policy, and deployment details in the distributor's overlay, without duplicating product behavior.

```bash
brew install steipete/tap/peekaboo
peekaboo permissions status
peekaboo permissions grant
```
Distributors that must work without a Peekaboo source checkout may copy a release-pinned snapshot. That copy is a distribution artifact: record its source release or commit, refresh it with Peekaboo releases, and send behavioral corrections upstream. A frozen snapshot must not be presented as live current-source guidance.

Agents should also use `peekaboo learn`, `peekaboo tools`, and `peekaboo <command> --help` for the current command surface.
## Maintenance and validation

## Canonical Docs
Keep frontmatter limited to `name` and `description`. Maintain a compact operational workflow rather than generated command catalogs or a mandatory build recipe. Check material claims against production source as well as live CLI help; copied help and older docs can themselves drift. Keep `peekaboo learn`, Commander metadata, and command documentation aligned when changing product behavior.

- Skill file: `skills/peekaboo/SKILL.md`
- Command index: `docs/commands/README.md`
- Command pages: `docs/commands/*.md`
- Permissions: `docs/permissions.md`
- Subprocess/OpenClaw integration: `docs/integrations/subprocess.md`
For skill/documentation edits, run from the repository root:

## Distribution Ownership
```bash
node scripts/docs-lint.mjs
ruby -e 'h=File.read("skills/peekaboo/SKILL.md").split(/^---\s*$/,3)[1]; keys=h.lines.grep(/^[A-Za-z0-9_-]+:/).map { |line| line.split(":",2).first }; abort("unexpected skill frontmatter") unless keys.sort == ["description","name"]'
git diff --check
```

Peekaboo owns product behavior, current command and workflow guidance, and validation documentation. Distributors such as OpenClaw may vendor a release-pinned snapshot so they work without a Peekaboo source checkout, but that snapshot is a distribution artifact rather than a second authority: it must declare its source release or commit, sync when Peekaboo releases, and keep host-specific installation or socket guidance as an overlay.
Source-code changes follow [AGENTS.md](../AGENTS.md) and the [building guide](building.md). Live checks should use a controlled target, inspect the actual screen or state readback, and preserve retry-unsafe outcomes. Image dimensions and command success alone are insufficient proof that the intended UI changed.

## Maintenance Rule
## Canonical references

Keep the skill compact and progressive. Its frontmatter should contain only `name` and `description`, and its body should explain observation strategy and validation flow without vendoring generated command catalogs. Update Commander metadata, `peekaboo learn`, or `docs/commands/*` when command behavior changes.
- [Command index](commands/README.md)
- [Permissions](permissions.md) and [Bridge hosts](bridge-host.md)
- [Subprocess integration](integrations/subprocess.md)
- `peekaboo <command> --help`, `peekaboo learn`, and `peekaboo tools describe <name> --json`
2 changes: 1 addition & 1 deletion docs/automation.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Every input command accepts one of three target shapes:

- **Element ID** — `--on <id>` from a fresh `peekaboo see` capture; preferred when available. Treat IDs as opaque strings and copy the exact value returned by the capture.
- **Label / role / app** — positional query text such as `peekaboo click "Send" --app Mail`; resolved via the AX tree.
- **Coordinates** — `--at 480,120`; target-relative when paired with `--app`, `--pid`, or `--window-*`, global otherwise. Add `--global` to force screen coordinates with a target.
- **Coordinates** — `--at 480,120`; background clicks use the explicit snapshot's window as their origin, even without app/window flags. Foreground coordinates are relative to a selected target window and global without one. Add `--global` to force global logical coordinates; background points must still lie inside the captured window.

Prefer IDs when you can capture them, labels when you can't, and coordinates only as a last resort. The agent and MCP tooling default to the first two.

Expand Down
4 changes: 2 additions & 2 deletions docs/commands/click.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ read_when:
| --- | --- |
| `[query]` | Optional positional text query (case-insensitive substring match). |
| `--on <id>` | Target an opaque Peekaboo element ID copied exactly from current `see` or MCP `inspect_ui` output. |
| `--at x,y` | Click coordinates. With target flags, coordinates are relative to the resolved target window; without target flags, they are global screen coordinates. |
| `--global` | Treat `--at` as global screen coordinates even when target flags are supplied. |
| `--at x,y` | Logical click coordinates. Background clicks are relative to the explicit snapshot's window, including when no target flags are supplied. Foreground clicks are relative to a selected target window, or global when no target window is selected. |
| `--global` | Treat `--at` as global screen coordinates in either delivery mode. Background clicks still require an exact-window snapshot and a point inside that window. |
| `--snapshot <id>` | Reuse a prior snapshot. A concrete ID must be `ps1_` plus 32 lowercase ASCII hexadecimal digits; element/query clicks otherwise default to the latest snapshot. Background coordinate clicks require an explicit reference from a fresh exact-window `see`. |
| Target flags | `--app <name>`, `--pid <pid>`, `--window-id <id>`, `--window-title <title>`, `--window-index <n>` — resolve the app/window that should receive the click. In background mode this does not focus the app; with `--foreground` it focuses before clicking. (`--window-title`/`--window-index` require `--app` or `--pid`; `--window-id` does not.) |
| `--wait-for <duration>` | Timeout while waiting for the element (default `5s`; bare values are milliseconds). |
Expand Down
Loading
Loading