From fec8ca3d5db18522ed86bad56f1b220bab71c86e Mon Sep 17 00:00:00 2001 From: openhands Date: Thu, 28 May 2026 16:29:08 -0600 Subject: [PATCH 01/17] fix(openhands-automation): warn about webhook limitations on local deployments and recommend polling Webhooks require an internet-accessible URL. When AGENT_SERVER_URL or is a local address (localhost, 127.0.0.1, etc.), external services cannot deliver events to the automation service. Changes: - Add CRITICAL rule 5: check for local URL before suggesting event triggers; recommend cron-based polling and reference the github-repo-monitor skill - Flag Event trigger type in the Trigger Types table as requiring a public URL - Add a callout box to the Trigger Types section linking to the new alternative - Add a new 'Polling as a Webhook Alternative' section with a GitHub example, a polling-vs-webhooks comparison table, and a tip to the github-repo-monitor skill - Add a public-URL-required warning at the top of the Event-Triggered Automations section Co-authored-by: openhands --- skills/openhands-automation/SKILL.md | 44 +++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index d6cb62e..c5e262e 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -53,6 +53,10 @@ The agent server typically runs inside a **sandbox** (a Docker or Kubernetes con > - **Custom script** — full control over code, with or without LLM; point them to `references/custom-automation.md` > - Let the user choose which approach to use. > 4. **Only create custom scripts after the user agrees to that path.** Refer to `references/custom-automation.md` for the full reference. +> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If the `AGENT_SERVER_URL` environment variable or the `` value in the system prompt is a local address — e.g., `http://localhost:...`, `http://127.0.0.1:...`, or `http://0.0.0.0:...` — the service cannot receive inbound webhook traffic from the public internet. In that case: +> - **Recommend a cron-based polling automation instead.** Have the automation run on a schedule and call the external service's API (e.g., the GitHub REST API) to check for new events since the last run. +> - Explain the limitation clearly to the user: "Because this is a local deployment, external services can't reach the webhook endpoint. I'll set up a polling automation using a cron schedule instead." +> - The `github-repo-monitor` skill provides a ready-made polling pattern for GitHub repositories. ### No-LLM Script Helpers @@ -138,7 +142,9 @@ Automations support two trigger types: | Trigger Type | Use Case | |--------------|----------| | **Cron** | Run on a schedule (daily, weekly, hourly, etc.) | -| **Event** | Run when a webhook event occurs (GitHub PR opened, issue commented, etc.) | +| **Event** | Run when a webhook event occurs (GitHub PR opened, issue commented, etc.) — **requires a publicly reachable deployment** | + +> **⚠️ Local deployments:** Event triggers rely on external services (GitHub, Slack, etc.) sending HTTP requests to the automation service. This only works when the service is hosted at a publicly accessible URL. If `AGENT_SERVER_URL` is a local address (`localhost`, `127.0.0.1`, etc.), use a **cron trigger** with polling instead — see [Polling as a Webhook Alternative](#polling-as-a-webhook-alternative). --- @@ -266,8 +272,44 @@ curl -X POST "${OPENHANDS_HOST}/api/automation/v1/preset/prompt" \ --- +## Polling as a Webhook Alternative + +When the deployment is local (i.e., `AGENT_SERVER_URL` or `` is `localhost`, `127.0.0.1`, `0.0.0.0`, or any other non-public address), external services cannot deliver webhook events to it. Use a **cron-triggered polling automation** instead. + +The polling automation runs on a schedule, calls the external service's API to fetch recent activity, and acts on anything new since the last run. + +### GitHub Polling Example + +The following automation checks for new pull requests opened in the last hour and processes them: + +```bash +curl -X POST "${OPENHANDS_HOST}/api/automation/v1/preset/prompt" \ + -H "Authorization: Bearer ${OPENHANDS_API_KEY}" \ + -H "Content-Type: application/json" \ + -d '{ + "name": "Poll GitHub for New PRs", + "prompt": "Use the GitHub API to fetch pull requests opened in the last hour for the repository owner/repo. For each new PR, review the code and post a structured review comment.", + "trigger": {"type": "cron", "schedule": "0 * * * *", "timezone": "UTC"} + }' +``` + +> **Tip:** The `github-repo-monitor` skill provides a more fully-featured pattern for polling a GitHub repository and reacting to new issues, PRs, or comments containing a trigger phrase. + +### Polling vs. Webhooks at a Glance + +| | Webhooks (Event trigger) | Polling (Cron trigger) | +|---|---|---| +| **Requires public URL** | Yes | No — works locally | +| **Latency** | Near-instant | Up to one poll interval | +| **API calls** | Only on real events | Every poll interval | +| **Best for** | Cloud / public deployments | Local or private deployments | + +--- + ## Event-Triggered Automations (Webhooks) +> **⚠️ Public URL required.** Event-triggered automations rely on external services posting HTTP events to the automation service. If your deployment is local (e.g., `AGENT_SERVER_URL` is `http://localhost:...` or `http://127.0.0.1:...`), use a [cron-based polling automation](#polling-as-a-webhook-alternative) instead. + Event-triggered automations run when a webhook event occurs — like a GitHub PR being opened, an issue receiving a comment, or a custom service sending a notification. ### Built-in Integrations From c74ba92cc5f010117e7f83152af340e48ac244e6 Mon Sep 17 00:00:00 2001 From: tofarr Date: Thu, 28 May 2026 17:29:12 -0600 Subject: [PATCH 02/17] Update skills --- skills/openhands-sdk/SKILL.md | 1 + 1 file changed, 1 insertion(+) diff --git a/skills/openhands-sdk/SKILL.md b/skills/openhands-sdk/SKILL.md index f0fe263..427bc21 100644 --- a/skills/openhands-sdk/SKILL.md +++ b/skills/openhands-sdk/SKILL.md @@ -188,6 +188,7 @@ Source: [`examples/`](https://github.com/OpenHands/software-agent-sdk/tree/main/ - [`48_conversation_fork.py`](https://github.com/OpenHands/software-agent-sdk/blob/main/examples/01_standalone_sdk/48_conversation_fork.py) - [`49_switch_llm_tool.py`](https://github.com/OpenHands/software-agent-sdk/blob/main/examples/01_standalone_sdk/49_switch_llm_tool.py) - [`50_async_cancellation.py`](https://github.com/OpenHands/software-agent-sdk/blob/main/examples/01_standalone_sdk/50_async_cancellation.py) +- [`51_agent_hooks`](https://github.com/OpenHands/software-agent-sdk/tree/main/examples/01_standalone_sdk/51_agent_hooks) ### [`02_remote_agent_server/`](https://github.com/OpenHands/software-agent-sdk/tree/main/examples/02_remote_agent_server) From ba4e98405d82c40399c7df626b75405c01b1e5e0 Mon Sep 17 00:00:00 2001 From: openhands Date: Fri, 29 May 2026 06:35:43 -0600 Subject: [PATCH 03/17] fix(openhands-automation): use RUNTIME_URL for locality check; remove github-repo-monitor refs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace all locality-check references to AGENT_SERVER_URL / with RUNTIME_URL — the variable that is actually present in production K8s pods. AGENT_SERVER_URL and HOST are absent there. Also remove the two mentions of the github-repo-monitor skill: the openhands-automation skill should stand alone without cross-referencing other skills. Changes: - Rule 5: check RUNTIME_URL (with explicit echo command), remove github-repo-monitor bullet - Trigger Types warning: RUNTIME_URL - Polling as a Webhook Alternative intro: RUNTIME_URL - Polling Tip blockquote: removed (referenced github-repo-monitor) - Event-Triggered Automations warning: RUNTIME_URL Co-authored-by: openhands --- skills/openhands-automation/SKILL.md | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index c5e262e..78323e6 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -53,10 +53,9 @@ The agent server typically runs inside a **sandbox** (a Docker or Kubernetes con > - **Custom script** — full control over code, with or without LLM; point them to `references/custom-automation.md` > - Let the user choose which approach to use. > 4. **Only create custom scripts after the user agrees to that path.** Refer to `references/custom-automation.md` for the full reference. -> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If the `AGENT_SERVER_URL` environment variable or the `` value in the system prompt is a local address — e.g., `http://localhost:...`, `http://127.0.0.1:...`, or `http://0.0.0.0:...` — the service cannot receive inbound webhook traffic from the public internet. In that case: +> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** Run `echo "RUNTIME_URL=${RUNTIME_URL}"`. Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, `0.0.0.0`, etc.), the service cannot receive inbound webhook traffic from the public internet. In that case: > - **Recommend a cron-based polling automation instead.** Have the automation run on a schedule and call the external service's API (e.g., the GitHub REST API) to check for new events since the last run. > - Explain the limitation clearly to the user: "Because this is a local deployment, external services can't reach the webhook endpoint. I'll set up a polling automation using a cron schedule instead." -> - The `github-repo-monitor` skill provides a ready-made polling pattern for GitHub repositories. ### No-LLM Script Helpers @@ -144,7 +143,7 @@ Automations support two trigger types: | **Cron** | Run on a schedule (daily, weekly, hourly, etc.) | | **Event** | Run when a webhook event occurs (GitHub PR opened, issue commented, etc.) — **requires a publicly reachable deployment** | -> **⚠️ Local deployments:** Event triggers rely on external services (GitHub, Slack, etc.) sending HTTP requests to the automation service. This only works when the service is hosted at a publicly accessible URL. If `AGENT_SERVER_URL` is a local address (`localhost`, `127.0.0.1`, etc.), use a **cron trigger** with polling instead — see [Polling as a Webhook Alternative](#polling-as-a-webhook-alternative). +> **⚠️ Local deployments:** Event triggers rely on external services (GitHub, Slack, etc.) sending HTTP requests to the automation service. This only works when the service is hosted at a publicly accessible URL. If `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, etc.), use a **cron trigger** with polling instead — see [Polling as a Webhook Alternative](#polling-as-a-webhook-alternative). --- @@ -274,7 +273,7 @@ curl -X POST "${OPENHANDS_HOST}/api/automation/v1/preset/prompt" \ ## Polling as a Webhook Alternative -When the deployment is local (i.e., `AGENT_SERVER_URL` or `` is `localhost`, `127.0.0.1`, `0.0.0.0`, or any other non-public address), external services cannot deliver webhook events to it. Use a **cron-triggered polling automation** instead. +When `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, `0.0.0.0`, etc.), external services cannot deliver webhook events to it. Use a **cron-triggered polling automation** instead. The polling automation runs on a schedule, calls the external service's API to fetch recent activity, and acts on anything new since the last run. @@ -293,8 +292,6 @@ curl -X POST "${OPENHANDS_HOST}/api/automation/v1/preset/prompt" \ }' ``` -> **Tip:** The `github-repo-monitor` skill provides a more fully-featured pattern for polling a GitHub repository and reacting to new issues, PRs, or comments containing a trigger phrase. - ### Polling vs. Webhooks at a Glance | | Webhooks (Event trigger) | Polling (Cron trigger) | @@ -308,7 +305,7 @@ curl -X POST "${OPENHANDS_HOST}/api/automation/v1/preset/prompt" \ ## Event-Triggered Automations (Webhooks) -> **⚠️ Public URL required.** Event-triggered automations rely on external services posting HTTP events to the automation service. If your deployment is local (e.g., `AGENT_SERVER_URL` is `http://localhost:...` or `http://127.0.0.1:...`), use a [cron-based polling automation](#polling-as-a-webhook-alternative) instead. +> **⚠️ Public URL required.** Event-triggered automations rely on external services posting HTTP events to the automation service. If `RUNTIME_URL` is unset, empty, or local, use a [cron-based polling automation](#polling-as-a-webhook-alternative) instead. Event-triggered automations run when a webhook event occurs — like a GitHub PR being opened, an issue receiving a comment, or a custom service sending a notification. From 0be327b2d2b16065f45cde073121b106b8650594 Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 06:38:20 -0600 Subject: [PATCH 04/17] Update SKILL.md --- skills/openhands-automation/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index 78323e6..5be55cf 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -53,7 +53,7 @@ The agent server typically runs inside a **sandbox** (a Docker or Kubernetes con > - **Custom script** — full control over code, with or without LLM; point them to `references/custom-automation.md` > - Let the user choose which approach to use. > 4. **Only create custom scripts after the user agrees to that path.** Refer to `references/custom-automation.md` for the full reference. -> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** Run `echo "RUNTIME_URL=${RUNTIME_URL}"`. Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, `0.0.0.0`, etc.), the service cannot receive inbound webhook traffic from the public internet. In that case: +> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** check the `RUNTIME_URL` environment variable. Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, `0.0.0.0`, etc.), the service cannot receive inbound webhook traffic from the public internet. In that case: > - **Recommend a cron-based polling automation instead.** Have the automation run on a schedule and call the external service's API (e.g., the GitHub REST API) to check for new events since the last run. > - Explain the limitation clearly to the user: "Because this is a local deployment, external services can't reach the webhook endpoint. I'll set up a polling automation using a cron schedule instead." From 548f41b3372fd2e752563b34662e8201757498e5 Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 06:45:05 -0600 Subject: [PATCH 05/17] Remove GitHub Polling Example section from SKILL.md Removed GitHub polling example and related details from SKILL.md. --- skills/openhands-automation/SKILL.md | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index 5be55cf..31db0c0 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -277,21 +277,6 @@ When `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1` The polling automation runs on a schedule, calls the external service's API to fetch recent activity, and acts on anything new since the last run. -### GitHub Polling Example - -The following automation checks for new pull requests opened in the last hour and processes them: - -```bash -curl -X POST "${OPENHANDS_HOST}/api/automation/v1/preset/prompt" \ - -H "Authorization: Bearer ${OPENHANDS_API_KEY}" \ - -H "Content-Type: application/json" \ - -d '{ - "name": "Poll GitHub for New PRs", - "prompt": "Use the GitHub API to fetch pull requests opened in the last hour for the repository owner/repo. For each new PR, review the code and post a structured review comment.", - "trigger": {"type": "cron", "schedule": "0 * * * *", "timezone": "UTC"} - }' -``` - ### Polling vs. Webhooks at a Glance | | Webhooks (Event trigger) | Polling (Cron trigger) | From f5b6c37230dfcb85aae202cbf394ad38569ba2b8 Mon Sep 17 00:00:00 2001 From: openhands Date: Fri, 29 May 2026 06:51:32 -0600 Subject: [PATCH 06/17] refactor(openhands-automation): remove repetition MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - RUNTIME_URL locality check: was stated in Rule 5, Trigger Types warning, Polling section intro, and Event-Triggered warning. Keep only Rule 5; remove the three restatements. - 'How It Works' implementation-detail blocks: removed from both Prompt Preset and Plugin Preset sections — not actionable for the agent. - 'Choosing the Right Preset' prose paragraphs: removed the three paragraphs that restated Rules 0 and 3; the decision table is sufficient. Net: -27 lines, no information lost. Co-authored-by: openhands --- skills/openhands-automation/SKILL.md | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index 31db0c0..55a14b5 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -143,8 +143,6 @@ Automations support two trigger types: | **Cron** | Run on a schedule (daily, weekly, hourly, etc.) | | **Event** | Run when a webhook event occurs (GitHub PR opened, issue commented, etc.) — **requires a publicly reachable deployment** | -> **⚠️ Local deployments:** Event triggers rely on external services (GitHub, Slack, etc.) sending HTTP requests to the automation service. This only works when the service is hosted at a publicly accessible URL. If `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, etc.), use a **cron trigger** with polling instead — see [Polling as a Webhook Alternative](#polling-as-a-webhook-alternative). - --- ## Creating Automations @@ -160,12 +158,6 @@ Two preset endpoints simplify automation creation by handling SDK boilerplate, t Use the **preset/prompt endpoint** for simple automations. Provide a natural language prompt describing the task. -#### How It Works - -1. Send a prompt describing the task (e.g., "Generate a weekly status report") -2. The automation service generates a Python script that: fetches LLM config and secrets from the agent server, starts an AI agent conversation with your prompt, and sends a completion callback when done -3. The script is packaged as a tarball and the automation is registered; on each trigger, the automation service uploads the tarball to the agent server, which unpacks and runs the script inside its environment - #### Request ```bash @@ -273,10 +265,6 @@ curl -X POST "${OPENHANDS_HOST}/api/automation/v1/preset/prompt" \ ## Polling as a Webhook Alternative -When `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, `0.0.0.0`, etc.), external services cannot deliver webhook events to it. Use a **cron-triggered polling automation** instead. - -The polling automation runs on a schedule, calls the external service's API to fetch recent activity, and acts on anything new since the last run. - ### Polling vs. Webhooks at a Glance | | Webhooks (Event trigger) | Polling (Cron trigger) | @@ -290,8 +278,6 @@ The polling automation runs on a schedule, calls the external service's API to f ## Event-Triggered Automations (Webhooks) -> **⚠️ Public URL required.** Event-triggered automations rely on external services posting HTTP events to the automation service. If `RUNTIME_URL` is unset, empty, or local, use a [cron-based polling automation](#polling-as-a-webhook-alternative) instead. - Event-triggered automations run when a webhook event occurs — like a GitHub PR being opened, an issue receiving a comment, or a custom service sending a notification. ### Built-in Integrations @@ -606,13 +592,6 @@ Use the **preset/plugin endpoint** when you need to load one or more plugins tha > **💡 Finding plugins:** Browse the [OpenHands/extensions](https://github.com/OpenHands/extensions) repository for available skills and plugins. When given a broad use case, check this directory first to see if something already exists that fits your needs. -#### How It Works - -1. Specify one or more plugins (from GitHub repos, git URLs, or monorepo subdirectories) -2. Provide a prompt that can invoke plugin commands (e.g., `/plugin-name:command`) -3. The service generates SDK boilerplate that loads all plugins at runtime, creates a conversation with plugin capabilities, and executes the prompt -4. The service packages everything into a tarball, uploads it, and creates the automation - #### Request ```bash @@ -844,12 +823,6 @@ Pick based on **what the task needs**, not just **what is technically possible** | **Deterministic task** (fixed data + scheduled action, e.g. healthcheck, Slack notification, rotating from a known list) — especially if it runs frequently | **Custom script, no LLM** — see `references/custom-automation.md#deterministic-script-no-llm` | | Custom Python dependencies, multi-file project, or direct SDK lifecycle control | **Custom script with SDK** — see `references/custom-automation.md#sdk-based-scripts` | -The **prompt preset** is the right default for genuinely agent-shaped work — anything that benefits from reasoning over context, calling tools dynamically, or producing a non-templated output. Use the **plugin preset** when you need extended capabilities from plugins (skills, MCP configurations, hooks, commands). - -**Watch for deterministic, high-frequency patterns.** Requests like "send a daily standup reminder", "ping a healthcheck URL every minute", "post a random quote every 5 minutes", or "rotate a fact-of-the-day message" do not need an LLM. Surface this to the user explicitly with a rough cost framing (e.g. "this schedule will invoke your LLM ~288 times/day") before defaulting to a preset. As a rule of thumb, any cron tighter than hourly deserves a deliberate "should this really be agent-driven?" check. - -**When neither preset is the right fit** (deterministic task, custom Python dependencies, non-Python entrypoint, multi-file project structure, direct SDK lifecycle control), explain the options to the user and let them decide. Do not attempt custom automation without explicit user agreement. If they choose the custom route, refer to `references/custom-automation.md`. - ## Reference Files - **`references/custom-automation.md`** — Detailed guide for custom automations: tarball uploads, code structure (SDK and no-LLM), environment variables, validation rules, and complete examples. Consult this whenever you need to evaluate or recommend the custom path (including for deterministic / cost-sensitive tasks per rule 0). Only *implement* a custom automation after the user agrees to that path. From 6df73f0545de8060f7d1f0278cdf053ed1ea6832 Mon Sep 17 00:00:00 2001 From: openhands Date: Fri, 29 May 2026 06:55:44 -0600 Subject: [PATCH 07/17] revert: restore 'How It Works' sections in Prompt and Plugin presets Co-authored-by: openhands --- skills/openhands-automation/SKILL.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index 55a14b5..116790d 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -158,6 +158,12 @@ Two preset endpoints simplify automation creation by handling SDK boilerplate, t Use the **preset/prompt endpoint** for simple automations. Provide a natural language prompt describing the task. +#### How It Works + +1. Send a prompt describing the task (e.g., "Generate a weekly status report") +2. The automation service generates a Python script that: fetches LLM config and secrets from the agent server, starts an AI agent conversation with your prompt, and sends a completion callback when done +3. The script is packaged as a tarball and the automation is registered; on each trigger, the automation service uploads the tarball to the agent server, which unpacks and runs the script inside its environment + #### Request ```bash @@ -592,6 +598,13 @@ Use the **preset/plugin endpoint** when you need to load one or more plugins tha > **💡 Finding plugins:** Browse the [OpenHands/extensions](https://github.com/OpenHands/extensions) repository for available skills and plugins. When given a broad use case, check this directory first to see if something already exists that fits your needs. +#### How It Works + +1. Specify one or more plugins (from GitHub repos, git URLs, or monorepo subdirectories) +2. Provide a prompt that can invoke plugin commands (e.g., `/plugin-name:command`) +3. The service generates SDK boilerplate that loads all plugins at runtime, creates a conversation with plugin capabilities, and executes the prompt +4. The service packages everything into a tarball, uploads it, and creates the automation + #### Request ```bash From e924ee47525f2841d0769359979934a0fd9ba9a0 Mon Sep 17 00:00:00 2001 From: openhands Date: Fri, 29 May 2026 06:57:38 -0600 Subject: [PATCH 08/17] revert: restore prompt-vs-plugin guidance paragraph after preset table Co-authored-by: openhands --- skills/openhands-automation/SKILL.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index 116790d..addd84f 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -836,6 +836,8 @@ Pick based on **what the task needs**, not just **what is technically possible** | **Deterministic task** (fixed data + scheduled action, e.g. healthcheck, Slack notification, rotating from a known list) — especially if it runs frequently | **Custom script, no LLM** — see `references/custom-automation.md#deterministic-script-no-llm` | | Custom Python dependencies, multi-file project, or direct SDK lifecycle control | **Custom script with SDK** — see `references/custom-automation.md#sdk-based-scripts` | +The **prompt preset** is the right default for genuinely agent-shaped work — anything that benefits from reasoning over context, calling tools dynamically, or producing a non-templated output. Use the **plugin preset** when you need extended capabilities from plugins (skills, MCP configurations, hooks, commands). + ## Reference Files - **`references/custom-automation.md`** — Detailed guide for custom automations: tarball uploads, code structure (SDK and no-LLM), environment variables, validation rules, and complete examples. Consult this whenever you need to evaluate or recommend the custom path (including for deterministic / cost-sensitive tasks per rule 0). Only *implement* a custom automation after the user agrees to that path. From 94385d2bda5ba3c6349a7cae07d06b334f5a0ad1 Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 06:58:56 -0600 Subject: [PATCH 09/17] Enhance SKILL.md with automation guidelines Added guidance on handling deterministic patterns and custom automation options. --- skills/openhands-automation/SKILL.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index addd84f..e424c36 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -823,6 +823,10 @@ When a run completes, the automation service receives a callback and marks the r The automation script itself controls when the callback fires (signalling completion). For simple synchronous scripts this happens naturally on exit. For scripts that start asynchronous conversations, the callback should be deferred until the conversation reaches an idle state (see `references/custom-automation.md` for patterns). +**Watch for deterministic, high-frequency patterns.** Requests like "send a daily standup reminder", "ping a healthcheck URL every minute", "post a random quote every 5 minutes", or "rotate a fact-of-the-day message" do not need an LLM. Surface this to the user explicitly with a rough cost framing (e.g. "this schedule will invoke your LLM ~288 times/day") before defaulting to a preset. As a rule of thumb, any cron tighter than hourly deserves a deliberate "should this really be agent-driven?" check. + +**When neither preset is the right fit** (deterministic task, custom Python dependencies, non-Python entrypoint, multi-file project structure, direct SDK lifecycle control), explain the options to the user and let them decide. Do not attempt custom automation without explicit user agreement. If they choose the custom route, refer to `references/custom-automation.md`. + --- ## Choosing the Right Preset From 6f328425d85b488d979a59070528c779a046832b Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 07:00:00 -0600 Subject: [PATCH 10/17] Refactor SKILL.md for clarity on automation patterns Removed redundant sections about handling deterministic patterns and custom automation options, and added them back in a more concise manner. --- skills/openhands-automation/SKILL.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index e424c36..68b404b 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -823,10 +823,6 @@ When a run completes, the automation service receives a callback and marks the r The automation script itself controls when the callback fires (signalling completion). For simple synchronous scripts this happens naturally on exit. For scripts that start asynchronous conversations, the callback should be deferred until the conversation reaches an idle state (see `references/custom-automation.md` for patterns). -**Watch for deterministic, high-frequency patterns.** Requests like "send a daily standup reminder", "ping a healthcheck URL every minute", "post a random quote every 5 minutes", or "rotate a fact-of-the-day message" do not need an LLM. Surface this to the user explicitly with a rough cost framing (e.g. "this schedule will invoke your LLM ~288 times/day") before defaulting to a preset. As a rule of thumb, any cron tighter than hourly deserves a deliberate "should this really be agent-driven?" check. - -**When neither preset is the right fit** (deterministic task, custom Python dependencies, non-Python entrypoint, multi-file project structure, direct SDK lifecycle control), explain the options to the user and let them decide. Do not attempt custom automation without explicit user agreement. If they choose the custom route, refer to `references/custom-automation.md`. - --- ## Choosing the Right Preset @@ -842,6 +838,10 @@ Pick based on **what the task needs**, not just **what is technically possible** The **prompt preset** is the right default for genuinely agent-shaped work — anything that benefits from reasoning over context, calling tools dynamically, or producing a non-templated output. Use the **plugin preset** when you need extended capabilities from plugins (skills, MCP configurations, hooks, commands). +**Watch for deterministic, high-frequency patterns.** Requests like "send a daily standup reminder", "ping a healthcheck URL every minute", "post a random quote every 5 minutes", or "rotate a fact-of-the-day message" do not need an LLM. Surface this to the user explicitly with a rough cost framing (e.g. "this schedule will invoke your LLM ~288 times/day") before defaulting to a preset. As a rule of thumb, any cron tighter than hourly deserves a deliberate "should this really be agent-driven?" check. + +**When neither preset is the right fit** (deterministic task, custom Python dependencies, non-Python entrypoint, multi-file project structure, direct SDK lifecycle control), explain the options to the user and let them decide. Do not attempt custom automation without explicit user agreement. If they choose the custom route, refer to `references/custom-automation.md`. + ## Reference Files - **`references/custom-automation.md`** — Detailed guide for custom automations: tarball uploads, code structure (SDK and no-LLM), environment variables, validation rules, and complete examples. Consult this whenever you need to evaluate or recommend the custom path (including for deterministic / cost-sensitive tasks per rule 0). Only *implement* a custom automation after the user agrees to that path. From 84881a2d9f49efb97365340c9af85f2e55e4849b Mon Sep 17 00:00:00 2001 From: openhands Date: Fri, 29 May 2026 07:26:18 -0600 Subject: [PATCH 11/17] docs(openhands-automation): clarify environment variables (RUNTIME_URL vs AGENT_SERVER_URL) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The review bot flagged RUNTIME_URL as undefined. This commit makes the environment variable landscape unambiguous: - RUNTIME_URL: ambient in cloud environments; the public-facing URL of the sandbox (both agent server and automation service). Used as the API host for all automation calls and to determine whether external webhook delivery is possible. - AGENT_SERVER_URL: NOT an ambient env var. Injected by the automation service into scripts at run time only. Only accessible inside a running script. - OPENHANDS_HOST: not a real environment variable. Used throughout examples as a shell-variable convention; now documented explicitly with the one-liner to resolve it from RUNTIME_URL. Changes: - Architecture section: add key env vars table; update component descriptions to match reality. - Rule 5: fix capitalisation (check → Check). - Determining the API Host: replace ' system prompt' instruction with RUNTIME_URL and the OPENHANDS_HOST shell-variable pattern. Co-authored-by: openhands --- skills/openhands-automation/SKILL.md | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index 68b404b..d26405a 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -24,14 +24,21 @@ Create and manage automations that run inside an OpenHands agent server — trig Two components work together to run automations: -**Automation Service** (API at `OPENHANDS_HOST/api/automation/v1`) +**Automation Service** (API at `{RUNTIME_URL}/api/automation/v1`) Manages the *when*: holds automation definitions, schedules cron-triggered runs, dispatches webhook-triggered runs, and receives completion callbacks to mark runs as done. This is the API you call to create, update, and manage automations. -**Agent Server** (reachable at `AGENT_SERVER_URL` inside a run) -Manages the *what*: the runtime environment where automation scripts execute and where conversations (AI agent interactions with tools, bash, file editing, etc.) run. When a run is triggered, the automation service uploads the automation's tarball to the agent server, which unpacks and runs the entrypoint script. The script runs inside the agent server and connects back to it using `AGENT_SERVER_URL` and a session API key to start, monitor, and stop conversations. +**Agent Server** (accessible as `AGENT_SERVER_URL` inside script runs) +Manages the *what*: the runtime environment where automation scripts execute and where conversations (AI agent interactions with tools, bash, file editing, etc.) run. When a run is triggered, the automation service uploads the automation's tarball to the agent server, which unpacks and runs the entrypoint script. The script connects back to the agent server using `AGENT_SERVER_URL` and a session API key to start, monitor, and stop conversations. The agent server typically runs inside a **sandbox** (a Docker or Kubernetes container). Some deployments use sandboxless mode, where the agent server runs directly on a host. +**Key environment variables:** + +| Variable | Availability | Description | +|---|---|---| +| `RUNTIME_URL` | Ambient in cloud environments | Public-facing URL of the sandbox (agent server + automation service). Use this as the API host for all automation service calls. Also use it to determine whether external webhook delivery is possible — if unset or local, webhooks cannot be received. | +| `AGENT_SERVER_URL` | Injected into scripts at run time only | Internal URL of the agent server. Available inside script execution context; **not** an ambient environment variable outside of a running script. | + > **⚠️ CRITICAL — Agent behavior rules:** > > 0. **Does this task need an LLM at all? Check first.** Before picking a preset, ask whether the task actually requires reasoning, judgment, summarization, or open-ended tool use. If it is fully deterministic — fixed data transforms, scheduled HTTP calls, healthcheck pings, file rotation, picking from a known list, posting a templated message — an LLM-driven preset is overkill. Every run will consume LLM tokens, which adds up fast at high frequencies (every 5 min ≈ 288 runs/day). Surface the trade-off to the user and offer the custom-script path (see `references/custom-automation.md`) as the cheaper, more reliable option. Be especially careful for cron schedules tighter than hourly. @@ -53,7 +60,7 @@ The agent server typically runs inside a **sandbox** (a Docker or Kubernetes con > - **Custom script** — full control over code, with or without LLM; point them to `references/custom-automation.md` > - Let the user choose which approach to use. > 4. **Only create custom scripts after the user agrees to that path.** Refer to `references/custom-automation.md` for the full reference. -> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** check the `RUNTIME_URL` environment variable. Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, `0.0.0.0`, etc.), the service cannot receive inbound webhook traffic from the public internet. In that case: +> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** Check `RUNTIME_URL`. Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, `0.0.0.0`, etc.), the service cannot receive inbound webhook traffic from the public internet. In that case: > - **Recommend a cron-based polling automation instead.** Have the automation run on a schedule and call the external service's API (e.g., the GitHub REST API) to check for new events since the last run. > - Explain the limitation clearly to the user: "Because this is a local deployment, external services can't reach the webhook endpoint. I'll set up a polling automation using a cron schedule instead." @@ -105,7 +112,11 @@ All requests require Bearer authentication: **Before making API calls, determine the correct host:** -Look for a `` value in the system prompt. If present, use that URL. Otherwise, default to `https://app.all-hands.dev`. +`RUNTIME_URL` is the public-facing URL of the sandbox and is the API host. In the examples throughout this skill, `${OPENHANDS_HOST}` is a shell-variable convention — set it before running any curl command: + +```bash +OPENHANDS_HOST="${RUNTIME_URL:-https://app.all-hands.dev}" +``` ### Automation Endpoints From cbe6ee77e6b80de49776f250cedd98ed94e15958 Mon Sep 17 00:00:00 2001 From: openhands Date: Fri, 29 May 2026 07:52:55 -0600 Subject: [PATCH 12/17] fix(openhands-automation): correct RUNTIME_URL scope to agent server only RUNTIME_URL is the public-facing URL of the agent server sandbox, not the automation service. The automation service may run at a separate URL. - Architecture: restore OPENHANDS_HOST convention in Automation Service description (RUNTIME_URL was incorrect there) - Env vars table: clarify RUNTIME_URL is the agent server URL; note the automation service may be at a separate URL - Determining the API Host: remove the incorrect OPENHANDS_HOST=RUNTIME_URL one-liner; restore the system-prompt lookup with app.all-hands.dev as default Co-authored-by: openhands --- skills/openhands-automation/SKILL.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index d26405a..cf8abaf 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -24,7 +24,7 @@ Create and manage automations that run inside an OpenHands agent server — trig Two components work together to run automations: -**Automation Service** (API at `{RUNTIME_URL}/api/automation/v1`) +**Automation Service** (API at `{OPENHANDS_HOST}/api/automation/v1`) Manages the *when*: holds automation definitions, schedules cron-triggered runs, dispatches webhook-triggered runs, and receives completion callbacks to mark runs as done. This is the API you call to create, update, and manage automations. **Agent Server** (accessible as `AGENT_SERVER_URL` inside script runs) @@ -36,7 +36,7 @@ The agent server typically runs inside a **sandbox** (a Docker or Kubernetes con | Variable | Availability | Description | |---|---|---| -| `RUNTIME_URL` | Ambient in cloud environments | Public-facing URL of the sandbox (agent server + automation service). Use this as the API host for all automation service calls. Also use it to determine whether external webhook delivery is possible — if unset or local, webhooks cannot be received. | +| `RUNTIME_URL` | Ambient in cloud environments | Public-facing URL of the **agent server** sandbox. Use this to determine whether external webhook delivery is possible — if unset or local, webhooks cannot be received. The automation service may run at a separate URL (see Determining the API Host). | | `AGENT_SERVER_URL` | Injected into scripts at run time only | Internal URL of the agent server. Available inside script execution context; **not** an ambient environment variable outside of a running script. | > **⚠️ CRITICAL — Agent behavior rules:** @@ -112,10 +112,13 @@ All requests require Bearer authentication: **Before making API calls, determine the correct host:** -`RUNTIME_URL` is the public-facing URL of the sandbox and is the API host. In the examples throughout this skill, `${OPENHANDS_HOST}` is a shell-variable convention — set it before running any curl command: +The automation service may run at a different URL from the agent server. In the examples throughout this skill, `${OPENHANDS_HOST}` is a shell-variable convention for the automation service base URL — it is **not** a real environment variable. Set it from context before running any curl command: + +- Look for a `` value in the system prompt. If present, use that URL. +- Otherwise default to `https://app.all-hands.dev`. ```bash -OPENHANDS_HOST="${RUNTIME_URL:-https://app.all-hands.dev}" +OPENHANDS_HOST="https://app.all-hands.dev" # replace with if provided ``` From d3d21707c084b81d4777ee8374fe7c7ac7c380a9 Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 07:54:38 -0600 Subject: [PATCH 13/17] Fix API URL formatting in SKILL.md Corrected formatting for API URL in documentation. --- skills/openhands-automation/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index cf8abaf..a2306ae 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -24,7 +24,7 @@ Create and manage automations that run inside an OpenHands agent server — trig Two components work together to run automations: -**Automation Service** (API at `{OPENHANDS_HOST}/api/automation/v1`) +**Automation Service** (API at `OPENHANDS_HOST/api/automation/v1`) Manages the *when*: holds automation definitions, schedules cron-triggered runs, dispatches webhook-triggered runs, and receives completion callbacks to mark runs as done. This is the API you call to create, update, and manage automations. **Agent Server** (accessible as `AGENT_SERVER_URL` inside script runs) From e8537e0fc9708d90a70fa41b542ba188853264ea Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 08:40:47 -0600 Subject: [PATCH 14/17] Update skills/openhands-automation/SKILL.md Co-authored-by: OpenHands Bot --- skills/openhands-automation/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index a2306ae..0604774 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -60,7 +60,7 @@ The agent server typically runs inside a **sandbox** (a Docker or Kubernetes con > - **Custom script** — full control over code, with or without LLM; point them to `references/custom-automation.md` > - Let the user choose which approach to use. > 4. **Only create custom scripts after the user agrees to that path.** Refer to `references/custom-automation.md` for the full reference. -> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** Check `RUNTIME_URL`. Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If `RUNTIME_URL` is unset, empty, or a local address (`localhost`, `127.0.0.1`, `0.0.0.0`, etc.), the service cannot receive inbound webhook traffic from the public internet. In that case: +> 5. **Before suggesting event-triggered (webhook) automations, check whether the deployment is publicly reachable.** Check `RUNTIME_URL`. Webhooks require an internet-accessible URL so that external services (GitHub, Slack, Linear, etc.) can deliver events to the automation service. If `RUNTIME_URL` is unset, empty, or resolves to a local or private address (`localhost`, `127.0.0.1`, `0.0.0.0`, or any RFC 1918 range: `10.x.x.x`, `192.168.x.x`, `172.16–31.x.x`), the service cannot receive inbound webhook traffic from the public internet. In that case: > - **Recommend a cron-based polling automation instead.** Have the automation run on a schedule and call the external service's API (e.g., the GitHub REST API) to check for new events since the last run. > - Explain the limitation clearly to the user: "Because this is a local deployment, external services can't reach the webhook endpoint. I'll set up a polling automation using a cron schedule instead." From fbeec8b006a62e1044790e86dff56eabe7f824f9 Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 08:44:52 -0600 Subject: [PATCH 15/17] Update skills/openhands-automation/SKILL.md Co-authored-by: OpenHands Bot --- skills/openhands-automation/SKILL.md | 1 + 1 file changed, 1 insertion(+) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index 0604774..ca66368 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -38,6 +38,7 @@ The agent server typically runs inside a **sandbox** (a Docker or Kubernetes con |---|---|---| | `RUNTIME_URL` | Ambient in cloud environments | Public-facing URL of the **agent server** sandbox. Use this to determine whether external webhook delivery is possible — if unset or local, webhooks cannot be received. The automation service may run at a separate URL (see Determining the API Host). | | `AGENT_SERVER_URL` | Injected into scripts at run time only | Internal URL of the agent server. Available inside script execution context; **not** an ambient environment variable outside of a running script. | +| `OPENHANDS_HOST` | Shell convention only — set manually | Base URL for the automation service API. **Not a real environment variable.** Set it from the `` system-prompt value, or default to `https://app.all-hands.dev`. Used in all `curl` examples throughout this skill. | > **⚠️ CRITICAL — Agent behavior rules:** > From a10e9f79340a63ee028614a01a9001a3cf8c1561 Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 08:45:43 -0600 Subject: [PATCH 16/17] Update skills/openhands-automation/SKILL.md Co-authored-by: OpenHands Bot --- skills/openhands-automation/SKILL.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index ca66368..1224247 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -286,6 +286,10 @@ curl -X POST "${OPENHANDS_HOST}/api/automation/v1/preset/prompt" \ ## Polling as a Webhook Alternative +When the deployment cannot receive inbound webhook traffic (see rule 5), use a cron-triggered automation that calls the external service’s API on a schedule to check for new events. + +### Polling vs. Webhooks at a Glance + ### Polling vs. Webhooks at a Glance | | Webhooks (Event trigger) | Polling (Cron trigger) | From e14f740c59b4bfd7369d4bb6aea5eeb33dd05909 Mon Sep 17 00:00:00 2001 From: Tim O'Farrell Date: Fri, 29 May 2026 08:46:24 -0600 Subject: [PATCH 17/17] Remove duplicate 'Polling vs. Webhooks' header Removed duplicate section header for clarity. --- skills/openhands-automation/SKILL.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/skills/openhands-automation/SKILL.md b/skills/openhands-automation/SKILL.md index 1224247..a255219 100644 --- a/skills/openhands-automation/SKILL.md +++ b/skills/openhands-automation/SKILL.md @@ -290,8 +290,6 @@ When the deployment cannot receive inbound webhook traffic (see rule 5), use a c ### Polling vs. Webhooks at a Glance -### Polling vs. Webhooks at a Glance - | | Webhooks (Event trigger) | Polling (Cron trigger) | |---|---|---| | **Requires public URL** | Yes | No — works locally |