Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 35 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -427,29 +427,59 @@ Package local skill directories (a directory containing `SKILL.md`) and run them

**`load` vs `run`:** `load` only publishes the agent (run it later with `agent run --name <skill>`); `run` starts local tool workers, launches the agent, and streams the execution. `serve` starts only the workers so the skill can be driven from elsewhere (e.g. the UI).

**Tool task types.** Each script in `scripts/` plus the built-in tools are served as the
Conductor task type `{skillName}__{tool}` — `read_skill_file`, and with a workspace enabled
`list_workspace_files`, `read_workspace_file`, `search_workspace`, `git_status`, `git_diff`.
`inputParameters.command` becomes the script's argv, stdout becomes `{"result": "<stdout>"}`,
and a non-zero exit fails the task. Script language is chosen by extension
(`.py .sh .js .mjs .ts .rb .go .bat .cmd`).

Because a tool is just a task type, a plain workflow can call one with no agent involved —
point a `SIMPLE` task at `{skillName}__{tool}` while `skill serve` is running. See
[WORKER_SKILL.md](./WORKER_SKILL.md).

### Worker Commands

Run task workers that poll Conductor and execute work locally.

| Command | Description | Required Args | Optional Flags | Example |
|---------|-------------|---------------|----------------|---------|
| `worker js <js_file>` | Run a JavaScript worker (EXPERIMENTAL) | JS file | `--type` (required), `--count`, `--worker-id`, `--domain`, `--timeout` | `conductor worker js worker.js --type my_task` |
| `worker js <js_file>` | Run a JavaScript worker (EXPERIMENTAL) | JS file | `--type` (required), `--count`, `--worker-id`, `--domain`, `--poll-timeout` | `conductor worker js worker.js --type my_task` |
| `worker stdio <command> [args...]` | Poll tasks and execute a command via stdin/stdout | command | `--type` (required), `--count`, `--worker-id`, `--domain`, `--poll-timeout`, `--exec-timeout`, `--verbose` | `conductor worker stdio ./handler.sh --type my_task` |
| `worker remote` | Run a worker from the job-runner registry (EXPERIMENTAL, Orkes only) | None | `--type` (required), `--count`, `--worker-id`, `--domain`, `--timeout`, `--refresh` | `conductor worker remote --type my_task` |
| `worker remote` | Run a worker from the job-runner registry (EXPERIMENTAL, Orkes only) | None | `--type` (required), `--count`, `--worker-id`, `--domain`, `--poll-timeout`, `--exec-timeout`, `--refresh` | `conductor worker remote --type my_task` |
| `worker list-remote` | List workers in the job-runner registry (EXPERIMENTAL, Orkes only) | None | `--namespace` | `conductor worker list-remote` |

**Flags:**
- `--type` - Task type to poll for (required)
- `--count` - Number of tasks to poll in each batch (default: 1)
- `--worker-id` - Worker ID reported to the server
- `--domain` - Task domain
- `--timeout` / `--poll-timeout` - Poll timeout in milliseconds (default: 100)
- `--exec-timeout` - Execution timeout in seconds for `stdio` (default: 0 = no timeout)
- `--poll-timeout` - Server-side long-poll wait in milliseconds (default: 100)
- `--exec-timeout` - Per-task execution timeout in seconds. `stdio` and `remote` only — a
JavaScript worker runs in-process with no interrupt, so there is nothing to time out.
Default 0 (no timeout) for `stdio`, 100 for `remote`.
- `--timeout` - Deprecated hidden alias for `--poll-timeout`
- `--verbose` - Print task and result JSON to stdout (`stdio` command)
- `--refresh` - Force refresh the worker from the registry, ignoring cache
- `--namespace` - Registry namespace to list workers from (default: `default`)

See [WORKER_JS.md](./WORKER_JS.md) and [WORKER_STDIO.md](./WORKER_STDIO.md) for the worker protocols.
All flavours share one poll loop; they differ only in how user code runs and in the result
shape it returns:

| Flavour | Worker returns | Failure carries |
|---------|----------------|-----------------|
| `stdio` | `{"status","output","logs","reason"}` on stdout | `reasonForIncompletion` + logs |
| `js` | `{status, body}` from the script; `$.task` holds the task | `output.error` |
| skill tools | bare stdout, wrapped as `{"result": ...}` | `reasonForIncompletion` |

Workers exit on Ctrl-C/SIGTERM once the in-flight batch finishes — a running task is left
to complete and report its real result rather than being killed, which would report a
failure the worker inflicted on itself and consume one of the task's retries. A second
signal exits immediately. Child processes receive `TASK_TYPE`, `TASK_ID`, `WORKFLOW_ID`, `EXECUTION_ID`,
`POLL_DOMAIN`, and the CLI's own `CONDUCTOR_SERVER_URL` and credentials.

See [WORKER_JS.md](./WORKER_JS.md), [WORKER_STDIO.md](./WORKER_STDIO.md) and
[WORKER_SKILL.md](./WORKER_SKILL.md) for the worker protocols.

### Development Commands

Expand Down
24 changes: 19 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ After installing, you'll get tab completion when typing `conductor <TAB>`.
* [Workers](#workers)
* [Stdio Workers](#stdio-workers)
* [JavaScript Workers (Built-in)](#javascript-workers--built-in-)
* [Skill Workers](#skill-workers)
* [Remote Workers (Registry-based)](#remote-workers--registry-based-)
* [Exit Codes](#exit-codes)
* [Error Handling](#error-handling)
Expand Down Expand Up @@ -548,8 +549,8 @@ conductor worker <command> [arguments] [flags]
| Command | Description |
|---------|-------------|
| `stdio <program> [args...]` | Run stdio worker (`--type`, `--count`, `--worker-id`, `--domain`, `--poll-timeout`, `--exec-timeout`, `--verbose`) |
| `js <file>` | Run JavaScript worker (`--type`, `--count`, `--worker-id`, `--domain`, `--timeout`) |
| `remote` | Run remote worker (`--type`, `--count`, `--worker-id`, `--domain`, `--refresh`) |
| `js <file>` | Run JavaScript worker (`--type`, `--count`, `--worker-id`, `--domain`, `--poll-timeout`) |
| `remote` | Run remote worker (`--type`, `--count`, `--worker-id`, `--domain`, `--poll-timeout`, `--exec-timeout`, `--refresh`) |
| `list-remote` | List remote workers (`--namespace`) |

**Worker Options:**
Expand Down Expand Up @@ -852,7 +853,7 @@ conductor --config /path/to/my-config.yaml workflow list

⚠️ **EXPERIMENTAL FEATURES**

The CLI supports two types of workers for processing Conductor tasks:
The CLI supports several types of workers for processing Conductor tasks:

### Stdio Workers

Expand All @@ -879,14 +880,27 @@ Execute tasks using **JavaScript** scripts with built-in utilities (HTTP, crypto

**Best for:** Prototyping, Lightweight tasks, quick scripts, HTTP integrations

👉 **[Complete JavaScript Worker Documentation →](WORKER_JS.md)**

**Quick example:**
```bash
# Run a JavaScript worker
conductor worker js --type greet_task worker.js
```

👉 **[Complete JavaScript Worker Documentation →](WORKER_JS.md)**

### Skill Workers

A skill directory (`SKILL.md` plus `scripts/`) serves each of its scripts as a
Conductor task type via `conductor skill serve`. Scripts take their arguments from
`inputParameters.command` and return bare stdout, so there is no result envelope to
emit. Works with or without an agent.

```bash
conductor skill serve ./myskill
```

👉 **[Complete Skill Worker Documentation →](WORKER_SKILL.md)**

### Remote Workers (Registry-based)

⚠️ **EXPERIMENTAL** - Download and execute workers directly from your Conductor Conductor instance without managing local files.
Expand Down
8 changes: 7 additions & 1 deletion WORKER_JS.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,11 @@ conductor worker js --type <task_type> <js_file>
- `--count` - Number of tasks to poll in each batch (default: 1)
- `--worker-id` - Worker ID for identification
- `--domain` - Domain for task polling
- `--timeout` - Timeout in milliseconds (default: 100)
- `--poll-timeout` - Poll timeout in milliseconds (default: 100)
- `--timeout` - Deprecated alias for `--poll-timeout`

There is no execution timeout for JavaScript workers: scripts run in-process and the
interpreter has no interrupt wired, so a script that loops forever blocks its worker.

### Example

Expand Down Expand Up @@ -874,3 +878,5 @@ if (parsed.error) {
| Custom Go functions | Modify `injectUtilities()` and rebuild |

The JavaScript worker is designed for lightweight task processing with HTTP integration. For heavy processing or complex dependencies, consider calling external services that have full library support.

See also [Stdio Workers](WORKER_STDIO.md) and [Skill Workers](WORKER_SKILL.md).
152 changes: 152 additions & 0 deletions WORKER_SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
# Skill Workers

A **skill** is a directory containing instructions and scripts. Registered with the server
it becomes an agent, but its tools still have to run on your machine — so the server
dispatches each tool call back to the CLI as a Conductor task, and the CLI runs it locally
and returns the result.

That local half is a worker. It shares the poll loop with
[stdio workers](WORKER_STDIO.md) and [JavaScript workers](WORKER_JS.md), and it works
with or without an agent: because a skill tool is just a Conductor task type, an ordinary
workflow can call one directly.

## Skill layout

```
myskill/
SKILL.md # required; frontmatter must set `name`
scripts/
greet.sh # each script becomes a tool
```

```markdown
---
name: greetskill
description: Says hello
---

# Greet skill

Instructions the agent reads.
```

```bash
#!/usr/bin/env bash
echo "Hello $1"
```

Script language is chosen by extension: `.py`, `.sh`, `.js`, `.mjs`, `.ts`, `.rb`, `.go`,
`.bat`, `.cmd`. Anything else is run with `bash`.

## Serving the tools

```bash
conductor skill serve ./myskill
# Serving workers for skill greetskill. Press Ctrl-C to stop.
```

`skill serve` starts one worker per tool and blocks. Use it when the skill is being run
somewhere else — from the UI, or by another process. `conductor skill run <skill> <prompt>`
starts the workers *and* runs the agent, stopping the workers when the run ends.

## Task types

Every tool is exposed as the task type:

```
{skillName}__{tool}
```

So `greetskill` with `scripts/greet.sh` serves `greetskill__greet`. Alongside the scripts,
these built-in tools are served too:

| Tool | Task type | Purpose |
|---|---|---|
| `read_skill_file` | `{skill}__read_skill_file` | Read a file bundled with the skill |
| `list_workspace_files` | `{skill}__list_workspace_files` | List files in the workspace |
| `read_workspace_file` | `{skill}__read_workspace_file` | Read a workspace file |
| `search_workspace` | `{skill}__search_workspace` | Search the workspace |
| `git_status` | `{skill}__git_status` | Workspace git status |
| `git_diff` | `{skill}__git_diff` | Workspace git diff |

The workspace tools are only served when a workspace is enabled; `--no-workspace` disables
them.

## Tool contract

Different from stdio workers, and simpler:

**Input** — `inputParameters.command` is passed to the script as **arguments**, not on
stdin. Only that field reaches the script.

**Output** — the script's **stdout** becomes the task output, wrapped as
`{"result": "<stdout>"}`. There is no envelope to emit.

**Failure** — a non-zero exit fails the task, with the captured output in the failure reason.

**stderr is merged into stdout.** `executeScript` gives the script a single buffer for both,
so anything a script logs to stderr ends up inside `{"result": …}` on success. Keep
diagnostics out of a script whose output you care about.

**Environment** — the skill root and the configured workspace roots are exported to the
script.

## Using a skill tool from a plain workflow

Nothing about this requires an agent. Point a `SIMPLE` task at the tool's task type:

```json
{
"name": "skill_as_worker",
"version": 1,
"tasks": [
{
"name": "greetskill__greet",
"taskReferenceName": "g",
"type": "SIMPLE",
"inputParameters": { "command": "${workflow.input.name}" }
}
]
}
```

```bash
conductor skill serve ./myskill &
conductor workflow start --workflow skill_as_worker --input '{"name":"Miguel"}' --sync
# { "result": "Hello Miguel\n" }
```

This makes a skill the lowest-ceremony way to run a script as a Conductor worker: no
result envelope, no SDK, and no protocol to implement.

Two constraints to know before relying on it:

- **The task type is fixed** as `{skill}__{tool}`, so an existing workflow cannot adopt a
skill tool without renaming its task.
- **Input is a single string.** Structured `inputData` does not reach the script; only
`command` does. Use a [stdio worker](WORKER_STDIO.md) when the task needs structured
input.

## Flags

| Flag | Applies to | Purpose |
|---|---|---|
| `--version` | run, serve | Skill version or checksum prefix |
| `--script-timeout` | run, serve | Per-script timeout in seconds (default 300) |
| `--script-output-limit` | run, serve | Max bytes captured from a script (default 10 MiB) |
| `--workspace` | run, serve | Workspace directory (default `.`) |
| `--no-workspace` | run, serve | Do not expose a workspace |
| `--filesystem name=path` | run, serve | Extra read-only root, repeatable |
| `--model` | run | Model for the agent (required for `run`) |
| `--param` | run | Skill parameter override, repeatable |

## Comparison with other worker types

| | Skill tools | [Stdio](WORKER_STDIO.md) | [JavaScript](WORKER_JS.md) |
|---|---|---|---|
| Task type | `{skill}__{tool}` | any | any |
| Input | `command` → argv | full task JSON on stdin | `$.task` |
| Output | bare stdout | `{status, output, logs, reason}` | `{status, body}` |
| Boilerplate | none | result envelope | result object |
| Structured input | no | yes | yes |
| Languages | by extension | any executable | JavaScript only |
6 changes: 4 additions & 2 deletions WORKER_STDIO.md
Original file line number Diff line number Diff line change
Expand Up @@ -260,9 +260,9 @@ func main() {
7. **Use environment variables**: Access `TASK_ID`, `WORKFLOW_ID` etc. when needed
8. **Exit with code 0**: Always exit with 0 and use status field for task outcome

## Comparison with JavaScript Workers
## Comparison with other worker types

| Feature | Generic Workers (exec) | JavaScript Workers (js) |
| Feature | Stdio Workers (stdio) | JavaScript Workers (js) |
|---------|----------------------|------------------------|
| Languages | Any (Python, Node, Go, etc.) | JavaScript only |
| Dependencies | Full access to language ecosystem | Limited (Goja ES5.1+) |
Expand All @@ -271,3 +271,5 @@ func main() {
| HTTP Calls | Use language's HTTP library | Built-in `http` object |
| File System | Full access | No access |
| Best For | Complex logic, heavy dependencies | Lightweight tasks, quick scripts |

See also [Skill Workers](WORKER_SKILL.md), which run a script with no result envelope at all.
20 changes: 12 additions & 8 deletions cmd/skill_run.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,18 +17,16 @@ import (
"context"
"encoding/json"
"fmt"
"os"
"os/signal"
"path/filepath"
"strings"
"syscall"
"time"

"github.com/spf13/cobra"

"github.com/conductor-oss/conductor-cli/internal"
"github.com/conductor-oss/conductor-cli/internal/agent"
"github.com/conductor-oss/conductor-cli/internal/skillworker"
"github.com/conductor-oss/conductor-cli/internal/taskworker"
)

// Skill run/serve flag defaults.
Expand Down Expand Up @@ -119,8 +117,11 @@ func runSkillRun(cmd *cobra.Command, args []string) error {

// One signal-aware context governs both the workers and the stream; cancelling
// it (Ctrl-C) stops everything. Workers are also cancelled when the execution
// ends normally.
ctx, stop := signal.NotifyContext(cmd.Context(), os.Interrupt, syscall.SIGTERM)
// ends normally. A second interrupt exits outright, so a stream or tool script
// that ignores cancellation cannot leave the process unkillable.
ctx, cancel := context.WithCancel(cmd.Context())
defer cancel()
stop := interruptWithEscalation(cancel)
defer stop()
workerCtx, cancelWorkers := context.WithCancel(ctx)
defer cancelWorkers()
Expand Down Expand Up @@ -152,7 +153,9 @@ func runSkillServe(cmd *cobra.Command, args []string) error {
return err
}

ctx, stop := signal.NotifyContext(cmd.Context(), os.Interrupt, syscall.SIGTERM)
ctx, cancel := context.WithCancel(cmd.Context())
defer cancel()
stop := interruptWithEscalation(cancel)
defer stop()
startSkillWorkers(ctx, buildSkillWorkerRegistry(cfg, local, ws, scriptOptions(), skillWorkspaceFileLimit))

Expand All @@ -173,9 +176,10 @@ func scriptOptions() skillworker.ScriptOptions {
// They run until ctx is cancelled.
func startSkillWorkers(ctx context.Context, registry map[string]skillworker.ToolHandler) {
taskClient := internal.GetTaskClient()
opts := skillworker.RunnerOptions()
for taskType, handler := range registry {
w := skillworker.NewWorker(skillworker.NewConductorRunner(taskClient))
go w.Run(ctx, taskType, handler)
w := taskworker.NewWorker(taskworker.NewConductorRunner(taskClient, opts), taskworker.Config{})
go w.Run(ctx, taskType, skillworker.AsTaskHandler(handler))
}
}

Expand Down
Loading
Loading