Skip to content

Latest commit

 

History

History
1197 lines (909 loc) · 28.4 KB

File metadata and controls

1197 lines (909 loc) · 28.4 KB

Orch8 Engine — API Reference

Base URL: http://localhost:8080 (configurable via ORCH8_HTTP_ADDR)

All request/response bodies are JSON. Dates use ISO 8601 / RFC 3339 format.


Health

Liveness Probe

GET /health/live

Always returns 200 if the process is running.

Response: 200 OK


Readiness Probe

GET /health/ready

Returns 200 if the database is reachable.

Response: 200 OK or 503 Service Unavailable


Sequences

Create Sequence

POST /sequences

Request Body:

{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "tenant_id": "acme",
  "namespace": "default",
  "name": "welcome-campaign",
  "version": 1,
  "blocks": [
    {
      "type": "step",
      "id": "send_welcome",
      "handler": "email_send",
      "params": { "template": "welcome" },
      "retry": {
        "max_attempts": 3,
        "initial_backoff": "1s",
        "max_backoff": "60s",
        "backoff_multiplier": 2.0
      },
      "timeout": "30s"
    },
    {
      "type": "step",
      "id": "wait_3_days",
      "handler": "sleep",
      "params": { "duration_ms": 259200000 }
    },
    {
      "type": "router",
      "id": "check_engagement",
      "routes": [
        {
          "condition": "opened == true",
          "blocks": [
            { "type": "step", "id": "send_followup", "handler": "email_send", "params": { "template": "followup" } }
          ]
        }
      ],
      "default": [
        { "type": "step", "id": "send_reminder", "handler": "email_send", "params": { "template": "reminder" } }
      ]
    }
  ],
  "created_at": "2024-01-15T10:00:00Z"
}

Response: 201 Created

{
  "id": "550e8400-e29b-41d4-a716-446655440000"
}

Get Sequence

GET /sequences/{id}

Response: 200 OK — full SequenceDefinition object.


Get Sequence by Name

GET /sequences/by-name?tenant_id=acme&namespace=default&name=welcome-campaign&version=1
Parameter Required Description
tenant_id Yes Tenant identifier
namespace Yes Namespace
name Yes Sequence name
version No Specific version (latest if omitted)

Response: 200 OK — full SequenceDefinition object.


Instances

Create Instance

POST /instances

Request Body:

{
  "sequence_id": "550e8400-e29b-41d4-a716-446655440000",
  "tenant_id": "acme",
  "namespace": "default",
  "priority": "normal",
  "timezone": "America/New_York",
  "metadata": { "campaign": "spring-2024", "segment": "enterprise" },
  "context": {
    "data": { "email": "john@acme.com", "name": "John" },
    "config": { "sender": "noreply@orch8.io" }
  },
  "next_fire_at": "2024-01-15T14:00:00Z",
  "concurrency_key": "contact:john@acme.com",
  "max_concurrency": 1,
  "idempotency_key": "welcome:john@acme.com:2024-01-15"
}
Field Type Default Description
sequence_id UUID required Sequence to execute
tenant_id string required Tenant identifier
namespace string required Namespace
priority string "normal" low, normal, high, critical
timezone string "UTC" IANA timezone
metadata object {} Arbitrary JSON metadata
context object {} Execution context (data, config sections)
next_fire_at datetime now When to start execution
concurrency_key string null Key for concurrency limiting
max_concurrency integer null Max parallel instances with same key
idempotency_key string null Deduplication key

Response: 201 Created

{
  "id": "a1b2c3d4-...",
  "deduplicated": false
}

If idempotency_key matches an existing instance:

{
  "id": "existing-instance-id",
  "deduplicated": true
}

Create Instances (Batch)

POST /instances/batch

Request Body:

{
  "instances": [
    { "sequence_id": "...", "tenant_id": "acme", "namespace": "default" },
    { "sequence_id": "...", "tenant_id": "acme", "namespace": "default" }
  ]
}

Response: 201 Created

{
  "count": 2
}

Get Instance

GET /instances/{id}

Response: 200 OK

{
  "id": "a1b2c3d4-...",
  "sequence_id": "550e8400-...",
  "tenant_id": "acme",
  "namespace": "default",
  "state": "running",
  "next_fire_at": "2024-01-15T14:00:00Z",
  "priority": "normal",
  "timezone": "America/New_York",
  "metadata": { "campaign": "spring-2024" },
  "context": {
    "data": { "email": "john@acme.com" },
    "config": {},
    "audit": [],
    "runtime": { "current_step": "send_welcome", "attempt": 0 }
  },
  "concurrency_key": "contact:john@acme.com",
  "max_concurrency": 1,
  "idempotency_key": "welcome:john@acme.com:2024-01-15",
  "created_at": "2024-01-15T10:00:00Z",
  "updated_at": "2024-01-15T14:00:05Z"
}

List Instances

GET /instances?tenant_id=acme&namespace=default&state=running,scheduled&limit=50&offset=0
Parameter Type Default Description
tenant_id string Filter by tenant
namespace string Filter by namespace
sequence_id UUID Filter by sequence
state string Comma-separated states to include
offset integer 0 Pagination offset
limit integer 100 Page size (max 1000)

Response: 200 OK — array of TaskInstance objects.


Update Instance State

PATCH /instances/{id}/state

Request Body:

{
  "state": "paused",
  "next_fire_at": "2024-01-16T09:00:00Z"
}

Valid state transitions:

From To
Scheduled Running, Paused, Cancelled
Running Scheduled, Waiting, Completed, Failed, Paused, Cancelled
Waiting Running, Scheduled, Cancelled, Failed
Paused Scheduled, Cancelled
Failed Scheduled (retry)
Completed (terminal)
Cancelled (terminal)

Response: 200 OK

Returns 400 Bad Request if the transition is invalid.


Update Instance Context

PATCH /instances/{id}/context

Request Body:

{
  "context": {
    "data": { "opened": true, "clicked_link": "pricing" }
  }
}

Response: 200 OK


Send Signal

POST /instances/{id}/signals

Request Body:

{
  "signal_type": "pause",
  "payload": {}
}
Signal Type Effect
pause Pause execution
resume Resume from Paused
cancel Cancel instance
update_context Merge payload into context.data
custom:* Application-defined signal

Response: 201 Created

{
  "signal_id": "b5c6d7e8-..."
}

Get Outputs

GET /instances/{id}/outputs

Response: 200 OK

[
  {
    "id": "f1e2d3c4-...",
    "instance_id": "a1b2c3d4-...",
    "block_id": "send_welcome",
    "output": { "email_id": "msg-abc123", "status": "sent" },
    "output_ref": null,
    "output_size": 52,
    "attempt": 0,
    "created_at": "2024-01-15T14:00:03Z"
  }
]

Retry Failed Instance

POST /instances/{id}/retry

Instance must be in failed state. Resets to scheduled with next_fire_at = now.

Response: 200 OK

{
  "id": "a1b2c3d4-...",
  "state": "scheduled"
}

Bulk Update State

PATCH /instances/bulk/state

Request Body:

{
  "filter": {
    "tenant_id": "acme",
    "namespace": "default",
    "sequence_id": "550e8400-...",
    "states": ["scheduled", "running"]
  },
  "state": "cancelled"
}

Response: 200 OK

{
  "count": 47
}

List Dead Letter Queue

GET /instances/dlq?tenant_id=acme&namespace=default&limit=50

Same parameters as List Instances. Returns only failed instances.

Response: 200 OK — array of TaskInstance objects.


Cron Schedules

Create Cron Schedule

POST /cron

Request Body:

{
  "tenant_id": "acme",
  "namespace": "default",
  "sequence_id": "550e8400-...",
  "cron_expr": "0 9 * * MON-FRI *",
  "timezone": "America/New_York",
  "metadata": { "type": "daily-report" },
  "enabled": true
}
Field Type Default Description
tenant_id string required Tenant identifier
namespace string required Namespace
sequence_id UUID required Sequence to instantiate
cron_expr string required 7-field cron expression
timezone string "UTC" IANA timezone for schedule
metadata object {} Passed to created instances
enabled boolean true Whether schedule is active

Cron expression format (7 fields):

second  minute  hour  day  month  day_of_week  year
  0       9      *     *     *      MON-FRI      *

Response: 201 Created

{
  "id": "c1d2e3f4-...",
  "next_fire_at": "2024-01-16T14:00:00Z"
}

Get Cron Schedule

GET /cron/{id}

Response: 200 OK — full CronSchedule object.


List Cron Schedules

GET /cron?tenant_id=acme

Response: 200 OK — array of CronSchedule objects.


Update Cron Schedule

PUT /cron/{id}

Request Body (all fields optional):

{
  "cron_expr": "0 10 * * * *",
  "timezone": "Europe/London",
  "enabled": false,
  "metadata": { "type": "weekly-digest" }
}

Response: 200 OK — updated CronSchedule.


Delete Cron Schedule

DELETE /cron/{id}

Response: 204 No Content


External Workers

Poll for Tasks

POST /workers/tasks/poll

Request Body:

{
  "handler_name": "email_send",
  "worker_id": "node-worker-42",
  "limit": 10
}
Field Type Default Description
handler_name string required Handler to claim tasks for
worker_id string required Unique worker identifier
limit integer 1 Max tasks to claim

Response: 200 OK

[
  {
    "id": "d4e5f6a7-...",
    "instance_id": "a1b2c3d4-...",
    "block_id": "send_welcome",
    "handler_name": "email_send",
    "params": { "template": "welcome", "to": "john@acme.com" },
    "context": { "data": { "name": "John" }, "config": {} },
    "attempt": 0,
    "timeout_ms": 30000,
    "state": "claimed",
    "worker_id": "node-worker-42",
    "claimed_at": "2024-01-15T14:00:00Z",
    "heartbeat_at": "2024-01-15T14:00:00Z",
    "completed_at": null,
    "output": null,
    "error_message": null,
    "error_retryable": null,
    "created_at": "2024-01-15T13:59:58Z"
  }
]

Returns empty array [] if no tasks available.

Mechanics:

  • Uses FOR UPDATE SKIP LOCKED — concurrent workers never get the same task.
  • Sets state = claimed, worker_id, claimed_at, and heartbeat_at.

Complete Task

POST /workers/tasks/{task_id}/complete

Request Body:

{
  "worker_id": "node-worker-42",
  "output": {
    "email_id": "msg-abc123",
    "delivered": true
  }
}
Field Type Description
worker_id string Must match the worker that claimed the task
output object Result JSON (saved as BlockOutput)

Response: 200 OK

Side effects:

  1. Worker task marked completed
  2. BlockOutput created with the output JSON
  3. Instance transitions Waiting -> Scheduled (immediate re-processing)
  4. If instance uses execution tree, corresponding node marked Completed

Fail Task

POST /workers/tasks/{task_id}/fail

Request Body:

{
  "worker_id": "node-worker-42",
  "message": "SMTP connection refused",
  "retryable": true
}
Field Type Default Description
worker_id string required Must match claimer
message string required Error description
retryable boolean false Whether the error is transient

Response: 200 OK

Retryable failure:

  • Worker task deleted (allows re-dispatch on next tick)
  • Instance reset to Scheduled

Permanent failure:

  • If instance has composite blocks: execution node marked Failed, instance re-scheduled for evaluator (try-catch can recover)
  • If instance is step-only: instance marked Failed (enters DLQ)

Heartbeat Task

POST /workers/tasks/{task_id}/heartbeat

Request Body:

{
  "worker_id": "node-worker-42"
}

Response: 200 OK

Send heartbeats every 15-30 seconds for long-running tasks. Tasks without a heartbeat for 60 seconds are reclaimed by the reaper and returned to the queue.


Metrics

Prometheus Metrics

GET /metrics

Response: 200 OK — Prometheus text format (v0.0.4)

Counters:

Metric Description
orch8_instances_claimed_total Instances claimed by scheduler
orch8_instances_completed_total Instances completed successfully
orch8_instances_failed_total Instances failed (entered DLQ)
orch8_steps_executed_total Steps executed
orch8_steps_failed_total Steps that failed
orch8_steps_retried_total Retry attempts
orch8_signals_delivered_total Signals processed
orch8_rate_limits_exceeded_total Rate limit deferrals
orch8_recovery_stale_instances_total Stale instances recovered at startup
orch8_webhooks_sent_total Webhooks delivered
orch8_webhooks_failed_total Webhook delivery failures
orch8_cron_triggered_total Cron instances created

Histograms:

Metric Description
orch8_tick_duration_seconds Scheduler tick latency
orch8_step_duration_seconds Individual step execution time
orch8_instance_processing_seconds Total instance processing time

Gauges:

Metric Description
orch8_queue_depth Instances claimed in current tick
orch8_active_tasks Currently in-flight step executions

Block Definition Reference

All blocks are defined in the blocks array of a sequence. Blocks can nest arbitrarily.

Step

{
  "type": "step",
  "id": "unique_block_id",
  "handler": "handler_name",
  "params": {},
  "delay": {
    "duration": "3600s",
    "business_days_only": false,
    "jitter": "300s"
  },
  "retry": {
    "max_attempts": 3,
    "initial_backoff": "1s",
    "max_backoff": "60s",
    "backoff_multiplier": 2.0
  },
  "timeout": "30s",
  "rate_limit_key": "resource:identifier"
}

Built-in handlers:

Handler Params Output
noop (none) {}
log message (string), level ("debug"/"info"/"warn") { "message": "..." }
sleep duration_ms (integer, default 100) { "slept_ms": N }
fail message (string) (step fails with the given message)
http_request url, method ("GET"/"POST"/"PUT"/"DELETE"), body, timeout_ms (default 10000) { "status": 200, "body": "..." }
llm_call provider, model, messages, temperature, max_tokens { "content": "...", "usage": {...} }
tool_call tool_name, arguments (tool-specific output)
human_review prompt, timeout_ms, escalation_handler { "approved": bool, "reviewer": "...", "comments": "..." }
self_modify blocks (array of block definitions to inject) { "injected": N }
emit_event See Workflow coordination handlers { instance_id, sequence_name, deduped }
send_signal See Workflow coordination handlers { signal_id }
query_instance See Workflow coordination handlers { found, state?, context?, created_at?, updated_at? }
set_state key (string), value (any) {}
get_state key (string) { "value": ... }
delete_state key (string) {}
merge_state data (object) {}
transform expression (string), target (string) { "result": ... }
assert condition (string), message (string) {}

Any handler name not registered as built-in is automatically dispatched to the external worker queue.


Workflow coordination handlers

Three built-in handlers let one workflow coordinate with another within the same tenant. Cross-tenant targets always fail with Permanent — the engine reads the caller's tenant_id from its running instance and rejects mismatches.

Template resolution: step handler params are resolved through the template engine before dispatch. Both the in-process scheduler and the external-worker path receive already-resolved params, so {{ context.data.target_id }}, {{ outputs.prev_step.id }}, and {{ foo | fallback }} all work inside any handler's params block — including the coordination handlers below.

emit_event

Fires an event trigger → spawns a new child workflow instance.

Params:

Field Type Required Notes
trigger_slug string yes Slug of an enabled event trigger in the caller's tenant
data object no Event payload; becomes the child's context.data (default {})
meta object no Extra metadata merged into the child's metadata. source and parent_instance_id are always set by the engine and cannot be spoofed
dedupe_key string no Non-empty string. When present, dedupe is recorded atomically — subsequent calls with the same (scope, key) pair return the existing child and set deduped: true. Empty string is rejected
dedupe_scope string no "parent" (default) scopes dedupe to (parent_instance_id, dedupe_key). "tenant" scopes it to (tenant_id, dedupe_key) so different parents in the same tenant cannot both spawn a child for the same key. Only meaningful when dedupe_key is present

Returns: { "instance_id": "<uuid>", "sequence_name": "<name>", "deduped": <bool> }

Errors:

Condition Variant
Missing trigger_slug; dedupe_key empty or non-string Permanent
Invalid dedupe_scope (not "parent" or "tenant") Permanent
Trigger not found, disabled, or in another tenant Permanent
Caller instance not found Permanent
Storage connection / pool / query failure Retryable
Other storage failure Permanent

Dedupe semantics:

  • dedupe_scope: "parent" (default) — key is (parent_instance_id, dedupe_key). Different parent instances can reuse the same key without colliding. Use this for the common fan-out case (e.g. one parent fires-once per external event).
  • dedupe_scope: "tenant" — key is (tenant_id, dedupe_key). All parents within the tenant share the namespace, so the child workflow fires at most once per key across the whole tenant. Use this when the dedupe identity is global (e.g. a webhook delivery id you refuse to process twice across the system).

Dedupe rows are swept by the TTL sweeper (default 30 days).

Example:

{
  "type": "step",
  "id": "fan_out_order",
  "handler": "emit_event",
  "params": {
    "trigger_slug": "on-order-created",
    "data": { "order_id": "ord_123", "total_cents": 4200 },
    "dedupe_key": "ord_123"
  }
}

send_signal

Enqueues a signal on an existing instance. Target must be non-terminal.

Params:

Field Type Required Notes
instance_id string (UUID) yes Target instance
signal_type string | object yes See variants below
payload any no Arbitrary JSON delivered with the signal (default null)

signal_type accepts all SignalType variants in snake_case:

  • "pause", "resume", "cancel", "update_context"
  • custom signals use the tagged form: { "custom": "my_signal_name" }

Returns: { "signal_id": "<uuid>" }

Errors:

Condition Variant
Missing / invalid instance_id; missing / invalid signal_type Permanent
Target not found; target in another tenant; target in terminal state Permanent
Enqueue conflict (UUID collision; indicates a bug) Permanent
Storage connection / pool / query failure Retryable

Example:

{
  "type": "step",
  "id": "cancel_child",
  "handler": "send_signal",
  "params": {
    "instance_id": "0194d2e6-7f44-7e2a-9c3e-1a2b3c4d5e6f",
    "signal_type": "cancel"
  }
}

Custom signal:

{
  "type": "step",
  "id": "notify_child",
  "handler": "send_signal",
  "params": {
    "instance_id": "0194d2e6-7f44-7e2a-9c3e-1a2b3c4d5e6f",
    "signal_type": { "custom": "approval_granted" },
    "payload": { "reviewer": "alice" }
  }
}

query_instance

Reads another instance's context and state.

Params:

Field Type Required
instance_id string (UUID) yes

Returns (found): { "found": true, "state": "<state>", "context": { ... }, "created_at": "<ts>", "updated_at": "<ts>" }

Returns (missing): { "found": false } — note this is NOT an error.

Errors:

Condition Variant
Missing / invalid instance_id Permanent
Caller instance not found Permanent
Target in another tenant Permanent (intentional — query_instance errors rather than masking cross-tenant as { found: false }, so a missing target and a cross-tenant target are distinguishable)
Storage connection / pool / query failure Retryable

Example:

{
  "type": "step",
  "id": "read_child",
  "handler": "query_instance",
  "params": {
    "instance_id": "0194d2e6-7f44-7e2a-9c3e-1a2b3c4d5e6f"
  }
}

Parallel

{
  "type": "parallel",
  "id": "notify_all",
  "branches": [
    [{ "type": "step", "id": "email", "handler": "email_send", "params": {} }],
    [{ "type": "step", "id": "sms", "handler": "sms_send", "params": {} }],
    [{ "type": "step", "id": "push", "handler": "push_send", "params": {} }]
  ]
}

All branches run concurrently. Completes when all finish. Fails if any branch fails.


Race

{
  "type": "race",
  "id": "fastest_provider",
  "branches": [
    [{ "type": "step", "id": "provider_a", "handler": "send_via_a", "params": {} }],
    [{ "type": "step", "id": "provider_b", "handler": "send_via_b", "params": {} }]
  ],
  "semantics": "first_to_succeed"
}
Semantics Behavior
first_to_resolve (default) First branch to complete (success or failure) wins
first_to_succeed First successful branch wins; failures ignored until all fail

Losing branches are cancelled.


Router

{
  "type": "router",
  "id": "segment_users",
  "routes": [
    {
      "condition": "plan == enterprise",
      "blocks": [{ "type": "step", "id": "vip_flow", "handler": "vip_onboard", "params": {} }]
    },
    {
      "condition": "plan == pro",
      "blocks": [{ "type": "step", "id": "pro_flow", "handler": "pro_onboard", "params": {} }]
    }
  ],
  "default": [
    { "type": "step", "id": "free_flow", "handler": "free_onboard", "params": {} }
  ]
}

Evaluates conditions in order against context.data. First match wins. Falls through to default if none match.

Condition syntax: path == value (equality) or path (truthy check). Dot notation for nested paths (e.g., user.plan == enterprise).


TryCatch

{
  "type": "try_catch",
  "id": "safe_send",
  "try_block": [
    { "type": "step", "id": "primary", "handler": "smtp_send", "params": {} }
  ],
  "catch_block": [
    { "type": "step", "id": "fallback", "handler": "ses_send", "params": {} }
  ],
  "finally_block": [
    { "type": "step", "id": "log_result", "handler": "log", "params": { "message": "send attempted" } }
  ]
}
  • try_block: Executes first. If all steps succeed, catch is skipped.
  • catch_block: Executes only if try failed. If catch succeeds, the overall block succeeds.
  • finally_block: Always executes, regardless of try/catch outcome. Optional.

Loop

{
  "type": "loop",
  "id": "poll_status",
  "condition": "status.pending",
  "body": [
    { "type": "step", "id": "check", "handler": "http_request", "params": { "url": "https://api.example.com/status" } }
  ],
  "max_iterations": 50
}

Repeats body while condition evaluates to truthy in context.data. Safety cap via max_iterations (default 1000).


ForEach

{
  "type": "for_each",
  "id": "process_batch",
  "collection": "items",
  "item_var": "item",
  "body": [
    { "type": "step", "id": "process", "handler": "process_item", "params": {} }
  ],
  "max_iterations": 500
}

Iterates over context.data[collection] (must be an array). Each iteration has item_var available in context. Empty or missing collection completes immediately.


ABSplit

{
  "type": "ab_split",
  "id": "experiment_onboarding",
  "variants": [
    {
      "name": "control",
      "weight": 70,
      "blocks": [
        { "type": "step", "id": "standard_flow", "handler": "standard_onboard", "params": {} }
      ]
    },
    {
      "name": "variant_a",
      "weight": 30,
      "blocks": [
        { "type": "step", "id": "new_flow", "handler": "new_onboard", "params": {} }
      ]
    }
  ]
}

Routes traffic to one of several variants by weight. Useful for A/B testing different workflow branches. The selected variant is recorded in the instance context for analytics.


CancellationScope

{
  "type": "cancellation_scope",
  "id": "critical_section",
  "blocks": [
    { "type": "step", "id": "charge", "handler": "payment_charge", "params": {} },
    { "type": "step", "id": "confirm", "handler": "send_receipt", "params": {} }
  ]
}

Child blocks inside a cancellation scope cannot be cancelled by external cancel signals. Provides subtree-level non-cancellability (Temporal-style structured concurrency). Use for critical sections like payment processing that must complete atomically.


Mobile Sync

Mobile sync endpoints require ORCH8_MOBILE_SYNC_ENABLED=true. All endpoints are tenant-scoped when X-Tenant-Id is provided.

Sync

POST /mobile/sync — Bidirectional sync endpoint. Devices push status updates, approval requests, and step delegations; server responds with pending commands.

Request body:

Field Type Description
device_id string Device identifier (required)
status_updates array Instance status updates from device
approval_requests array Human-in-the-loop approval requests
step_delegations array Step delegation requests
command_acks array Command IDs the device has processed

Response: { "commands": [...], "sync_interval_secs": 30 }

Register Device

POST /mobile/devices/register — Register a mobile device for push notifications.

List Devices

GET /mobile/devices — List registered mobile devices.

List Approvals

GET /mobile/approvals — List pending human-in-the-loop approval requests.

Resolve Approval

POST /mobile/approvals/{id}/resolve — Resolve a pending approval request.

List Status

GET /mobile/status — List mobile instance status reports.

Create Command

POST /mobile/commands — Queue a command for a mobile device.


Additional Endpoints

The following endpoint groups are available via the OpenAPI spec (Swagger UI at /swagger-ui/) but not yet fully documented here:

  • Sessions — Stateful multi-instance coordination
  • Pools — Resource pool management with weighted allocation
  • Credentials — Encrypted credential vault with OAuth2 refresh
  • Triggers — Webhook and event-driven instance creation
  • Circuit Breakers — Per-tenant/handler state, manual reset
  • Plugins — WASM and gRPC plugin registration
  • Approvals — Human-in-the-loop approval inbox
  • Cluster — Node listing, heartbeat, drain
  • Webhooks — Webhook subscription management
  • Telemetry — Execution telemetry and rollback history

Error Responses

All error responses follow this format:

{
  "error": "Human-readable error message"
}
Status Code Meaning
400 Bad request (invalid JSON, missing required field, invalid argument)
401 Unauthorized (missing or invalid API key)
403 Forbidden (insufficient permissions)
404 Resource not found (instance, sequence, cron schedule, worker task)
409 Conflict (state transition not allowed, duplicate resource)
413 Payload too large (context exceeds max_context_bytes)
500 Internal server error
503 Service unavailable (storage connection failure)