diff --git a/AGENTS.md b/AGENTS.md index e975958..da8ce0a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,7 @@ pnpm install --frozen-lockfile pnpm typecheck pnpm build pnpm test -bash -n ops/provision.sh +bash -n ops/provision.sh ops/claudex ``` ## Architecture diff --git a/daemon/README.md b/daemon/README.md index 91e7cd9..17bea14 100644 --- a/daemon/README.md +++ b/daemon/README.md @@ -20,14 +20,17 @@ pnpm install --frozen-lockfile pnpm typecheck pnpm build pnpm test -bash -n ops/provision.sh +bash -n ops/provision.sh ops/claudex ops/claudex-fable ops/proxy-accounts.sh ops/codex-provider-gate.sh ``` The Vitest suite is hermetic: it uses loopback HTTP servers and temporary real SQLite databases, requires no environment variables, and makes no internet or Linear requests. Tests cover phase-1 ingress plus durable planner queues, real temporary git worktrees, and a fake stream-json Claude executable. They require no network, Linear credentials, or Claude -account. Real Linear/Claude/systemd acceptance remains a deploy-time gate in `ops/runbook.md`. +account. When `CLIPROXY_BIN` points to the pinned CLIProxyAPI binary, the proxy integration +suite additionally checks aliases, credential management, hot-loading, disabling, and log +redaction. Real account, Linear, Claude, and systemd acceptance remains a deploy-time gate in +`ops/runbook.md`. ## Run locally @@ -53,9 +56,39 @@ app prefix are test-only static overrides and are ignored unless `DAEMON_TEST_MO production uses client credentials. Optional settings are `PORT`, `BIND_ADDR`, `REPLAY_WINDOW_MS`, `LINEAR_GRAPHQL_URL`, and `LINEAR_TOKEN_URL`. +Set `ARTIFACT_TOKEN` to enable artifact hosting. An authenticated `POST /a` creates a +bundle with a server-generated id; authenticated `PUT /a/` atomically replaces an +existing bundle. Both accept a JSON manifest whose file contents are base64 encoded: + +```json +{ + "files": [ + { "path": "item.md", "contentBase64": "IyBJdGVtCg==" }, + { "path": "refs/explainer.html", "contentBase64": "PGgxPkV4cGxhaW5lcjwvaDE+" } + ] +} +``` + +Writes require `Authorization: Bearer `. `GET /a//` is an +unauthenticated, self-contained viewer; `GET /a//index.json` returns the live version's +file paths as a no-cache JSON array, and `GET /a//` serves raw files. The name +`index.json` is reserved at the bundle root. Nothing above the unguessable `/a//` URL +enumerates bundles. `ARTIFACTS_DIR` defaults to an `artifacts` directory beside the database, +and `ARTIFACT_MAX_BODY_BYTES` defaults to 32 MiB. Provisioning creates the default directory +under `/var/lib/linear-agent-daemon`, outside the deployed application tree, so content +survives provision and deploy reruns. It is not backed up yet; loss of the VM disk loses +stored bundles. + Planner sessions default on. `TARGET_REPO_PATH` and `LINEAR_API_KEY` are required when enabled. Optional session settings are `WORKTREES_ROOT` (defaults beside the database), `CLAUDE_BIN` (default `claude`, whitespace-split for a command prefix), +`CLAUDEX_BIN` (optional, whitespace-split; enables a one-shot retry of validated +Claude usage/rate-limit failures through the Claudex proxy runtime — point it at +the provisioned `claudex` wrapper), `CLAUDEX_ENV` (optional JSON string map of +extra child env for `CLAUDEX_BIN`; requires `CLAUDEX_BIN`), +`FABLE_BIN` (optional; normally the installed `ops/claudex-fable` launcher), +`CLIPROXY_ENV_FILE`, `CLIPROXY_URL`, `PROVIDER_PROBE_INTERVAL_MS`, +`PROVIDER_STATE_STALE_MS`, and `PROVIDER_INITIAL_PROBE_TIMEOUT_MS`, `CLAUDE_PERMISSION_MODE` (`bypassPermissions`), `CLAUDE_MAX_TURNS` (100), `DO_PERMISSION_MODE` (`bypassPermissions`; production rejects every other value), `DO_MAX_TURNS` (300), `DO_MAX_BUDGET_USD` (optional positive number), diff --git a/daemon/ops/claudex b/daemon/ops/claudex new file mode 100755 index 0000000..28cdb66 --- /dev/null +++ b/daemon/ops/claudex @@ -0,0 +1,18 @@ +#!/bin/sh +set -eu + +. /etc/linear-agent-daemon/cliproxyapi.env + +export CLIPROXY_API_KEY +export ANTHROPIC_BASE_URL=http://127.0.0.1:8317 +export ANTHROPIC_AUTH_TOKEN="${CLIPROXY_API_KEY}" +export ANTHROPIC_DEFAULT_HAIKU_MODEL=gpt-5.6-sol-low +export ANTHROPIC_DEFAULT_SONNET_MODEL=gpt-5.6-sol-low +export ANTHROPIC_DEFAULT_OPUS_MODEL=gpt-5.6-sol-medium +export ANTHROPIC_DEFAULT_FABLE_MODEL=gpt-5.6-sol-xhigh +export CLAUDE_CODE_MAX_CONTEXT_TOKENS=250000 +export CLAUDE_CODE_ALWAYS_ENABLE_EFFORT=1 +export CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY=3 +export ENABLE_TOOL_SEARCH=true + +exec /var/lib/linear-agent-daemon/.local/bin/claude --model gpt-5.6-sol "$@" diff --git a/daemon/ops/claudex-fable b/daemon/ops/claudex-fable new file mode 100755 index 0000000..e7cd1c3 --- /dev/null +++ b/daemon/ops/claudex-fable @@ -0,0 +1,55 @@ +#!/bin/sh +set -eu + +CLIPROXY_ENV_FILE="${CLIPROXY_ENV_FILE:-/etc/linear-agent-daemon/cliproxyapi.env}" +FABLE_MODELS_ENV_FILE="${FABLE_MODELS_ENV_FILE:-/etc/linear-agent-daemon/fable-models.env}" +PROXY_URL="${PROXY_URL:-http://127.0.0.1:8317}" +FABLE_CLAUDE_BIN="${FABLE_CLAUDE_BIN:-/var/lib/linear-agent-daemon/.local/bin/claude}" + +fail() { echo "claudex-fable: $1" >&2; exit 1; } +[ -r "$CLIPROXY_ENV_FILE" ] || fail "missing proxy environment file: $CLIPROXY_ENV_FILE" +[ -r "$FABLE_MODELS_ENV_FILE" ] || fail "missing Fable model file: $FABLE_MODELS_ENV_FILE" +# shellcheck disable=SC1090 +. "$CLIPROXY_ENV_FILE" +# shellcheck disable=SC1090 +. "$FABLE_MODELS_ENV_FILE" + +validate_model() { + variable="$1" model="$2" + [ -n "$model" ] || fail "missing $variable" + case "$model" in claude-*) ;; *) fail "$variable must name a claude-* model, got: $model" ;; esac +} +catalog_has_model() { + variable="$1" model="$2" + printf '%s' "$MODEL_CATALOG" | python3 -c \ + 'import json,sys; p=json.load(sys.stdin); needle=sys.argv[1]; raise SystemExit(0 if any(x.get("id")==needle for x in p.get("data",[])) else 1)' \ + "$model" || fail "$variable model absent from proxy catalog: $model" +} + +[ -n "${CLIPROXY_API_KEY:-}" ] || fail "CLIPROXY_API_KEY is missing" +validate_model FABLE_MAIN_MODEL "${FABLE_MAIN_MODEL:-}" +validate_model FABLE_HAIKU_MODEL "${FABLE_HAIKU_MODEL:-}" +validate_model FABLE_SONNET_MODEL "${FABLE_SONNET_MODEL:-}" +validate_model FABLE_OPUS_MODEL "${FABLE_OPUS_MODEL:-}" +validate_model FABLE_FABLE_MODEL "${FABLE_FABLE_MODEL:-}" +MODEL_CATALOG="$(printf 'header = "Authorization: Bearer %s"\n' "$CLIPROXY_API_KEY" | + curl -fsS --connect-timeout 2 --max-time 15 -K - "${PROXY_URL%/}/v1/models")" || fail "proxy model catalog unavailable" +catalog_has_model FABLE_MAIN_MODEL "$FABLE_MAIN_MODEL" +catalog_has_model FABLE_HAIKU_MODEL "$FABLE_HAIKU_MODEL" +catalog_has_model FABLE_SONNET_MODEL "$FABLE_SONNET_MODEL" +catalog_has_model FABLE_OPUS_MODEL "$FABLE_OPUS_MODEL" +catalog_has_model FABLE_FABLE_MODEL "$FABLE_FABLE_MODEL" + +export CLIPROXY_API_KEY +export ANTHROPIC_BASE_URL="${PROXY_URL%/}" +export ANTHROPIC_AUTH_TOKEN="$CLIPROXY_API_KEY" +export ANTHROPIC_DEFAULT_HAIKU_MODEL="$FABLE_HAIKU_MODEL" +export ANTHROPIC_DEFAULT_SONNET_MODEL="$FABLE_SONNET_MODEL" +export ANTHROPIC_DEFAULT_OPUS_MODEL="$FABLE_OPUS_MODEL" +export ANTHROPIC_DEFAULT_FABLE_MODEL="$FABLE_FABLE_MODEL" +export CLAUDE_CODE_MAX_CONTEXT_TOKENS=250000 +export CLAUDE_CODE_ALWAYS_ENABLE_EFFORT=1 +export CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY=3 +export ENABLE_TOOL_SEARCH=true + +exec "$FABLE_CLAUDE_BIN" --model "$FABLE_MAIN_MODEL" "$@" diff --git a/daemon/ops/cliproxyapi.service b/daemon/ops/cliproxyapi.service new file mode 100644 index 0000000..fb2e2fa --- /dev/null +++ b/daemon/ops/cliproxyapi.service @@ -0,0 +1,37 @@ +[Unit] +Description=CLIProxyAPI for Claude Code +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +User=linear-daemon +Group=linear-daemon +Environment=HOME=/var/lib/linear-agent-daemon +WorkingDirectory=/var/lib/linear-agent-daemon +ExecStart=/usr/local/bin/cliproxyapi -config /etc/linear-agent-daemon/cliproxyapi.yaml +Restart=always +RestartSec=2 +NoNewPrivileges=true +PrivateTmp=true +PrivateDevices=true +ProtectHome=true +ProtectSystem=strict +ProtectKernelTunables=true +ProtectKernelModules=true +ProtectControlGroups=true +ProtectKernelLogs=true +RestrictSUIDSGID=true +RestrictNamespaces=true +RestrictRealtime=true +LockPersonality=true +RemoveIPC=true +CapabilityBoundingSet= +RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX +SystemCallFilter=@system-service +SystemCallArchitectures=native +UMask=0077 +ReadWritePaths=/var/lib/linear-agent-daemon/.cli-proxy-api + +[Install] +WantedBy=multi-user.target diff --git a/daemon/ops/codex-provider-gate.sh b/daemon/ops/codex-provider-gate.sh new file mode 100755 index 0000000..4914a77 --- /dev/null +++ b/daemon/ops/codex-provider-gate.sh @@ -0,0 +1,228 @@ +#!/usr/bin/env bash +set -euo pipefail + +usage() { + cat <<'EOF' +usage: codex-provider-gate.sh [--help] + +Prove CLIProxyAPI Responses compatibility, identity, resume, and affinity, +then install the daemon-owned Codex provider config. Failures remove only a +gate-managed target config. + +Environment variables: + PROXY_URL Proxy origin (default: http://127.0.0.1:8317) + CLIPROXY_ENV_FILE API-key env file (default: /etc/linear-agent-daemon/cliproxyapi.env) + CLIPROXY_VERSION_MARKER Version marker (default: /usr/local/share/cliproxyapi-version) + CLIPROXY_BIN Proxy binary (default: /usr/local/bin/cliproxyapi) + CODEX_BIN Codex executable (default: codex) + CODEX_HOME Temporary Codex state directory (default: generated) + TARGET_CONFIG Installed config path (default: /var/lib/linear-agent-daemon/.codex/config.toml) + EXPECTED_PROXY_VERSION Required proxy version (default: 7.2.93) + GATE_TIMEOUT_SECONDS Per-Codex-command timeout (default: 900) + GATE_COUNTER_WINDOW_SECONDS Identity-counter poll window (default: 15) +EOF +} + +if [[ $# -gt 0 ]]; then + case "$1" in + --help|-h) + if [[ $# -ne 1 ]]; then usage >&2; exit 2; fi + usage + exit 0 + ;; + *) usage >&2; exit 2 ;; + esac +fi + +# Coupled intentionally to claude/skills/codex/SKILL.md's detached dispatch and +# `codex exec resume --last` shapes. A pass proves the unchanged workflow. +EXPECTED_PROXY_VERSION="${EXPECTED_PROXY_VERSION:-7.2.93}" +PROXY_URL="${PROXY_URL:-http://127.0.0.1:8317}" +CLIPROXY_ENV_FILE="${CLIPROXY_ENV_FILE:-/etc/linear-agent-daemon/cliproxyapi.env}" +CLIPROXY_VERSION_MARKER="${CLIPROXY_VERSION_MARKER:-/usr/local/share/cliproxyapi-version}" +CLIPROXY_BIN="${CLIPROXY_BIN:-/usr/local/bin/cliproxyapi}" +CODEX_BIN="${CODEX_BIN:-codex}" +TARGET_CONFIG="${TARGET_CONFIG:-/var/lib/linear-agent-daemon/.codex/config.toml}" +GATE_TIMEOUT_SECONDS="${GATE_TIMEOUT_SECONDS:-900}" +GATE_COUNTER_WINDOW_SECONDS="${GATE_COUNTER_WINDOW_SECONDS:-15}" +MARKER='# managed by codex-provider-gate.sh — removed on gate failure' +work_dir="$(mktemp -d)" +# Relative path overrides keep their caller-cwd meaning even though the gate +# changes directory below. +caller_pwd="$PWD" +absolutize() { case "$1" in /*) printf '%s' "$1" ;; *) printf '%s/%s' "${caller_pwd}" "$1" ;; esac; } +CLIPROXY_ENV_FILE="$(absolutize "${CLIPROXY_ENV_FILE}")" +CLIPROXY_VERSION_MARKER="$(absolutize "${CLIPROXY_VERSION_MARKER}")" +CLIPROXY_BIN="$(absolutize "${CLIPROXY_BIN}")" +TARGET_CONFIG="$(absolutize "${TARGET_CONFIG}")" +if [[ -n "${CODEX_HOME:-}" ]]; then CODEX_HOME="$(absolutize "${CODEX_HOME}")"; fi +case "${CODEX_BIN}" in */*) CODEX_BIN="$(absolutize "${CODEX_BIN}")" ;; esac +gate_home="${CODEX_HOME:-${work_dir}/codex-home}" +mkdir -p "${gate_home}" +# The caller's cwd may be unreadable for the invoking user (e.g. provision +# launches this via runuser from an operator home dir); run from the gate's +# own workdir so codex never inherits an unusable working directory. +cd "${work_dir}" + +cleanup() { + local status=$? + rm -rf "${work_dir}" + return "${status}" +} +trap cleanup EXIT + +rollback() { + if [[ -f "${TARGET_CONFIG}" ]] && head -n 1 "${TARGET_CONFIG}" | grep -Fqx "${MARKER}"; then + rm -f "${TARGET_CONFIG}" + echo "rollback: removed gate-managed provider config" >&2 + fi +} +fail() { rollback; echo "FAIL: $1" >&2; exit 1; } +on_error() { + local status=$? + rollback + echo "FAIL: unexpected-error" >&2 + return "${status}" +} +trap on_error ERR + +[[ -r "${CLIPROXY_ENV_FILE}" ]] || fail "environment-file" +CLIPROXY_API_KEY="$(grep -m1 '^CLIPROXY_API_KEY=' "${CLIPROXY_ENV_FILE}" | cut -d= -f2- || true)" +CLIPROXY_MANAGEMENT_KEY="$(grep -m1 '^CLIPROXY_MANAGEMENT_KEY=' "${CLIPROXY_ENV_FILE}" | cut -d= -f2- || true)" +if [[ -z "${CLIPROXY_API_KEY:-}" ]]; then + fail "environment-file: ${CLIPROXY_ENV_FILE} is missing CLIPROXY_API_KEY" +fi +if [[ -z "${CLIPROXY_MANAGEMENT_KEY:-}" ]]; then + fail "environment-file: ${CLIPROXY_ENV_FILE} is missing CLIPROXY_MANAGEMENT_KEY" +fi +export CLIPROXY_API_KEY + +if [[ -r "${CLIPROXY_VERSION_MARKER}" ]]; then + [[ "$(<"${CLIPROXY_VERSION_MARKER}")" == "${EXPECTED_PROXY_VERSION}" ]] || fail "proxy-version" +else + version_output="$("${CLIPROXY_BIN}" --version 2>&1 || true)" + grep -Fq "CLIProxyAPI Version: ${EXPECTED_PROXY_VERSION}" <<<"${version_output}" || fail "proxy-version" +fi + +api_get() { + printf 'header = "Authorization: Bearer %s"\n' "${CLIPROXY_API_KEY}" \ + | curl -fsS --connect-timeout 2 --max-time 15 -K - "$1" +} +management_get() { + printf 'header = "Authorization: Bearer %s"\n' "${CLIPROXY_MANAGEMENT_KEY}" \ + | curl -fsS --connect-timeout 2 --max-time 15 -K - "${PROXY_URL%/}/v0/management/auth-files" +} + +models="" +for _attempt in {1..20}; do + if models="$(api_get "${PROXY_URL%/}/v1/models" 2>/dev/null)"; then break; fi + sleep 1 +done +[[ -n "${models}" ]] || fail "responses-model-catalog" +for model in gpt-5.6-sol gpt-5.6-sol-low gpt-5.6-sol-medium gpt-5.6-sol-xhigh; do + python3 -c 'import json,sys; p=json.load(sys.stdin); needle=sys.argv[1]; raise SystemExit(0 if any(x.get("id")==needle for x in p.get("data",[])) else 1)' \ + "${model}" <<<"${models}" || fail "responses-model-catalog" +done + +snapshot() { + management_get | python3 -c ' +import json, sys +p=json.load(sys.stdin); xs=p.get("files",p.get("data",p if isinstance(p,list) else [])) +# recent_requests success counters are known as success, successful, or succeeded. +def successes(value): + if isinstance(value, dict): + return sum((v if isinstance(v,(int,float)) else 0) for k,v in value.items() if k in ("success","successful","succeeded")) + sum(successes(v) for k,v in value.items() if k not in ("success","successful","succeeded")) + if isinstance(value, list): return sum(successes(v) for v in value) + return 0 +out={} +for x in xs: + if x.get("provider")=="codex" and not x.get("disabled",False) and (x.get("account") or x.get("email")): + out[str(x.get("name"))]=successes(x.get("recent_requests",{})) +print(json.dumps(out,sort_keys=True)) +' +} + +before="$(snapshot)" || fail "codex-credential-identity" +enabled_count="$(python3 -c 'import json,sys; print(len(json.load(sys.stdin)))' <<<"${before}")" +(( enabled_count >= 1 )) || fail "codex-credential-identity" + +candidate="${gate_home}/config.toml" +cat > "${candidate}" < "$6.tmp" && mv "$6.tmp" "$6"' \ + gate-launch "${GATE_TIMEOUT_SECONDS}" "${CODEX_BIN}" "$(pwd)" "${report}" "$1" "${done_file}" >"${log}" 2>&1 "$5.tmp" && mv "$5.tmp" "$5"' \ + gate-resume "${GATE_TIMEOUT_SECONDS}" "${CODEX_BIN}" "${report}" "$1" "${done_file}" >"${log}" 2>&1 before.get(name, 0))) +PY +} +after="$(snapshot)" || fail "codex-credential-identity" +changed_count="$(changed_credentials "${before}" "${after}")" +counter_deadline=$((SECONDS + GATE_COUNTER_WINDOW_SECONDS)) +while (( changed_count < 1 && SECONDS < counter_deadline )); do + sleep 1 + after="$(snapshot)" || fail "codex-credential-identity" + changed_count="$(changed_credentials "${before}" "${after}")" +done +(( changed_count >= 1 )) || fail "codex-credential-identity" +if (( enabled_count >= 2 )); then + (( changed_count == 1 )) || fail "session-affinity" +else + echo "session-affinity: N/A — single credential; rerun after enrollment" +fi +echo "effort-survival: request logging does not expose reasoning effort; client command used medium" + +if [[ "${candidate}" == "${TARGET_CONFIG}" ]]; then + chmod 0600 "${TARGET_CONFIG}" +else + install -d -m 0700 "$(dirname "${TARGET_CONFIG}")" + install -m 0600 "${candidate}" "${TARGET_CONFIG}" +fi +echo "PASS: standalone Codex provider installed" diff --git a/daemon/ops/linear-agent-daemon.service b/daemon/ops/linear-agent-daemon.service index 5a3cd4a..9e47630 100644 --- a/daemon/ops/linear-agent-daemon.service +++ b/daemon/ops/linear-agent-daemon.service @@ -1,7 +1,7 @@ [Unit] Description=Linear agent webhook daemon -After=network-online.target -Wants=network-online.target +After=network-online.target cliproxyapi.service +Wants=network-online.target cliproxyapi.service [Service] Type=simple diff --git a/daemon/ops/provision.sh b/daemon/ops/provision.sh index b9d00bc..7384aeb 100755 --- a/daemon/ops/provision.sh +++ b/daemon/ops/provision.sh @@ -12,7 +12,7 @@ SOURCE_DIR="${1:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}" export DEBIAN_FRONTEND=noninteractive apt-get update apt-get install -y \ - build-essential ca-certificates curl git gnupg libatomic1 pkg-config python3 \ + build-essential ca-certificates curl git gnupg libatomic1 openssl pkg-config python3 \ rsync sqlite3 ufw \ libcairo2 libdrm2 libgbm1 libnss3 libpango-1.0-0 libxcomposite1 \ libxdamage1 libxfixes3 libxkbcommon0 libxrandr2 @@ -44,6 +44,33 @@ if ! command -v gh >/dev/null || [[ "$(gh --version | head -1)" != "gh version $ rm -rf "${tmp}"; trap - EXIT fi +CLIPROXY_VERSION="7.2.93" +case "${ARCH}" in + amd64) + CLIPROXY_ARCH="amd64" + CLIPROXY_SHA256="3ca18073c87a7d21391dcc437558c37ee9b98ce1eb1cd2c013e064a236664322" + ;; + arm64) + CLIPROXY_ARCH="aarch64" + CLIPROXY_SHA256="fc9d27799c97950614e98f191c3a6fea5c1b61bd390c44d2977090678b1c5794" + ;; + *) echo "unsupported CLIProxyAPI architecture: ${ARCH}" >&2; exit 1 ;; +esac +CLIPROXY_MARKER="/usr/local/share/cliproxyapi-version" +if [[ ! -x /usr/local/bin/cliproxyapi ]] || [[ ! -f "${CLIPROXY_MARKER}" ]] \ + || [[ "$(<"${CLIPROXY_MARKER}")" != "${CLIPROXY_VERSION}" ]]; then + tmp="$(mktemp -d)"; trap 'rm -rf "${tmp}"' EXIT + archive="CLIProxyAPI_${CLIPROXY_VERSION}_linux_${CLIPROXY_ARCH}.tar.gz" + curl -fsSL "https://github.com/router-for-me/CLIProxyAPI/releases/download/v${CLIPROXY_VERSION}/${archive}" \ + -o "${tmp}/${archive}" + printf '%s %s\n' "${CLIPROXY_SHA256}" "${tmp}/${archive}" | sha256sum -c - + tar -xzf "${tmp}/${archive}" -C "${tmp}" + install -m 0755 "${tmp}/cli-proxy-api" /usr/local/bin/cliproxyapi + install -d -m 0755 "$(dirname "${CLIPROXY_MARKER}")" + printf '%s\n' "${CLIPROXY_VERSION}" > "${CLIPROXY_MARKER}" + rm -rf "${tmp}"; trap - EXIT +fi + if ! command -v node >/dev/null || [[ "$(node --version)" != v22.* ]]; then curl -fsSL https://deb.nodesource.com/setup_22.x | bash - apt-get install -y nodejs @@ -72,11 +99,11 @@ if ! id linear-daemon >/dev/null 2>&1; then useradd --system --home-dir /var/lib/linear-agent-daemon --create-home --shell /usr/sbin/nologin linear-daemon fi install -d -o linear-daemon -g linear-daemon -m 0750 /opt/linear-agent-daemon /var/lib/linear-agent-daemon -install -d -o linear-daemon -g linear-daemon -m 0750 /var/lib/linear-agent-daemon/worktrees /var/lib/linear-agent-daemon/repos +install -d -o linear-daemon -g linear-daemon -m 0750 /var/lib/linear-agent-daemon/worktrees /var/lib/linear-agent-daemon/repos /var/lib/linear-agent-daemon/artifacts install -d -o root -g linear-daemon -m 0750 /etc/linear-agent-daemon if [[ ! -f /etc/linear-agent-daemon/env ]]; then install -o linear-daemon -g linear-daemon -m 0600 /dev/null /etc/linear-agent-daemon/env - echo "created /etc/linear-agent-daemon/env; populate it before starting the service" >&2 + echo "created /etc/linear-agent-daemon/env; populate it before starting the service (see README.md Environment; optional ARTIFACT_TOKEN enables artifact hosting)" >&2 fi if [[ ! -x /var/lib/linear-agent-daemon/.local/bin/claude ]]; then @@ -84,6 +111,90 @@ if [[ ! -x /var/lib/linear-agent-daemon/.local/bin/claude ]]; then 'curl -fsSL https://claude.ai/install.sh | bash' fi +install -d -o linear-daemon -g linear-daemon -m 0700 \ + /var/lib/linear-agent-daemon/.cli-proxy-api \ + /var/lib/linear-agent-daemon/.local/bin +CLIPROXY_ENV="/etc/linear-agent-daemon/cliproxyapi.env" +if [[ ! -f "${CLIPROXY_ENV}" ]]; then + install -o root -g linear-daemon -m 0640 /dev/null "${CLIPROXY_ENV}" + printf 'CLIPROXY_API_KEY=%s\nCLIPROXY_MANAGEMENT_KEY=%s\n' \ + "$(openssl rand -hex 24)" "$(openssl rand -hex 24)" > "${CLIPROXY_ENV}" +elif ! grep -q '^CLIPROXY_MANAGEMENT_KEY=' "${CLIPROXY_ENV}"; then + printf 'CLIPROXY_MANAGEMENT_KEY=%s\n' "$(openssl rand -hex 24)" >> "${CLIPROXY_ENV}" +fi +chown root:linear-daemon "${CLIPROXY_ENV}" +chmod 0640 "${CLIPROXY_ENV}" +CLIPROXY_API_KEY="$(grep -E '^CLIPROXY_API_KEY=[0-9a-f]{48}$' "${CLIPROXY_ENV}" | cut -d= -f2- || true)" +if [[ "$(grep -c '^CLIPROXY_API_KEY=' "${CLIPROXY_ENV}" || true)" -ne 1 ]] \ + || [[ ! "${CLIPROXY_API_KEY}" =~ ^[0-9a-f]{48}$ ]]; then + echo "${CLIPROXY_ENV} must contain one CLIPROXY_API_KEY=<48 lowercase hex characters> entry" >&2 + exit 1 +fi +CLIPROXY_MANAGEMENT_KEY="$(grep -E '^CLIPROXY_MANAGEMENT_KEY=[0-9a-f]{48}$' "${CLIPROXY_ENV}" | cut -d= -f2- || true)" +if [[ "$(grep -c '^CLIPROXY_MANAGEMENT_KEY=' "${CLIPROXY_ENV}" || true)" -ne 1 ]] \ + || [[ ! "${CLIPROXY_MANAGEMENT_KEY}" =~ ^[0-9a-f]{48}$ ]]; then + echo "${CLIPROXY_ENV} must contain one CLIPROXY_MANAGEMENT_KEY=<48 lowercase hex characters> entry" >&2 + exit 1 +fi +cat > /etc/linear-agent-daemon/cliproxyapi.yaml < /etc/caddy/Caddyfile </dev/null)"; then + python3 -c 'import json, sys; payload = json.load(sys.stdin); files = payload.get("files", payload.get("data", [])); raise SystemExit(0 if any(item.get("provider") == "codex" and not item.get("disabled", False) and (item.get("account") or item.get("email")) for item in files) else 1)' \ + <<<"${management_json}" + return $? + fi + sleep 1 + done + return 2 +} +GATE_CREDENTIAL_STATUS=0 +cliproxy_has_enabled_codex_credential || GATE_CREDENTIAL_STATUS=$? +if [[ "${GATE_CREDENTIAL_STATUS}" -eq 0 ]]; then + if ! runuser -u linear-daemon -- env \ + HOME=/var/lib/linear-agent-daemon \ + CLIPROXY_ENV_FILE="${CLIPROXY_ENV}" \ + CLIPROXY_VERSION_MARKER="${CLIPROXY_MARKER}" \ + EXPECTED_PROXY_VERSION="${CLIPROXY_VERSION}" \ + TARGET_CONFIG="${GATE_TARGET_CONFIG}" \ + /opt/linear-agent-daemon/ops/codex-provider-gate.sh; then + echo "standalone Codex provider gate failed; direct authentication remains selected" >&2 + fi +else + if [[ -f "${GATE_TARGET_CONFIG}" ]] \ + && head -n 1 "${GATE_TARGET_CONFIG}" | grep -Fqx '# managed by codex-provider-gate.sh — removed on gate failure'; then + rm -f "${GATE_TARGET_CONFIG}" + fi + if [[ "${GATE_CREDENTIAL_STATUS}" -eq 1 ]]; then + echo "standalone Codex provider gate skipped: no enabled Codex OAuth credentials; direct authentication remains selected" >&2 + else + echo "standalone Codex provider gate skipped: management API unavailable; direct authentication remains selected" >&2 + fi +fi env_has_key() { local key="$1" grep -Eq "^[[:space:]]*${key}=[^[:space:]]+" /etc/linear-agent-daemon/env @@ -187,6 +338,18 @@ env_has_key() { env_sessions_enabled() { ! grep -Eq '^[[:space:]]*SESSIONS_ENABLED=0([[:space:]]*(#.*)?)?$' /etc/linear-agent-daemon/env } +cliproxy_has_default_model() { + local _attempt + for _attempt in {1..10}; do + if printf 'header = "Authorization: Bearer %s"\n' "${CLIPROXY_API_KEY}" \ + | curl -fs --connect-timeout 2 --max-time 10 -K - http://127.0.0.1:8317/v1/models \ + | python3 -c 'import json, sys; data = json.load(sys.stdin).get("data", []); raise SystemExit(0 if any(model.get("id") == "gpt-5.6-sol" for model in data) else 1)' 2>/dev/null; then + return 0 + fi + sleep 1 + done + return 1 +} env_ready_for_restart() { if [[ ! -s /etc/linear-agent-daemon/env ]]; then echo "service enabled but not started: populate /etc/linear-agent-daemon/env, then systemctl restart linear-agent-daemon" >&2 @@ -201,6 +364,10 @@ env_ready_for_restart() { echo "service enabled but not restarted: SESSIONS_ENABLED=1 requires ${missing[*]} in /etc/linear-agent-daemon/env" >&2 return 1 fi + if ! systemctl is-active --quiet cliproxyapi || ! cliproxy_has_default_model; then + echo "service enabled but not started: authenticate CLIProxyAPI as linear-daemon and verify gpt-5.6-sol, then systemctl restart linear-agent-daemon" >&2 + return 1 + fi fi return 0 } diff --git a/daemon/ops/proxy-accounts.sh b/daemon/ops/proxy-accounts.sh new file mode 100755 index 0000000..70e38a3 --- /dev/null +++ b/daemon/ops/proxy-accounts.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +set -euo pipefail + +usage() { + cat <<'EOF' +usage: proxy-accounts.sh list | add codex|claude [--dry-run] + proxy-accounts.sh --help +EOF +} + +if [[ "${1:-}" == --help || "${1:-}" == -h ]]; then + if [[ $# -ne 1 ]]; then usage >&2; exit 2; fi + usage + exit 0 +fi + +PROXY_URL="${PROXY_URL:-http://127.0.0.1:8317}" +CLIPROXY_ENV_FILE="${CLIPROXY_ENV_FILE:-/etc/linear-agent-daemon/cliproxyapi.env}" +CLIPROXY_BIN="${CLIPROXY_BIN:-/usr/local/bin/cliproxyapi}" +CLIPROXY_CONFIG="${CLIPROXY_CONFIG:-/etc/linear-agent-daemon/cliproxyapi.yaml}" + +if [[ ! -r "${CLIPROXY_ENV_FILE}" ]]; then + echo "cannot read CLIProxyAPI environment file: ${CLIPROXY_ENV_FILE}" >&2 + exit 1 +fi +CLIPROXY_MANAGEMENT_KEY="$(grep -m1 '^CLIPROXY_MANAGEMENT_KEY=' "${CLIPROXY_ENV_FILE}" | cut -d= -f2- || true)" +if [[ -z "${CLIPROXY_MANAGEMENT_KEY}" ]]; then + echo "${CLIPROXY_ENV_FILE} is missing CLIPROXY_MANAGEMENT_KEY" >&2 + exit 1 +fi + +management_get() { + printf 'header = "Authorization: Bearer %s"\n' "${CLIPROXY_MANAGEMENT_KEY}" \ + | curl -fsS -K - "${PROXY_URL%/}/v0/management/auth-files" +} + +list_accounts() { + management_get | python3 -c ' +import json, sys +payload = json.load(sys.stdin) +items = payload.get("files", payload.get("data", payload if isinstance(payload, list) else [])) +for item in sorted(items, key=lambda value: (str(value.get("provider", "")), str(value.get("name", "")))): + safe = {key: item.get(key) for key in ("name", "email", "provider", "disabled", "failed", "recent_requests")} + print(json.dumps(safe, sort_keys=True, separators=(",", ":"))) +' +} + +case "${1:-}" in + list) + if [[ $# -ne 1 ]]; then usage >&2; exit 2; fi + list_accounts + ;; + add) + provider="${2:-}" + if [[ "${provider}" != codex && "${provider}" != claude ]]; then usage >&2; exit 2; fi + if [[ $# -gt 3 ]]; then usage >&2; exit 2; fi + dry_run=0 + if [[ "${3:-}" == --dry-run ]]; then dry_run=1; elif [[ $# -eq 3 ]]; then usage >&2; exit 2; fi + login_flag="--${provider}-login" + if (( dry_run )); then + printf 'would run: %q -config %q %q --no-browser\n' \ + "${CLIPROXY_BIN}" "${CLIPROXY_CONFIG}" "${login_flag}" + list_accounts + exit 0 + fi + "${CLIPROXY_BIN}" -config "${CLIPROXY_CONFIG}" "${login_flag}" --no-browser + sleep 2 + list_accounts + ;; + *) usage >&2; exit 2 ;; +esac diff --git a/daemon/ops/runbook.md b/daemon/ops/runbook.md index 525ae3e..60d445d 100644 --- a/daemon/ops/runbook.md +++ b/daemon/ops/runbook.md @@ -1,8 +1,8 @@ # Linear agent daemon provisioning and operations Provisioning and OAuth registration are human-controlled deploy gates. Do not run this -runbook from an automated agent. Real Linear, Claude, and systemd acceptance is not closed -until the live smoke checks below are captured. +runbook from an automated agent. Real Linear, Claude Code through CLIProxyAPI, and systemd +acceptance is not closed until the live smoke checks below are captured. ## Host and DNS @@ -26,12 +26,14 @@ Copy this `daemon/` directory to the host, then run the idempotent provisioner a ```bash cd daemon -bash -n ops/provision.sh +bash -n ops/provision.sh ops/claudex ops/claudex-fable sudo DAEMON_HOST=linear-agent.example.com ops/provision.sh "$PWD" ``` -It installs Node 22, pnpm 11.8, Git, pinned GitHub CLI and Codex CLI releases, Claude Code, Caddy, UFW, the dedicated `linear-daemon` user, and the -systemd unit. It also installs the `sqlite3` CLI used by the smoke checks. It deploys with +It installs Node 22, pnpm 11.8, Git, pinned GitHub CLI, Codex CLI, and CLIProxyAPI releases, +the Claude Code harness, a real `claudex` executable, Caddy, UFW, the dedicated +`linear-daemon` user, and the daemon/proxy systemd units. It also installs the `sqlite3` CLI +used by the smoke checks. It deploys with `pnpm install --frozen-lockfile && pnpm build && pnpm prune --prod`. Caddy terminates TLS, enforces a 1 MB request-body cap, and proxies only to the daemon's loopback listener. @@ -40,8 +42,9 @@ for NodeSource and pnpm so the script remains runnable on a clean host. Before p use, fetch the scripts once, review and pin their checksums in your deployment notes where practical, and rely on the pinned pnpm version plus `pnpm install --frozen-lockfile` for the application dependency graph. Caddy packages are installed from the signed Cloudsmith apt -repository. The Claude native installer is another vendor `curl | bash` path: review and -checksum the downloaded installer before production provisioning when practical. +repository. CLIProxyAPI is pinned by release and SHA-256 for both supported architectures. +The Claude native installer is another vendor `curl | bash` path: review and checksum the +downloaded installer before production provisioning when practical. The default firewall permits only SSH and HTTPS. Linear currently publishes these webhook source IPs: `35.231.147.226`, `35.243.134.228`, `34.140.253.14`, `34.38.87.206`, @@ -97,6 +100,13 @@ TARGET_REPO_PATH=/var/lib/linear-agent-daemon/repos/bloom-mono WORKTREES_ROOT=/var/lib/linear-agent-daemon/worktrees LINEAR_API_KEY=... CLAUDE_BIN=/var/lib/linear-agent-daemon/.local/bin/claude +# Capacity fallback: a validated Claude usage/rate-limit failure retries once through +# the Claudex proxy wrapper (ops/claudex, installed by provision.sh; it carries the +# proxy env itself, so CLAUDEX_ENV is not needed with it): +CLAUDEX_BIN=/var/lib/linear-agent-daemon/.local/bin/claudex +# CLAUDEX_ENV optionally supplies extra child env as a JSON string map when CLAUDEX_BIN +# points at a bare claude binary instead of the wrapper; it requires CLAUDEX_BIN: +# CLAUDEX_ENV={"ANTHROPIC_BASE_URL":"http://127.0.0.1:8317","ANTHROPIC_AUTH_TOKEN":"..."} CLAUDE_PERMISSION_MODE=bypassPermissions CLAUDE_MAX_TURNS=100 DO_PERMISSION_MODE=bypassPermissions @@ -150,16 +160,64 @@ sudo chmod 600 /var/lib/linear-agent-daemon/.git-credentials ``` Use a fine-grained PAT or GitHub App installation token limited to the target repository. -The `read -s` flow keeps the token out of shell history and process argv. Install the -dedicated Anthropic credential as `linear-daemon` using the current Claude Code headless -authentication procedure, and put the scoped `LINEAR_API_KEY` only in the mode-0600 env -file. Confirm `sudo -u linear-daemon -H claude --version` and that `git remote -v` is HTTPS. -The target repository must contain its own `AGENTS.md` work-item tracker configuration; the -daemon guarantees `/do` runs on its required non-default `agents/` branch. +The `read -s` flow keeps the token out of shell history and process argv. The provisioner +installs the Claude Code harness but does not authenticate it directly with Anthropic. +Instead, it creates a loopback-only CLIProxyAPI service and a `claudex` executable that +routes the harness and every subagent through GPT-5.6 Sol via the Codex OAuth pool. The main +loop uses high reasoning; Claude model pins map to low, medium, or xhigh proxy aliases, and +Claude Code compacts against a 250k context budget. Separate local API and management keys +are generated once in `/etc/linear-agent-daemon/cliproxyapi.env`; do not copy either key into +the main daemon env file. + +Enroll both founders in both provider pools as `linear-daemon`. On a headless host, open each +printed URL on another device and paste the callback URL or authorization code as prompted. +The helper prints only a redacted pool summary. Re-running `add` is an intentional re-login +and leaves one current pool record for that identity; use `--dry-run` before checking the +flow: -Authenticate GitHub and Codex for the service user, then verify them from the systemd user -context. Use only the repository-scoped bot token. `GH_TOKEN`/`GITHUB_TOKEN` may instead be -placed in the mode-0600 env file and is passed to `/do` without webhook/OAuth secrets. +```bash +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh add codex --dry-run +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh add codex +# Repeat the preceding command for the second founder. +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh add claude +# Repeat the preceding command for the second founder. +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list +sudo systemctl restart cliproxyapi +sudo -u linear-daemon -H /var/lib/linear-agent-daemon/.local/bin/claudex \ + -p "Reply with exactly: claudex works." +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/codex-provider-gate.sh +sudo systemctl restart linear-agent-daemon +``` + +The Claude enrollment is viable only after one real provider-native request completes +through an enrolled Claude credential. Choose a Claude-provider model from the proxy catalog, +then record the redacted management counters before and after this Messages-protocol probe: + +```bash +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list +read -r -p 'Claude provider model ID from /v1/models: ' CLAUDE_MODEL +sudo -u linear-daemon -H env CLAUDE_MODEL="$CLAUDE_MODEL" bash -c ' + . /etc/linear-agent-daemon/cliproxyapi.env + python3 -c "import json,os; print(json.dumps({\"model\":os.environ[\"CLAUDE_MODEL\"],\"max_tokens\":16,\"messages\":[{\"role\":\"user\",\"content\":\"Reply with exactly: claude pool works.\"}]}))" \ + | curl -fsS -K <(printf "header = \"x-api-key: %s\"\nheader = \"anthropic-version: 2023-06-01\"\nheader = \"content-type: application/json\"\n" "$CLIPROXY_API_KEY") \ + --data-binary @- http://127.0.0.1:8317/v1/messages' +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list +unset CLAUDE_MODEL +``` + +If Anthropic rejects subscription OAuth, disable/remove the Claude +credentials, record the probe as failed, and leave Phase 2 on Sol; do not claim the Claude +half of AC1. The `claudex` command must return exactly `claudex works.` and the provider gate +must print `PASS`. A warning that claude.ai connectors +are disabled is expected because the local proxy token replaces Claude authentication for +that process. Confirm that `git remote -v` is HTTPS. The target repository must contain its +own `AGENTS.md` work-item tracker configuration; the daemon guarantees `/do` runs on its +required non-default `agents/` branch. + +Authenticate GitHub and the standalone Codex CLI for the service user, then verify them from +the systemd user context. Use only the repository-scoped bot token. +`GH_TOKEN`/`GITHUB_TOKEN` may instead be placed in the mode-0600 env file and is passed to +`/do` without webhook/OAuth secrets. ```bash sudo -u linear-daemon -H gh auth login --git-protocol https @@ -173,18 +231,46 @@ sudo -u linear-daemon -H gh pr create --repo dcouple/bloom-mono --draft --fill - ## Host checks ```bash -cd /opt/linear-agent-daemon && bash -n ops/provision.sh -sudo systemd-analyze verify /etc/systemd/system/linear-agent-daemon.service +cd /opt/linear-agent-daemon && bash -n ops/provision.sh ops/claudex ops/claudex-fable ops/proxy-accounts.sh ops/codex-provider-gate.sh +sudo systemd-analyze verify \ + /etc/systemd/system/cliproxyapi.service \ + /etc/systemd/system/linear-agent-daemon.service sudo caddy validate --config /etc/caddy/Caddyfile sudo ufw status verbose sshd -T | grep -i passwordauthentication -sudo stat -c '%a %U %G' /etc/linear-agent-daemon/env -sudo ss -ltnp | grep -E ':(443|8787) ' -systemctl status linear-agent-daemon caddy +sudo stat -c '%a %U %G' \ + /etc/linear-agent-daemon/env \ + /etc/linear-agent-daemon/cliproxyapi.env \ + /etc/linear-agent-daemon/cliproxyapi.yaml +sudo ss -ltnp | grep -E ':(443|8317|8787) ' +systemctl status cliproxyapi linear-agent-daemon caddy +``` + +Expected: the daemon env is `600 linear-daemon linear-daemon`; proxy secret/config files are +`640 root linear-daemon`; ports 8317 and 8787 listen on `127.0.0.1` only; Caddy listens on +443; password authentication is `no`; and all three services are active. Verify the proxy's +model inventory without exposing its key in process argv: + +```bash +sudo -u linear-daemon -H sh -c '. /etc/linear-agent-daemon/cliproxyapi.env + printf "header = \\"Authorization: Bearer %s\\"\\n" "${CLIPROXY_API_KEY}" \ + | curl -fsS -K - http://127.0.0.1:8317/v1/models \ + | python3 -m json.tool' | grep -F 'gpt-5.6-sol' +``` + +The inventory should include `gpt-5.6-sol` plus the `-low`, `-medium`, and `-xhigh` +aliases. An empty `data` array means the Codex OAuth flow did not complete for +`linear-daemon`. + +Check the credential pool only through the redacting management helper: + +```bash +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list ``` -Expected: env mode/owner `600 linear-daemon linear-daemon`, port 8787 on `127.0.0.1` -only, Caddy on 443, password authentication `no`, and both services active. +Each record includes only name, email, provider, disabled, failed, and recent request +counters. It must show two enabled Codex and two enabled Claude identities after enrollment. +Never paste raw credential files into logs or artifacts. The SQLite database is secret material because it stores raw webhook payloads and OAuth access tokens. Keep `/var/lib/linear-agent-daemon` owned by `linear-daemon:linear-daemon` @@ -195,6 +281,123 @@ live database file directly. ## Deploy-gate smoke evidence +### Phase 1 multi-account evidence + +Capture command output in the private deploy record after redacting account names and email +addresses. These are human red-tier checks because they use founder subscription accounts. + +AC1 — aliases, two identities in each pool, and no credential-value logging: + +```bash +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list +sudo journalctl -u cliproxyapi --since today --no-pager +``` + +AC2 — record `recent_requests` before and after six independent Sol conversations; successful +counters must increase for at least two enabled Codex identities: + +```bash +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list +for run in 1 2 3 4 5 6; do + sudo -u linear-daemon -H /var/lib/linear-agent-daemon/.local/bin/claudex \ + -p "Reply with exactly: routing-${run}." +done +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list +``` + +AC3 — the gate exercises a tool call, streaming output, detached marker pickup, and +`resume --last` through the host-pinned Codex CLI: + +```bash +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/codex-provider-gate.sh +``` + +AC4 — verify the 168-hour config and continue both protocol conversations. Record the same +credential's counter increment on each continuation. The Codex continuation is part of the +gate; this command makes and resumes a Claude-protocol conversation: + +```bash +grep -F 'session-affinity-ttl: "168h"' /etc/linear-agent-daemon/cliproxyapi.yaml +sudo -u linear-daemon -H sh -c ' + result="$(claudex -p --output-format json "Reply with exactly: affinity-start.")" + session="$(printf "%s" "$result" | python3 -c "import json,sys; print(json.load(sys.stdin)[\"session_id\"])")" + claudex --resume "$session" -p "Reply with exactly: affinity-resume."' +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/codex-provider-gate.sh +``` + +There is no finite maximum daemon session lifetime, so the literal “TTL longer than maximum +lifecycle” wording remains unmet. The 168-hour value exceeds observed runs; expiry affects +provider-side affinity/cache cost, while both harnesses retain local resume state. Record +human acceptance or require the criterion to be reworded. If Claude-protocol counters do not +expose affinity, record that evidence limitation rather than passing it silently. + +AC5 — choose one enabled Codex filename from the redacted list, disable it through the body +form, run a new conversation, verify only another enabled identity increments, then re-enable +the credential: + +```bash +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list +read -r -p 'Codex credential filename to disable: ' ACCOUNT_FILE +sudo -u linear-daemon -H bash -c ' + set -euo pipefail + account_file="$1" + account_body="$(mktemp)" + . /etc/linear-agent-daemon/cliproxyapi.env + set_account_status() { + python3 -c '\''import json,sys; print(json.dumps({"name":sys.argv[1],"disabled":sys.argv[2] == "true"}))'\'' \ + "$account_file" "$1" > "$account_body" + printf "header = \"Authorization: Bearer %s\"\n" "$CLIPROXY_MANAGEMENT_KEY" \ + | curl -fsS -K - -H "Content-Type: application/json" -X PATCH \ + --data-binary "@$account_body" http://127.0.0.1:8317/v0/management/auth-files/status + } + account_disabled=0 + cleanup() { + status=$? + if (( account_disabled )); then set_account_status false || true; fi + rm -f "$account_body" + return "$status" + } + trap cleanup EXIT + account_disabled=1 + set_account_status true + /var/lib/linear-agent-daemon/.local/bin/claudex -p "Reply with exactly: failover-ok." + /opt/linear-agent-daemon/ops/proxy-accounts.sh list + set_account_status false + account_disabled=0 +' gate-account-status "$ACCOUNT_FILE" +unset ACCOUNT_FILE +``` + +AC6 — onboarding needs no daemon code change; dry-run twice for identical output, perform the +interactive login, and observe hot-loading in the list without restarting the proxy: + +```bash +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh add codex --dry-run +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh add codex --dry-run +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh add codex +sudo -u linear-daemon -H /opt/linear-agent-daemon/ops/proxy-accounts.sh list +``` + +AC7 — the passing gate above installs the provider. A dead-port run must name the failed gate, +exit nonzero, and remove only a gate-marked target; use a disposable target for this proof: + +```bash +sudo -u linear-daemon -H bash -c ' + set -eu + gate_tmp="$(mktemp -d)" + trap '\''rm -rf "$gate_tmp"'\'' EXIT + target="$gate_tmp/config.toml" + printf "%s\n" "# managed by codex-provider-gate.sh — removed on gate failure" > "$target" + set +e + PROXY_URL=http://127.0.0.1:1 TARGET_CONFIG="$target" \ + /opt/linear-agent-daemon/ops/codex-provider-gate.sh + gate_status=$? + set -e + test "$gate_status" -ne 0 + test ! -e "$target" +' +``` + External HTTPS health: ```bash @@ -241,6 +444,75 @@ and confirm Linear reuses `Linear-Delivery`; record the two delivery IDs. Linear about 1 minute, 1 hour, and 6 hours and may disable repeated failures. On restart, confirm the daemon re-enables each app webhook; use the app settings only as the manual fallback. +## Enable and verify Fable routing + +Fable remains disabled until the enrolled Claude accounts expose confirmed real Anthropic +model IDs. Enroll the accounts with `cliproxyapi --claude-login`, then inspect the catalog: + +```bash +. /etc/linear-agent-daemon/cliproxyapi.env +printf 'header = "Authorization: Bearer %s"\n' "$CLIPROXY_API_KEY" | + curl -fsS -K - http://127.0.0.1:8317/v1/models | + python3 -m json.tool | grep '"id": "claude-' +``` + +If no usable `claude-*` model completes a request, leave `FABLE_BIN` unset and report Phase +2 AC2 unmet. Otherwise author the mapping only after that request succeeds: + +```bash +sudo install -o root -g linear-daemon -m 0640 /dev/null /etc/linear-agent-daemon/fable-models.env +sudoedit /etc/linear-agent-daemon/fable-models.env +# FABLE_MAIN_MODEL=claude- +# FABLE_HAIKU_MODEL=claude- +# FABLE_SONNET_MODEL=claude- +# FABLE_OPUS_MODEL=claude- +# FABLE_FABLE_MODEL=claude- +sudo sh -c 'printf "\nFABLE_BIN=/var/lib/linear-agent-daemon/.local/bin/claudex-fable\n" >> /etc/linear-agent-daemon/env' +sudo systemctl restart linear-agent-daemon +sudo -u linear-daemon -H sh -c ' + . /etc/linear-agent-daemon/cliproxyapi.env + printf "header = \"Authorization: Bearer %s\"\n" "$CLIPROXY_MANAGEMENT_KEY" | + curl -fsS -K - http://127.0.0.1:8787/healthz +' | python3 -m json.tool +``` + +The health response must show `providers.claude.status` as `ready` before a new Fable +session is created (AC1/AC3). For AC2, create one Fable workflow, complete one Claude-side +request and one detached `codex exec -m gpt-5.6-sol` role, then preserve the model catalog, +management counters, and redacted proxy journal proving Claude and Codex used their +respective pools. For AC4, record the session profile and Claude ID, restart, prompt again, +and verify both remain unchanged: + +```bash +sudo sqlite3 /var/lib/linear-agent-daemon/events.db \ + 'select linear_session_id,profile,claude_session_id from sessions order by last_seen_at desc limit 5;' +sudo systemctl restart linear-agent-daemon +sudo journalctl -u linear-agent-daemon --since '-10 min' -o cat | + grep -E 'session_profile_assigned|provider_state_changed|provider_failure_classified|profile_fallback|profile_launcher_unconfigured' +sudo -u linear-daemon -H sh -c ' + . /etc/linear-agent-daemon/cliproxyapi.env + printf "header = \"Authorization: Bearer %s\"\n" "$CLIPROXY_MANAGEMENT_KEY" | + curl -fsS -K - http://127.0.0.1:8787/healthz +' | python3 -m json.tool +``` + +The public health response is only `{ "ok": true }`; provider details require the management-key +header used above. Those journal and authorized-health commands are the supporting AC6 schema/redaction evidence; verify +they contain no key, token, or account email. For AC7, temporarily put a `gpt-*` alias in one +`FABLE_*_MODEL` entry and run `claudex-fable -p test`: it must exit nonzero naming that +variable before sending a model request. Restore the confirmed mapping afterward. Legacy +rows with a NULL profile intentionally route as Sol and log `legacy_session_profile_defaulted`. + +Preserve the automated host evidence for the fixture-drivable criteria as well: + +```bash +cd /opt/linear-agent-daemon +pnpm vitest run test/sessions.test.ts -t 'persists Fable for planner and implementer|probes readiness' # AC1, AC3 +pnpm vitest run test/sessions.test.ts -t 'reopens SQLite|child-restart' # AC4 +pnpm vitest run test/sessions.test.ts -t 'falls back once|launcher is unconfigured' # AC5, AC7 +pnpm vitest run test/server.test.ts test/sessions.test.ts -t 'keeps provider health private|probes readiness' # AC6 +``` + ## Planner-session smoke Assign bloom-planner to a test issue. Within ten seconds, confirm the ack and session-start @@ -321,20 +593,28 @@ artifact path. Do not run the emulator smoke from an automated implementation ag | Two OAuth client IDs/secrets | 1 / `linear-daemon` | env, `0600` | Agent scopes only | Rotate secret or revoke app; update and restart | | SQLite event/token database | 1 / `linear-daemon` | `/var/lib/linear-agent-daemon/events.db*`, `0600`/dir `0750` | Raw payloads and OAuth access tokens | Encrypt backups; delete per retention; revoke OAuth apps if exposed | | Bot git/gh identity + HTTPS credential | 2–3 / `linear-daemon` | `~/.gitconfig`, `~/.git-credentials` or env, `0600` | One repository, least privilege | Revoke PAT/App token and replace | -| Codex authentication | 3 / `linear-daemon` | provider config under service HOME, `0600` | Dedicated bot project | Revoke provider token, replace | +| Standalone Codex provider selection | 3 / `linear-daemon` | `~/.codex/config.toml`, `0600` | Loopback Responses provider; no token stored | Rerun gate; failed gate removes only its marked config | +| CLIProxyAPI Codex OAuth pool | 2 / `linear-daemon` | `~/.cli-proxy-api/codex-*.json`, `0600` | Both founders' ChatGPT/Codex subscriptions | Revoke OpenAI authorization, rerun `proxy-accounts.sh add codex` | +| CLIProxyAPI Claude OAuth pool | 2 / `linear-daemon` | provider-reported files under `~/.cli-proxy-api/`, `0600` | Both founders' Claude subscriptions, subject to viability probe | Revoke Anthropic authorization, rerun `proxy-accounts.sh add claude` | +| CLIProxyAPI local API + management keys | 2 / root + `linear-daemon` group | `/etc/linear-agent-daemon/cliproxyapi.env` and generated `.yaml`, `0640` | Loopback proxy and loopback management API only | Stop services, replace both env values, rerun provisioner, restart | | `LINEAR_API_KEY` for spawned sessions | 2 / `linear-daemon` | env, `0600` | Scoped bot access | Revoke in Linear, replace env | -| Anthropic authentication | 2 / `linear-daemon` | provider config, `0600` | Dedicated bot billing/project | Revoke provider token, replace | -Never install personal credentials on the host. The latter three credentials are inventory -only in phase 1 and are not installed until their owning phases. +Install founder subscription OAuth only through the documented interactive proxy flow; never +copy raw token values or unrelated personal credentials onto the host. All other credentials +are installed only in their owning phase. ## Logs and recovery ```bash journalctl -u linear-agent-daemon -f +journalctl -u cliproxyapi --since today journalctl -u caddy --since today ``` +The daemon orders itself after CLIProxyAPI and wants it started, but does not stop when the +proxy briefly restarts. This keeps webhook ingestion online; any in-flight Claude turn that +fails during the outage is recorded normally and can be resumed after the proxy recovers. + Logs contain delivery/session/issue IDs but no raw webhook bodies or tokens. Back up `/var/lib/linear-agent-daemon/events.db` using SQLite's backup mechanism. The service uses WAL and `Restart=always`; investigate named `ack_failed`, `terminal_activity_delivery_failed`, diff --git a/daemon/src/artifacts.ts b/daemon/src/artifacts.ts new file mode 100644 index 0000000..28909d0 --- /dev/null +++ b/daemon/src/artifacts.ts @@ -0,0 +1,165 @@ +import { randomBytes } from "node:crypto"; +import { mkdir, readFile, readdir, rename, rm, stat, writeFile } from "node:fs/promises"; +import { extname, isAbsolute, relative, resolve as resolvePath, sep } from "node:path"; + +export interface ArtifactFile { + path: string; + content: Buffer; +} + +export class InvalidArtifactError extends Error {} +export class ArtifactNotFoundError extends Error {} + +const ID_PATTERN = /^[A-Za-z0-9_-]{22,64}$/; +const VERSION_PATTERN = /^v-[A-Za-z0-9_-]+$/; +const SEGMENT_PATTERN = /^[A-Za-z0-9._ -]+$/; + +export class ArtifactStore { + constructor(readonly artifactsDir: string) {} + + createId(): string { + return randomBytes(16).toString("base64url"); + } + + isValidId(id: string): boolean { + return ID_PATTERN.test(id); + } + + async create(files: readonly ArtifactFile[]): Promise { + await mkdir(this.artifactsDir, { recursive: true }); + for (;;) { + const id = this.createId(); + try { + await mkdir(this.bundleDir(id)); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "EEXIST") continue; + throw error; + } + try { + await this.write(id, files); + return id; + } catch (error) { + await rm(this.bundleDir(id), { recursive: true, force: true }); + throw error; + } + } + } + + async replace(id: string, files: readonly ArtifactFile[]): Promise { + if (!this.isValidId(id) || !(await this.currentVersion(id))) throw new ArtifactNotFoundError("artifact not found"); + await this.write(id, files); + } + + async resolve(id: string, relPath: string): Promise { + if (!this.isValidId(id) || !validRelativePath(relPath)) return undefined; + const version = await this.currentVersion(id); + if (!version) return undefined; + const root = resolvePath(this.bundleDir(id), version); + const candidate = resolvePath(root, relPath); + if (candidate !== root && !candidate.startsWith(`${root}${sep}`)) return undefined; + try { + return (await stat(candidate)).isFile() ? candidate : undefined; + } catch { + return undefined; + } + } + + async list(id: string): Promise { + if (!this.isValidId(id)) return []; + const version = await this.currentVersion(id); + if (!version) return []; + const root = resolvePath(this.bundleDir(id), version); + const files: string[] = []; + const walk = async (dir: string): Promise => { + for (const entry of await readdir(dir, { withFileTypes: true })) { + const absolute = resolvePath(dir, entry.name); + if (entry.isDirectory()) await walk(absolute); + else if (entry.isFile()) files.push(relative(root, absolute).split(sep).join("/")); + } + }; + try { + await walk(root); + return files; + } catch { + return []; + } + } + + contentTypeFor(path: string): string { + const types: Record = { + ".html": "text/html; charset=utf-8", + ".md": "text/markdown; charset=utf-8", + ".png": "image/png", + ".svg": "image/svg+xml", + ".jpg": "image/jpeg", + ".jpeg": "image/jpeg", + ".gif": "image/gif", + ".webp": "image/webp", + ".css": "text/css; charset=utf-8", + ".js": "text/javascript; charset=utf-8", + ".json": "application/json; charset=utf-8", + ".txt": "text/plain; charset=utf-8", + ".pdf": "application/pdf", + }; + return types[extname(path).toLowerCase()] ?? "application/octet-stream"; + } + + private bundleDir(id: string): string { + return resolvePath(this.artifactsDir, id); + } + + private async currentVersion(id: string): Promise { + try { + const version = (await readFile(resolvePath(this.bundleDir(id), "current"), "utf8")).trim(); + return VERSION_PATTERN.test(version) ? version : undefined; + } catch { + return undefined; + } + } + + private async write(id: string, files: readonly ArtifactFile[]): Promise { + validateFiles(files); + const bundleDir = this.bundleDir(id); + const previous = await this.currentVersion(id); + const version = `v-${randomBytes(12).toString("base64url")}`; + const versionDir = resolvePath(bundleDir, version); + const pointerTemp = resolvePath(bundleDir, `.current.tmp-${randomBytes(8).toString("base64url")}`); + await mkdir(versionDir); + try { + for (const file of files) { + const destination = resolvePath(versionDir, file.path); + await mkdir(resolvePath(destination, ".."), { recursive: true }); + await writeFile(destination, file.content); + } + await writeFile(pointerTemp, version, { flag: "wx" }); + await rename(pointerTemp, resolvePath(bundleDir, "current")); + } catch (error) { + await rm(pointerTemp, { force: true }); + await rm(versionDir, { recursive: true, force: true }); + throw error; + } + if (previous && previous !== version) { + const cleanup = setTimeout(() => { + void rm(resolvePath(bundleDir, previous), { recursive: true, force: true }).catch(() => undefined); + }, 30_000); + cleanup.unref(); + } + } +} + +function validateFiles(files: readonly ArtifactFile[]): void { + if (files.length === 0 || files.length > 100) throw new InvalidArtifactError("bundle must contain 1 to 100 files"); + const paths = new Set(); + for (const file of files) { + if (!validRelativePath(file.path)) throw new InvalidArtifactError("invalid artifact path"); + if (paths.has(file.path)) throw new InvalidArtifactError("duplicate artifact path"); + paths.add(file.path); + if (!Buffer.isBuffer(file.content)) throw new InvalidArtifactError("invalid artifact content"); + } +} + +function validRelativePath(path: string): boolean { + if (!path || isAbsolute(path) || path.includes("\\")) return false; + const segments = path.split("/"); + return segments.length <= 8 && segments.every(segment => segment !== "" && segment !== "." && segment !== ".." && SEGMENT_PATTERN.test(segment)); +} diff --git a/daemon/src/claude.ts b/daemon/src/claude.ts index 38a016e..6ca1726 100644 --- a/daemon/src/claude.ts +++ b/daemon/src/claude.ts @@ -11,6 +11,7 @@ export type ClaudeEvent = export interface RunTurnOptions { cwd: string; prompt: string; resumeSessionId?: string; argv: string[]; permissionMode: string; maxTurns: number; maxBudgetUsd?: number; mcpConfigJson: string; env?: NodeJS.ProcessEnv; + trustedEnv?: Record; onEvent?: (event: ClaudeEvent) => void | Promise; onSessionId?: (id: string) => void | Promise; signal?: AbortSignal; @@ -18,14 +19,14 @@ export interface RunTurnOptions { export interface RunTurnResult { ok: boolean; sessionId?: string; resultText?: string; isError: boolean; exitCode: number | null; signal: NodeJS.Signals | null; spawnError?: string; permissionDenials: unknown[]; sawResult: boolean; - stderrTail?: string; + stderrTail?: string; capacityEvidence: string[]; } function record(value: unknown): Record | undefined { return value !== null && typeof value === "object" && !Array.isArray(value) ? value as Record : undefined; } -function childEnv(extra: NodeJS.ProcessEnv | undefined): NodeJS.ProcessEnv { +function childEnv(extra: NodeJS.ProcessEnv | undefined, trusted: Record | undefined): NodeJS.ProcessEnv { const allowed: NodeJS.ProcessEnv = {}; const include = (key: string, value: string | undefined): void => { if (value === undefined) return; @@ -39,9 +40,38 @@ function childEnv(extra: NodeJS.ProcessEnv | undefined): NodeJS.ProcessEnv { if (key !== "ORCHESTRA_DISPATCH_OWNER") include(key, value); } for (const [key, value] of Object.entries(extra ?? {})) include(key, value); + for (const [key, value] of Object.entries(trusted ?? {})) allowed[key] = value; return allowed; } +function collectCapacityEvidence(event: Record, evidence: Set): void { + if (event.type === "rate_limit_event") { + const info = record(event.rate_limit_info); + const rejected = info?.status === "rejected" || info?.overageStatus === "rejected"; + const disabled = typeof info?.overageDisabledReason === "string" && !!info.overageDisabledReason; + const status429 = event.apiErrorStatus === 429 || info?.apiErrorStatus === 429; + if (rejected || disabled || status429) { + const cause = rejected ? "rejected" : disabled + ? (info?.overageDisabledReason === "out_of_credits" ? "out_of_credits" : "overage_disabled") : "429"; + evidence.add(`rate_limit_event:${cause}`); + } + } + if (event.type === "system" && event.subtype === "api_retry") { + const error = typeof event.error === "string" ? event.error : undefined; + const status = event.error_status; + if (["rate_limit", "overloaded", "billing_error"].includes(error ?? "") || status === 429 || status === 529) + evidence.add(`api_retry:${error ?? status}`); + } + if (event.type === "result") { + if (event.error_status === 429) evidence.add("result:429"); + if (Array.isArray(event.errors)) for (const raw of event.errors) { + const type = record(raw)?.type; + if (type === "rate_limit_error" || type === "overloaded_error") evidence.add(`result:${type}`); + } + } + if (event.type === "assistant" && event.error === "rate_limit") evidence.add("assistant:rate_limit"); +} + function appendTail(current: string, chunk: Buffer): string { const next = current + chunk.toString("utf8"); return next.length > 8192 ? next.slice(next.length - 8192) : next; @@ -57,7 +87,7 @@ export async function runTurn(options: RunTurnOptions): Promise { if (options.resumeSessionId) args.push("--resume", options.resumeSessionId); args.push("--permission-mode", options.permissionMode, "--max-turns", String(options.maxTurns), "--mcp-config", configPath); if (options.maxBudgetUsd !== undefined) args.push("--max-budget-usd", String(options.maxBudgetUsd)); - const child = spawn(bin, args, { cwd: options.cwd, env: childEnv(options.env), detached: true, stdio: ["ignore", "pipe", "pipe"] }); + const child = spawn(bin, args, { cwd: options.cwd, env: childEnv(options.env, options.trustedEnv), detached: true, stdio: ["ignore", "pipe", "pipe"] }); let latestId: string | undefined; let resultText: string | undefined; let isError = false; @@ -65,12 +95,14 @@ export async function runTurn(options: RunTurnOptions): Promise { let denials: unknown[] = []; let spawnError: string | undefined; let stderrTail = ""; + const capacityEvidence = new Set(); const pending: Promise[] = []; const lines = createInterface({ input: child.stdout }); lines.on("line", line => { let value: unknown; try { value = JSON.parse(line); } catch { return; } const event = record(value); if (!event) return; + collectCapacityEvidence(event, capacityEvidence); if (typeof event.session_id === "string" && event.session_id !== latestId) { latestId = event.session_id; if (options.onSessionId) pending.push(Promise.resolve(options.onSessionId(latestId))); @@ -118,7 +150,7 @@ export async function runTurn(options: RunTurnOptions): Promise { return { ok, ...(latestId ? { sessionId: latestId } : {}), ...(resultText !== undefined ? { resultText } : {}), isError, exitCode: closed.code, signal: closed.signal, ...(spawnError ? { spawnError } : {}), permissionDenials: denials, sawResult, - ...(stderrTail ? { stderrTail } : {}), + ...(stderrTail ? { stderrTail } : {}), capacityEvidence: [...capacityEvidence], }; } finally { if (killTimer) clearTimeout(killTimer); diff --git a/daemon/src/config.ts b/daemon/src/config.ts index 11575c9..9c0dc21 100644 --- a/daemon/src/config.ts +++ b/daemon/src/config.ts @@ -19,6 +19,9 @@ export interface Config { linearGraphqlUrl: string; linearTokenUrl: string; webhookBaseUrl: string; + artifactToken?: string; + artifactsDir: string; + artifactMaxBodyBytes: number; reconcileIntervalMs: number; reconcileRequestTimeoutMs: number; apps: Record; @@ -26,6 +29,14 @@ export interface Config { worktreesRoot: string; targetRepoPath?: string; claudeArgv: string[]; + claudexArgv?: string[]; + claudexEnv?: Record; + fableArgv?: string[]; + cliproxyEnvFile: string; + cliproxyUrl: string; + providerProbeIntervalMs: number; + providerStateStaleMs: number; + providerInitialProbeTimeoutMs: number; claudePermissionMode: string; claudeMaxTurns: number; doPermissionMode: string; @@ -61,6 +72,27 @@ function enabled(env: NodeJS.ProcessEnv, name: string, fallback = true): boolean throw new Error(`${name} must be 0 or 1`); } +function optionalArgv(env: NodeJS.ProcessEnv, name: string): string[] | undefined { + const raw = env[name]; + if (raw === undefined) return undefined; + const value = raw.trim(); + if (!value) throw new Error(`${name} must not be empty`); + return value.split(/\s+/); +} + +function stringMap(env: NodeJS.ProcessEnv, name: string): Record | undefined { + if (env[name] === undefined) return undefined; + const raw = env[name]!.trim(); + if (!raw) throw new Error(`${name} must be valid JSON`); + let value: unknown; + try { value = JSON.parse(raw); } catch { throw new Error(`${name} must be valid JSON`); } + if (value === null || typeof value !== "object" || Array.isArray(value) + || Object.values(value as Record).some(entry => typeof entry !== "string")) { + throw new Error(`${name} must be a JSON object with string values`); + } + return value as Record; +} + function appConfig(env: NodeJS.ProcessEnv, name: AppName, testMode: boolean): AppConfig { const prefix = name.toUpperCase(); const staticToken = env[`${prefix}_LINEAR_TOKEN`]?.trim(); @@ -80,10 +112,16 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config { const sessionsEnabled = enabled(env, "SESSIONS_ENABLED"); const targetRepoPath = env.TARGET_REPO_PATH?.trim(); const linearApiKey = env.LINEAR_API_KEY?.trim(); + const artifactToken = env.ARTIFACT_TOKEN?.trim(); const webhookBaseUrl = env.WEBHOOK_BASE_URL?.trim() || (testMode ? "http://127.0.0.1:8787" : required(env, "WEBHOOK_BASE_URL")); if (sessionsEnabled && !targetRepoPath) required(env, "TARGET_REPO_PATH"); if (sessionsEnabled && !linearApiKey) required(env, "LINEAR_API_KEY"); const claudeArgv = (env.CLAUDE_BIN?.trim() || "claude").split(/\s+/); + const claudexArgv = optionalArgv(env, "CLAUDEX_BIN"); + const claudexEnv = stringMap(env, "CLAUDEX_ENV"); + if (claudexEnv && !claudexArgv) throw new Error("CLAUDEX_ENV requires CLAUDEX_BIN"); + const fableBin = env.FABLE_BIN?.trim(); + const providerProbeIntervalMs = positiveInteger(env, "PROVIDER_PROBE_INTERVAL_MS", 60_000); const doPermissionMode = env.DO_PERMISSION_MODE?.trim() || "bypassPermissions"; if (!testMode && doPermissionMode !== "bypassPermissions") { throw new Error("DO_PERMISSION_MODE must be bypassPermissions unless DAEMON_TEST_MODE=1"); @@ -101,6 +139,9 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config { linearGraphqlUrl: env.LINEAR_GRAPHQL_URL?.trim() || "https://api.linear.app/graphql", linearTokenUrl: env.LINEAR_TOKEN_URL?.trim() || "https://api.linear.app/oauth/token", webhookBaseUrl: webhookBaseUrl.replace(/\/+$/, ""), + ...(artifactToken ? { artifactToken } : {}), + artifactsDir: env.ARTIFACTS_DIR?.trim() || `${dirname(dbPath)}/artifacts`, + artifactMaxBodyBytes: positiveInteger(env, "ARTIFACT_MAX_BODY_BYTES", 32 * 1024 * 1024), reconcileIntervalMs: positiveInteger(env, "RECONCILE_INTERVAL_MS", 60_000), reconcileRequestTimeoutMs: positiveInteger(env, "RECONCILE_REQUEST_TIMEOUT_MS", 10_000), apps: { planner: appConfig(env, "planner", testMode), implementer: appConfig(env, "implementer", testMode) }, @@ -108,6 +149,14 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config { worktreesRoot: env.WORKTREES_ROOT?.trim() || `${dirname(dbPath)}/worktrees`, ...(targetRepoPath ? { targetRepoPath } : {}), claudeArgv, + ...(claudexArgv ? { claudexArgv } : {}), + ...(claudexEnv ? { claudexEnv } : {}), + ...(fableBin ? { fableArgv: fableBin.split(/\s+/) } : {}), + cliproxyEnvFile: env.CLIPROXY_ENV_FILE?.trim() || "/etc/linear-agent-daemon/cliproxyapi.env", + cliproxyUrl: (env.CLIPROXY_URL?.trim() || "http://127.0.0.1:8317").replace(/\/+$/, ""), + providerProbeIntervalMs, + providerStateStaleMs: positiveInteger(env, "PROVIDER_STATE_STALE_MS", 5 * providerProbeIntervalMs), + providerInitialProbeTimeoutMs: positiveInteger(env, "PROVIDER_INITIAL_PROBE_TIMEOUT_MS", 5_000), claudePermissionMode: env.CLAUDE_PERMISSION_MODE?.trim() || "bypassPermissions", claudeMaxTurns: positiveInteger(env, "CLAUDE_MAX_TURNS", 100), doPermissionMode, diff --git a/daemon/src/eventlog.ts b/daemon/src/eventlog.ts index b656369..fb27a55 100644 --- a/daemon/src/eventlog.ts +++ b/daemon/src/eventlog.ts @@ -21,8 +21,17 @@ export interface AppendEvent { export interface SessionRow { linearSessionId: string; app: AppName; issueId: string | null; issueIdentifier: string | null; worktreePath: string | null; branch: string | null; claudeSessionId: string | null; + runtime: "claude" | "claudex"; fallbackCause: string | null; + profile: "fable" | "sol" | null; profileFallback: number | null; mode: string; status: string; lastSeenAt: number; lastSeenActivityAt: number | null; } +export interface ProviderStateRow { + provider: string; status: string; reason: string | null; cooldownUntil: number | null; updatedAt: number; +} +export interface AppendResult { + inserted: boolean; deliveryId: string; assignedProfile?: "fable" | "sol"; assignmentReason?: string; + stop?: { agentSessionId: string; app: AppName }; +} export interface TurnRow { id: number; eventId: number; app: AppName; linearSessionId: string; issueId: string; kind: "created" | "prompted"; prompt: string | null; status: "pending" | "running" | "awaiting_activity" | "done" | "failed" | "interrupted"; @@ -69,7 +78,8 @@ export interface AckState { export class EventLog { private readonly db: Database.Database; - constructor(path: string) { + constructor(path: string, private readonly selectProfile: () => { profile: "fable" | "sol"; reason: string } = + () => ({ profile: "sol", reason: "fable_not_configured" })) { this.db = new Database(path); this.db.pragma("journal_mode = WAL"); this.db.pragma("foreign_keys = ON"); @@ -110,6 +120,10 @@ export class EventLog { worktree_path TEXT, branch TEXT, claude_session_id TEXT, + runtime TEXT NOT NULL DEFAULT 'claude', + fallback_cause TEXT, + profile TEXT CHECK(profile IS NULL OR profile IN ('fable','sol')), + profile_fallback INTEGER, mode TEXT NOT NULL DEFAULT 'planner', status TEXT NOT NULL DEFAULT 'active', last_seen_at INTEGER NOT NULL, @@ -165,6 +179,13 @@ export class EventLog { status TEXT NOT NULL CHECK(status IN ('pending','posted','failed')), attempts INTEGER NOT NULL DEFAULT 0, next_attempt_at INTEGER NOT NULL DEFAULT 0, created_at INTEGER NOT NULL ); + CREATE TABLE IF NOT EXISTS provider_state ( + provider TEXT PRIMARY KEY, + status TEXT NOT NULL, + reason TEXT, + cooldown_until INTEGER, + updated_at INTEGER NOT NULL + ); `); this.migrateEventColumns(); this.migrateSessionColumns(); @@ -185,6 +206,10 @@ export class EventLog { private migrateSessionColumns(): void { const columns = new Set((this.db.prepare("PRAGMA table_info(sessions)").all() as Array<{ name: string }>).map(column => column.name)); if (!columns.has("last_seen_activity_at")) this.db.prepare("ALTER TABLE sessions ADD COLUMN last_seen_activity_at INTEGER").run(); + if (!columns.has("runtime")) this.db.prepare("ALTER TABLE sessions ADD COLUMN runtime TEXT NOT NULL DEFAULT 'claude'").run(); + if (!columns.has("fallback_cause")) this.db.prepare("ALTER TABLE sessions ADD COLUMN fallback_cause TEXT").run(); + if (!columns.has("profile")) this.db.prepare("ALTER TABLE sessions ADD COLUMN profile TEXT CHECK(profile IS NULL OR profile IN ('fable','sol'))").run(); + if (!columns.has("profile_fallback")) this.db.prepare("ALTER TABLE sessions ADD COLUMN profile_fallback INTEGER").run(); } private migrateTurnColumns(): void { @@ -235,7 +260,7 @@ export class EventLog { if (!columns.has("progress_barrier")) this.db.prepare("ALTER TABLE turn_activities ADD COLUMN progress_barrier INTEGER NOT NULL DEFAULT 0").run(); } - append(event: AppendEvent): { inserted: boolean; deliveryId: string; stop?: { agentSessionId: string; app: AppName } } { + append(event: AppendEvent): AppendResult { const deliveryId = event.deliveryId?.trim() || `sha256:${createHash("sha256").update(event.rawBody).digest("hex")}`; const run = this.db.transaction(() => { const result = this.db.prepare(`INSERT OR IGNORE INTO events @@ -260,20 +285,22 @@ export class EventLog { } return { inserted: true } as const; } + let assignment: { profile: "fable" | "sol"; reason: string } | undefined; const createsTurn = event.agentSessionId && (event.action === "created" || event.action === "prompted"); if (createsTurn) { const existing = this.db.prepare("SELECT issue_id issueId, issue_identifier issueIdentifier FROM sessions WHERE linear_session_id=?") .get(event.agentSessionId) as { issueId: string | null; issueIdentifier: string | null } | undefined; const issueId = event.issueId ?? existing?.issueId ?? event.agentSessionId; const issueIdentifier = event.issueIdentifier ?? existing?.issueIdentifier ?? event.issueId ?? event.agentSessionId; + assignment = existing ? undefined : this.selectProfile(); this.db.prepare(`INSERT INTO sessions - (linear_session_id, app, issue_id, issue_identifier, mode, status, last_seen_at) - VALUES (?, ?, ?, ?, ?, 'active', ?) + (linear_session_id, app, issue_id, issue_identifier, profile, mode, status, last_seen_at) + VALUES (?, ?, ?, ?, ?, ?, 'active', ?) ON CONFLICT(linear_session_id) DO UPDATE SET issue_id=COALESCE(excluded.issue_id, sessions.issue_id), issue_identifier=COALESCE(excluded.issue_identifier, sessions.issue_identifier), last_seen_at=excluded.last_seen_at`) - .run(event.agentSessionId, event.app, issueId, issueIdentifier, event.app, event.receivedAt); + .run(event.agentSessionId, event.app, issueId, issueIdentifier, assignment?.profile ?? null, event.app, event.receivedAt); if (event.action === "created" && event.issueId) { this.db.prepare(`UPDATE sessions SET issue_id=?, issue_identifier=?, last_seen_at=? WHERE linear_session_id=?`) @@ -298,6 +325,7 @@ export class EventLog { if (event.type === "Issue" && event.stateType === "completed" && event.issueId && event.issueIdentifier) { this.enqueueCleanup(event.issueId, event.issueIdentifier, event.receivedAt); } + if (assignment) return { inserted: true, assignedProfile: assignment.profile, assignmentReason: assignment.reason } as const; return { inserted: true } as const; }); const result = run(); @@ -344,11 +372,13 @@ export class EventLog { getSession(linearSessionId: string): SessionRow | undefined { return this.db.prepare(`SELECT linear_session_id linearSessionId, app, issue_id issueId, issue_identifier issueIdentifier, worktree_path worktreePath, branch, claude_session_id claudeSessionId, + runtime, fallback_cause fallbackCause, profile, profile_fallback profileFallback, mode, status, last_seen_at lastSeenAt, last_seen_activity_at lastSeenActivityAt FROM sessions WHERE linear_session_id=?`).get(linearSessionId) as SessionRow | undefined; } sessionByIssueIdentifier(identifier: string): SessionRow | undefined { const query = (mode: string) => this.db.prepare(`SELECT linear_session_id linearSessionId, app, issue_id issueId, issue_identifier issueIdentifier, worktree_path worktreePath, branch, claude_session_id claudeSessionId, + runtime, fallback_cause fallbackCause, profile, profile_fallback profileFallback, mode, status, last_seen_at lastSeenAt, last_seen_activity_at lastSeenActivityAt FROM sessions WHERE issue_identifier=? AND mode=? ORDER BY last_seen_at DESC LIMIT 1`) .get(identifier, mode) as SessionRow | undefined; return query("implementer") ?? query("planner"); @@ -356,6 +386,7 @@ export class EventLog { plannerSessionsForReconcile(): SessionRow[] { return this.db.prepare(`SELECT linear_session_id linearSessionId, app, issue_id issueId, issue_identifier issueIdentifier, worktree_path worktreePath, branch, claude_session_id claudeSessionId, + runtime, fallback_cause fallbackCause, profile, profile_fallback profileFallback, mode, status, last_seen_at lastSeenAt, last_seen_activity_at lastSeenActivityAt FROM sessions WHERE app='planner' AND mode='planner' ORDER BY last_seen_at`) .all() as SessionRow[]; @@ -363,6 +394,7 @@ export class EventLog { sessionsWithWorktrees(): SessionRow[] { return this.db.prepare(`SELECT linear_session_id linearSessionId, app, issue_id issueId, issue_identifier issueIdentifier, worktree_path worktreePath, branch, claude_session_id claudeSessionId, + runtime, fallback_cause fallbackCause, profile, profile_fallback profileFallback, mode, status, last_seen_at lastSeenAt, last_seen_activity_at lastSeenActivityAt FROM sessions WHERE worktree_path IS NOT NULL ORDER BY last_seen_at`).all() as SessionRow[]; } @@ -384,6 +416,35 @@ export class EventLog { updateClaudeSessionId(linearSessionId: string, id: string, now = Date.now()): void { this.db.prepare(`UPDATE sessions SET claude_session_id=?, last_seen_at=? WHERE linear_session_id=?`).run(id, now, linearSessionId); } + clearClaudeSessionId(linearSessionId: string, now = Date.now()): void { + this.db.prepare("UPDATE sessions SET claude_session_id=NULL, last_seen_at=? WHERE linear_session_id=?").run(now, linearSessionId); + } + recordRuntimeFallback(linearSessionId: string, claudexSessionId: string | undefined, cause: string, now = Date.now()): void { + this.db.prepare(`UPDATE sessions SET runtime='claudex', fallback_cause=?, claude_session_id=?, last_seen_at=? + WHERE linear_session_id=?`).run(cause, claudexSessionId ?? null, now, linearSessionId); + } + applyPreEstablishmentFallback(linearSessionId: string): boolean { + // The four-way WHERE predicate is the atomicity guarantee: only one unestablished Fable row can flip once. + return this.db.prepare(`UPDATE sessions SET profile='sol', profile_fallback=1 + WHERE linear_session_id=? AND claude_session_id IS NULL AND profile='fable' AND profile_fallback IS NULL`) + .run(linearSessionId).changes === 1; + } + getProviderState(provider: string): ProviderStateRow | undefined { + return this.db.prepare(`SELECT provider, status, reason, cooldown_until cooldownUntil, updated_at updatedAt + FROM provider_state WHERE provider=?`).get(provider) as ProviderStateRow | undefined; + } + setProviderState(provider: string, status: string, reason: string | null, updatedAt = Date.now(), cooldownUntil?: number | null): void { + this.db.prepare(`INSERT INTO provider_state(provider,status,reason,cooldown_until,updated_at) VALUES(?,?,?,?,?) + ON CONFLICT(provider) DO UPDATE SET status=excluded.status, reason=excluded.reason, + cooldown_until=excluded.cooldown_until, updated_at=excluded.updated_at`) + .run(provider, status, reason, cooldownUntil ?? null, updatedAt); + } + setProviderCooldown(provider: string, cooldownUntil: number, reason: string, updatedAt = Date.now()): void { + this.db.prepare(`INSERT INTO provider_state(provider,status,reason,cooldown_until,updated_at) VALUES(?,'cooldown',?,?,?) + ON CONFLICT(provider) DO UPDATE SET status='cooldown', reason=excluded.reason, + cooldown_until=excluded.cooldown_until, updated_at=excluded.updated_at`) + .run(provider, reason, cooldownUntil, updatedAt); + } touchSession(linearSessionId: string, now = Date.now()): void { this.db.prepare("UPDATE sessions SET last_seen_at=? WHERE linear_session_id=?").run(now, linearSessionId); } diff --git a/daemon/src/index.ts b/daemon/src/index.ts index 7a8c65b..c8b4bb7 100644 --- a/daemon/src/index.ts +++ b/daemon/src/index.ts @@ -3,12 +3,14 @@ import { loadConfig } from "./config.js"; import { EventLog } from "./eventlog.js"; import { LinearGateway } from "./linear.js"; import { WebhookServer } from "./server.js"; -import { SessionWorker } from "./sessions.js"; +import { ProviderReadinessPoller, SessionWorker, selectSessionProfile } from "./sessions.js"; import { CleanupWorker } from "./cleanup.js"; import { ReconcileWorker } from "./reconcile.js"; +import { ArtifactStore } from "./artifacts.js"; const config = loadConfig(); -const log = new EventLog(config.dbPath); +let log: EventLog; +log = new EventLog(config.dbPath, () => selectSessionProfile(log, config)); const gateway = new LinearGateway(log, config.apps, config.linearGraphqlUrl, config.linearTokenUrl); const worker = new AckWorker(log, gateway); const cleanupWorker = config.sessionsEnabled ? new CleanupWorker(log,gateway,config.worktreesRoot,config.targetRepoPath!) : undefined; @@ -16,7 +18,20 @@ const sessionWorker = config.sessionsEnabled ? new SessionWorker(log, gateway, c const triggerWorkers = () => { worker.trigger(); sessionWorker?.trigger(); void cleanupWorker?.trigger(); }; const onStop = (id: string) => sessionWorker?.stopSession(id); const reconcileWorker = hasLinearApiCreds() ? new ReconcileWorker(log, gateway, config, { onInserted: triggerWorkers, onStop }) : undefined; -const server = new WebhookServer({ config, log, onInserted: triggerWorkers, onStop }); +const artifactStore = config.artifactToken ? new ArtifactStore(config.artifactsDir) : undefined; +const server = new WebhookServer({ config, log, onInserted: triggerWorkers, onStop, ...(artifactStore ? { artifactStore } : {}) }); +const providerPoller = config.sessionsEnabled ? new ProviderReadinessPoller(log, config) : undefined; + +if (providerPoller) { + let initialTimer: NodeJS.Timeout | undefined; + await Promise.race([providerPoller.probe(), new Promise(resolve => { + initialTimer = setTimeout(() => { + log.setProviderState("claude", "not_ready", "initial_probe_timeout"); resolve(); + }, config.providerInitialProbeTimeoutMs); + })]); + if (initialTimer) clearTimeout(initialTimer); + providerPoller.start(); +} worker.start(); sessionWorker?.start(); @@ -35,6 +50,7 @@ async function shutdown(signal: string): Promise { await worker.stop(); await sessionWorker?.stop(); await cleanupWorker?.stop(); + providerPoller?.stop(); log.close(); } diff --git a/daemon/src/reconcile.ts b/daemon/src/reconcile.ts index d5fc4c9..fd630e5 100644 --- a/daemon/src/reconcile.ts +++ b/daemon/src/reconcile.ts @@ -86,13 +86,16 @@ export class ReconcileWorker { for (const session of sessions) { if (this.log.getSession(session.id)) continue; - const inserted = this.appendCreated(session); - if (inserted) this.options.onInserted?.(); + const result = this.appendCreated(session); + if (result.assignedProfile) this.logger.log(JSON.stringify({ event: "session_profile_assigned", + linearSessionId: session.id, profile: result.assignedProfile, reason: result.assignmentReason })); + if (result.inserted) this.options.onInserted?.(); } } } private async reconcilePlannerPrompts(): Promise { + // Prompt reconciliation only visits durable sessions, so it cannot assign a profile. for (const session of this.log.plannerSessionsForReconcile()) { let activities: AgentPromptActivity[]; try { @@ -116,7 +119,7 @@ export class ReconcileWorker { } } - private appendCreated(session: AgentSessionSummary): boolean { + private appendCreated(session: AgentSessionSummary): ReturnType { const raw = { action: "created", agentSession: { @@ -134,7 +137,7 @@ export class ReconcileWorker { issueIdentifier: session.issueIdentifier, receivedAt: this.now(), rawBody: Buffer.from(JSON.stringify(raw)), - }).inserted; + }); } private appendPrompted(session: SessionRow, activity: AgentPromptActivity): ReturnType { diff --git a/daemon/src/server.ts b/daemon/src/server.ts index 3e056b9..e539af1 100644 --- a/daemon/src/server.ts +++ b/daemon/src/server.ts @@ -1,14 +1,20 @@ +import { createHash, timingSafeEqual } from "node:crypto"; +import { readFile } from "node:fs/promises"; import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http"; import type { AddressInfo } from "node:net"; +import { ArtifactNotFoundError, type ArtifactFile, ArtifactStore, InvalidArtifactError } from "./artifacts.js"; import type { AppName, Config } from "./config.js"; import type { EventLog } from "./eventlog.js"; +import { readCliproxyManagementKey } from "./sessions.js"; import { verifyWebhook } from "./verify.js"; +import { renderViewer } from "./viewer.js"; const MAX_BODY_BYTES = 1024 * 1024; export interface WebhookServerOptions { config: Config; log: EventLog; + artifactStore?: ArtifactStore; onInserted?: () => void; onStop?: (agentSessionId: string) => void; logger?: Pick; @@ -45,22 +51,30 @@ export class WebhookServer { private async handle(request: IncomingMessage, response: ServerResponse): Promise { try { - if (this.contentLengthTooLarge(request)) { - this.json(response, 413, { error: "payload_too_large" }, undefined, () => request.socket.destroy()); - return; - } if (request.method === "GET" && request.url === "/healthz") { - this.earlyJson(request, response, 200, { ok: true }); return; + if (!await this.managementAuthorized(request)) { + this.earlyJson(request, response, 200, { ok: true }); return; + } + this.earlyJson(request, response, 200, this.providerHealth()); return; } - const routeMatch = /^\/webhook\/(planner|implementer)$/.exec(request.url ?? ""); + const pathname = (request.url ?? "").split("?", 1)[0] ?? ""; + if (pathname === "/a" || pathname === "/a/" || pathname.startsWith("/a/")) { + await this.handleArtifact(request, response, pathname); + return; + } + const routeMatch = /^\/webhook\/(planner|implementer)$/.exec(pathname); if (routeMatch && request.method !== "POST") { this.earlyJson(request, response, 405, { error: "method_not_allowed" }, { Allow: "POST" }); return; } const match = request.method === "POST" ? routeMatch : null; if (!match) { this.earlyJson(request, response, 404, { error: "not_found" }); return; } + if (this.contentLengthTooLarge(request, MAX_BODY_BYTES)) { + this.json(response, 413, { error: "payload_too_large" }, undefined, () => request.socket.destroy()); + return; + } const app = match[1] as AppName; - const rawBody = await this.readBody(request, response); + const rawBody = await this.readBody(request, response, MAX_BODY_BYTES); if (!rawBody) return; const signatureHeader = Array.isArray(request.headers["linear-signature"]) ? request.headers["linear-signature"][0] : request.headers["linear-signature"]; @@ -92,6 +106,8 @@ export class WebhookServer { rawBody, }; const result = this.options.log.append(event); + if (result.assignedProfile) this.logger.log(JSON.stringify({ event: "session_profile_assigned", + linearSessionId: event.agentSessionId, profile: result.assignedProfile, reason: result.assignmentReason })); this.logger.log(JSON.stringify({ event: "webhook", deliveryId: result.deliveryId, app, action: event.action ?? null, signal: event.signal ?? null, sessionId: event.agentSessionId ?? null, issueId: event.issueId ?? null, inserted: result.inserted })); @@ -104,7 +120,139 @@ export class WebhookServer { } } - private readBody(request: IncomingMessage, response: ServerResponse): Promise { + private async handleArtifact(request: IncomingMessage, response: ServerResponse, pathname: string): Promise { + const store = this.options.artifactStore; + if (!store) { this.earlyJson(request, response, 404, { error: "not_found" }); return; } + + const createRoute = pathname === "/a" && request.method === "POST"; + const putMatch = request.method === "PUT" ? /^\/a\/([^/]+)$/.exec(pathname) : null; + if (createRoute || putMatch) { + let bundleId = putMatch ? safeDecode(putMatch[1]!) : undefined; + let fileCount: number | undefined; + if (!this.authorized(request)) { + this.logArtifactWrite(request.method!, bundleId, fileCount, "unauthorized", 401); + this.earlyJson(request, response, 401, { error: "unauthorized" }); return; + } + if (this.contentLengthTooLarge(request, this.options.config.artifactMaxBodyBytes)) { + this.logArtifactWrite(request.method!, bundleId, fileCount, "payload_too_large", 413); + this.json(response, 413, { error: "payload_too_large" }, undefined, () => request.socket.destroy()); + return; + } + const rawBody = await this.readBody(request, response, this.options.config.artifactMaxBodyBytes); + if (!rawBody) { + this.logArtifactWrite(request.method!, bundleId, fileCount, "payload_too_large", 413); + return; + } + let files: ArtifactFile[]; + try { + files = parseManifest(rawBody); + fileCount = files.length; + const id = createRoute ? await store.create(files) : decodeURIComponent(putMatch![1]!); + bundleId = id; + if (!createRoute) await store.replace(id, files); + const url = `${this.options.config.webhookBaseUrl}/a/${id}/`; + this.logArtifactWrite(request.method!, bundleId, fileCount, "success", createRoute ? 201 : 200); + this.json(response, createRoute ? 201 : 200, { url }); + } catch (error) { + if (error instanceof ArtifactNotFoundError || error instanceof URIError) { + this.logArtifactWrite(request.method!, bundleId, fileCount, "not_found", 404); + this.json(response, 404, { error: "not_found" }); + } else if (error instanceof InvalidArtifactError || error instanceof SyntaxError) { + this.logArtifactWrite(request.method!, bundleId, fileCount, "invalid_manifest", 400); + this.json(response, 400, { error: "invalid_manifest" }); + } else { + this.logArtifactWrite(request.method!, bundleId, fileCount, "internal_error", 500); + throw error; + } + } + return; + } + + if (request.method !== "GET") { + this.earlyJson(request, response, 405, { error: "method_not_allowed" }, { Allow: pathname === "/a" ? "POST" : "GET, PUT" }); + return; + } + if (pathname === "/a" || pathname === "/a/") { this.earlyJson(request, response, 404, { error: "not_found" }); return; } + + const redirectMatch = /^\/a\/([^/]+)$/.exec(pathname); + if (redirectMatch) { + const id = safeDecode(redirectMatch[1]!); + if (!id || !store.isValidId(id) || (await store.list(id)).length === 0) { + this.earlyJson(request, response, 404, { error: "not_found" }); return; + } + response.writeHead(301, { Location: `/a/${encodeURIComponent(id)}/`, "Cache-Control": "no-cache" }); + response.end(); return; + } + const viewerMatch = /^\/a\/([^/]+)\/$/.exec(pathname); + if (viewerMatch) { + const id = safeDecode(viewerMatch[1]!); + const files = id && store.isValidId(id) ? await store.list(id) : []; + if (!id || files.length === 0) { this.earlyJson(request, response, 404, { error: "not_found" }); return; } + const html = renderViewer(id, files); + response.writeHead(200, { "Content-Type": "text/html; charset=utf-8", "Content-Length": Buffer.byteLength(html), "Cache-Control": "no-cache" }); + response.end(html); return; + } + const indexMatch = /^\/a\/([^/]+)\/index\.json$/.exec(pathname); + if (indexMatch) { + const id = safeDecode(indexMatch[1]!); + const files = id && store.isValidId(id) ? await store.list(id) : []; + if (!id || files.length === 0) { this.earlyJson(request, response, 404, { error: "not_found" }); return; } + const body = JSON.stringify(files.sort()); + response.writeHead(200, { "Content-Type": "application/json; charset=utf-8", "Content-Length": Buffer.byteLength(body), "Cache-Control": "no-cache" }); + response.end(body); return; + } + const rawMatch = /^\/a\/([^/]+)\/(.+)$/.exec(pathname); + if (!rawMatch) { this.earlyJson(request, response, 404, { error: "not_found" }); return; } + const id = safeDecode(rawMatch[1]!); + const relPath = safeDecode(rawMatch[2]!); + if (!id || relPath === undefined) { this.earlyJson(request, response, 404, { error: "not_found" }); return; } + const absolute = await store.resolve(id, relPath); + if (!absolute) { this.earlyJson(request, response, 404, { error: "not_found" }); return; } + try { + const content = await readFile(absolute); + response.writeHead(200, { "Content-Type": store.contentTypeFor(relPath), "Content-Length": content.length, "Cache-Control": "no-cache" }); + response.end(content); + } catch { + this.earlyJson(request, response, 404, { error: "not_found" }); + } + } + + private authorized(request: IncomingMessage): boolean { + const expected = this.options.config.artifactToken; + const authorization = Array.isArray(request.headers.authorization) ? request.headers.authorization[0] : request.headers.authorization; + const supplied = authorization?.startsWith("Bearer ") ? authorization.slice(7) : ""; + if (!expected) return false; + const expectedHash = createHash("sha256").update(expected).digest(); + const suppliedHash = createHash("sha256").update(supplied).digest(); + return timingSafeEqual(expectedHash, suppliedHash); + } + + private async managementAuthorized(request: IncomingMessage): Promise { + const authorization = Array.isArray(request.headers.authorization) ? request.headers.authorization[0] : request.headers.authorization; + const supplied = authorization?.startsWith("Bearer ") ? authorization.slice(7) : ""; + let expected: string; + try { expected = await readCliproxyManagementKey(this.options.config.cliproxyEnvFile); } catch { expected = ""; } + const expectedHash = createHash("sha256").update(expected).digest(); + const suppliedHash = createHash("sha256").update(supplied).digest(); + return expected.length > 0 && timingSafeEqual(expectedHash, suppliedHash); + } + + private providerHealth(): Record { + const claude = this.options.log.getProviderState("claude"); + const codex = this.options.log.getProviderState("codex"); + return { ok: true, providers: { + claude: { status: claude?.status ?? "not_ready", reason: claude?.reason ?? "state_missing", + ...(claude?.cooldownUntil != null ? { cooldownUntil: claude.cooldownUntil } : {}), updatedAt: claude?.updatedAt ?? 0 }, + codex: { status: codex?.status ?? "ready", ...(codex?.cooldownUntil != null ? { cooldownUntil: codex.cooldownUntil } : {}) }, + } }; + } + + private logArtifactWrite(method: string, bundleId: string | undefined, fileCount: number | undefined, outcome: string, status: number): void { + this.logger.log(JSON.stringify({ event: "artifact_write", method, bundleId: bundleId ?? null, + fileCount: fileCount ?? null, outcome, status })); + } + + private readBody(request: IncomingMessage, response: ServerResponse, maxBytes: number): Promise { return new Promise((resolve, reject) => { const chunks: Buffer[] = []; let size = 0; @@ -112,7 +260,7 @@ export class WebhookServer { request.on("data", (chunk: Buffer) => { if (tooLarge) return; size += chunk.length; - if (size > MAX_BODY_BYTES) { + if (size > maxBytes) { tooLarge = true; this.json(response, 413, { error: "payload_too_large" }, undefined, () => request.socket.destroy()); resolve(undefined); @@ -123,12 +271,12 @@ export class WebhookServer { }); } - private contentLengthTooLarge(request: IncomingMessage): boolean { + private contentLengthTooLarge(request: IncomingMessage, maxBytes: number): boolean { const header = request.headers["content-length"]; const value = Array.isArray(header) ? header[0] : header; if (!value) return false; const length = Number(value); - return Number.isFinite(length) && length > MAX_BODY_BYTES; + return Number.isFinite(length) && length > maxBytes; } private hasRequestBody(request: IncomingMessage): boolean { @@ -149,3 +297,28 @@ export class WebhookServer { response.end(encoded, done); } } + +function safeDecode(value: string): string | undefined { + try { return decodeURIComponent(value); } catch { return undefined; } +} + +function parseManifest(rawBody: Buffer): ArtifactFile[] { + const manifest = JSON.parse(rawBody.toString("utf8")) as unknown; + if (!manifest || typeof manifest !== "object" || Array.isArray(manifest)) throw new InvalidArtifactError("manifest must be an object"); + const files = (manifest as { files?: unknown }).files; + if (!Array.isArray(files)) throw new InvalidArtifactError("manifest files must be an array"); + return files.map(file => { + if (!file || typeof file !== "object" || Array.isArray(file)) throw new InvalidArtifactError("invalid file entry"); + const { path, contentBase64 } = file as { path?: unknown; contentBase64?: unknown }; + if (typeof path !== "string" || typeof contentBase64 !== "string" || !validBase64(contentBase64)) { + throw new InvalidArtifactError("invalid file entry"); + } + return { path, content: Buffer.from(contentBase64, "base64") }; + }); +} + +function validBase64(value: string): boolean { + if (value === "") return true; + if (!/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(value)) return false; + return Buffer.from(value, "base64").toString("base64") === value; +} diff --git a/daemon/src/sessions.ts b/daemon/src/sessions.ts index bcbc109..df25dc6 100644 --- a/daemon/src/sessions.ts +++ b/daemon/src/sessions.ts @@ -1,8 +1,8 @@ import { randomUUID } from "node:crypto"; -import { mkdir, open, readdir, realpath, unlink } from "node:fs/promises"; +import { mkdir, open, readFile, readdir, realpath, unlink } from "node:fs/promises"; import { resolve, sep } from "node:path"; import type { Config } from "./config.js"; -import { runTurn, type ClaudeEvent } from "./claude.js"; +import { runTurn, type ClaudeEvent, type RunTurnResult } from "./claude.js"; import type { EventLog, ExternalUrlRow, StopAckRow, TurnActivityRow, TurnRow } from "./eventlog.js"; import type { LinearGateway, PostResult, ProgressContent } from "./linear.js"; import { WorktreeManager } from "./worktrees.js"; @@ -14,6 +14,80 @@ export interface SessionWorkerOptions { onTurnComplete?: () => void; } +const PROVIDER_COOLDOWN_MS = 10 * 60_000; + +export function classifyProviderFailure(result: RunTurnResult): { state: string; reason: string } | undefined { + const spawn = result.spawnError ?? ""; + if (/ECONNREFUSED/i.test(spawn)) return { state: "transport_failure", reason: "spawn_econnrefused" }; + if (/ENOTFOUND/i.test(spawn)) return { state: "transport_failure", reason: "spawn_enotfound" }; + const stderr = result.stderrTail ?? ""; + if (/connection\s+refused/i.test(stderr)) return { state: "transport_failure", reason: "connection_refused" }; + const status = /(?:HTTP(?:\/\d(?:\.\d)?)?\s*|status(?:\s+code)?[=: ]+)(401|403|5\d\d)\b[^\n]*(?:base\s*URL|proxy|anthropic)/i.exec(stderr)?.[1]; + if (status) return { state: status.startsWith("5") ? "transport_failure" : "auth_failure", reason: `http_${status}` }; + return undefined; +} + +export function selectSessionProfile(log: EventLog, config: Config, now = Date.now()): { profile: "fable" | "sol"; reason: string } { + if (!config.fableArgv) return { profile: "sol", reason: "fable_not_configured" }; + const state = log.getProviderState("claude"); + if (!state) return { profile: "sol", reason: "claude_state_missing" }; + if (state.cooldownUntil !== null && state.cooldownUntil > now) return { profile: "sol", reason: "claude_cooldown" }; + if (now - state.updatedAt > config.providerStateStaleMs) return { profile: "sol", reason: "claude_state_stale" }; + if (state.status !== "ready") return { profile: "sol", reason: state.reason ?? "claude_not_ready" }; + return { profile: "fable", reason: "claude_ready" }; +} + +export async function readCliproxyManagementKey(path: string): Promise { + const source = await readFile(path, "utf8").catch(() => { throw new Error("probe_env_unreadable"); }); + const key = /^\s*(?:export\s+)?CLIPROXY_MANAGEMENT_KEY=(?:['"]?)([^'"\s#]+)(?:['"]?)\s*(?:#.*)?$/m.exec(source)?.[1]; + if (!key) throw new Error("probe_management_key_missing"); + return key; +} + +export class ProviderReadinessPoller { + private timer?: NodeJS.Timeout; + private probing: Promise | undefined; + constructor(private readonly log: EventLog, private readonly config: Config, + private readonly logger: Logger = console, private readonly probeOverride?: () => Promise) {} + probe(now = Date.now()): Promise { + this.probing ??= this.performProbe(now).finally(() => { this.probing = undefined; }); + return this.probing; + } + private async performProbe(now: number): Promise { + const before = this.log.getProviderState("claude"); + let status = "not_ready"; let reason = "probe_error"; + try { + const payload = this.probeOverride ? await this.probeOverride() : await this.fetchAuthFiles(); + const rows = Array.isArray(payload) ? payload : Array.isArray(object(payload)?.files) ? object(payload)!.files as unknown[] : []; + const claude = rows.map(object).filter((row): row is Record => !!row && row.provider === "claude"); + const eligible = claude.filter(row => row.disabled === false); + const failed = claude.filter(row => row.failed === true).length; + status = eligible.length > 0 ? "ready" : "not_ready"; + reason = eligible.length > 0 ? `eligible_${eligible.length}_failed_${failed}` : `no_eligible_claude_failed_${failed}`; + } catch (error) { + reason = error instanceof Error && error.message.startsWith("probe_") ? error.message : "probe_error"; + } + const activeCooldown = before?.cooldownUntil != null && before.cooldownUntil > now; + if (activeCooldown) { status = "cooldown"; reason = before.reason ?? "provider_cooldown"; } + this.log.setProviderState("claude", status, reason, now, activeCooldown ? before.cooldownUntil : null); + const after = this.log.getProviderState("claude")!; + if (!before || before.status !== after.status || before.reason !== after.reason || before.cooldownUntil !== after.cooldownUntil) + this.logger.log(jsonLog({ event: "provider_state_changed", provider: "claude", status: after.status, + reason: after.reason, cooldownUntil: after.cooldownUntil })); + } + private async fetchAuthFiles(): Promise { + const key = await readCliproxyManagementKey(this.config.cliproxyEnvFile); + const signal = AbortSignal.timeout(this.config.providerInitialProbeTimeoutMs); + const response = await fetch(`${this.config.cliproxyUrl}/v0/management/auth-files`, { + headers: { Authorization: `Bearer ${key}` }, signal, + }).catch(error => { if (signal.aborted) throw new Error("probe_timeout"); throw error; }); + if (!response.ok) { await response.body?.cancel(); throw new Error(`probe_http_${response.status}`); } + return response.json(); + } + start(): void { this.timer = setInterval(() => void this.probe(), this.config.providerProbeIntervalMs); this.timer.unref(); } + stop(): void { if (this.timer) clearInterval(this.timer); } +} + const DISPATCH_OWNER_PATTERN = /^[0-9a-fA-F][0-9a-fA-F-]{7,63}$/; function object(value: unknown): Record | undefined { @@ -74,6 +148,7 @@ export class SessionWorker { private readonly now: () => number; private readonly logger: Logger; private readonly worktrees: WorktreeManager; + private readonly legacyProfileLogged = new Set(); constructor(private readonly log: EventLog, private readonly gateway: LinearGateway, private readonly config: Config, private readonly options: SessionWorkerOptions = {}) { @@ -120,7 +195,12 @@ export class SessionWorker { const worktree = await this.worktrees.ensureWorktree(identifier); this.log.updateSessionWorktree(turn.linearSessionId, worktree.path, worktree.branch, this.now()); const implementer = session.mode === "implementer"; - const resuming = turn.kind === "prompted" && !!session.claudeSessionId; + const runtimeDegraded = session.runtime === "claudex" && !this.config.claudexArgv; + const runtime = runtimeDegraded ? "claude" : session.runtime; + if (runtimeDegraded) this.logger.error(jsonLog({ event: "session_runtime_degraded", turnId: turn.id, + linearSessionId: turn.linearSessionId, configuredRuntime: "claudex", effectiveRuntime: "claude", + reason: "CLAUDEX_BIN is not configured" })); + const resuming = turn.kind === "prompted" && !!session.claudeSessionId && !runtimeDegraded; let prompt = implementer && !resuming ? `/do ${identifier}` : this.composePrompt(turn, identifier); if ((!implementer || resuming) && this.config.attachmentsEnabled) prompt += await this.downloadAttachments(turn.rawBody, worktree.path); this.log.setTurnPrompt(turn.id, prompt); @@ -137,22 +217,98 @@ export class SessionWorker { keepalive.unref(); const mcpConfigJson = JSON.stringify({ mcpServers: { linear: { type: "http", url: "https://mcp.linear.app/mcp", headers: { Authorization: `Bearer ${this.config.linearApiKey}` } } } }); - const result = await runTurn({ cwd: worktree.path, prompt, - ...(resuming ? { resumeSessionId: session.claudeSessionId! } : {}), - argv: this.config.claudeArgv, permissionMode: implementer ? this.config.doPermissionMode : this.config.claudePermissionMode, - maxTurns: implementer ? this.config.doMaxTurns : this.config.claudeMaxTurns, - ...(implementer && this.config.doMaxBudgetUsd !== undefined ? { maxBudgetUsd: this.config.doMaxBudgetUsd } : {}), - mcpConfigJson, env: { LINEAR_API_KEY: this.config.linearApiKey!, GH_TOKEN: process.env.GH_TOKEN, - GITHUB_TOKEN: process.env.GITHUB_TOKEN, - ...(DISPATCH_OWNER_PATTERN.test(turn.linearSessionId) ? { ORCHESTRA_DISPATCH_OWNER: turn.linearSessionId } : {}) }, signal, - onSessionId: id => this.log.updateClaudeSessionId(turn.linearSessionId, id, this.now()), - onEvent: event => progress.push(event.type === "text" - ? { type: "thought", body: event.text } : toolUseContent(event)), - }).catch(async error => { - clearInterval(keepalive); - await progress.cancelAndWait(); - throw error; - }); + const durableProfile = session.profile ?? "sol"; + if (session.profile === null && !this.legacyProfileLogged.has(turn.linearSessionId)) { + this.legacyProfileLogged.add(turn.linearSessionId); + this.logger.log(jsonLog({ event: "legacy_session_profile_defaulted", linearSessionId: turn.linearSessionId, profile: "sol" })); + } + const common = { + cwd: worktree.path, permissionMode: implementer ? this.config.doPermissionMode : this.config.claudePermissionMode, + maxTurns: implementer ? this.config.doMaxTurns : this.config.claudeMaxTurns, + ...(implementer && this.config.doMaxBudgetUsd !== undefined ? { maxBudgetUsd: this.config.doMaxBudgetUsd } : {}), + mcpConfigJson, env: { LINEAR_API_KEY: this.config.linearApiKey!, GH_TOKEN: process.env.GH_TOKEN, + GITHUB_TOKEN: process.env.GITHUB_TOKEN, + ...(DISPATCH_OWNER_PATTERN.test(turn.linearSessionId) ? { ORCHESTRA_DISPATCH_OWNER: turn.linearSessionId } : {}) }, signal, + onEvent: (event: ClaudeEvent) => progress.push(event.type === "text" + ? { type: "thought", body: event.text } : toolUseContent(event)), + }; + // CLAUDE_BIN may name the host's Sol launcher; CLAUDEX_BIN is the distinct #101 capacity-fallback target. + const runtimeArgv = runtime === "claudex" ? this.config.claudexArgv! + : durableProfile === "fable" ? this.config.fableArgv : this.config.claudeArgv; + const cleanupRejectedRun = async (error: unknown): Promise => { + clearInterval(keepalive); + await progress.cancelAndWait(); + throw error; + }; + const run = (argv: string[], runPrompt = prompt, resume = resuming, + trustedEnv = runtime === "claudex" ? this.config.claudexEnv : undefined) => runTurn({ ...common, prompt: runPrompt, + ...(resume ? { resumeSessionId: session.claudeSessionId! } : {}), argv, + ...(trustedEnv ? { trustedEnv } : {}), + onSessionId: id => { if (!runtimeDegraded) this.log.updateClaudeSessionId(turn.linearSessionId, id, this.now()); }, + }).catch(cleanupRejectedRun); + let result: RunTurnResult; + if (!runtimeArgv) { + this.logger.error(jsonLog({ event: "profile_launcher_unconfigured", linearSessionId: turn.linearSessionId, profile: "fable" })); + result = { ok: false, isError: true, exitCode: null, signal: null, + spawnError: "Fable profile launcher is not configured; set FABLE_BIN after validating fable-models.env", + permissionDenials: [], sawResult: false, capacityEvidence: [] }; + } else result = await run(runtimeArgv); + let fallbackCause: string | undefined; + let fallbackAttempted = false; + const applyCapacityFallback = async (candidate: RunTurnResult, + originalRuntime = durableProfile === "fable" ? "fable" : "claude"): Promise => { + if (candidate.ok || !candidate.capacityEvidence.length || runtime !== "claude") return candidate; + fallbackCause = candidate.capacityEvidence.join(", "); + // Structured capacity evidence is narrower than provider unavailability and always wins classification. + if (!this.config.claudexArgv || this.stopRequested.has(turn.id)) return candidate; + fallbackAttempted = true; + this.log.clearClaudeSessionId(turn.linearSessionId, this.now()); + progress.push({ type: "thought", body: `${originalRuntime === "fable" ? "Fable" : "Claude"} capacity limit detected (${fallbackCause}); retrying once with Claudex.` }); + this.logger.log(jsonLog({ event: "session_capacity_fallback", turnId: turn.id, issueIdentifier: identifier, + linearSessionId: turn.linearSessionId, originalRuntime, fallbackRuntime: "claudex", evidence: candidate.capacityEvidence })); + const fallbackContext = `Runtime fallback context: original runtime ${originalRuntime}; fallback runtime claudex; classified cause ${fallbackCause}; effective review lanes are single/Codex-only regardless of any dual request.`; + const retryPrompt = implementer + ? `/do ${identifier}\n\nResume where we left off.\n\n${fallbackContext}` + : `${prompt}\n\nResume where we left off.\n\n${fallbackContext}`; + let fallbackSessionId: string | undefined; + const fallbackResult = await runTurn({ ...common, prompt: retryPrompt, argv: this.config.claudexArgv, + ...(this.config.claudexEnv ? { trustedEnv: this.config.claudexEnv } : {}), + onSessionId: id => { fallbackSessionId = id; }, + }).catch(cleanupRejectedRun); + if (fallbackResult.ok && !this.stopRequested.has(turn.id)) + this.log.recordRuntimeFallback(turn.linearSessionId, fallbackSessionId, fallbackCause, this.now()); + return fallbackResult; + }; + result = await applyCapacityFallback(result); + const recordProviderFailure = (profile: "fable" | "sol", provider: "claude" | "codex", + classified: { state: string; reason: string }): number => { + const cooldownUntil = this.now() + PROVIDER_COOLDOWN_MS; + this.log.setProviderCooldown(provider, cooldownUntil, classified.reason, this.now()); + this.logger.log(jsonLog({ event: "provider_state_changed", provider, status: "cooldown", + reason: classified.reason, cooldownUntil })); + this.logger.error(jsonLog({ event: "provider_failure_classified", linearSessionId: turn.linearSessionId, + profile, provider, classifiedState: classified.state, reason: classified.reason, cooldownUntil })); + return cooldownUntil; + }; + if (!result.ok && !fallbackCause) { + const classified = classifyProviderFailure(result); + if (classified) { + const provider = runtime === "claudex" ? "codex" : durableProfile === "fable" ? "claude" : "codex"; + const cooldownUntil = recordProviderFailure(durableProfile, provider, classified); + const fresh = this.log.getSession(turn.linearSessionId); + if (runtime === "claude" && durableProfile === "fable" && !fresh?.claudeSessionId + && this.log.applyPreEstablishmentFallback(turn.linearSessionId)) { + this.logger.log(jsonLog({ event: "profile_fallback", linearSessionId: turn.linearSessionId, from: "fable", to: "sol", + provider, classifiedState: classified.state, reason: classified.reason, cooldownUntil })); + result = await run(this.config.claudeArgv, prompt, false, undefined); + result = await applyCapacityFallback(result, "claude"); + if (!result.ok && !fallbackCause) { + const solClassified = classifyProviderFailure(result); + if (solClassified) recordProviderFailure("sol", "codex", solClassified); + } + } + } + } clearInterval(keepalive); const finishedAt = this.now(); if (this.stopRequested.has(turn.id)) { @@ -174,8 +330,12 @@ export class SessionWorker { linearSessionId: turn.linearSessionId, attempts: turn.attempts, durationMs: Math.max(0, finishedAt - (turn.startedAt ?? turn.receivedAt)) })); } else { - const detail = result.spawnError ?? (result.permissionDenials.length ? "Claude permission was denied" : - result.signal ? `Claude exited on ${result.signal}` : !result.sawResult ? "Claude exited without a result" : `Claude exited with code ${result.exitCode}`); + const failedRuntime = fallbackAttempted ? "Claudex" : runtime === "claudex" ? "Claudex" : "Claude"; + const runtimeDetail = result.spawnError ?? (result.permissionDenials.length ? `${failedRuntime} permission was denied` : + result.signal ? `${failedRuntime} exited on ${result.signal}` : !result.sawResult ? `${failedRuntime} exited without a result` : `${failedRuntime} exited with code ${result.exitCode}`); + const detail = fallbackCause + ? `Claude hit a usage limit (${fallbackCause})${fallbackAttempted ? `; Claudex fallback failed: ${runtimeDetail}` : ""}` + : runtimeDetail; this.log.finishTurn(turn.id, "error", `${implementer ? "Implementer" : "Planner"} turn failed: ${detail}`, finishedAt, randomUUID(), true); this.logger.error(jsonLog({ event: "session_turn_failed", turnId: turn.id, issueIdentifier: identifier, linearSessionId: turn.linearSessionId, attempts: turn.attempts, durationMs: Math.max(0, finishedAt - (turn.startedAt ?? turn.receivedAt)), @@ -212,6 +372,7 @@ export class SessionWorker { private async scanDispatchMarkers(): Promise { if (this.stopped) return; try { + // Dispatch scanning only visits sessions with durable worktrees, so it cannot assign a profile. for (const session of this.log.sessionsWithWorktrees()) { if (!session.worktreePath || this.log.hasOpenTurn(session.linearSessionId)) continue; if (!DISPATCH_OWNER_PATTERN.test(session.linearSessionId)) { diff --git a/daemon/src/viewer.ts b/daemon/src/viewer.ts new file mode 100644 index 0000000..6fbaffd --- /dev/null +++ b/daemon/src/viewer.ts @@ -0,0 +1,219 @@ +import { extname } from "node:path"; + +const IMAGE_EXTENSIONS = new Set([".png", ".svg", ".jpg", ".jpeg", ".gif", ".webp"]); + +export function renderViewer(id: string, files: readonly string[]): string { + const ordered = [...files].sort((left, right) => rank(left) - rank(right) || left.localeCompare(right)); + const fileJson = JSON.stringify(ordered).replace(/<\//g, "<\\/"); + const root = `/a/${encodeURIComponent(id)}/`; + return String.raw` + + + + +Artifact bundle + + + + + + + + +`; +} + +function rank(path: string): number { + if (path === "refs/explainer.html") return 0; + const extension = extname(path).toLowerCase(); + if (extension === ".html") return 10; + if (path === "plan.md") return 20; + if (path === "wrapup.md") return 21; + if (path === "item.md") return 22; + if (extension === ".md") return 30; + if (IMAGE_EXTENSIONS.has(extension)) return 40; + return 50; +} diff --git a/daemon/test/artifacts.test.ts b/daemon/test/artifacts.test.ts new file mode 100644 index 0000000..f72823b --- /dev/null +++ b/daemon/test/artifacts.test.ts @@ -0,0 +1,310 @@ +import { spawn, type ChildProcess } from "node:child_process"; +import { once } from "node:events"; +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { request as httpRequest } from "node:http"; +import { createConnection, createServer as createNetServer } from "node:net"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { ArtifactStore } from "../src/artifacts.js"; +import { loadConfig } from "../src/config.js"; +import { EventLog } from "../src/eventlog.js"; +import { WebhookServer } from "../src/server.js"; +import { renderViewer } from "../src/viewer.js"; + +const dirs: string[] = []; +afterEach(() => { for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); }); + +function setup(maxBodyBytes = 32 * 1024 * 1024) { + const dir = mkdtempSync(join(tmpdir(), "artifact-server-")); dirs.push(dir); + const artifactsDir = join(dir, "artifacts"); + const config = loadConfig({ + DAEMON_TEST_MODE: "1", SESSIONS_ENABLED: "0", DB_PATH: join(dir, "events.db"), + PLANNER_WEBHOOK_SECRET: "planner-secret", PLANNER_LINEAR_TOKEN: "p", + IMPLEMENTER_WEBHOOK_SECRET: "implementer-secret", IMPLEMENTER_LINEAR_TOKEN: "i", + ARTIFACT_TOKEN: "artifact-secret", ARTIFACTS_DIR: artifactsDir, + ARTIFACT_MAX_BODY_BYTES: String(maxBodyBytes), WEBHOOK_BASE_URL: "https://artifacts.example.test", + }); + config.port = 0; + const log = new EventLog(config.dbPath); + const store = new ArtifactStore(artifactsDir); + const logger = { log: vi.fn(), error: vi.fn() }; + const server = new WebhookServer({ config, log, artifactStore: store, logger }); + return { dir, artifactsDir, config, log, store, server, logger }; +} + +function manifest(files: Array<{ path: string; content: string | Buffer }>): string { + return JSON.stringify({ files: files.map(file => ({ path: file.path, + contentBase64: (typeof file.content === "string" ? Buffer.from(file.content) : file.content).toString("base64") })) }); +} + +function auth(token = "artifact-secret"): Record { + return { Authorization: `Bearer ${token}`, "Content-Type": "application/json" }; +} + +async function freePort(): Promise { + const server = createNetServer(); + await new Promise((resolveListen, reject) => { + server.once("error", reject); + server.listen(0, "127.0.0.1", resolveListen); + }); + const port = (server.address() as { port: number }).port; + await new Promise(resolveClose => server.close(() => resolveClose())); + return port; +} + +async function waitForHealth(port: number, child: ChildProcess): Promise { + for (let attempt = 0; attempt < 100; attempt++) { + if (child.exitCode !== null) throw new Error(`child exited before health check: ${child.exitCode}`); + try { if ((await fetch(`http://127.0.0.1:${port}/healthz`)).ok) return; } catch { /* starting */ } + await new Promise(resolveWait => setTimeout(resolveWait, 25)); + } + throw new Error("child daemon did not become healthy"); +} + +async function expectJsonError(response: Response, status: number, error: string): Promise { + expect(response.status).toBe(status); + await expect(response.json()).resolves.toEqual({ error }); +} + +async function rawGet(port: number, path: string): Promise<{ status: number; body: string }> { + return new Promise((resolve, reject) => { + const request = httpRequest({ host: "127.0.0.1", port, path, method: "GET" }, response => { + const chunks: Buffer[] = []; + response.on("data", chunk => chunks.push(chunk)); + response.on("end", () => resolve({ status: response.statusCode ?? 0, body: Buffer.concat(chunks).toString() })); + }); + request.on("error", reject); request.end(); + }); +} + +async function oversizedWebhook(port: number): Promise { + return new Promise((resolve, reject) => { + const socket = createConnection({ host: "127.0.0.1", port }); let raw = ""; + socket.on("connect", () => socket.write([ + "POST /webhook/planner HTTP/1.1", "Host: 127.0.0.1", "Content-Length: 1048577", + "Linear-Signature: oversized", "", "", + ].join("\r\n"))); + socket.on("data", chunk => { raw += chunk.toString(); }); + socket.on("end", () => resolve(raw)); socket.on("close", () => resolve(raw)); socket.on("error", reject); + }); +} + +function expectExecutableScriptsToParse(html: string): void { + const scripts = [...html.matchAll(/]*)>([\s\S]*?)<\/script>/gi)] + .filter(match => !/\btype=["']application\/json["']/i.test(match[1] ?? "")) + .map(match => match[2] ?? ""); + expect(scripts.length).toBeGreaterThan(0); + for (const source of scripts) expect(() => new Function(source)).not.toThrow(); +} + +describe("artifact store", () => { + it("generates distinct 128-bit base64url ids and rejects traversal", async () => { + const { store } = setup(); + const first = store.createId(); const second = store.createId(); + expect(first).toMatch(/^[A-Za-z0-9_-]{22}$/); expect(second).not.toBe(first); + const id = await store.create([{ path: "item.md", content: Buffer.from("safe") }]); + await expect(store.resolve(id, "../secret.txt")).resolves.toBeUndefined(); + await expect(store.resolve(id, "refs\\secret.txt")).resolves.toBeUndefined(); + }); +}); + +describe("artifact viewer", () => { + it("emits syntactically valid executable script blocks", () => { + const html = renderViewer("AAAAAAAAAAAAAAAAAAAAAA", ["refs/explainer.html", "plan.md", "refs/image.png"]); + expectExecutableScriptsToParse(html); + }); +}); + +describe("artifact HTTP integration", () => { + it("AC1: lists a known bundle without exposing a bundle enumeration route", async () => { + const { log, server } = setup(); const address = await server.listen(); + const created = await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth(), + body: manifest([{ path: "refs/z.md", content: "z" }, { path: "item.md", content: "item" }, + { path: "refs/a.md", content: "a" }]) }); + const id = /\/a\/([^/]+)\/$/.exec(((await created.json()) as { url: string }).url)![1]!; + const index = await fetch(`http://127.0.0.1:${address.port}/a/${id}/index.json`); + expect(index.status).toBe(200); + expect(index.headers.get("content-type")).toBe("application/json; charset=utf-8"); + await expect(index.json()).resolves.toEqual(["item.md", "refs/a.md", "refs/z.md"]); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/AAAAAAAAAAAAAAAAAAAAAA/index.json`), 404, "not_found"); + const malformed = await rawGet(address.port, "/a/%ZZ/index.json"); + expect(malformed.status).toBe(404); expect(JSON.parse(malformed.body)).toEqual({ error: "not_found" }); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a`), 404, "not_found"); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/`), 404, "not_found"); + await server.close(); log.close(); + }); + + it("AC2: index refetch lists only the replacement version", async () => { + const { log, server } = setup(); const address = await server.listen(); + const created = await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth(), + body: manifest([{ path: "item.md", content: "old" }, { path: "refs/old.md", content: "old" }]) }); + const id = /\/a\/([^/]+)\/$/.exec(((await created.json()) as { url: string }).url)![1]!; + const replaced = await fetch(`http://127.0.0.1:${address.port}/a/${id}`, { method: "PUT", headers: auth(), + body: manifest([{ path: "item.md", content: "new" }, { path: "plan.md", content: "plan" }]) }); + expect(replaced.status).toBe(200); + const index = await fetch(`http://127.0.0.1:${address.port}/a/${id}/index.json`); + expect(index.headers.get("cache-control")).toBe("no-cache"); + await expect(index.json()).resolves.toEqual(["item.md", "plan.md"]); + await server.close(); log.close(); + }); + + it("AC3: spawned daemon pull contract reconstructs every file byte-identically", async () => { + const dir = mkdtempSync(join(tmpdir(), "artifact-child-")); dirs.push(dir); + const port = await freePort(); + const files = [ + { path: "item.md", content: Buffer.from("# Item\n") }, + { path: "refs/discussion.md", content: Buffer.from("decision\n") }, + { path: "refs/blob.bin", content: Buffer.from([0x00, 0xff, 0x80, 0x41, 0x0a]) }, + ]; + const env = { ...process.env, PORT: String(port), BIND_ADDR: "127.0.0.1", DB_PATH: join(dir, "events.db"), + DAEMON_TEST_MODE: "1", SESSIONS_ENABLED: "0", WEBHOOK_BASE_URL: `http://127.0.0.1:${port}`, + PLANNER_WEBHOOK_SECRET: "planner-secret", PLANNER_LINEAR_TOKEN: "p", + IMPLEMENTER_WEBHOOK_SECRET: "implementer-secret", IMPLEMENTER_LINEAR_TOKEN: "i", + ARTIFACT_TOKEN: "test-token" }; + const child = spawn(process.execPath, [resolve("dist/index.js")], { env, stdio: "ignore" }); + try { + await waitForHealth(port, child); + const created = await fetch(`http://127.0.0.1:${port}/a`, { method: "POST", headers: auth("test-token"), + body: manifest(files) }); + expect(created.status).toBe(201); + const bundleUrl = ((await created.json()) as { url: string }).url; + const index = await fetch(`${bundleUrl}index.json`); + expect(index.status).toBe(200); + const paths = await index.json() as string[]; + expect(paths).toEqual(files.map(file => file.path).sort()); + for (const path of paths) { + const response = await fetch(`${bundleUrl}${path}`); + expect(response.status).toBe(200); + expect(Buffer.from(await response.arrayBuffer())).toEqual(files.find(file => file.path === path)!.content); + } + } finally { + if (child.exitCode === null) { child.kill("SIGTERM"); await once(child, "exit"); } + } + }, 15_000); + + it("AC1/AC3/AC4: creates a multi-file bundle with a server id and serves the viewer", async () => { + const { artifactsDir, log, server, logger } = setup(); const address = await server.listen(); + const body = manifest([ + { path: "item.md", content: "# Item" }, + { path: "refs/explainer.html", content: "

Explainer

" }, + { path: "refs/pixel.png", content: Buffer.from([0x89, 0x50, 0x4e, 0x47]) }, + ]); + const response = await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth(), body }); + expect(response.status).toBe(201); + const { url } = await response.json() as { url: string }; + const id = /\/a\/([^/]+)\/$/.exec(url)?.[1]; + expect(id).toMatch(/^[A-Za-z0-9_-]{22}$/); + expect(readFileSync(join(artifactsDir, id!, "current"), "utf8")).toMatch(/^v-/); + expect((await fetch(`http://127.0.0.1:${address.port}/a/${id}/item.md`)).status).toBe(200); + const viewer = await fetch(`http://127.0.0.1:${address.port}/a/${id}/`); + const html = await viewer.text(); + expect(viewer.status).toBe(200); expect(html).toContain("refs/explainer.html"); + expectExecutableScriptsToParse(html); + expect(html).toContain('setAttribute("sandbox", "allow-scripts allow-popups")'); + expect(html.indexOf("refs/explainer.html")).toBeLessThan(html.indexOf("item.md")); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a`), 404, "not_found"); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/`), 404, "not_found"); + const redirect = await fetch(`http://127.0.0.1:${address.port}/a/${id}`, { redirect: "manual" }); + expect(redirect.status).toBe(301); expect(redirect.headers.get("location")).toBe(`/a/${id}/`); + expect(logger.log).toHaveBeenCalledWith(JSON.stringify({ event: "artifact_write", method: "POST", + bundleId: id, fileCount: 3, outcome: "success", status: 201 })); + await server.close(); log.close(); + }); + + it("AC2/AC3: rejects unauthorized writes and client-minted ids without creating storage", async () => { + const { artifactsDir, log, server, logger } = setup(); const address = await server.listen(); + const body = manifest([{ path: "item.md", content: "private" }]); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", body }), 401, "unauthorized"); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth("wrong"), body }), 401, "unauthorized"); + expect(existsSync(artifactsDir)).toBe(false); + const chosen = "AAAAAAAAAAAAAAAAAAAAAA"; + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/${chosen}`, { method: "PUT", headers: auth(), body }), 404, "not_found"); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/${chosen}`, { method: "PUT", headers: auth("wrong"), body }), 401, "unauthorized"); + expect(existsSync(artifactsDir)).toBe(false); + const records = logger.log.mock.calls.map(([line]) => JSON.parse(line as string) as Record); + expect(records).toContainEqual({ event: "artifact_write", method: "POST", bundleId: null, + fileCount: null, outcome: "unauthorized", status: 401 }); + expect(records).toContainEqual({ event: "artifact_write", method: "PUT", bundleId: chosen, + fileCount: 1, outcome: "not_found", status: 404 }); + expect(JSON.stringify(records)).not.toContain("artifact-secret"); + await server.close(); log.close(); + }); + + it("rejects unauthorized replacement and malformed paths without changing the live bundle", async () => { + const { log, server } = setup(); const address = await server.listen(); + const created = await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth(), + body: manifest([{ path: "item.md", content: "original" }]) }); + const id = /\/a\/([^/]+)\/$/.exec(((await created.json()) as { url: string }).url)![1]!; + const replacement = manifest([{ path: "item.md", content: "changed" }]); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/${id}`, { method: "PUT", body: replacement }), 401, "unauthorized"); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/${id}`, { method: "PUT", headers: auth("wrong"), body: replacement }), 401, "unauthorized"); + const malformed = manifest([{ path: "../secret.txt", content: "bad" }]); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/${id}`, { method: "PUT", headers: auth(), body: malformed }), 400, "invalid_manifest"); + await expectJsonError(await fetch(`http://127.0.0.1:${address.port}/a/${id}`, { method: "PUT", headers: auth(), body: "{" }), 400, "invalid_manifest"); + expect(await (await fetch(`http://127.0.0.1:${address.port}/a/${id}/item.md`)).text()).toBe("original"); + await server.close(); log.close(); + }); + + it("AC6: serves raw files with correct content types and no-cache", async () => { + const { log, server } = setup(); const address = await server.listen(); + const body = manifest([ + { path: "refs/page.html", content: "

x

" }, { path: "item.md", content: "# x" }, + { path: "refs/image.png", content: "png" }, { path: "refs/image.svg", content: "" }, + ]); + const created = await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth(), body }); + const id = /\/a\/([^/]+)\/$/.exec(((await created.json()) as { url: string }).url)![1]!; + const expected = { "refs/page.html": "text/html; charset=utf-8", "item.md": "text/markdown; charset=utf-8", + "refs/image.png": "image/png", "refs/image.svg": "image/svg+xml" }; + for (const [path, type] of Object.entries(expected)) { + const response = await fetch(`http://127.0.0.1:${address.port}/a/${id}/${path}`); + expect(response.status).toBe(200); expect(response.headers.get("content-type")).toBe(type); + expect(response.headers.get("cache-control")).toBe("no-cache"); + } + await server.close(); log.close(); + }); + + it("AC7: atomically replaces contents at the stable URL", async () => { + const { log, server } = setup(); const address = await server.listen(); + const oldContent = "old-".repeat(20_000); const newContent = "new-".repeat(20_000); + const created = await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth(), + body: manifest([{ path: "item.md", content: oldContent }]) }); + const url = ((await created.json()) as { url: string }).url; + const id = /\/a\/([^/]+)\/$/.exec(url)![1]!; + const reads: Array> = []; + const replacing = fetch(`http://127.0.0.1:${address.port}/a/${id}`, { method: "PUT", headers: auth(), + body: manifest([{ path: "item.md", content: newContent }]) }); + for (let index = 0; index < 20; index++) reads.push(fetch(`http://127.0.0.1:${address.port}/a/${id}/item.md`).then(response => { + expect(response.status).toBe(200); return response.text(); + })); + const replaced = await replacing; expect(replaced.status).toBe(200); + expect(((await replaced.json()) as { url: string }).url).toBe(url); + for (const content of await Promise.all(reads)) expect([oldContent, newContent]).toContain(content); + const latest = await fetch(`http://127.0.0.1:${address.port}/a/${id}/item.md`); + expect(await latest.text()).toBe(newContent); expect(latest.headers.get("cache-control")).toBe("no-cache"); + await server.close(); log.close(); + }); + + it("AC8: returns 404 for literal and encoded traversal without exposing a sibling secret", async () => { + const { dir, log, server } = setup(); writeFileSync(join(dir, "secret.txt"), "outside-secret"); + const address = await server.listen(); + const created = await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth(), + body: manifest([{ path: "item.md", content: "safe" }]) }); + const id = /\/a\/([^/]+)\/$/.exec(((await created.json()) as { url: string }).url)![1]!; + for (const path of [`/a/${id}/../secret.txt`, `/a/${id}/%2e%2e/%2e%2e/secret.txt`]) { + const response = await rawGet(address.port, path); + expect(response.status).toBe(404); expect(JSON.parse(response.body)).toEqual({ error: "not_found" }); + expect(response.body).not.toContain("outside-secret"); + } + await server.close(); log.close(); + }); + + it("keeps artifact and webhook body limits route-specific", async () => { + const { log, server } = setup(256); const address = await server.listen(); + const artifactResponse = await fetch(`http://127.0.0.1:${address.port}/a`, { method: "POST", headers: auth(), + body: manifest([{ path: "item.md", content: "x".repeat(512) }]) }); + await expectJsonError(artifactResponse, 413, "payload_too_large"); + expect((await oversizedWebhook(address.port)).startsWith("HTTP/1.1 413")).toBe(true); + await server.close(); log.close(); + }); +}); diff --git a/daemon/test/claude.test.ts b/daemon/test/claude.test.ts index 99cb627..3a1dad1 100644 --- a/daemon/test/claude.test.ts +++ b/daemon/test/claude.test.ts @@ -19,6 +19,7 @@ describe("runTurn", () => { const events: unknown[] = []; const ids: string[] = []; const result = await runTurn(options({ onEvent: (event: unknown) => events.push(event), onSessionId: (id: string) => ids.push(id) })); expect(result).toMatchObject({ ok: true, sessionId: "claude-session-1", resultText: "planner answer", sawResult: true }); + expect(result.capacityEvidence).toEqual([]); expect(events).toEqual([{ type: "text", text: "thinking" }, { type: "toolUse", name: "Read", input: { description: "ticket" } }]); expect(ids).toEqual(["claude-session-1"]); }); @@ -29,7 +30,27 @@ describe("runTurn", () => { expect(result.resultText).toBe("resumed prior-id"); }); it.each([["crash", false], ["no-result", false], ["denied", false]])("classifies %s as failure", async (mode, ok) => { - const result = await runTurn(options({ env: { CLAUDE_FAKE_MODE: mode } })); expect(result.ok).toBe(ok); + const result = await runTurn(options({ env: { CLAUDE_FAKE_MODE: mode } })); + expect(result.ok).toBe(ok); expect(result.capacityEvidence).toEqual([]); + }); + it.each([ + ["rate-limit-rejected", "rate_limit_event:rejected"], + ["out-of-credits", "rate_limit_event:out_of_credits"], + ["api-retry-exhausted", "api_retry:rate_limit"], + ["result-429", "result:429"], + ["assistant-rate-limit", "assistant:rate_limit"], + ])("collects structured capacity evidence from %s", async (mode, evidence) => { + const result = await runTurn(options({ env: { CLAUDE_FAKE_MODE: mode } })); + expect(result.ok).toBe(false); expect(result.capacityEvidence).toContain(evidence); + }); + it("does not turn recovered retry evidence into a runner failure", async () => { + const result = await runTurn(options({ env: { CLAUDE_FAKE_MODE: "api-retry-recovered" } })); + expect(result.ok).toBe(true); expect(result.capacityEvidence).toContain("api_retry:overloaded"); + }); + it("does not classify generic API result errors as capacity", async () => { + const result = await runTurn(options({ env: { CLAUDE_FAKE_MODE: "non-capacity-api-error" } })); + expect(result).toMatchObject({ ok: false, sawResult: true, exitCode: 1 }); + expect(result.capacityEvidence).toEqual([]); }); it("classifies ENOENT and abort signal death", async () => { expect(await runTurn(options({ argv: ["/missing/claude"] }))).toMatchObject({ ok: false, sawResult: false }); @@ -54,6 +75,14 @@ describe("runTurn", () => { expect(row.args).toEqual(expect.arrayContaining(["--max-budget-usd","12.5"])); expect(JSON.stringify(row.args)).not.toContain("secret-token"); }); + it("merges trusted runtime environment after the allowlist", async () => { + const dir = cwd(); const envFile = join(dir, "env.jsonl"); + const result = await runTurn(options({ cwd: dir, env: { CLAUDE_FAKE_ENV_FILE: envFile }, + trustedEnv: { ENABLE_TOOL_SEARCH: "true", CLAUDE_FAKE_MODE: "happy" } })); + expect(result.ok).toBe(true); + const row = JSON.parse(readFileSync(envFile, "utf8").trim()) as { env: Record }; + expect(row.env.ENABLE_TOOL_SEARCH).toBe("true"); + }); it("drains noisy stderr and returns a bounded tail on failure", async () => { const result = await runTurn(options({ env: { CLAUDE_FAKE_MODE: "stderr-fail" } })); expect(result.ok).toBe(false); diff --git a/daemon/test/config.test.ts b/daemon/test/config.test.ts index 5c4dce9..82fd03d 100644 --- a/daemon/test/config.test.ts +++ b/daemon/test/config.test.ts @@ -14,13 +14,30 @@ describe("loadConfig", () => { expect(config.bindAddr).toBe("127.0.0.1"); expect(config.replayWindowMs).toBe(60_000); expect(config.webhookBaseUrl).toBe("http://127.0.0.1:8787"); + expect(config.artifactToken).toBeUndefined(); + expect(config.artifactsDir).toBe("/var/lib/linear-agent-daemon/artifacts"); + expect(config.artifactMaxBodyBytes).toBe(32 * 1024 * 1024); expect(config.reconcileIntervalMs).toBe(60_000); expect(config.reconcileRequestTimeoutMs).toBe(10_000); expect(config.apps.planner.staticToken).toBe("pt"); expect(config.sessionsEnabled).toBe(false); expect(config.claudeArgv).toEqual(["claude"]); + expect(config.claudexArgv).toBeUndefined(); + expect(config.fableArgv).toBeUndefined(); + expect(config).toMatchObject({ cliproxyEnvFile: "/etc/linear-agent-daemon/cliproxyapi.env", + cliproxyUrl: "http://127.0.0.1:8317", providerProbeIntervalMs: 60_000, + providerStateStaleMs: 300_000, providerInitialProbeTimeoutMs: 5_000 }); expect(config).toMatchObject({doPermissionMode:"bypassPermissions",doMaxTurns:300}); }); + it("loads Fable and provider probe overrides", () => { + expect(loadConfig({ ...base, FABLE_BIN: "node fable.mjs", CLIPROXY_ENV_FILE: "/tmp/proxy.env", + CLIPROXY_URL: "http://proxy:8317/", PROVIDER_PROBE_INTERVAL_MS: "2000", + PROVIDER_STATE_STALE_MS: "9000", PROVIDER_INITIAL_PROBE_TIMEOUT_MS: "750" })).toMatchObject({ + fableArgv: ["node", "fable.mjs"], cliproxyEnvFile: "/tmp/proxy.env", cliproxyUrl: "http://proxy:8317", + providerProbeIntervalMs: 2000, providerStateStaleMs: 9000, providerInitialProbeTimeoutMs: 750, + }); + expect(() => loadConfig({ ...base, PROVIDER_INITIAL_PROBE_TIMEOUT_MS: "0" })).toThrow("PROVIDER_INITIAL_PROBE_TIMEOUT_MS"); + }); it("forces production do-mode autonomy and parses its budget",()=>{ expect(()=>loadConfig({...base,DAEMON_TEST_MODE:undefined,WEBHOOK_BASE_URL:"https://agent.example.com",DO_PERMISSION_MODE:"plan"})).toThrow("DO_PERMISSION_MODE"); expect(loadConfig({...base,DO_PERMISSION_MODE:"plan",DO_MAX_TURNS:"400",DO_MAX_BUDGET_USD:"25.5"})) @@ -38,6 +55,18 @@ describe("loadConfig", () => { it("names missing variables", () => { expect(() => loadConfig({ ...base, PLANNER_WEBHOOK_SECRET: "" })).toThrow("PLANNER_WEBHOOK_SECRET"); }); + it("parses and validates the optional Claudex runtime", () => { + expect(loadConfig({ ...base, CLAUDEX_BIN: "claude --model gpt-5.6-sol", + CLAUDEX_ENV: '{"ANTHROPIC_BASE_URL":"http://proxy","ENABLE_TOOL_SEARCH":"true"}' })) + .toMatchObject({ claudexArgv: ["claude", "--model", "gpt-5.6-sol"], + claudexEnv: { ANTHROPIC_BASE_URL: "http://proxy", ENABLE_TOOL_SEARCH: "true" } }); + expect(() => loadConfig({ ...base, CLAUDEX_BIN: " " })).toThrow("CLAUDEX_BIN must not be empty"); + expect(() => loadConfig({ ...base, CLAUDEX_ENV: "{}" })).toThrow("requires CLAUDEX_BIN"); + expect(() => loadConfig({ ...base, CLAUDEX_BIN: "claude", CLAUDEX_ENV: "[]" })).toThrow("JSON object"); + expect(() => loadConfig({ ...base, CLAUDEX_BIN: "claude", CLAUDEX_ENV: '{"X":1}' })).toThrow("string values"); + expect(() => loadConfig({ ...base, CLAUDEX_BIN: "claude", CLAUDEX_ENV: "{" })).toThrow("valid JSON"); + expect(() => loadConfig({ ...base, CLAUDEX_BIN: "claude", CLAUDEX_ENV: " " })).toThrow("valid JSON"); + }); it("requires client credentials without the test-only token override", () => { const env = { ...base }; delete (env as Partial).DAEMON_TEST_MODE; (env as Record).WEBHOOK_BASE_URL = "https://agent.example.com"; @@ -52,6 +81,11 @@ describe("loadConfig", () => { expect(config.apps.planner.appActorId).toBe("planner-actor"); expect(config.apps.implementer.appActorId).toBe("implementer-actor"); }); + it("loads artifact settings", () => { + const config = loadConfig({ ...base, DB_PATH: "/state/events.db", ARTIFACT_TOKEN: " secret ", + ARTIFACTS_DIR: "/srv/artifacts", ARTIFACT_MAX_BODY_BYTES: "4096" }); + expect(config).toMatchObject({ artifactToken: "secret", artifactsDir: "/srv/artifacts", artifactMaxBodyBytes: 4096 }); + }); it("requires WEBHOOK_BASE_URL outside test mode", () => { const env = { ...base, PLANNER_LINEAR_CLIENT_ID: "p-id", PLANNER_LINEAR_CLIENT_SECRET: "p-secret", IMPLEMENTER_LINEAR_CLIENT_ID: "i-id", IMPLEMENTER_LINEAR_CLIENT_SECRET: "i-secret" }; diff --git a/daemon/test/eventlog.test.ts b/daemon/test/eventlog.test.ts index d8fa637..7fdac6a 100644 --- a/daemon/test/eventlog.test.ts +++ b/daemon/test/eventlog.test.ts @@ -175,7 +175,11 @@ describe("EventLog", () => { log.updateClaudeSessionId("session-1", "claude-1", 1002); log.append(event({ deliveryId: "delivery-2", action: "prompted", issueId: undefined, issueIdentifier: undefined })); expect(log.getSession("session-1")).toMatchObject({ issueId: "issue-uuid-1", issueIdentifier: "ENG-42", - worktreePath: "/worktree", branch: "agents/ENG-42", claudeSessionId: "claude-1" }); + worktreePath: "/worktree", branch: "agents/ENG-42", claudeSessionId: "claude-1", runtime: "claude", fallbackCause: null }); + log.clearClaudeSessionId("session-1", 1003); + expect(log.getSession("session-1")?.claudeSessionId).toBeNull(); + log.recordRuntimeFallback("session-1", "claudex-1", "rate_limit_event:rejected", 1004); + expect(log.getSession("session-1")).toMatchObject({ runtime: "claudex", fallbackCause: "rate_limit_event:rejected", claudeSessionId: "claudex-1" }); expect(log.claimNextTurn(1100)?.id).toBe(1); expect(log.interruptStaleRunning(1200)).toEqual([1]); expect(log.pendingTurnActivities(1200)[0]).toMatchObject({ kind: "error", turnId: 1 }); @@ -185,7 +189,7 @@ describe("EventLog", () => { expect(log.claimNextTurn(1301)).toMatchObject({ id: 2, kind: "prompted" }); log.close(); const reopened = new EventLog(dbPath); - expect(reopened.getSession("session-1")?.claudeSessionId).toBe("claude-1"); + expect(reopened.getSession("session-1")).toMatchObject({ runtime: "claudex", claudeSessionId: "claudex-1" }); reopened.close(); }); @@ -243,4 +247,48 @@ describe("EventLog", () => { expect(log.append(event({ deliveryId: undefined })).inserted).toBe(false); log.close(); }); + it("assigns a profile only on first insert and preserves it in every projection", () => { + let selections = 0; + const log = new EventLog(path(), () => { selections++; return { profile: "fable", reason: "claude_ready" }; }); + const first = log.append(event()); + const duplicate = log.append(event({ deliveryId: "delivery-2", action: "prompted" })); + expect(first).toMatchObject({ assignedProfile: "fable", assignmentReason: "claude_ready" }); + expect(duplicate.assignedProfile).toBeUndefined(); expect(selections).toBe(1); + log.updateSessionWorktree("session-1", "/worktree", "branch"); + expect(log.getSession("session-1")?.profile).toBe("fable"); + expect(log.sessionByIssueIdentifier("ENG-42")?.profile).toBe("fable"); + expect(log.plannerSessionsForReconcile()[0]?.profile).toBe("fable"); + expect(log.sessionsWithWorktrees()[0]?.profile).toBe("fable"); + log.close(); + }); + it("stores provider state and atomically permits only one pre-establishment fallback", () => { + const log = new EventLog(path(), () => ({ profile: "fable", reason: "ready" })); + log.append(event()); + expect(log.applyPreEstablishmentFallback("session-1")).toBe(true); + expect(log.applyPreEstablishmentFallback("session-1")).toBe(false); + expect(log.getSession("session-1")).toMatchObject({ profile: "sol", profileFallback: 1 }); + log.setProviderState("claude", "ready", "eligible_1_failed_0", 100); + log.setProviderCooldown("claude", 900, "http_503", 200); + expect(log.getProviderState("claude")).toEqual({ provider: "claude", status: "cooldown", reason: "http_503", cooldownUntil: 900, updatedAt: 200 }); + log.close(); + const established = new EventLog(path(), () => ({ profile: "fable", reason: "ready" })); + established.append(event({ deliveryId: "other", agentSessionId: "other" })); + established.updateClaudeSessionId("other", "claude-id"); + expect(established.applyPreEstablishmentFallback("other")).toBe(false); + established.close(); + }); + it("migrates legacy session rows with NULL profiles", () => { + const dbPath = path(); const old = new Database(dbPath); + old.exec(`CREATE TABLE sessions ( + linear_session_id TEXT PRIMARY KEY, app TEXT NOT NULL, issue_id TEXT, issue_identifier TEXT, + worktree_path TEXT, branch TEXT, claude_session_id TEXT, mode TEXT NOT NULL DEFAULT 'planner', + status TEXT NOT NULL DEFAULT 'active', last_seen_at INTEGER NOT NULL, last_seen_activity_at INTEGER + ); INSERT INTO sessions(linear_session_id,app,mode,status,last_seen_at) VALUES('legacy','planner','planner','active',1);`); + old.close(); const log = new EventLog(dbPath); + expect(log.getSession("legacy")).toMatchObject({ profile: null, profileFallback: null }); + const db = new Database(dbPath, { readonly: true }); + expect((db.prepare("PRAGMA table_info(sessions)").all() as Array<{ name: string }>).map(row => row.name)) + .toEqual(expect.arrayContaining(["profile", "profile_fallback"])); + db.close(); log.close(); + }); }); diff --git a/daemon/test/fixtures/fake-claude.mjs b/daemon/test/fixtures/fake-claude.mjs index c5d5601..c2ba06b 100644 --- a/daemon/test/fixtures/fake-claude.mjs +++ b/daemon/test/fixtures/fake-claude.mjs @@ -12,11 +12,53 @@ if (argsFile) await appendFile(argsFile, if (process.env.CLAUDE_FAKE_ENV_FILE) await appendFile(process.env.CLAUDE_FAKE_ENV_FILE, `${JSON.stringify({ args, env: process.env, at: Date.now(), phase: "env" })}\n`); const emit = value => process.stdout.write(`${JSON.stringify(value)}\n`); +if (mode === "provider-fail-pre-id") { + process.stderr.write("connection refused by provider base URL\n"); + process.exit(7); +} const session = resumed || (mode === "do-pr" || mode === "do-pr-error" ? "claude-do-session" : "claude-session-1"); emit({ type: "system", subtype: "init", session_id: session, uuid: "init" }); +if (mode === "rate-limit-rejected" || mode === "capacity-after-session") { + emit({ type: "rate_limit_event", session_id: session, rate_limit_info: { status: "rejected", rateLimitType: "five_hour" } }); + process.exit(1); +} +if (mode === "out-of-credits") { + emit({ type: "rate_limit_event", session_id: session, apiErrorStatus: 429, + rate_limit_info: { status: "allowed", overageDisabledReason: "out_of_credits" } }); + process.exit(1); +} +if (mode === "api-retry-exhausted") { + emit({ type: "system", subtype: "api_retry", session_id: session, attempt: 1, max_retries: 2, error: "rate_limit", error_status: 429 }); + emit({ type: "system", subtype: "api_retry", session_id: session, attempt: 2, max_retries: 2, error: "rate_limit", error_status: 429 }); + emit({ type: "result", subtype: "error_during_execution", is_error: true, result: "request failed", session_id: session }); + process.exit(1); +} +if (mode === "result-429") { + emit({ type: "result", subtype: "error_during_execution", is_error: true, error_status: 429, + errors: [{ type: "rate_limit_error" }], result: "request failed", session_id: session }); + process.exit(1); +} +if (mode === "assistant-rate-limit") { + emit({ type: "assistant", session_id: session, error: "rate_limit", message: { content: [] } }); + process.exit(1); +} +if (mode === "api-retry-recovered") + emit({ type: "system", subtype: "api_retry", session_id: session, attempt: 1, max_retries: 2, error: "overloaded", error_status: 529 }); +if (mode === "non-capacity-api-error") { + emit({ type: "result", subtype: "error_during_execution", terminal_reason: "api_error", is_error: true, + errors: [{ type: "authentication_error" }, { type: "api_error" }], result: "request failed", session_id: session }); + process.exit(1); +} +if (mode === "provider-fail-post-id") { + process.stderr.write("HTTP 503 from provider base URL\n"); + process.exit(7); +} if (mode === "crash") process.exit(7); if (mode === "no-result") process.exit(0); -if (mode === "hang") await new Promise(() => {}); +if (mode === "hang") { + setInterval(() => {}, 1_000); + await new Promise(() => {}); +} if (mode === "grandchild-hang") { const file = process.env.CLAUDE_FAKE_HEARTBEAT_FILE; spawn(process.execPath, ["-e", `const fs=require("fs"); setInterval(() => fs.appendFileSync(${JSON.stringify(file)}, Date.now()+"\\n"), 25);`], @@ -35,9 +77,11 @@ emit({ type: "assistant", session_id: session, message: { content: [ if (mode === "new-id") emit({ type: "assistant", session_id: "claude-session-2", message: { content: [{ type: "text", text: "compacted" }] } }); if (mode === "slow") await new Promise(resolve => setTimeout(resolve, Number(process.env.CLAUDE_FAKE_DELAY_MS || process.env.FAKE_DELAY_MS || 100))); const finalSession = mode === "new-id" ? "claude-session-2" : session; -emit({ type: "result", subtype: mode === "denied" || mode === "do-pr-error" ? "error" : "success", is_error: mode === "denied" || mode === "do-pr-error", +const errorResult = mode === "denied" || mode === "do-pr-error" || mode === "error-result-exit"; +emit({ type: "result", subtype: errorResult ? "error" : "success", is_error: errorResult, result: mode === "do-pr" || mode === "do-pr-error" ? "Opened https://github.com/dcouple/example/pull/42" : resumed ? `resumed ${resumed}` : "planner answer", session_id: finalSession, permission_denials: mode === "denied" ? [{ tool: "Bash" }] : [] }); +if (mode === "error-result-exit") process.exit(11); if (mode === "touch-file" && process.env.CLAUDE_FAKE_TOUCH_FILE) await writeFile(process.env.CLAUDE_FAKE_TOUCH_FILE, "done"); if (argsFile) await appendFile(argsFile, `${JSON.stringify({ args, cwd: process.cwd(), at: Date.now(), phase: "end" })}\n`); diff --git a/daemon/test/provision.test.ts b/daemon/test/provision.test.ts new file mode 100644 index 0000000..9fd8a4c --- /dev/null +++ b/daemon/test/provision.test.ts @@ -0,0 +1,158 @@ +import { spawn, spawnSync } from "node:child_process"; +import { chmodSync, existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { createServer } from "node:http"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { describe, expect, it } from "vitest"; + +const provision = readFileSync(resolve("ops/provision.sh"), "utf8"); +const claudex = readFileSync(resolve("ops/claudex"), "utf8"); +const claudexFable = readFileSync(resolve("ops/claudex-fable"), "utf8"); +const proxyAccounts = readFileSync(resolve("ops/proxy-accounts.sh"), "utf8"); +const providerGate = readFileSync(resolve("ops/codex-provider-gate.sh"), "utf8"); +const proxyUnit = readFileSync(resolve("ops/cliproxyapi.service"), "utf8"); +const daemonUnit = readFileSync(resolve("ops/linear-agent-daemon.service"), "utf8"); + +describe("daemon provisioning", () => { + it("pins and checksum-verifies CLIProxyAPI for supported architectures", () => { + expect(provision).toContain('CLIPROXY_VERSION="7.2.93"'); + expect(provision).toContain('CLIPROXY_ARCH="amd64"'); + expect(provision).toContain('CLIPROXY_ARCH="aarch64"'); + expect(provision).toContain("sha256sum -c -"); + }); + + it("installs a claudex executable with the GPT-5.6 Sol defaults", () => { + expect(provision).toContain('"${SOURCE_DIR}/ops/claudex"'); + expect(claudex).toContain(". /etc/linear-agent-daemon/cliproxyapi.env"); + expect(claudex).toContain("export ANTHROPIC_BASE_URL=http://127.0.0.1:8317"); + expect(claudex).toContain("export CLIPROXY_API_KEY"); + expect(claudex).toContain('export ANTHROPIC_AUTH_TOKEN="${CLIPROXY_API_KEY}"'); + expect(claudex).toContain("export ANTHROPIC_DEFAULT_HAIKU_MODEL=gpt-5.6-sol-low"); + expect(claudex).toContain("export ANTHROPIC_DEFAULT_SONNET_MODEL=gpt-5.6-sol-low"); + expect(claudex).toContain("export ANTHROPIC_DEFAULT_OPUS_MODEL=gpt-5.6-sol-medium"); + expect(claudex).toContain("export ANTHROPIC_DEFAULT_FABLE_MODEL=gpt-5.6-sol-xhigh"); + expect(claudex).toContain("export CLAUDE_CODE_MAX_CONTEXT_TOKENS=250000"); + expect(claudex).not.toContain("CLAUDE_CODE_SUBAGENT_MODEL"); + expect(claudex).toContain("export CLAUDE_CODE_ALWAYS_ENABLE_EFFORT=1"); + expect(claudex).toContain("export CLAUDE_CODE_MAX_TOOL_USE_CONCURRENCY=3"); + expect(claudex).toContain("export ENABLE_TOOL_SEARCH=true"); + expect(claudex).toContain("claude --model gpt-5.6-sol"); + }); + it("installs a fail-closed Fable launcher and validates model identity before exec", async () => { + expect(provision).toContain('"${SOURCE_DIR}/ops/claudex-fable"'); + expect(provision).not.toContain("cat > /etc/linear-agent-daemon/fable-models.env"); + expect(claudexFable).toContain("claude-*"); + expect(claudexFable).toContain("/v1/models"); + const dir = mkdtempSync(join(tmpdir(), "claudex-fable-")); + const proxyEnv = join(dir, "proxy.env"), modelsEnv = join(dir, "models.env"), argsFile = join(dir, "args"); + const fakeClaude = join(dir, "claude"); + writeFileSync(proxyEnv, "CLIPROXY_API_KEY=test-key\n"); + writeFileSync(fakeClaude, `#!/bin/sh\nprintf '%s\\n' "$*" > ${argsFile}\n`); chmodSync(fakeClaude, 0o755); + const server = createServer((_request, response) => { response.setHeader("content-type", "application/json"); + response.end(JSON.stringify({ data: [{ id: "claude-real" }] })); }); + await new Promise(resolveListen => server.listen(0, "127.0.0.1", resolveListen)); + const port = (server.address() as { port: number }).port; + const run = (models: string) => new Promise<{ code: number | null; stderr: string }>(resolveRun => { + writeFileSync(modelsEnv, models); + const child = spawn("sh", [resolve("ops/claudex-fable"), "--flag"], { env: { ...process.env, + CLIPROXY_ENV_FILE: proxyEnv, FABLE_MODELS_ENV_FILE: modelsEnv, PROXY_URL: `http://127.0.0.1:${port}`, + FABLE_CLAUDE_BIN: fakeClaude }, stdio: ["ignore", "ignore", "pipe"] }); + let stderr = ""; child.stderr.on("data", chunk => { stderr += String(chunk); }); + child.on("close", code => resolveRun({ code, stderr })); + }); + const valid = "FABLE_MAIN_MODEL=claude-real\nFABLE_HAIKU_MODEL=claude-real\nFABLE_SONNET_MODEL=claude-real\nFABLE_OPUS_MODEL=claude-real\nFABLE_FABLE_MODEL=claude-real\n"; + expect((await run(valid)).code).toBe(0); expect(readFileSync(argsFile, "utf8")).toContain("--model claude-real --flag"); + const wrong = await run(valid.replace("FABLE_MAIN_MODEL=claude-real", "FABLE_MAIN_MODEL=gpt-wrong")); + expect(wrong.code).not.toBe(0); expect(wrong.stderr).toContain("FABLE_MAIN_MODEL must name a claude-*"); + const missing = await run(valid.replace("FABLE_MAIN_MODEL=claude-real", "FABLE_MAIN_MODEL=claude-missing")); + expect(missing.code).not.toBe(0); expect(missing.stderr).toContain("FABLE_MAIN_MODEL model absent"); + await new Promise(resolveClose => server.close(() => resolveClose())); rmSync(dir, { recursive: true, force: true }); + }); + + it("configures model effort tiers and starts the loopback proxy before the daemon", () => { + expect(provision).toContain('alias: "gpt-5.6-sol-low"'); + expect(provision).toContain('alias: "gpt-5.6-sol-medium"'); + expect(provision).toContain('alias: "gpt-5.6-sol-xhigh"'); + expect(provision).toContain('"reasoning.effort": "xhigh"'); + expect(provision).toContain('session-affinity-ttl: "168h"'); + expect(provision).toContain("save-cooldown-status: true"); + expect(provision).toContain("remote-management:"); + expect(provision).toContain("allow-remote: false"); + expect(provision).toMatch(/payload:\n default:[\s\S]*name: "gpt-5\.6-sol"[\s\S]*"reasoning\.effort": "high"[\s\S]* override:/); + expect(proxyUnit).toContain("ExecStart=/usr/local/bin/cliproxyapi -config /etc/linear-agent-daemon/cliproxyapi.yaml"); + expect(proxyUnit).toContain("User=linear-daemon"); + expect(daemonUnit).toContain("After=network-online.target cliproxyapi.service"); + expect(daemonUnit).toContain("Wants=network-online.target cliproxyapi.service"); + expect(daemonUnit).not.toContain("Requires=cliproxyapi.service"); + expect(provision).toContain("systemctl enable caddy cliproxyapi linear-agent-daemon"); + expect(provision).toContain("cliproxy_has_default_model"); + expect(provision).toContain('model.get("id") == "gpt-5.6-sol"'); + }); + + it("manages separate API and management keys and deploys fail-closed account tooling", () => { + expect(provision).toContain("CLIPROXY_MANAGEMENT_KEY=<48 lowercase hex characters>"); + expect(provision).toContain('CLIPROXY_MANAGEMENT_KEY="$(grep -E'); + expect(provision).toContain("ops/proxy-accounts.sh"); + expect(provision).toContain("ops/codex-provider-gate.sh"); + expect(provision).toContain('EXPECTED_PROXY_VERSION="${CLIPROXY_VERSION}"'); + expect(provision).toContain("standalone Codex provider gate skipped"); + expect(provision).toContain("/v0/management/auth-files"); + expect(provision).toContain('item.get("provider") == "codex"'); + expect(provision).not.toContain("-name 'codex-*.json'"); + expect(proxyAccounts).toContain("/v0/management/auth-files"); + expect(providerGate).toContain("codex exec resume --last"); + expect(providerGate).toContain("rollback"); + expect(providerGate).toContain('echo "FAIL: unexpected-error"'); + expect(providerGate).toContain('GATE_COUNTER_WINDOW_SECONDS="${GATE_COUNTER_WINDOW_SECONDS:-15}"'); + }); + + it("exposes help before secret validation and returns nonzero for gate environment failures", () => { + const gatePath = resolve("ops/codex-provider-gate.sh"); + const accountsPath = resolve("ops/proxy-accounts.sh"); + const missingSecrets = { ...process.env, CLIPROXY_ENV_FILE: "/dev/null" }; + + const gateHelp = spawnSync("bash", [gatePath, "--help"], { encoding: "utf8", env: missingSecrets }); + expect(gateHelp.status).toBe(0); + expect(gateHelp.stdout).toContain("usage: codex-provider-gate.sh [--help]"); + expect(gateHelp.stdout).toContain("Environment variables:"); + + const accountsHelp = spawnSync("bash", [accountsPath, "--help"], { encoding: "utf8", env: missingSecrets }); + expect(accountsHelp.status).toBe(0); + expect(accountsHelp.stdout).toContain("usage: proxy-accounts.sh"); + + const gateFailure = spawnSync("bash", [gatePath], { encoding: "utf8", env: missingSecrets }); + expect(gateFailure.status).not.toBe(0); + expect(gateFailure.stderr).toContain("missing CLIPROXY_API_KEY"); + + const unknownArgument = spawnSync("bash", [gatePath, "--unknown"], { encoding: "utf8", env: missingSecrets }); + expect(unknownArgument.status).not.toBe(0); + expect(unknownArgument.stderr).toContain("usage: codex-provider-gate.sh"); + }); + + it("removes only gate-managed provider configs when environment validation fails", () => { + const gatePath = resolve("ops/codex-provider-gate.sh"); + const dir = mkdtempSync(join(tmpdir(), "provider-gate-rollback-")); + try { + const managed = join(dir, "managed.toml"); + writeFileSync(managed, "# managed by codex-provider-gate.sh — removed on gate failure\nmodel = \"old\"\n"); + const managedFailure = spawnSync("bash", [gatePath], { + encoding: "utf8", + env: { ...process.env, CLIPROXY_ENV_FILE: "/dev/null", TARGET_CONFIG: managed }, + }); + expect(managedFailure.status).not.toBe(0); + expect(existsSync(managed)).toBe(false); + + const unmarked = join(dir, "unmarked.toml"); + const unmarkedContent = "model = \"direct\"\n# preserve byte-identically\n"; + writeFileSync(unmarked, unmarkedContent); + const unmarkedFailure = spawnSync("bash", [gatePath], { + encoding: "utf8", + env: { ...process.env, CLIPROXY_ENV_FILE: "/dev/null", TARGET_CONFIG: unmarked }, + }); + expect(unmarkedFailure.status).not.toBe(0); + expect(readFileSync(unmarked, "utf8")).toBe(unmarkedContent); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/daemon/test/proxy-integration.test.ts b/daemon/test/proxy-integration.test.ts new file mode 100644 index 0000000..8e9d3ce --- /dev/null +++ b/daemon/test/proxy-integration.test.ts @@ -0,0 +1,181 @@ +import { execFileSync, spawn, type ChildProcess } from "node:child_process"; +import { once } from "node:events"; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { createServer } from "node:net"; +import { join } from "node:path"; +import { tmpdir } from "node:os"; +import { afterEach, describe, expect, it } from "vitest"; + +const proxyBin = process.env.CLIPROXY_BIN; +const dirs: string[] = []; +let child: ChildProcess | undefined; + +afterEach(async () => { + if (child?.exitCode === null) { + child.kill("SIGTERM"); + await once(child, "exit"); + } + child = undefined; + for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); +}); + +async function freePort(): Promise { + const server = createServer(); + server.listen(0, "127.0.0.1"); + await once(server, "listening"); + const address = server.address(); + if (!address || typeof address === "string") throw new Error("no test port"); + await new Promise((resolve, reject) => server.close(error => error ? reject(error) : resolve())); + return address.port; +} + +function credential(type: "codex" | "claude", email: string, token: string): string { + return JSON.stringify({ + type, + email, + account_id: `fixture-${email}`, + access_token: token, + refresh_token: `${token}-refresh`, + disabled: false, + }); +} + +async function poll(fn: () => Promise, timeoutMs = 12_000): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + const value = await fn().catch(() => undefined); + if (value !== undefined) return value; + await new Promise(resolve => setTimeout(resolve, 100)); + } + throw new Error("timed out waiting for proxy"); +} + +describe.skipIf(!proxyBin)("CLIProxyAPI integration", () => { + it("loads aliases and credentials, disables one, hot-loads one, and redacts tokens", async () => { + const dir = mkdtempSync(join(tmpdir(), "cliproxy-integration-")); dirs.push(dir); + const authDir = join(dir, "auth"); mkdirSync(authDir); + const apiKey = "a".repeat(48); + const managementKey = "b".repeat(48); + const fixtureTokens = ["fixture-codex-one-secret", "fixture-codex-two-secret", "fixture-claude-one-secret", "fixture-claude-two-secret"]; + const fixtures = [ + ["codex-one.json", "codex", "codex-one@example.test", fixtureTokens[0]], + ["codex-two.json", "codex", "codex-two@example.test", fixtureTokens[1]], + ["claude-one.json", "claude", "claude-one@example.test", fixtureTokens[2]], + ["claude-two.json", "claude", "claude-two@example.test", fixtureTokens[3]], + ] as const; + for (const [name, type, email, token] of fixtures) writeFileSync(join(authDir, name), credential(type, email, token)); + const port = await freePort(); + const config = join(dir, "config.yaml"); + writeFileSync(config, `host: "127.0.0.1" +port: ${port} +auth-dir: "${authDir}" +api-keys: + - "${apiKey}" +routing: + strategy: "round-robin" + session-affinity: true + session-affinity-ttl: "168h" +save-cooldown-status: true +remote-management: + secret-key: "${managementKey}" + allow-remote: false +oauth-model-alias: + codex: + - name: "gpt-5.6-sol" + alias: "gpt-5.6-sol-low" + fork: true + - name: "gpt-5.6-sol" + alias: "gpt-5.6-sol-medium" + fork: true + - name: "gpt-5.6-sol" + alias: "gpt-5.6-sol-xhigh" + fork: true +payload: + default: + - models: + - name: "gpt-5.6-sol" + protocol: "codex" + params: + "reasoning.effort": "high" + override: + - models: + - name: "gpt-5.6-sol-low" + protocol: "codex" + params: + "reasoning.effort": "low" + - models: + - name: "gpt-5.6-sol-medium" + protocol: "codex" + params: + "reasoning.effort": "medium" + - models: + - name: "gpt-5.6-sol-xhigh" + protocol: "codex" + params: + "reasoning.effort": "xhigh" +`); + let logs = ""; + child = spawn(proxyBin!, ["-config", config], { stdio: ["ignore", "pipe", "pipe"] }); + child.stdout?.on("data", chunk => { logs += chunk.toString(); }); + child.stderr?.on("data", chunk => { logs += chunk.toString(); }); + const base = `http://127.0.0.1:${port}`; + const expectedModels = ["gpt-5.6-sol", "gpt-5.6-sol-low", "gpt-5.6-sol-medium", "gpt-5.6-sol-xhigh"]; + const models = await poll(async () => { + const response = await fetch(`${base}/v1/models`, { headers: { authorization: `Bearer ${apiKey}` } }); + if (!response.ok) return undefined; + const payload = await response.json() as { data: Array<{ id: string }> }; + const ids = payload.data.map(model => model.id); + return expectedModels.every(model => ids.includes(model)) ? payload : undefined; + }); + expect(models.data.map(model => model.id)).toEqual(expect.arrayContaining(expectedModels)); + + const managementHeaders = { authorization: `Bearer ${managementKey}` }; + const list = async () => { + const response = await fetch(`${base}/v0/management/auth-files`, { headers: managementHeaders }); + expect(response.status).toBe(200); + return await response.json() as { files: Array> }; + }; + const initial = await poll(async () => { + const payload = await list(); + return payload.files.length >= 4 ? payload : undefined; + }); + for (const [name, type, email] of fixtures) { + expect(initial.files).toContainEqual(expect.objectContaining({ name, provider: type, email, disabled: false })); + } + const initialJson = JSON.stringify(initial); + for (const token of fixtureTokens) expect(initialJson).not.toContain(token); + + const envFile = join(dir, "cliproxy.env"); + writeFileSync(envFile, `CLIPROXY_API_KEY=${apiKey}\nCLIPROXY_MANAGEMENT_KEY=${managementKey}\n`); + const helperEnv = { + ...process.env, + PROXY_URL: base, + CLIPROXY_ENV_FILE: envFile, + CLIPROXY_BIN: proxyBin!, + CLIPROXY_CONFIG: config, + }; + const helper = join(process.cwd(), "ops/proxy-accounts.sh"); + const helperList = execFileSync(helper, ["list"], { env: helperEnv, encoding: "utf8" }); + expect(helperList).toContain('"provider":"codex"'); + for (const token of fixtureTokens) expect(helperList).not.toContain(token); + const dryRunOne = execFileSync(helper, ["add", "codex", "--dry-run"], { env: helperEnv, encoding: "utf8" }); + const dryRunTwo = execFileSync(helper, ["add", "codex", "--dry-run"], { env: helperEnv, encoding: "utf8" }); + expect(dryRunTwo).toBe(dryRunOne); + + const disabledName = fixtures[0][0]; + const disabled = await fetch(`${base}/v0/management/auth-files/status`, { + method: "PATCH", + headers: { ...managementHeaders, "content-type": "application/json" }, + body: JSON.stringify({ name: disabledName, disabled: true }), + }); + expect(disabled.status).toBe(200); + await poll(async () => (await list()).files.find(item => item.name === disabledName && item.disabled === true)); + + const hotToken = "fixture-codex-hot-secret"; + fixtureTokens.push(hotToken); + writeFileSync(join(authDir, "codex-hot.json"), credential("codex", "codex-hot@example.test", hotToken)); + await poll(async () => (await list()).files.find(item => item.name === "codex-hot.json")); + await new Promise(resolve => setTimeout(resolve, 100)); + for (const token of fixtureTokens) expect(logs).not.toContain(token); + }, 20_000); +}); diff --git a/daemon/test/reconcile.test.ts b/daemon/test/reconcile.test.ts index 32e826b..ff02602 100644 --- a/daemon/test/reconcile.test.ts +++ b/daemon/test/reconcile.test.ts @@ -196,6 +196,7 @@ describe("ReconcileWorker", () => { expect(migrated.prepare("SELECT source_key sourceKey FROM turns WHERE id=1").get()) .toEqual({ sourceKey: "created:planner-session" }); const seeded = log.getSession("planner-session")?.lastSeenActivityAt; + expect(log.getSession("planner-session")).toMatchObject({ runtime: "claude", fallbackCause: null }); expect(seeded).toEqual(expect.any(Number)); expect(seeded!).toBeGreaterThan(2_000); migrated.close(); diff --git a/daemon/test/server.test.ts b/daemon/test/server.test.ts index d37bd57..105faf3 100644 --- a/daemon/test/server.test.ts +++ b/daemon/test/server.test.ts @@ -1,7 +1,7 @@ import { createHmac } from "node:crypto"; import { spawn, type ChildProcess } from "node:child_process"; import { once } from "node:events"; -import { mkdtempSync, rmSync } from "node:fs"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { createServer as createHttpServer } from "node:http"; import { createConnection, createServer as createNetServer } from "node:net"; import { tmpdir } from "node:os"; @@ -11,6 +11,7 @@ import { AckWorker } from "../src/ack.js"; import type { Config } from "../src/config.js"; import { EventLog } from "../src/eventlog.js"; import { LinearGateway } from "../src/linear.js"; +import { ReconcileWorker } from "../src/reconcile.js"; import { WebhookServer } from "../src/server.js"; const dirs: string[] = []; @@ -18,18 +19,21 @@ afterEach(() => { for (const dir of dirs.splice(0)) rmSync(dir, { recursive: tru function setup() { const dir = mkdtempSync(join(tmpdir(), "linear-server-")); dirs.push(dir); + const managementKey = "management-secret"; const cliproxyEnvFile = join(dir, "cliproxy.env"); + writeFileSync(cliproxyEnvFile, `CLIPROXY_MANAGEMENT_KEY=${managementKey}\n`); const log = new EventLog(join(dir, "events.db")); const config: Config = { port: 0, bindAddr: "127.0.0.1", dbPath: join(dir, "events.db"), replayWindowMs: 60_000, - linearGraphqlUrl: "http://unused", linearTokenUrl: "http://unused", + linearGraphqlUrl: "http://unused", linearTokenUrl: "http://unused", cliproxyEnvFile, apps: { planner: { name: "planner", webhookSecret: "planner-secret", staticToken: "p" }, implementer: { name: "implementer", webhookSecret: "implementer-secret", staticToken: "i" }, }, }; const onInserted = vi.fn(); const onStop = vi.fn(); - const server = new WebhookServer({ config, log, onInserted, onStop, logger: { log: vi.fn(), error: vi.fn() } }); - return { config, log, server, onInserted, onStop }; + const logger = { log: vi.fn(), error: vi.fn() }; + const server = new WebhookServer({ config, log, onInserted, onStop, logger }); + return { config, log, server, onInserted, onStop, logger, managementKey }; } function signed(body: string, secret = "planner-secret", delivery = "delivery-1") { @@ -73,6 +77,32 @@ describe("webhook HTTP integration", () => { expect(log.getSession("session")).toMatchObject({ issueId: "issue", issueIdentifier: "ENG-42" }); await server.close(); log.close(); }); + it("logs one assignment for a prompted-first session and none for later conflicts", async () => { + const { log, server, logger } = setup(); const address = await server.listen(); + const prompted = JSON.stringify({ webhookTimestamp: Date.now(), action: "prompted", + agentActivity: { id: "a1", body: "start" }, agentSession: { id: "prompt-first" } }); + await fetch(`http://127.0.0.1:${address.port}/webhook/planner`, { method: "POST", headers: signed(prompted, "planner-secret", "p1"), body: prompted }); + const created = JSON.stringify({ webhookTimestamp: Date.now(), action: "created", agentSession: { id: "prompt-first" } }); + await fetch(`http://127.0.0.1:${address.port}/webhook/planner`, { method: "POST", headers: signed(created, "planner-secret", "c1"), body: created }); + const assignments = logger.log.mock.calls.map(call => JSON.parse(String(call[0]))).filter(entry => entry.event === "session_profile_assigned"); + expect(assignments).toEqual([{ event: "session_profile_assigned", linearSessionId: "prompt-first", profile: "sol", reason: "fable_not_configured" }]); + await server.close(); log.close(); + }); + it("logs one assignment when reconciliation creates the durable session", async () => { + const base = setup(); const config = base.config; base.log.close(); const dbPath = join(dirs.at(-1)!, "reconcile.db"); + const log = new EventLog(dbPath, () => ({ profile: "fable", reason: "claude_ready" })); + config.apps.planner.appActorId = "planner-actor"; config.apps.implementer.appActorId = "implementer-actor"; + config.reconcileRequestTimeoutMs = 1000; config.webhookBaseUrl = "http://daemon"; + const gateway = { ensureWebhookEnabled: async () => ({ matched: true, updated: false }), + listAgentSessions: async (app: string) => app === "planner" ? [{ id: "reconciled", app: "planner", issueId: "issue", issueIdentifier: "ENG-1" }] : [], + listDelegatedIssueAgentSessions: async () => [], listSessionActivitiesSince: async () => [] }; + const logger = { log: vi.fn(), error: vi.fn() }; + const worker = new ReconcileWorker(log, gateway as unknown as LinearGateway, config, { logger, now: () => 1000 }); + await worker.trigger(); await worker.trigger(); + const assignments = logger.log.mock.calls.map(call => JSON.parse(String(call[0]))).filter(entry => entry.event === "session_profile_assigned"); + expect(assignments).toEqual([{ event: "session_profile_assigned", linearSessionId: "reconciled", profile: "fable", reason: "claude_ready" }]); + await worker.stop(); log.close(); + }); it.each(["missing", "tampered", "stale", "malformed"])("AC2: rejects %s payloads without persistence", async kind => { const { log, server } = setup(); const address = await server.listen(); @@ -170,6 +200,21 @@ describe("webhook HTTP integration", () => { expect(await health.json()).toEqual({ ok: true }); expect(log.count()).toBe(1); expect(log.ackCount()).toBe(0); await server.close(); log.close(); }); + it("keeps provider health private and reports it only for the management key", async () => { + const { log, server, managementKey } = setup(); + log.setProviderState("claude", "ready", "eligible_2_failed_0", 1000); + log.setProviderCooldown("codex", 9000, "http_503", 2000); + const address = await server.listen(); + expect(await (await fetch(`http://127.0.0.1:${address.port}/healthz`)).json()).toEqual({ ok: true }); + expect(await (await fetch(`http://127.0.0.1:${address.port}/healthz`, + { headers: { Authorization: "Bearer wrong-key" } })).json()).toEqual({ ok: true }); + expect(await (await fetch(`http://127.0.0.1:${address.port}/healthz`, + { headers: { Authorization: `Bearer ${managementKey}` } })).json()).toEqual({ ok: true, providers: { + claude: { status: "ready", reason: "eligible_2_failed_0", updatedAt: 1000 }, + codex: { status: "cooldown", cooldownUntil: 9000 }, + } }); + await server.close(); log.close(); + }); it("rejects bodies over 1 MB with 413", async () => { const { log, server } = setup(); const address = await server.listen(); diff --git a/daemon/test/sessions.test.ts b/daemon/test/sessions.test.ts index fa0426e..61d5c5a 100644 --- a/daemon/test/sessions.test.ts +++ b/daemon/test/sessions.test.ts @@ -9,7 +9,8 @@ import { afterEach, describe, expect, it } from "vitest"; import type { Config } from "../src/config.js"; import { EventLog } from "../src/eventlog.js"; import type { LinearGateway, PostResult, ProgressContent, TerminalContent } from "../src/linear.js"; -import { SessionWorker } from "../src/sessions.js"; +import { WebhookServer } from "../src/server.js"; +import { ProviderReadinessPoller, SessionWorker, classifyProviderFailure, selectSessionProfile } from "../src/sessions.js"; const dirs: string[] = []; const oldMode = process.env.CLAUDE_FAKE_MODE; @@ -37,6 +38,8 @@ function setup() { planner: { name: "planner", webhookSecret: "p", staticToken: "p" }, implementer: { name: "implementer", webhookSecret: "i", staticToken: "i" } }, sessionsEnabled: true, worktreesRoot: join(dir, "trees"), targetRepoPath: repo, claudeArgv: [process.execPath, resolve("test/fixtures/fake-claude.mjs")], claudePermissionMode: "plan", claudeMaxTurns: 5, + cliproxyEnvFile: join(dir, "proxy.env"), cliproxyUrl: "http://127.0.0.1:1", providerProbeIntervalMs: 60_000, + providerStateStaleMs: 300_000, providerInitialProbeTimeoutMs: 5000, doPermissionMode:"plan",doMaxTurns:50,doMaxBudgetUsd:10, sessionConcurrency: 2, keepaliveMs: 30, linearApiKey: "linear-key", attachmentsEnabled: false, attachmentHosts: ["uploads.linear.app"] }; return { dir, log, config }; @@ -89,6 +92,206 @@ async function healthy(port: number, child: ChildProcess): Promise { } describe("SessionWorker", () => { + it("classifies only conservative provider failures", () => { + const base = { ok: false, isError: true, exitCode: 1, signal: null, permissionDenials: [], sawResult: false } as const; + expect(classifyProviderFailure({ ...base, spawnError: "spawn ECONNREFUSED" })).toMatchObject({ reason: "spawn_econnrefused" }); + expect(classifyProviderFailure({ ...base, stderrTail: "HTTP 403 from base URL" })).toMatchObject({ state: "auth_failure", reason: "http_403" }); + expect(classifyProviderFailure({ ...base, stderrTail: "ordinary model error" })).toBeUndefined(); + }); + it("probes readiness and selects Fable only for fresh ready state without cooldown", async () => { + const { log, config } = setup(); config.fableArgv = ["fable"]; + const logger = new CapturingLogger(); + await new ProviderReadinessPoller(log, config, logger, async () => ({ files: [ + { provider: "claude", disabled: false, failed: false, email: "secret@example.com" }, + ] })).probe(1000); + expect(selectSessionProfile(log, config, 1001)).toEqual({ profile: "fable", reason: "claude_ready" }); + expect(logger.entries()[0]).toEqual({ event: "provider_state_changed", provider: "claude", status: "ready", + reason: "eligible_1_failed_0", cooldownUntil: null }); + expect(logger.lines.join("\n")).not.toContain("secret@example.com"); + log.setProviderCooldown("claude", 5000, "http_503", 1100); + expect(selectSessionProfile(log, config, 1200)).toEqual({ profile: "sol", reason: "claude_cooldown" }); + log.close(); + }); + it("probes the management auth-files endpoint with the key read from disk", async () => { + const { log, config } = setup(); writeFileSync(config.cliproxyEnvFile, "CLIPROXY_MANAGEMENT_KEY=management-secret\n"); + let authorization: string | undefined; + const server = createServer((request, response) => { authorization = request.headers.authorization; + response.setHeader("content-type", "application/json"); response.end(JSON.stringify({ files: [{ provider: "claude", disabled: false, failed: true }] })); }); + await new Promise(resolveListen => server.listen(0, "127.0.0.1", resolveListen)); + config.cliproxyUrl = `http://127.0.0.1:${(server.address() as { port: number }).port}`; + const logger = new CapturingLogger(); await new ProviderReadinessPoller(log, config, logger).probe(1000); + expect(authorization).toBe("Bearer management-secret"); + expect(log.getProviderState("claude")).toMatchObject({ status: "ready", reason: "eligible_1_failed_1" }); + expect(logger.lines.join("\n")).not.toContain("management-secret"); + await new Promise(resolveClose => server.close(() => resolveClose())); log.close(); + }); + it("times out a stalled readiness fetch and never overlaps interval probes", async () => { + const { log, config } = setup(); writeFileSync(config.cliproxyEnvFile, "CLIPROXY_MANAGEMENT_KEY=management-secret\n"); + config.providerInitialProbeTimeoutMs = 50; config.providerProbeIntervalMs = 10; + let requests = 0; let active = 0; let maxActive = 0; + const server = createServer(request => { + requests++; active++; maxActive = Math.max(maxActive, active); + request.on("close", () => { active--; }); + // Intentionally never respond: AbortSignal.timeout must release the single-flight probe. + }); + await new Promise(resolveListen => server.listen(0, "127.0.0.1", resolveListen)); + config.cliproxyUrl = `http://127.0.0.1:${(server.address() as { port: number }).port}`; + const poller = new ProviderReadinessPoller(log, config, new CapturingLogger()); const startedAt = Date.now(); + poller.start(); const inFlight = poller.probe(); expect(poller.probe()).toBe(inFlight); await inFlight; poller.stop(); + expect(log.getProviderState("claude")).toMatchObject({ status: "not_ready", reason: "probe_timeout" }); + expect(Date.now() - startedAt).toBeLessThan(500); expect(requests).toBe(1); expect(maxActive).toBe(1); + server.closeAllConnections(); await new Promise(resolveClose => server.close(() => resolveClose())); log.close(); + }); + it("routes Fable, falls back once before establishment, and never flips after an id", async () => { + // AC5's pre/post-establishment states cannot both be reached from one webhook payload, so seed them directly. + const seeded = setup(); seeded.log.close(); + const log = new EventLog(seeded.config.dbPath, () => ({ profile: "fable", reason: "claude_ready" })); + const fixtureEmail = "secret@example.com"; const fixtureKey = "management-secret"; + seeded.config.linearApiKey = fixtureKey; + writeFileSync(seeded.config.cliproxyEnvFile, `CLIPROXY_MANAGEMENT_KEY=${fixtureKey}\n`); + const logger = new CapturingLogger(); + await new ProviderReadinessPoller(log, seeded.config, logger, async () => ({ files: [ + { provider: "claude", disabled: false, failed: false, email: fixtureEmail }, + ] })).probe(); + seeded.config.fableArgv = [process.execPath, resolve("test/fixtures/fake-claude.mjs"), "--fable-launcher"]; + process.env.CLAUDE_FAKE_ARGS_FILE = join(seeded.dir, "routing.jsonl"); + process.env.CLAUDE_FAKE_MODE = "provider-fail-pre-id"; + append(log, "fable-created", "fable-session", "created"); + let worker = new SessionWorker(log, new Poster() as unknown as LinearGateway, seeded.config, { pollMs: 10, logger }); + worker.start(); await waitFor(() => log.turnStates()[0]?.status === "failed"); await worker.stop(); + expect(log.getSession("fable-session")).toMatchObject({ profile: "sol", profileFallback: 1 }); + let starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n").map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(2); expect(starts[0].args).toContain("--fable-launcher"); expect(starts[1].args).not.toContain("--fable-launcher"); + const firstFailureEntries = logger.entries(); + const classified = firstFailureEntries.find(entry => entry.event === "provider_failure_classified"); + const fallback = firstFailureEntries.find(entry => entry.event === "profile_fallback"); + expect(classified).toEqual({ event: "provider_failure_classified", linearSessionId: "fable-session", + profile: "fable", provider: "claude", classifiedState: "transport_failure", + reason: "connection_refused", cooldownUntil: expect.any(Number) }); + expect(fallback).toEqual({ event: "profile_fallback", linearSessionId: "fable-session", from: "fable", to: "sol", + provider: "claude", classifiedState: "transport_failure", reason: "connection_refused", + cooldownUntil: classified?.cooldownUntil }); + + process.env.CLAUDE_FAKE_MODE = "happy"; + append(log, "established-created", "established-fable", "created", "issue-2", "ENG-43"); + worker = new SessionWorker(log, new Poster() as unknown as LinearGateway, seeded.config, { pollMs: 10, logger }); + worker.start(); await waitFor(() => log.turnStates()[1]?.status === "done"); await worker.stop(); + expect(log.getSession("established-fable")).toMatchObject({ profile: "fable", claudeSessionId: "claude-session-1" }); + process.env.CLAUDE_FAKE_MODE = "provider-fail-post-id"; + append(log, "established-prompt", "established-fable", "prompted", "issue-2", "ENG-43"); + worker = new SessionWorker(log, new Poster() as unknown as LinearGateway, seeded.config, { pollMs: 10, logger }); + worker.start(); await waitFor(() => log.turnStates()[2]?.status === "failed"); await worker.stop(); + expect(log.getSession("established-fable")?.profile).toBe("fable"); + starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n").map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(4); + const healthServer = new WebhookServer({ config: seeded.config, log, logger }); + const healthAddress = await healthServer.listen(); + const healthBody = await (await fetch(`http://127.0.0.1:${healthAddress.port}/healthz`, + { headers: { Authorization: `Bearer ${fixtureKey}` } })).json(); + const routingNames = new Set(["session_profile_assigned", "provider_state_changed", "provider_failure_classified", + "profile_fallback", "profile_launcher_unconfigured", "legacy_session_profile_defaulted"]); + const capturedRoutingEvents = logger.entries().filter(entry => routingNames.has(String(entry.event))); + expect([...capturedRoutingEvents, healthBody].every(payload => { + const serialized = JSON.stringify(payload); return !serialized.includes(fixtureEmail) && !serialized.includes(fixtureKey); + })).toBe(true); + await healthServer.close(); + log.close(); + }); + it("classifies both pools when Fable and its one Sol retry fail before establishment", async () => { + const seeded = setup(); seeded.log.close(); const poster = new Poster(); const logger = new CapturingLogger(); + const log = new EventLog(seeded.config.dbPath, () => ({ profile: "fable", reason: "claude_ready" })); + seeded.config.fableArgv = [process.execPath, resolve("test/fixtures/fake-claude.mjs"), "--fable-launcher"]; + process.env.CLAUDE_FAKE_ARGS_FILE = join(seeded.dir, "two-provider-failures.jsonl"); + process.env.CLAUDE_FAKE_MODE = "provider-fail-pre-id"; + append(log, "d1", "session", "created"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, seeded.config, { pollMs: 10, logger }); + worker.start(); await waitFor(() => log.turnStates()[0]?.status === "failed"); await worker.stop(); + expect(log.getSession("session")).toMatchObject({ profile: "sol", profileFallback: 1, claudeSessionId: null }); + expect(log.getProviderState("claude")).toMatchObject({ status: "cooldown", reason: "connection_refused" }); + expect(log.getProviderState("codex")).toMatchObject({ status: "cooldown", reason: "connection_refused" }); + expect(logger.entries().filter(entry => entry.event === "provider_failure_classified")) + .toEqual([expect.objectContaining({ linearSessionId: "session", profile: "fable", provider: "claude" }), + expect.objectContaining({ linearSessionId: "session", profile: "sol", provider: "codex" })]); + expect(logger.entries().filter(entry => entry.event === "profile_fallback")).toHaveLength(1); + expect(poster.posts.filter(post => !post.ephemeral)).toHaveLength(1); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n") + .map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(2); expect(starts[0].args).toContain("--fable-launcher"); + expect(starts[1].args).not.toContain("--fable-launcher"); log.close(); + }); + it("persists Fable for planner and implementer before their first launch", async () => { + const seeded = setup(); seeded.log.close(); + const log = new EventLog(seeded.config.dbPath, () => ({ profile: "fable", reason: "claude_ready" })); + seeded.config.fableArgv = [process.execPath, resolve("test/fixtures/fake-claude.mjs"), "--fable-launcher"]; + process.env.CLAUDE_FAKE_ARGS_FILE = join(seeded.dir, "fable-starts.jsonl"); process.env.CLAUDE_FAKE_MODE = "happy"; + append(log, "planner-fable", "planner-fable", "created", "issue-p", "ENG-41"); + appendImplementer(log, "implementer-fable", "implementer-fable", "issue-i", "ENG-42"); + const worker = new SessionWorker(log, new Poster() as unknown as LinearGateway, seeded.config, { pollMs: 10 }); + worker.start(); await waitFor(() => log.turnStates().every(turn => turn.status === "done")); await worker.stop(); + expect(log.getSession("planner-fable")?.profile).toBe("fable"); expect(log.getSession("implementer-fable")?.profile).toBe("fable"); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n").map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(2); expect(starts.every(row => row.args.includes("--fable-launcher"))).toBe(true); log.close(); + }); + it("persists Sol and uses the Sol launcher when Claude is cooling down", async () => { + const seeded = setup(); seeded.log.close(); seeded.config.fableArgv = [process.execPath, resolve("test/fixtures/fake-claude.mjs"), "--fable-launcher"]; + let log: EventLog; log = new EventLog(seeded.config.dbPath, () => selectSessionProfile(log, seeded.config, 1000)); + log.setProviderCooldown("claude", 5000, "http_503", 900); + process.env.CLAUDE_FAKE_ARGS_FILE = join(seeded.dir, "sol-starts.jsonl"); process.env.CLAUDE_FAKE_MODE = "happy"; + append(log, "cooldown-created", "sol-session", "created"); + const worker = new SessionWorker(log, new Poster() as unknown as LinearGateway, seeded.config, { pollMs: 10 }); + worker.start(); await waitFor(() => log.turnStates()[0]?.status === "done"); await worker.stop(); + expect(log.getSession("sol-session")?.profile).toBe("sol"); + const start = JSON.parse(readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").split("\n")[0]!); + expect(start.args).not.toContain("--fable-launcher"); log.close(); + }); + it("persists Sol and uses the Sol launcher when Claude is explicitly not ready", async () => { + const seeded = setup(); seeded.log.close(); seeded.config.fableArgv = [process.execPath, resolve("test/fixtures/fake-claude.mjs"), "--fable-launcher"]; + let log: EventLog; log = new EventLog(seeded.config.dbPath, () => selectSessionProfile(log, seeded.config)); + log.setProviderState("claude", "not_ready", "no_eligible_claude_failed_0", Date.now()); + process.env.CLAUDE_FAKE_ARGS_FILE = join(seeded.dir, "not-ready-starts.jsonl"); process.env.CLAUDE_FAKE_MODE = "happy"; + append(log, "not-ready-created", "not-ready-session", "created"); + const worker = new SessionWorker(log, new Poster() as unknown as LinearGateway, seeded.config, { pollMs: 10 }); + worker.start(); await waitFor(() => log.turnStates()[0]?.status === "done"); await worker.stop(); + expect(log.getSession("not-ready-session")?.profile).toBe("sol"); + const start = JSON.parse(readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").split("\n")[0]!); + expect(start.args).not.toContain("--fable-launcher"); log.close(); + }); + it.each([ + { criterion: "AC1", status: "ready", reason: "eligible_1_failed_0", expectedProfile: "fable", usesFable: true }, + { criterion: "AC3", status: "not_ready", reason: "no_eligible_claude_failed_0", expectedProfile: "sol", usesFable: false }, + ])("$criterion signed webhook persists $expectedProfile and launches it", async ({ status, reason, expectedProfile, usesFable }) => { + const seeded = setup(); seeded.log.close(); seeded.config.fableArgv = [process.execPath, resolve("test/fixtures/fake-claude.mjs"), "--fable-launcher"]; + let log: EventLog; log = new EventLog(seeded.config.dbPath, () => selectSessionProfile(log, seeded.config)); + log.setProviderState("claude", status, reason, Date.now()); + process.env.CLAUDE_FAKE_ARGS_FILE = join(seeded.dir, `webhook-${expectedProfile}.jsonl`); process.env.CLAUDE_FAKE_MODE = "happy"; + const worker = new SessionWorker(log, new Poster() as unknown as LinearGateway, seeded.config, { pollMs: 10 }); + const server = new WebhookServer({ config: seeded.config, log, onInserted: () => worker.trigger(), logger: new CapturingLogger() }); + worker.start(); const address = await server.listen(); + const body = JSON.stringify({ webhookTimestamp: Date.now(), action: "created", promptContext: "route me", + agentSession: { id: `webhook-${expectedProfile}`, issue: { id: `issue-${expectedProfile}`, identifier: expectedProfile === "fable" ? "ENG-51" : "ENG-52" } } }); + const signature = createHmac("sha256", seeded.config.apps.planner.webhookSecret).update(body).digest("hex"); + expect((await fetch(`http://127.0.0.1:${address.port}/webhook/planner`, { method: "POST", body, + headers: { "Linear-Signature": signature, "Linear-Delivery": `delivery-${expectedProfile}` } })).status).toBe(200); + await waitFor(() => log.turnStates()[0]?.status === "done"); await worker.stop(); await server.close(); + expect(log.getSession(`webhook-${expectedProfile}`)?.profile).toBe(expectedProfile); + const start = JSON.parse(readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").split("\n")[0]!); + expect(start.args.includes("--fable-launcher")).toBe(usesFable); log.close(); + }); + it("fails an established Fable session closed when its launcher is unconfigured", async () => { + // AC7 requires an already-established Fable row, which a single webhook payload cannot create while FABLE_BIN is unset. + const seeded = setup(); seeded.log.close(); + const log = new EventLog(seeded.config.dbPath, () => ({ profile: "fable", reason: "ready" })); + append(log, "created-fable", "fable-session", "created"); + log.updateClaudeSessionId("fable-session", "established-id"); + const logger = new CapturingLogger(); + const poster = new Poster(); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, seeded.config, { pollMs: 10, logger }); + worker.start(); await waitFor(() => log.turnStates()[0]?.status === "failed"); await worker.stop(); + expect(log.getSession("fable-session")).toMatchObject({ profile: "fable", claudeSessionId: "established-id" }); + expect(logger.entries()).toContainEqual({ event: "profile_launcher_unconfigured", linearSessionId: "fable-session", profile: "fable" }); + expect(poster.posts.map(post => activityBody(post.content)).filter(Boolean).join("\n")).toContain("FABLE_BIN"); + log.close(); + }); it("phase3 AC1/AC2/AC3: do-mode reuses worktree, starts fresh, uses literal prompt, and posts PR URL",async()=>{ const {dir,log,config}=setup(); const poster=new Poster(); process.env.CLAUDE_FAKE_ARGS_FILE=join(dir,"args.jsonl"); append(log,"planner","planner-session","created"); let worker=new SessionWorker(log,poster as unknown as LinearGateway,config,{pollMs:10,reconcileMs:20});worker.start(); @@ -597,11 +800,17 @@ describe("SessionWorker", () => { request.on("end", () => { requests.push(JSON.parse(Buffer.concat(chunks).toString())); response.writeHead(200, { "Content-Type": "application/json" }); response.end(JSON.stringify({ data: { agentActivityCreate: { success: true, agentActivity: { id: "a" } } } })); }); }); await new Promise(resolve => graphql.listen(0, "127.0.0.1", resolve)); const graphqlPort = (graphql.address() as { port: number }).port; + const proxy = createServer((_request, response) => { response.setHeader("content-type", "application/json"); + response.end(JSON.stringify({ files: [{ provider: "claude", disabled: false, failed: false }] })); }); + await new Promise(resolve => proxy.listen(0, "127.0.0.1", resolve)); const proxyPort = (proxy.address() as { port: number }).port; + const proxyEnv = join(dir, "proxy.env"); writeFileSync(proxyEnv, "CLIPROXY_MANAGEMENT_KEY=management-key\n"); process.env.CLAUDE_FAKE_ARGS_FILE = join(dir, "args.jsonl"); const env = { ...process.env, DAEMON_TEST_MODE: "1", PORT: String(port), BIND_ADDR: "127.0.0.1", DB_PATH: config.dbPath, PLANNER_WEBHOOK_SECRET: "planner-secret", PLANNER_LINEAR_TOKEN: "p", IMPLEMENTER_WEBHOOK_SECRET: "i", IMPLEMENTER_LINEAR_TOKEN: "i", SESSIONS_ENABLED: "1", TARGET_REPO_PATH: config.targetRepoPath!, WORKTREES_ROOT: config.worktreesRoot, LINEAR_API_KEY: "key", CLAUDE_BIN: `${process.execPath} ${resolve("test/fixtures/fake-claude.mjs")}`, CLAUDE_PERMISSION_MODE: "plan", ATTACHMENTS_ENABLED: "0", + FABLE_BIN: `${process.execPath} ${resolve("test/fixtures/fake-claude.mjs")} --fable-launcher`, CLIPROXY_ENV_FILE: proxyEnv, + CLIPROXY_URL: `http://127.0.0.1:${proxyPort}`, PROVIDER_INITIAL_PROBE_TIMEOUT_MS: "1000", LINEAR_GRAPHQL_URL: `http://127.0.0.1:${graphqlPort}/graphql` }; const launch = () => spawn(process.execPath, [resolve("dist/index.js")], { env, stdio: "ignore" }); const send = async (delivery: string, body: Record) => { const encoded = JSON.stringify({ webhookTimestamp: Date.now(), ...body }); @@ -611,13 +820,18 @@ describe("SessionWorker", () => { let child = launch(); await healthy(port, child); await send("d1", { action: "created", promptContext: "plan", agentSession: { id: "session", issue: { id: "issue", identifier: "ENG-7" } } }); await waitFor(() => { const db = new EventLog(config.dbPath); const done = db.turnStates()[0]?.status === "done"; db.close(); return done; }); + { const db = new EventLog(config.dbPath); expect(db.getProviderState("claude")).toMatchObject({ status: "ready" }); + expect(db.getSession("session")?.profile).toBe("fable"); db.close(); } child.kill("SIGKILL"); await new Promise(resolve => child.once("close", resolve)); child = launch(); await healthy(port, child); await send("d2", { action: "prompted", agentActivity: { body: "continue" }, agentSession: { id: "session" } }); await waitFor(() => { const db = new EventLog(config.dbPath); const done = db.turnStates()[1]?.status === "done"; db.close(); return done; }); child.kill("SIGTERM"); await new Promise(resolve => child.once("close", resolve)); const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n").map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts.map(row => row.args.includes("--fable-launcher"))).toEqual([true, true]); expect(starts[1].args).toContain("--resume"); expect(starts[1].args).toContain("claude-session-1"); expect(requests.length).toBeGreaterThan(2); + const persisted = new EventLog(config.dbPath); expect(persisted.getSession("session")?.profile).toBe("fable"); persisted.close(); await new Promise(resolve => graphql.close(() => resolve())); + await new Promise(resolve => proxy.close(() => resolve())); }, 10_000); it("persists a compacted session id and never posts progress after the terminal response", async () => { const { dir, log, config } = setup(); process.env.CLAUDE_FAKE_MODE = "new-id"; process.env.CLAUDE_FAKE_ARGS_FILE = join(dir, "args.jsonl"); @@ -636,4 +850,181 @@ describe("SessionWorker", () => { expect(poster.posts.at(-1)?.ephemeral).toBe(false); await worker.stop(); log.close(); }); + it.each(["rate-limit-rejected", "out-of-credits", "api-retry-exhausted", "result-429"])( + "AC3/AC4: starts on Claude and retries %s exactly once on Claudex", async mode => { + const { dir, log, config } = setup(); const poster = new Poster(); + process.env.CLAUDE_FAKE_ARGS_FILE = join(dir, "args.jsonl"); process.env.CLAUDE_FAKE_MODE = mode; + config.claudexArgv = [...config.claudeArgv, "--claudex-runtime"]; + config.claudexEnv = { CLAUDE_FAKE_MODE: "happy" }; + append(log, "d1", "session", "created"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20 }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "done"); await worker.stop(); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n") + .map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(2); expect(starts[0].args).not.toContain("--claudex-runtime"); + expect(starts[1].args).toContain("--claudex-runtime"); expect(starts[1].args).not.toContain("--resume"); + expect(log.getSession("session")).toMatchObject({ runtime: "claudex", fallbackCause: expect.any(String) }); + log.close(); + }); + it("routes Fable first, gives structured capacity fallback priority, and persists Claudex", async () => { + const seeded = setup(); seeded.log.close(); const poster = new Poster(); const logger = new CapturingLogger(); + const log = new EventLog(seeded.config.dbPath, () => ({ profile: "fable", reason: "claude_ready" })); + process.env.CLAUDE_FAKE_ARGS_FILE = join(seeded.dir, "fable-capacity.jsonl"); process.env.CLAUDE_FAKE_MODE = "result-429"; + seeded.config.fableArgv = [...seeded.config.claudeArgv, "--fable-launcher"]; + seeded.config.claudexArgv = [...seeded.config.claudeArgv, "--claudex-runtime"]; + seeded.config.claudexEnv = { CLAUDE_FAKE_MODE: "happy" }; + append(log, "d1", "session", "created"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, seeded.config, { pollMs: 10, reconcileMs: 20, logger }); + worker.start(); await waitFor(() => log.turnStates()[0]?.status === "done"); + process.env.CLAUDE_FAKE_MODE = "crash"; append(log, "d2", "session", "prompted"); worker.trigger(); + await waitFor(() => log.turnStates()[1]?.status === "done"); await worker.stop(); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n") + .map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(3); expect(starts[0].args).toContain("--fable-launcher"); + expect(starts[1].args).toContain("--claudex-runtime"); expect(starts[1].args).not.toContain("--resume"); + expect(starts[2].args).toEqual(expect.arrayContaining(["--claudex-runtime", "--resume", "claude-session-1"])); + expect(log.getSession("session")).toMatchObject({ profile: "fable", profileFallback: null, runtime: "claudex" }); + expect(logger.entries().find(entry => entry.event === "session_capacity_fallback")) + .toMatchObject({ originalRuntime: "fable", fallbackRuntime: "claudex" }); + expect(logger.entries().filter(entry => entry.event === "provider_failure_classified" || entry.event === "profile_fallback")).toHaveLength(0); + log.close(); + }); + it("AC5/AC6/AC8: resumed /do fallback is fresh, records the downgrade, and persists Claudex", async () => { + const { dir, log, config } = setup(); const poster = new Poster(); const logger = new CapturingLogger(); + process.env.CLAUDE_FAKE_ARGS_FILE = join(dir, "args.jsonl"); process.env.CLAUDE_FAKE_MODE = "do-pr"; + config.claudexArgv = [...config.claudeArgv, "--claudex-runtime"]; + config.claudexEnv = { CLAUDE_FAKE_MODE: "happy", ENABLE_TOOL_SEARCH: "true" }; + appendImplementer(log, "i1", "implementer-session"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20, logger }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "done"); + process.env.CLAUDE_FAKE_MODE = "capacity-after-session"; + log.append({ deliveryId: "i2", app: "implementer", action: "prompted", agentSessionId: "implementer-session", receivedAt: Date.now(), + rawBody: Buffer.from(JSON.stringify({ action: "prompted", agentActivity: { body: "continue" }, agentSession: { id: "implementer-session" } })) }); + worker.trigger(); await waitFor(() => log.turnStates()[1]?.status === "done"); + process.env.CLAUDE_FAKE_MODE = "crash"; + log.append({ deliveryId: "i3", app: "implementer", action: "prompted", agentSessionId: "implementer-session", receivedAt: Date.now(), + rawBody: Buffer.from(JSON.stringify({ action: "prompted", agentActivity: { body: "next" }, agentSession: { id: "implementer-session" } })) }); + worker.trigger(); await waitFor(() => log.turnStates()[2]?.status === "done"); await worker.stop(); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n") + .map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(4); + const retryArgs = starts[2].args as string[]; expect(retryArgs).toContain("--claudex-runtime"); expect(retryArgs).not.toContain("--resume"); + const retryPrompt = retryArgs[retryArgs.indexOf("-p") + 1]; + expect(retryPrompt).toContain("/do ENG-42\n\nResume where we left off."); + expect(retryPrompt).toContain("original runtime claude; fallback runtime claudex"); + expect(retryPrompt).toContain("effective review lanes are single/Codex-only regardless of any dual request"); + expect(starts[3].args).toEqual(expect.arrayContaining(["--claudex-runtime", "--resume", "claude-session-1"])); + expect(log.getSession("implementer-session")).toMatchObject({ runtime: "claudex", claudeSessionId: "claude-session-1" }); + expect(poster.posts.filter(post => post.ephemeral && activityBody(post.content)?.includes("retrying once with Claudex"))).toHaveLength(1); + expect(logger.entries().filter(entry => entry.event === "session_capacity_fallback")).toHaveLength(1); + log.close(); + }); + it.each([ + ["error-result-exit", "Planner turn failed: Claude exited with code 11"], + ["denied", "Planner turn failed: Claude permission was denied"], + ["no-result", "Planner turn failed: Claude exited without a result"], + ["non-capacity-api-error", "Planner turn failed: Claude exited with code 1"], + ])("AC7: %s stays on Claude and preserves its terminal classification", async (mode, detail) => { + const { dir, log, config } = setup(); const poster = new Poster(); const logger = new CapturingLogger(); + process.env.CLAUDE_FAKE_ARGS_FILE = join(dir, "args.jsonl"); process.env.CLAUDE_FAKE_MODE = mode; + config.claudexArgv = [...config.claudeArgv, "--claudex-runtime"]; config.claudexEnv = { CLAUDE_FAKE_MODE: "happy" }; + append(log, "d1", "session", "created"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20, logger }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "failed"); await worker.stop(); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n") + .map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(1); expect(starts[0].args).not.toContain("--claudex-runtime"); + expect(poster.posts.filter(post => !post.ephemeral).map(post => activityBody(post.content))).toEqual([detail]); + expect(logger.entries().filter(entry => entry.event === "session_capacity_fallback")).toHaveLength(0); + expect(logger.entries().filter(entry => entry.event === "session_turn_failed")) + .toEqual([expect.objectContaining({ attempts: 1, error: detail.replace("Planner turn failed: ", "") })]); + expect(log.getSession("session")).toMatchObject({ runtime: "claude", fallbackCause: null }); log.close(); + }); + it("AC7: signal death stays on Claude and preserves the queued terminal classification", async () => { + const { dir, log, config } = setup(); const poster = new Poster(); const logger = new CapturingLogger(); + process.env.CLAUDE_FAKE_ARGS_FILE = join(dir, "args.jsonl"); process.env.CLAUDE_FAKE_MODE = "hang"; + config.claudexArgv = [...config.claudeArgv, "--claudex-runtime"]; config.claudexEnv = { CLAUDE_FAKE_MODE: "happy" }; + append(log, "d1", "session", "created"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20, logger }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "running" && readdirSync(dir).includes("args.jsonl")); + await worker.stop(); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n") + .map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(1); expect(starts[0].args).not.toContain("--claudex-runtime"); + expect(log.pendingTurnActivities().map(activity => activity.body)) + .toEqual(["Planner turn failed: Claude exited on SIGTERM"]); + expect(logger.entries().filter(entry => entry.event === "session_capacity_fallback")).toHaveLength(0); + expect(logger.entries().filter(entry => entry.event === "session_turn_failed")) + .toEqual([expect.objectContaining({ attempts: 1, error: "Claude exited on SIGTERM" })]); + expect(log.getSession("session")).toMatchObject({ runtime: "claude", fallbackCause: null }); log.close(); + }); + it("AC7: missing Claude binary does not start Claudex and preserves the spawn failure", async () => { + const { dir, log, config } = setup(); const poster = new Poster(); const logger = new CapturingLogger(); + const argsFile = join(dir, "args.jsonl"); process.env.CLAUDE_FAKE_ARGS_FILE = argsFile; + config.claudeArgv = [join(dir, "missing-claude")]; + config.claudexArgv = [process.execPath, resolve("test/fixtures/fake-claude.mjs"), "--claudex-runtime"]; + config.claudexEnv = { CLAUDE_FAKE_MODE: "happy" }; + append(log, "d1", "session", "created"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20, logger }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "failed"); await worker.stop(); + expect(readdirSync(dir)).not.toContain("args.jsonl"); + const terminals = poster.posts.filter(post => !post.ephemeral).map(post => activityBody(post.content)); + expect(terminals).toHaveLength(1); expect(terminals[0]).toContain("Planner turn failed: spawn"); expect(terminals[0]).toContain("ENOENT"); + expect(logger.entries().filter(entry => entry.event === "session_capacity_fallback")).toHaveLength(0); + expect(logger.entries().filter(entry => entry.event === "session_turn_failed")) + .toEqual([expect.objectContaining({ attempts: 1, error: expect.stringMatching(/^spawn .* ENOENT$/) })]); + expect(log.getSession("session")).toMatchObject({ runtime: "claude", fallbackCause: null }); log.close(); + }); + it("reports a sanitized usage limit when fallback is unavailable", async () => { + const { log, config } = setup(); const poster = new Poster(); process.env.CLAUDE_FAKE_MODE = "rate-limit-rejected"; + append(log, "d1", "session", "created"); const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20 }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "failed"); await worker.stop(); + expect(log.turnStates()[0]?.status).toBe("failed"); + expect(poster.posts.some(post => !post.ephemeral && activityBody(post.content)?.includes("Claude hit a usage limit"))).toBe(true); + log.close(); + }); + it("cleans up progress and keepalive when the Claudex fallback runner rejects", async () => { + const { log, config } = setup(); const poster = new Poster(); const logger = new CapturingLogger(); + process.env.CLAUDE_FAKE_MODE = "rate-limit-rejected"; + config.claudexArgv = []; config.keepaliveMs = 10; + append(log, "d1", "session", "created"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20, logger }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "failed"); + const postsAtTerminal = poster.posts.length; + expect(poster.posts.at(-1)?.ephemeral).toBe(false); + await new Promise(resolve => setTimeout(resolve, 120)); + expect(poster.posts).toHaveLength(postsAtTerminal); + expect(logger.entries().find(entry => entry.event === "session_turn_unhandled")) + .toMatchObject({ attempts: 1, error: expect.stringContaining("Claude argv is empty") }); + await worker.stop(); log.close(); + }); + it("degrades a persisted Claudex session to a fresh Claude turn when CLAUDEX_BIN is removed", async () => { + const { dir, log, config } = setup(); const poster = new Poster(); const logger = new CapturingLogger(); + process.env.CLAUDE_FAKE_ARGS_FILE = join(dir, "args.jsonl"); + append(log, "d1", "session", "created"); + const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20, logger }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "done"); + log.recordRuntimeFallback("session", "persisted-claudex-id", "prior capacity", Date.now()); + append(log, "d2", "session", "prompted"); worker.trigger(); + await waitFor(() => log.turnStates()[1]?.status === "done"); await worker.stop(); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n") + .map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts).toHaveLength(2); expect(starts[1].args).not.toContain("--resume"); + expect(logger.entries().find(entry => entry.event === "session_runtime_degraded")) + .toMatchObject({ configuredRuntime: "claudex", effectiveRuntime: "claude", reason: "CLAUDEX_BIN is not configured" }); + expect(log.getSession("session")).toMatchObject({ runtime: "claudex", claudeSessionId: "persisted-claudex-id" }); + log.close(); + }); + it("discards a failed fallback session id and retries Claude fresh next turn", async () => { + const { dir, log, config } = setup(); const poster = new Poster(); + process.env.CLAUDE_FAKE_ARGS_FILE = join(dir, "args.jsonl"); process.env.CLAUDE_FAKE_MODE = "rate-limit-rejected"; + config.claudexArgv = [...config.claudeArgv, "--claudex-runtime"]; config.claudexEnv = { CLAUDE_FAKE_MODE: "capacity-after-session" }; + append(log, "d1", "session", "created"); const worker = new SessionWorker(log, poster as unknown as LinearGateway, config, { pollMs: 10, reconcileMs: 20 }); worker.start(); + await waitFor(() => log.turnStates()[0]?.status === "failed"); + expect(log.getSession("session")).toMatchObject({ runtime: "claude", claudeSessionId: null }); + process.env.CLAUDE_FAKE_MODE = "happy"; append(log, "d2", "session", "prompted"); worker.trigger(); + await waitFor(() => log.turnStates()[1]?.status === "done"); await worker.stop(); + const starts = readFileSync(process.env.CLAUDE_FAKE_ARGS_FILE, "utf8").trim().split("\n").map(line => JSON.parse(line)).filter(row => row.phase === "start"); + expect(starts[2].args).not.toContain("--resume"); expect(starts[2].args).not.toContain("--claudex-runtime"); log.close(); + }); }); diff --git a/docs/linear-agent-daemon-setup.md b/docs/linear-agent-daemon-setup.md index 496d9b0..361ae66 100644 --- a/docs/linear-agent-daemon-setup.md +++ b/docs/linear-agent-daemon-setup.md @@ -24,8 +24,9 @@ either agent resume the same Claude session. - Admin access to the Linear workspace where the agents will live. - A GitHub bot identity with a fine-grained PAT scoped to the target repository (contents + pull-requests write). -- Anthropic (Claude Code) and OpenAI (Codex CLI) accounts for the agent - sessions to bill against. +- An OpenAI account with Codex access (ChatGPT Plus/Pro) for both the + `claudex` proxy and standalone Codex CLI lanes. The provisioner installs + Claude Code as the harness; it does not require Anthropic authentication. - A target repository that meets the requirements in [Target repository requirements](#target-repository-requirements). @@ -39,10 +40,11 @@ cd daemon sudo DAEMON_HOST=linear-agent.example.com ops/provision.sh "$PWD" ``` -This installs Node 22, pnpm, git, the GitHub and Codex CLIs, Claude Code, -Caddy (TLS termination, 1 MB body cap, loopback-only proxy to the -daemon), UFW (SSH + HTTPS only), the dedicated low-privilege -`linear-daemon` user, and the systemd unit. Follow the **Host and DNS** +This installs Node 22, pnpm, git, the GitHub and Codex CLIs, the Claude +Code harness, pinned CLIProxyAPI, a real `claudex` executable, Caddy (TLS +termination, 1 MB body cap, loopback-only proxy to the daemon), UFW (SSH + +HTTPS only), the dedicated low-privilege `linear-daemon` user, and both +systemd units. Follow the **Host and DNS** and **Host checks** sections of `daemon/ops/runbook.md` for the SSH hardening steps and post-provision verification commands. @@ -140,11 +142,17 @@ commands; the checklist is: 1. Bot git identity + GitHub fine-grained PAT (HTTPS credential store, mode 600), verified with a dry-run push and one disposable draft PR. -2. `claude` authenticated as the service user (headless flow), verified - with `sudo -u linear-daemon -H claude --version` and a trivial `-p` - turn. -3. `codex` authenticated, verified with `codex exec --version`. -4. The Linear API key — only ever in the env file below. +2. CLIProxyAPI's one-time OpenAI Codex OAuth completed as `linear-daemon` + with `--codex-login --no-browser`; credentials must appear under + `/var/lib/linear-agent-daemon/.cli-proxy-api/`. +3. `claudex` verified with a trivial `-p` turn that returns through + GPT-5.6 Sol. No Anthropic login is installed on this host. +4. The standalone `codex` CLI authenticated and verified with + `codex exec --version`. +5. The Linear API key — only ever in the env file below. + +The runbook's **Planner credentials and repository** section contains the +exact OAuth, model-list, and `claudex` smoke commands. ## 5. Configure and start @@ -167,7 +175,7 @@ LINEAR_API_KEY=... SESSIONS_ENABLED=1 TARGET_REPO_PATH=/var/lib/linear-agent-daemon/repos/ WORKTREES_ROOT=/var/lib/linear-agent-daemon/worktrees -CLAUDE_BIN=/var/lib/linear-agent-daemon/.local/bin/claude +CLAUDE_BIN=/var/lib/linear-agent-daemon/.local/bin/claudex CLAUDE_PERMISSION_MODE=bypassPermissions CLAUDE_MAX_TURNS=100 DO_PERMISSION_MODE=bypassPermissions