From af61ebd09122144610a5c17168c34c3ed6e983f9 Mon Sep 17 00:00:00 2001 From: Blake Bauman Date: Wed, 11 Nov 2026 15:29:18 -0700 Subject: [PATCH 1/2] Refuse host shell on a Docker agent by default, and always for a fork's pull request A Docker agent ran a step in a container only when the step named an image and dropped every other step to its own shell, with no way to refuse. On the agent-docker Compose profile that shell is root beside the engine's socket, so a project writer, or a fork's author through an imageless step, had root on the host. FIBER_AGENT_ALLOW_HOST_SHELL now decides it: unset, it is false on a Docker agent and true on a shell agent, and anything but true/false/1/0 stops the agent at startup. The offer carries `untrusted` (serde default), and the agent never runs such a step on its host shell whatever the flag says, so a fork's pull request fails on a shell agent. The decision is one pure function, execution_mode, taken before any workspace prep. PROTOCOL_VERSION is 3. The agent-docker profile pins the flag to false and adds the docker=true label. smoke-runs expects the refusal and runs the fork's pull request in a container where an engine is available. Co-Authored-By: Claude Fable 5.2 --- .env.example | 3 + CHANGELOG.md | 47 +++++ SECURITY.md | 22 ++- crates/fiber-agent/src/main.rs | 304 ++++++++++++++++++++++++++++++++- crates/fiber-api/src/ws.rs | 3 + crates/fiber-proto/src/lib.rs | 60 ++++++- deploy/.env.example | 5 +- deploy/docker-compose.yml | 10 +- docs/agents.md | 65 ++++++- docs/configuration.md | 6 +- docs/pipeline-yaml.md | 2 +- docs/triggers.md | 35 +++- scripts/dev-env.sh | 4 + scripts/install-agent.sh | 16 +- scripts/smoke_runs.py | 71 ++++++-- 15 files changed, 600 insertions(+), 53 deletions(-) diff --git a/.env.example b/.env.example index 1692918..f5c6287 100644 --- a/.env.example +++ b/.env.example @@ -25,6 +25,9 @@ FIBER_ADMIN_PASSWORD=fiber FIBER_API_URL=http://127.0.0.1:18080 FIBER_AGENT_USE_DOCKER=false +# Unset = true for a shell agent (USE_DOCKER=false), false for a Docker agent, which then +# refuses a step without `image:`. Fork pull request steps never run on the host shell. +# FIBER_AGENT_ALLOW_HOST_SHELL=true FIBER_AGENT_NAME=local FIBER_AGENT_LABELS=os=linux FIBER_AGENT_CONCURRENCY=1 diff --git a/CHANGELOG.md b/CHANGELOG.md index 7d1da45..fef245a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,8 +6,49 @@ minor versions may carry breaking changes. ## [Unreleased] +**Breaking: a Docker agent no longer runs a step without `image:`.** It used to run it on +the agent's host shell without being asked; now the step fails, saying so. Add `image:` to +the step, or put the old behaviour back with one line on the agent: +`FIBER_AGENT_ALLOW_HOST_SHELL=true`. `FIBER_AGENT_USE_DOCKER` defaults to `true` in the +binary, so an agent started without it is a Docker agent and is affected. Shell agents +(`FIBER_AGENT_USE_DOCKER=false` — the published image, the plain Compose `agent` profile, +`install-agent.sh` without `--docker`, `make agent`) are unchanged for the project's own +runs. **A fork's pull request now runs in containers or +fails** — on a shell agent it fails, every step, and no setting changes that. + +Wire: `Offer` gains `untrusted` (defaulted, so 0.6.7 agents and servers keep working with +the new ones) and `PROTOCOL_VERSION` is now `3`. The fork-PR control is the agent's: +upgrade the agents, not only the API. + ### Security +- **A step without `image:` ran on the host of a Docker agent, as the agent's user.** An + agent with `FIBER_AGENT_USE_DOCKER=true` ran a step in a container only when the step + named an image, and dropped every other step to its own shell, with no way to refuse. + On the `agent-docker` Compose profile that shell is root with the engine's socket + mounted, so any project `writer` had root on the Docker host by leaving `image:` out — + and so did the author of a fork's pull request, through any imageless step that runs a + script from the repository. + - `FIBER_AGENT_ALLOW_HOST_SHELL` decides whether an agent runs steps on its host. Unset, + it is `false` on a Docker agent and `true` on a shell agent. It takes `true`, `false`, + `1` or `0`; anything else stops the agent at startup rather than meaning one of them. + - A refused step fails before anything is cloned or restored, with the remedy in its + log and its error: ``this agent runs steps in containers only: add `image:` to the + step, or set FIBER_AGENT_ALLOW_HOST_SHELL=true on the agent``. + - The `agent-docker` Compose profile pins it to `false`. + - An agent installed with `install-agent.sh --docker` gets the refusing default with no + extra flag. A `FIBER_AGENT_ALLOW_HOST_SHELL` line added to `/etc/fiber/agent.env` is + kept across re-runs of the installer. + + [agents](docs/agents.md#steps-that-would-run-on-the-host). +- **A fork's pull request never runs on an agent's host shell.** The offer now tells the + agent the run is untrusted, and the agent runs such a step in a container or fails it — + `FIBER_AGENT_ALLOW_HOST_SHELL=true` does not apply. A fork PR therefore needs an + `image:` on every step and a project agent with Docker on; on a shell agent it fails + with `fork pull request steps only run in containers, and this is a shell agent …`. + That is the intended outcome: a shell agent has nowhere to run the code but its own + account. An agent older than this release ignores the flag. + [triggers](docs/triggers.md#pull-requests-from-forks). - **Seven RustSec advisories in the lockfile, all fixed by upgrades.** `rustls` 0.23.46 (RUSTSEC-2026-0346 to -0349), `rustls-webpki` 0.103.16 (RUSTSEC-2026-0382: certificate verification could accept a distrusted intermediate), `tokio` 1.54.1 (RUSTSEC-2026-0386) @@ -23,6 +64,12 @@ minor versions may carry breaking changes. workflow runs `cargo deny check` before it builds anything. - The release workflow passes the tag to its version check through the environment rather than interpolating it into the script. +- **The `agent-docker` Compose profile registers with `docker=true`** on top of + `FIBER_AGENT_LABELS`, so a step can ask for it with `labels: [docker=true]`. The plain + `agent` profile is unchanged. +- **`smoke-runs` covers the refusal.** Its fork-PR check now expects a shell agent to + refuse the run, then — where a Docker engine is available — runs the same pull request + in a container to check it gets no secrets. ## [0.6.7] — 2026-10-01 diff --git a/SECURITY.md b/SECURITY.md index a1aeeab..8b70f1f 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -23,8 +23,11 @@ boundaries are. reaching instance-level surfaces, a non-admin managing global agents or users. - Authentication or session flaws: forging a session or agent token, bypassing webhook signature verification, session fixation. -- A **fork's** pull request obtaining project secrets, or its run being scheduled onto an - agent outside that project's own pool. +- A **fork's** pull request obtaining project secrets, its run being scheduled onto an + agent outside that project's own pool, or a step of it running on an agent's host shell + instead of in a container. +- A Docker agent running a step on its host shell without + `FIBER_AGENT_ALLOW_HOST_SHELL=true`. - Secrets escaping where they should not go: into logs, into a process list, into an artifact, into an API response. - Anything unauthenticated that starts a run, mutates state, or discloses data. @@ -36,12 +39,15 @@ boundaries are. `fiber.yml`, and see [docs/agents.md](docs/agents.md) for how a step is confined. - **An instance admin is effectively root.** They manage the global agent pool, and an agent token receives the secrets of the projects it serves. -- **Fork pull requests still execute untrusted code.** Fiber withholds secrets and - restricts them to project-dedicated agents, which limits the blast radius; it is not a - sandbox. Run them on a disposable agent with containerised steps — - [docs/triggers.md](docs/triggers.md#pull-requests-from-forks). -- **Steps run without a container unless the step sets `image:`.** A host-shell step runs - as the agent's user. +- **Fork pull requests still execute untrusted code.** Fiber withholds secrets, restricts + them to project-dedicated agents, and runs their steps in containers or not at all, + which limits the blast radius; a container is not a sandbox. Run them on a disposable + agent — [docs/triggers.md](docs/triggers.md#pull-requests-from-forks). +- **A host-shell step runs as the agent's user.** That is every step on a shell agent + (`FIBER_AGENT_USE_DOCKER=false`), and a step without `image:` on a Docker agent whose + operator set `FIBER_AGENT_ALLOW_HOST_SHELL=true` — where the agent's user can reach the + Docker engine, so such a step is root on the host. A Docker agent refuses those steps + by default. - **Defaults are for a local trial, not the internet.** `admin` / `fiber`, no secrets encryption key, everything on loopback. [docs/operations.md](docs/operations.md#deployment) describes what to change before exposing an instance. diff --git a/crates/fiber-agent/src/main.rs b/crates/fiber-agent/src/main.rs index 964d11b..43f5678 100644 --- a/crates/fiber-agent/src/main.rs +++ b/crates/fiber-agent/src/main.rs @@ -53,6 +53,18 @@ struct Args { #[arg(long, env = "FIBER_AGENT_USE_DOCKER", default_value_t = true)] use_docker: bool, + /// Whether a step may run on this host's shell: `true`/`false`/`1`/`0`. Unset, it is + /// `false` for a Docker agent (a step without `image:` is refused) and `true` for a + /// shell agent. A fork pull request's steps never run on the host shell, whatever + /// this says. Kept as text and checked at startup (`host_shell_policy`), so a typo + /// stops the agent instead of quietly meaning one or the other. + #[arg(long, env = "FIBER_AGENT_ALLOW_HOST_SHELL")] + allow_host_shell: Option, + + /// `allow_host_shell` resolved against `use_docker`, once, in `main`. + #[arg(skip)] + host_shell_allowed: bool, + #[arg( long, env = "FIBER_AGENT_WORKSPACE_DIR", @@ -160,6 +172,8 @@ fn docker_client_reads(k: &str) -> bool { #[derive(Clone)] struct ExecConfig { use_docker: bool, + /// Whether a trusted step may run on the host shell (`host_shell_policy`). + allow_host_shell: bool, /// `--label`s on every step container: this agent, and this process of it. The /// startup sweep uses them to find its own orphans without touching another agent's /// running containers. @@ -176,6 +190,7 @@ impl ExecConfig { fn from_args(args: &Args, container_labels: ContainerLabels) -> Self { Self { use_docker: args.use_docker, + allow_host_shell: args.host_shell_allowed, container_labels, env_passthrough: args .env_passthrough @@ -192,6 +207,112 @@ impl ExecConfig { } } +/// Where one step runs. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ExecutionMode<'a> { + /// `docker run` of this image (trimmed), with the step's workspace mounted. + Container(&'a str), + /// The agent's own shell, as the agent's user. + HostShell, +} + +/// Why a step that would have run on the host shell was refused instead. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum HostShellRefusal { + /// The run builds code from outside the project. Not the operator's to override. + Untrusted, + /// This agent does not run steps on its host (`FIBER_AGENT_ALLOW_HOST_SHELL`). + NotAllowed, +} + +impl HostShellRefusal { + /// What the step's log and `error` say. `has_image` and `use_docker` pick the remedy: + /// telling someone to add an `image:` the step already has, or that a shell agent + /// will never use, sends them the wrong way. + fn message(self, has_image: bool, use_docker: bool) -> String { + match (self, use_docker) { + (HostShellRefusal::Untrusted, true) => "fork pull request steps only run in \ + containers: this run builds code from outside the project and the step has \ + no `image:`, so it would have run on the agent's host. Add `image:` to the \ + step (FIBER_AGENT_ALLOW_HOST_SHELL does not apply to fork pull requests)" + .into(), + (HostShellRefusal::Untrusted, false) => format!( + "fork pull request steps only run in containers, and this is a shell agent \ + (FIBER_AGENT_USE_DOCKER=false) that runs every step on its host: run fork \ + pull requests on a project agent with FIBER_AGENT_USE_DOCKER=true{}", + if has_image { + "" + } else { + ", and add `image:` to the step" + } + ), + (HostShellRefusal::NotAllowed, true) => "this agent runs steps in containers \ + only: add `image:` to the step, or set FIBER_AGENT_ALLOW_HOST_SHELL=true on \ + the agent" + .into(), + // `host_shell_policy` refuses to start an agent configured like this, so no + // offer reaches here; worded anyway rather than left to mislead. + (HostShellRefusal::NotAllowed, false) => "this agent has host shell turned off \ + (FIBER_AGENT_ALLOW_HOST_SHELL=false) and Docker turned off \ + (FIBER_AGENT_USE_DOCKER=false), so it cannot run any step" + .into(), + } + } +} + +/// Decide where a step runs, or that it must not run here at all. +/// +/// A step runs in a container when it names an image and the agent uses Docker. +/// Everything else is the host shell — a step with no `image:`, and any step on a shell +/// agent, which ignores `image:` — and the host shell is the agent's own user, token and +/// (on a Docker agent) engine socket. So it is refused for a fork's pull request always, +/// and otherwise unless the agent allows it. +fn execution_mode( + image: Option<&str>, + use_docker: bool, + allow_host_shell: bool, + untrusted: bool, +) -> std::result::Result, HostShellRefusal> { + // Trimmed here as at compile time, so the value docker sees is the value that was + // checked. Empty means "no image". + let image = image.map(str::trim).filter(|i| !i.is_empty()); + match image { + Some(img) if use_docker => Ok(ExecutionMode::Container(img)), + // Before `allow_host_shell`: that flag is the operator vouching for the people + // who can write the pipeline, and a fork's author is not one of them. + _ if untrusted => Err(HostShellRefusal::Untrusted), + _ if !allow_host_shell => Err(HostShellRefusal::NotAllowed), + _ => Ok(ExecutionMode::HostShell), + } +} + +/// Resolve `FIBER_AGENT_ALLOW_HOST_SHELL` at startup. Unset (or empty, which is what +/// Compose passes for a variable nobody set) takes the default: allowed on a shell +/// agent, where the host is all there is, and refused on a Docker agent. Anything that is +/// not a boolean is an error rather than a guess, since guessing wrong in one direction +/// runs pipeline code on the host. +fn host_shell_policy(raw: Option<&str>, use_docker: bool) -> std::result::Result { + let explicit = match raw.map(str::trim).filter(|v| !v.is_empty()) { + None => None, + Some("true" | "1") => Some(true), + Some("false" | "0") => Some(false), + Some(other) => { + return Err(format!( + "FIBER_AGENT_ALLOW_HOST_SHELL must be true, false, 1 or 0 (got `{other}`)" + )); + } + }; + let allowed = explicit.unwrap_or(!use_docker); + if !allowed && !use_docker { + return Err( + "FIBER_AGENT_ALLOW_HOST_SHELL=false with FIBER_AGENT_USE_DOCKER=false leaves the \ + agent unable to run any step: enable Docker, or allow host shell" + .into(), + ); + } + Ok(allowed) +} + /// Per-run workspace refcount: the last step of a run to finish on this agent deletes /// the run's tree. Without it every run leaks a checkout for the agent's lifetime. #[derive(Default)] @@ -1428,6 +1549,28 @@ async fn main() -> Result<()> { } } info!(workspace_dir = %args.workspace_dir.display(), "workspace root"); + match host_shell_policy(args.allow_host_shell.as_deref(), args.use_docker) { + Ok(allowed) => args.host_shell_allowed = allowed, + Err(e) => { + error!("{e}"); + std::process::exit(2); + } + } + match (args.use_docker, args.host_shell_allowed) { + (true, false) => info!( + "steps run in containers only: a step without `image:` is refused \ + (FIBER_AGENT_ALLOW_HOST_SHELL=true allows it)" + ), + (true, true) => warn!( + "FIBER_AGENT_ALLOW_HOST_SHELL=true: a step without `image:` runs on this host as \ + the agent's user, with whatever that user can reach — including the Docker \ + engine. Fork pull request steps are still refused" + ), + (false, _) => info!( + "shell agent: every step runs on this host as the agent's user, `image:` is \ + ignored, and fork pull request steps are refused" + ), + } // A macOS workspace reaches step containers through Docker Desktop's file sharing, // which intermittently loses files a build has just written — `cargo` fails with // `can't find crate for …` naming a crate it compiled seconds earlier. Same image and @@ -1842,6 +1985,7 @@ async fn run_session( working_directory, shell, attempt, + untrusted, }) => { info!(%step_id, %step_name, %run_id, ?attempt, "offered step"); // Anything still queued about this step_run_id is from an @@ -1898,6 +2042,7 @@ async fn run_session( traceparent.as_deref(), working_directory.as_deref(), shell.as_deref(), + untrusted, ).await; if let Ok(mut g) = cancels.lock() { @@ -2051,6 +2196,7 @@ async fn execute_step( traceparent: Option<&str>, working_directory: Option<&str>, shell: Option<&str>, + untrusted: bool, ) -> Result { let run_dir = workspace_root.join(run_id.to_string()); // A fresh directory per execution, never a reused one. A previous attempt's container @@ -2096,6 +2242,7 @@ async fn execute_step( workspaces, working_directory, shell, + untrusted, ) // `.instrument`, not `span.enter()`: a guard held across an await point attributes // whatever else the runtime schedules on this thread to this step. @@ -2138,6 +2285,7 @@ async fn execute_step_inner( workspaces: &Workspaces, working_directory: Option<&str>, shell: Option<&str>, + untrusted: bool, ) -> Result { // One sequence for system/stdout/stderr so the server's `ORDER BY seq` interleaves // streams in emission order (the old per-stream bases collided after 1000 lines). @@ -2202,6 +2350,18 @@ async fn execute_step_inner( log("system", format!("step timeout: {m} min")); } + // Decided before anything touches the disk or the network: a step this agent will + // not run gets no checkout, no artifact restore, and no directory. + let mode = match execution_mode(image, exec.use_docker, exec.allow_host_shell, untrusted) { + Ok(mode) => mode, + Err(refusal) => { + let has_image = image.is_some_and(|i| !i.trim().is_empty()); + let msg = refusal.message(has_image, exec.use_docker); + log("system", format!("step refused: {msg}")); + bail!("{msg}"); + } + }; + tokio::fs::create_dir_all(work_dir).await?; // Workspace prep can be interrupted by cancel. @@ -2318,13 +2478,8 @@ async fn execute_step_inner( } let mut env_file: Option = None; - // Trimmed here as at compile time, so the value docker sees is the value that was - // checked. Empty means "no image". - let mut child = if let Some(img) = image - .map(str::trim) - .filter(|i| !i.is_empty()) - .filter(|_| exec.use_docker) - { + // `mode` was settled at the top (`execution_mode`), before the workspace existed. + let mut child = if let ExecutionMode::Container(img) = mode { let mount = format!("{}:/workspace", work_dir.display()); log( "system", @@ -3788,6 +3943,141 @@ mod tests { } } + #[test] + fn execution_mode_covers_every_combination() { + use ExecutionMode::{Container, HostShell}; + use HostShellRefusal::{NotAllowed, Untrusted}; + let img = Some("alpine:3"); + // (image, use_docker, allow_host_shell, untrusted) -> outcome. All sixteen rows, + // so a change to any arm of the decision moves at least one of them. + let table = [ + // An image on a Docker agent is a container, whoever asks and whatever the flag. + (img, true, true, false, Ok(Container("alpine:3"))), + (img, true, false, false, Ok(Container("alpine:3"))), + (img, true, true, true, Ok(Container("alpine:3"))), + (img, true, false, true, Ok(Container("alpine:3"))), + // No image on a Docker agent: the host, so only when the operator allows it, + // and never for a fork's pull request. + (None, true, true, false, Ok(HostShell)), + (None, true, false, false, Err(NotAllowed)), + (None, true, true, true, Err(Untrusted)), + (None, true, false, true, Err(Untrusted)), + // A shell agent ignores `image:` — the documented behaviour — so both of + // these are the host too. + (img, false, true, false, Ok(HostShell)), + (None, false, true, false, Ok(HostShell)), + (img, false, true, true, Err(Untrusted)), + (None, false, true, true, Err(Untrusted)), + // Not a configuration the agent starts with (`host_shell_policy`), but the + // decision still has to refuse rather than run. + (img, false, false, false, Err(NotAllowed)), + (None, false, false, false, Err(NotAllowed)), + (img, false, false, true, Err(Untrusted)), + (None, false, false, true, Err(Untrusted)), + ]; + for (image, use_docker, allow, untrusted, want) in table { + assert_eq!( + execution_mode(image, use_docker, allow, untrusted), + want, + "image={image:?} use_docker={use_docker} allow_host_shell={allow} untrusted={untrusted}" + ); + } + } + + #[test] + fn a_blank_image_is_no_image_and_a_padded_one_is_trimmed() { + // The server trims at compile time; a snapshot from before that can hold either. + assert_eq!( + execution_mode(Some(" "), true, false, false), + Err(HostShellRefusal::NotAllowed) + ); + assert_eq!( + execution_mode(Some(""), true, true, false), + Ok(ExecutionMode::HostShell) + ); + assert_eq!( + execution_mode(Some(" alpine:3 "), true, false, true), + Ok(ExecutionMode::Container("alpine:3")) + ); + } + + #[test] + fn a_refusal_names_the_remedy_that_applies() { + let not_allowed = HostShellRefusal::NotAllowed.message(false, true); + assert!(not_allowed.contains("add `image:` to the step")); + assert!(not_allowed.contains("FIBER_AGENT_ALLOW_HOST_SHELL=true")); + + // The flag is no way out for a fork, so the message must not offer it as one. + let fork = HostShellRefusal::Untrusted.message(false, true); + assert!(fork.starts_with("fork pull request steps only run in containers")); + assert!(fork.contains("add `image:`") || fork.contains("Add `image:`")); + assert!(!fork.contains("FIBER_AGENT_ALLOW_HOST_SHELL=true")); + + // On a shell agent an `image:` changes nothing; the agent is what has to change. + let shell = HostShellRefusal::Untrusted.message(true, false); + assert!(shell.starts_with("fork pull request steps only run in containers")); + assert!(shell.contains("FIBER_AGENT_USE_DOCKER=true")); + assert!(!shell.contains("`image:`")); + assert!( + HostShellRefusal::Untrusted + .message(false, false) + .contains("add `image:`") + ); + + // The completion handler reads these two prefixes as a cancel and a timeout. + for refusal in [HostShellRefusal::Untrusted, HostShellRefusal::NotAllowed] { + for (has_image, use_docker) in + [(false, false), (false, true), (true, false), (true, true)] + { + let m = refusal.message(has_image, use_docker); + assert!( + !m.contains("cancelled") && !m.starts_with("timed out"), + "{m}" + ); + } + } + } + + #[test] + fn host_shell_defaults_off_for_a_docker_agent_and_on_for_a_shell_agent() { + assert_eq!(host_shell_policy(None, true), Ok(false)); + assert_eq!(host_shell_policy(None, false), Ok(true)); + // Compose passes an unset variable as empty; that is unset, not a value. + assert_eq!(host_shell_policy(Some(""), true), Ok(false)); + assert_eq!(host_shell_policy(Some(" "), false), Ok(true)); + } + + #[test] + fn an_explicit_host_shell_setting_overrides_the_default() { + for yes in ["true", "1", " true "] { + assert_eq!(host_shell_policy(Some(yes), true), Ok(true), "{yes}"); + assert_eq!(host_shell_policy(Some(yes), false), Ok(true), "{yes}"); + } + for no in ["false", "0"] { + assert_eq!(host_shell_policy(Some(no), true), Ok(false), "{no}"); + } + } + + #[test] + fn a_host_shell_setting_that_is_not_a_boolean_stops_the_agent() { + for bad in ["yes", "no", "TRUE", "on", "2", "tru"] { + let err = host_shell_policy(Some(bad), true).unwrap_err(); + assert!( + err.contains("FIBER_AGENT_ALLOW_HOST_SHELL") && err.contains(bad), + "{err}" + ); + assert!(host_shell_policy(Some(bad), false).is_err(), "{bad}"); + } + } + + #[test] + fn an_agent_that_could_run_nothing_does_not_start() { + for no in ["false", "0"] { + let err = host_shell_policy(Some(no), false).unwrap_err(); + assert!(err.contains("FIBER_AGENT_USE_DOCKER=false"), "{err}"); + } + } + #[test] fn a_normal_line_loses_its_newline_and_nothing_else() { assert_eq!(truncate_line(b"hello world\n"), "hello world"); diff --git a/crates/fiber-api/src/ws.rs b/crates/fiber-api/src/ws.rs index c59301d..d26b276 100644 --- a/crates/fiber-api/src/ws.rs +++ b/crates/fiber-api/src/ws.rs @@ -292,6 +292,9 @@ async fn offer_for_step(state: &AppState, step: StepRun) -> Result, + /// The run builds code from outside the project (a fork's pull request). An + /// agent that reads this never runs such a step on its host shell, whatever + /// `FIBER_AGENT_ALLOW_HOST_SHELL` says: it runs in a container or fails. Absent + /// from older servers, which the agent reads as the old contract — a trusted + /// run. An agent below revision 3 ignores it and runs the step as before, so the + /// server may rely on it only for an agent whose `Hello` said 3 or later. + #[serde(default)] + untrusted: bool, }, Cancel { step_run_id: Uuid, @@ -716,8 +730,9 @@ mod tests { #[test] fn an_offer_from_an_older_server_omits_the_newer_fields() { - // timeout_minutes / secret_keys / traceparent / working_directory / shell are all - // `#[serde(default)]` precisely so a mixed-version deploy keeps working. + // timeout_minutes / secret_keys / traceparent / working_directory / shell / + // untrusted are all `#[serde(default)]` precisely so a mixed-version deploy keeps + // working. let minimal = json!({ "type": "offer", "step_run_id": Uuid::nil(), @@ -738,11 +753,15 @@ mod tests { shell, artifacts, restore, + untrusted, .. } = msg else { panic!("expected an offer"); }; + // An older server cannot say a run is a fork's: absent reads as trusted, which is + // what that server's agents did with it anyway. + assert!(!untrusted); assert_eq!(timeout_minutes, None); assert!(secret_keys.is_empty()); assert_eq!(traceparent, None); @@ -752,6 +771,41 @@ mod tests { assert!(restore.is_empty()); } + #[test] + fn an_untrusted_offer_says_so_on_the_wire() { + // The agent refuses host shell on this field alone, so it has to survive the trip + // under this exact name; an older agent skips a key it does not know. + let offer = ServerMessage::Offer { + step_run_id: Uuid::nil(), + run_id: Uuid::nil(), + step_id: "build".into(), + step_name: "Build".into(), + image: None, + run: "make".into(), + workspace: None, + env: vec![], + attempt: Some(1), + artifacts: vec![], + restore: vec![], + timeout_minutes: None, + working_directory: None, + shell: None, + secret_keys: vec![], + traceparent: None, + untrusted: true, + }; + let v = serde_json::to_value(&offer).unwrap(); + assert_eq!(v["untrusted"], json!(true)); + let back: ServerMessage = serde_json::from_value(v).unwrap(); + assert!(matches!( + back, + ServerMessage::Offer { + untrusted: true, + .. + } + )); + } + #[test] fn a_log_batch_round_trips_with_its_lines_in_order() { let batch = AgentMessage::LogBatch { diff --git a/deploy/.env.example b/deploy/.env.example index 7413fe9..ecf4b47 100644 --- a/deploy/.env.example +++ b/deploy/.env.example @@ -119,7 +119,10 @@ FIBER_AGENT_CONCURRENCY=2 # ------------------------------------------------- docker agent (profile: agent-docker) # docker compose -f deploy/docker-compose.yml --profile agent-docker up -d fiber-agent-docker -# For pipelines with `image:` steps; uses the token and labels above. Bound at the same +# For pipelines with `image:` steps; uses the token above, and the labels above plus +# `docker=true`. It runs steps in containers only (FIBER_AGENT_ALLOW_HOST_SHELL is pinned +# to false in the Compose file, not read from here): a step without `image:` is refused, +# because on this agent it would run as root beside the Docker socket. Bound at the same # path inside the agent and on the Docker host. On Docker Desktop use a path inside its # VM, e.g. `docker volume create fiber_docker_ws` then # /var/lib/docker/volumes/fiber_docker_ws/_data (docs/agents.md). diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml index 4b44849..6efa515 100644 --- a/deploy/docker-compose.yml +++ b/deploy/docker-compose.yml @@ -299,7 +299,12 @@ services: # Optional worker for pipelines with `image:` steps: # `docker compose --profile agent-docker up -d fiber-agent-docker`. Same token and labels - # as fiber-agent (run one or the other). It gets the Docker socket — root on the host. + # as fiber-agent (run one or the other), plus `docker=true` so a step can ask for it. + # It gets the Docker socket — root on the host. + # + # Which is why FIBER_AGENT_ALLOW_HOST_SHELL is pinned to false here rather than read + # from .env: a step without `image:` would run in THIS container, as root, next to that + # socket. Such a step is refused; give it an `image:` or run it on the plain agent. # # The workspace is bound at the SAME absolute path inside this container and on the # Docker host. Steps run as `docker run -v /:/workspace`, and the daemon @@ -321,9 +326,10 @@ services: FIBER_API_URL: ws://fiber-api:8080 FIBER_AGENT_TOKEN: ${FIBER_AGENT_TOKEN:-} FIBER_AGENT_NAME: ${FIBER_AGENT_NAME:-compose} - FIBER_AGENT_LABELS: ${FIBER_AGENT_LABELS:-os=linux} + FIBER_AGENT_LABELS: ${FIBER_AGENT_LABELS:-os=linux},docker=true FIBER_AGENT_CONCURRENCY: ${FIBER_AGENT_CONCURRENCY:-2} FIBER_AGENT_USE_DOCKER: "true" + FIBER_AGENT_ALLOW_HOST_SHELL: "false" FIBER_AGENT_WORKSPACE_DIR: ${FIBER_AGENT_DOCKER_WORKSPACE:-/var/lib/fiber/workspaces} RUST_LOG: ${RUST_LOG:-info,fiber_agent=info} volumes: diff --git a/docs/agents.md b/docs/agents.md index b68f584..211b905 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -6,7 +6,7 @@ An agent's identity is the token it connected with. The server binds `agent_id` A lease belongs to the agent, not to the socket. A step keeps running through a lost connection — an API restart, a proxy timeout, a network blip — and the agent renews its leases on its first heartbeat back; the server judges every late message by the row (is the step still `running` under this agent, on the attempt the message names?), never by which session it arrived on. Every `Offer` carries the attempt number and the agent echoes it on each log batch, artifact, and completion — and on the HTTP artifact routes, as `X-Fiber-Attempt` on the upload and an `attempt` field in the presign and complete bodies ([api](./api.md#agent-endpoints)) — so output an agent held through a reclaim can never be filed under, overwrite, or close the attempt that replaced it; a message whose attempt is not the row's is dropped with a warning, and an upload for one is refused with `401`. A log line the agent re-sends after an aborted write can appear twice: `log_lines` is append-only with no dedupe. See [Lifecycle](#lifecycle) for the limits. -`Hello` carries `protocol_version` (`fiber_proto::PROTOCOL_VERSION`, currently **2**; absent from older agents, read as `0`). The server logs it and uses it to tell an agent that keeps its steps across sessions (1 and up) from one that cancels them on any close (0), and a later server may refuse a revision it no longer supports. It is not yet stored on the agent row. `Welcome` carries the **server's** revision the same way, and the agent reads its absence as "older than `log_batch`" and sends one `log_chunk` per line to that server — so a rolling deploy in either order keeps a build's log, though **upgrade the API before the agents** remains the supported direction. Revision 2 adds batched log messages; see [Log path](#log-path). +`Hello` carries `protocol_version` (`fiber_proto::PROTOCOL_VERSION`, currently **3**; absent from older agents, read as `0`). The server logs it and uses it to tell an agent that keeps its steps across sessions (1 and up) from one that cancels them on any close (0), and a later server may refuse a revision it no longer supports. It is not yet stored on the agent row. `Welcome` carries the **server's** revision the same way, and the agent reads its absence as "older than `log_batch`" and sends one `log_chunk` per line to that server — so a rolling deploy in either order keeps a build's log, though **upgrade the API before the agents** remains the supported direction. Revision 2 adds batched log messages; see [Log path](#log-path). Revision 3 means the agent reads `untrusted` on an `Offer` and never runs a fork pull request's step on its host shell, and that a Docker agent refuses a step without `image:` unless told otherwise; see [Executor](#executor). `untrusted` is absent from an older server, which the agent reads as a trusted run, and an older agent ignores it — nothing gates on the revision yet. ## Pools @@ -79,7 +79,10 @@ Two details that are not decoration: Downloads the release binary for the host architecture, creates the `fiber` system user, writes `/etc/fiber/agent.env` (mode `0640`, root-owned so the token is not world-readable), installs [`deploy/fiber-agent.service`](../deploy/fiber-agent.service), and enables it. Add -`--docker` to allow `image:` steps (adds `fiber` to the `docker` group). `--uninstall` +`--docker` to allow `image:` steps (adds `fiber` to the `docker` group); an agent installed +that way runs steps in containers only and refuses one without `image:` — see +[Executor](#executor). A `FIBER_AGENT_ALLOW_HOST_SHELL` line you add to `agent.env` is +kept by the next run of the installer. `--uninstall` removes the service, keeping `/etc/fiber` and `/var/lib/fiber`. #### What it verifies @@ -275,7 +278,11 @@ docker compose -f deploy/docker-compose.yml --profile agent up -d fiber-agent Pipelines with `image:` steps need the `agent-docker` profile instead. It builds the published agent plus the Docker CLI locally (`deploy/agent-docker.Dockerfile`), mounts the -engine's socket, and runs as root, because the socket is root on the host anyway: +engine's socket, and runs as root, because the socket is root on the host anyway. For +that reason it runs steps **in containers only**: `FIBER_AGENT_ALLOW_HOST_SHELL` is pinned +to `false` in the Compose file, so a step without `image:` is refused rather than run as +root inside the agent's container, next to the socket. It registers with the labels of +the plain agent plus `docker=true`, so a step can ask for it with `labels: [docker=true]`: ```bash echo "FIBER_AGENT_DOCKER_WORKSPACE=/var/lib/fiber/workspaces" >> deploy/.env @@ -309,6 +316,8 @@ export FIBER_AGENT_NAME=local export FIBER_AGENT_LABELS=os=linux,docker=true export FIBER_AGENT_CONCURRENCY=1 export FIBER_AGENT_USE_DOCKER=true # false = host shell +# With Docker on, a step without `image:` is refused. To run those on this host instead: +# export FIBER_AGENT_ALLOW_HOST_SHELL=true export FIBER_AGENT_WORKSPACE_DIR=./data/workspaces cargo run -p fiber-agent # or: cargo run -p fiber-cli -- agent --token "$FIBER_AGENT_TOKEN" @@ -420,8 +429,52 @@ repeat is identifiable. ## Executor -- **Shell** (`FIBER_AGENT_USE_DOCKER=false`): runs `run` in the step's workspace directory. -- **Docker**: if `image` is set and docker mode is on, runs inside the image with the workspace mounted, as a named container (`fiber-step-`) so cancel and timeout `docker kill` it rather than only the client. The image is checked against the docker reference grammar and passed after `--`, so a pipeline cannot smuggle a `docker run` flag through it. +- **Shell** (`FIBER_AGENT_USE_DOCKER=false`): runs `run` in the step's workspace directory, on the host, as the agent's user. A step's `image:` is ignored. +- **Docker** (`FIBER_AGENT_USE_DOCKER=true`, the binary's default): if `image` is set, runs inside the image with the workspace mounted, as a named container (`fiber-step-`) so cancel and timeout `docker kill` it rather than only the client. The image is checked against the docker reference grammar and passed after `--`, so a pipeline cannot smuggle a `docker run` flag through it. + +### Steps that would run on the host + +A step runs on the host shell when it has no `image:`, or when the agent is a shell agent. +The host shell is the agent's own user, with whatever that user can reach — on a Docker +agent that includes the Docker engine, which is root on the host. So the agent decides, +before it clones anything, whether a step may run there: + +| Step | Agent | `FIBER_AGENT_ALLOW_HOST_SHELL` | Run from a fork's pull request? | Outcome | +|---|---|---|---|---| +| has `image:` | Docker | any | either | container | +| no `image:` | Docker | unset or `false` | no | **refused** | +| no `image:` | Docker | `true` | no | host shell | +| no `image:` | Docker | any | yes | **refused** | +| either | shell | unset or `true` | no | host shell | +| either | shell | any | yes | **refused** | + +`FIBER_AGENT_ALLOW_HOST_SHELL` defaults to `true` on a shell agent, where the host is all +there is, and to `false` on a Docker agent. Set it to `true` on a Docker agent to run +steps without `image:` on its host — that is the operator saying everyone who can write +this agent's pipelines may act as the agent's user. It accepts `true`, `false`, `1` and +`0`; anything else stops the agent at startup, and so does `false` on a shell agent, which +could then run nothing. + +A fork's pull request is never run on the host shell, and the setting does not change +that: its author is not someone the operator vouched for. Its steps need an `image:` and a +project agent with Docker on. On a shell agent they fail, by design — see +[triggers](./triggers.md#pull-requests-from-forks). + +A refused step fails, and a retry will fail the same way. Its log and its `error` say +which case it was and what to change; nothing is cloned or restored first: + +```text +this agent runs steps in containers only: add `image:` to the step, or set FIBER_AGENT_ALLOW_HOST_SHELL=true on the agent +fork pull request steps only run in containers: this run builds code from outside the project and the step has no `image:` … +fork pull request steps only run in containers, and this is a shell agent (FIBER_AGENT_USE_DOCKER=false) … +``` + +To keep steps without `image:` away from a Docker agent in the first place, give the two +kinds of agent different labels and ask for one in the step's `labels:` — the Compose +`agent-docker` profile adds `docker=true` for this. + +Releases up to 0.6.7 ran a step without `image:` on a Docker agent's host without being +asked, and ran a fork's pull request on the host shell like any other run. Process groups: cancel kills the step's process group so grandchildren die too. @@ -471,7 +524,7 @@ Steps run repo-supplied shell, so the agent narrows what is reachable: | **Container limits** | `--security-opt no-new-privileges` and `--pids-limit 512` by default, plus `--user`, `--network`, `--memory`, `--cpus` from `FIBER_AGENT_DOCKER_*`. Memory and CPU limits are off unless set; whatever applies is printed as a `system` log line so an exit 137 is diagnosable | | **Workspace** | One directory per step, deleted when the step finishes — including on cancel, timeout, or failure. The run's tree goes when its last step on this agent finishes; anything older than `FIBER_AGENT_WORKSPACE_TTL_HOURS` (24) is swept at startup | | **Artifacts** | Only those produced by the steps this one transitively `needs` — enforced when the offer is built *and* again when the agent asks to download one | -| **Untrusted runs** | A fork's pull request is offered only to agents bound to that project, never to the global pool, and receives no secrets — see [triggers](./triggers.md#pull-requests-from-forks) | +| **Untrusted runs** | A fork's pull request is offered only to agents bound to that project, never to the global pool, receives no secrets, and runs in a container or not at all ([Executor](#steps-that-would-run-on-the-host)) — see [triggers](./triggers.md#pull-requests-from-forks) | A step with a git workspace is cloned from one reference clone per run, so a second step costs a local object copy rather than another fetch, and the checkout is self-contained diff --git a/docs/configuration.md b/docs/configuration.md index 301e184..21805ad 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -106,6 +106,7 @@ file passes the variable through as an empty string and each reader falls back. | `FIBER_AGENT_LABELS` | `os=linux,docker=true` | Comma-separated | | `FIBER_AGENT_CONCURRENCY` | `1` | Max parallel steps | | `FIBER_AGENT_USE_DOCKER` | `true` | Docker vs host shell | +| `FIBER_AGENT_ALLOW_HOST_SHELL` | `false` with Docker, `true` without | Whether a step may run on the agent's host shell. On a Docker agent the default refuses a step without `image:`; `true` runs it on the host as the agent's user, which can reach the Docker engine. `true`, `false`, `1` or `0` — anything else, or `false` on a shell agent, stops the agent at startup. A fork pull request's steps are never run on the host shell, whatever this is set to. [agents](./agents.md#steps-that-would-run-on-the-host) | | `FIBER_AGENT_WORKSPACE_DIR` | `./data/workspaces` | Root for per-step work dirs | | `FIBER_AGENT_ENV_PASSTHROUGH` | empty | Comma-separated names to pass from the agent's environment into steps, on top of the built-in allowlist (shell basics, proxy and CA settings). Everything else is cleared — use this for `SSH_AUTH_SOCK`, `CARGO_HOME`, `JAVA_HOME`, `NVM_DIR` and similar. **Under systemd, `SSH_AUTH_SOCK` needs a drop-in** — see below | | `FIBER_AGENT_WORKSPACE_TTL_HOURS` | `24` | Sweep run workspaces older than this at startup; `0` disables | @@ -118,7 +119,10 @@ file passes the variable through as an empty string and each reader falls back. Installed by `scripts/install-agent.sh` into `/etc/fiber/agent.env` (root-owned, mode `0640`); in Compose they come from `deploy/.env` (`FIBER_AGENT_TOKEN`, `FIBER_AGENT_NAME`, -`FIBER_AGENT_LABELS`, `FIBER_AGENT_CONCURRENCY`, `FIBER_AGENT_USE_DOCKER`). The installer also writes +`FIBER_AGENT_LABELS`, `FIBER_AGENT_CONCURRENCY`, `FIBER_AGENT_USE_DOCKER`). The Compose +`agent-docker` profile pins `FIBER_AGENT_ALLOW_HOST_SHELL=false` and appends `docker=true` +to the labels; the installer writes `FIBER_AGENT_ALLOW_HOST_SHELL` only to carry over a +line already in `agent.env`. The installer also writes `FIBER_AGENT_INSTALLED_VERSION` and `FIBER_AGENT_INSTALLED_SHA256` there so a re-run can say what it is replacing; no crate reads them, and they are not deployment configuration. diff --git a/docs/pipeline-yaml.md b/docs/pipeline-yaml.md index bf195c8..2b950a4 100644 --- a/docs/pipeline-yaml.md +++ b/docs/pipeline-yaml.md @@ -104,7 +104,7 @@ API replicas still leave exactly one run in flight. | `run` | — | Shell script (required for execution) | | `needs` | `[]` | Upstream step ids (DAG edges) | | `labels` | `[]` | Matched against agent labels (all required labels must be present) | -| `image` | — | Optional Docker image; agent runs in container when Docker enabled. A docker image reference only (`name[:tag][@digest]`, optionally registry-qualified): the value becomes a `docker run` argument, so anything docker could read as a flag (`-v/:/host`, `--privileged`) is rejected when the pipeline compiles and again by the agent, which also passes it after `--` | +| `image` | — | Docker image to run the step in, on an agent with Docker enabled; a shell agent ignores it and runs on its host. **Without it the step runs on the agent's host shell, which a Docker agent refuses by default and every agent refuses for a fork's pull request** — see [agents](./agents.md#steps-that-would-run-on-the-host). A docker image reference only (`name[:tag][@digest]`, optionally registry-qualified): the value becomes a `docker run` argument, so anything docker could read as a flag (`-v/:/host`, `--privileged`) is rejected when the pipeline compiles and again by the agent, which also passes it after `--` | | `retries` | `0` | Extra attempts after failure (exponential backoff `2^attempt` s, max 60 s, persisted on the row). An attempt lost to an agent disconnect or lease expiry also counts, and gets one extra try: a step is failed once it has lost more than `retries + 1` leases | | `timeout_minutes` | server default (60) | Per-attempt wall-clock limit incl. workspace prep and restores. The agent kills the step and fails the attempt (retries still apply); the server independently fails it `FIBER_STEP_TIMEOUT_GRACE_MINUTES` later if the agent did not | | `secrets` | all | Project secrets to inject, by name. Omit for every secret (the default); `secrets: []` for none. Naming them keeps credentials out of steps that have no use for them, which matters most for a step running a third-party `image:` | diff --git a/docs/triggers.md b/docs/triggers.md index fa2e6d3..31ff566 100644 --- a/docs/triggers.md +++ b/docs/triggers.md @@ -86,7 +86,7 @@ access to the fork. A fork's pull request is code written by someone outside the project, while the pipeline and its secrets belong to the base repository. Fiber marks such a run **untrusted** -(`runs.untrusted`) and applies two controls: +(`runs.untrusted`) and applies three controls: 1. **No project secrets.** No step of the run receives any, whatever its `secrets:` says. Provenance that cannot be determined from the payload counts as untrusted, and a retry @@ -94,13 +94,32 @@ and its secrets belong to the base repository. Fiber marks such a run **untruste 2. **Project-dedicated agents only.** Untrusted steps are never offered to the global pool — only to an agent bound to that project. If the project has no such agent, the steps stay queued. - -**These reduce the blast radius; they do not sandbox the code.** A step still executes as -the agent's user, so on a shared agent it could read that agent's token out of `/proc` and -reach other projects' work. Run untrusted pull requests only on an agent you are willing -to treat as compromised: dedicated to the project, disposable, and ideally running every -step in a container (`image:`), which is what keeps the agent's own environment out of -reach. +3. **Containers only.** The offer tells the agent the run is untrusted, and the agent + never runs such a step on its host shell: a step with an `image:`, on an agent with + Docker on, runs in its container; anything else **fails** before a line of the fork's + code is checked out. `FIBER_AGENT_ALLOW_HOST_SHELL=true` does not change this — that + setting is the operator vouching for the people who can write the pipeline, and a + fork's author is not one of them. + +So a pipeline that builds fork pull requests needs an `image:` on every step and a +project agent with `FIBER_AGENT_USE_DOCKER=true`. Two consequences worth knowing before +the first fork PR arrives: + +- **On a shell agent (`FIBER_AGENT_USE_DOCKER=false`) a fork's pull request fails**, + every step, with `fork pull request steps only run in containers, and this is a shell + agent …` in the step's log and error. That is intended: a shell agent has nowhere to put + the code except its own user account. Same-repository pull requests are unaffected. +- A step without `image:` fails the same way on a Docker agent, even one that allows host + shell for the project's own runs. + +The third control is the agent's, so it holds only on agents that have it: an agent older +than protocol revision 3 ignores the flag and runs the step as before. Upgrade the agents +that serve projects with fork pull requests. + +**These reduce the blast radius; they do not sandbox the code.** A container shares the +host's kernel, has the network unless `FIBER_AGENT_DOCKER_NETWORK=none`, and writes to a +workspace the agent later reads. Run untrusted pull requests only on an agent you are +willing to treat as compromised: dedicated to the project and disposable. A step that needs a secret will fail on a fork's pull request. That is the intended outcome — do the privileged part on `push` to a branch you control, after review. diff --git a/scripts/dev-env.sh b/scripts/dev-env.sh index eaadd83..e193763 100755 --- a/scripts/dev-env.sh +++ b/scripts/dev-env.sh @@ -44,6 +44,10 @@ export FIBER_AGENT_WORKSPACE_DIR="${FIBER_AGENT_WORKSPACE_DIR:-$ROOT/data/worksp # export FIBER_AGENT_ENV_PASSTHROUGH=SSH_AUTH_SOCK # export FIBER_AGENT_DOCKER_NETWORK=none export FIBER_AGENT_USE_DOCKER="${FIBER_AGENT_USE_DOCKER:-false}" +# Left unset on purpose: the agent's default follows FIBER_AGENT_USE_DOCKER (host shell +# allowed for a shell agent, refused for a Docker one). Set it only to run steps without +# `image:` on a Docker agent. +# export FIBER_AGENT_ALLOW_HOST_SHELL=true export FIBER_AGENT_NAME="${FIBER_AGENT_NAME:-local}" export FIBER_AGENT_LABELS="${FIBER_AGENT_LABELS:-os=linux}" export FIBER_AGENT_CONCURRENCY="${FIBER_AGENT_CONCURRENCY:-1}" diff --git a/scripts/install-agent.sh b/scripts/install-agent.sh index 1cb2e57..1884014 100755 --- a/scripts/install-agent.sh +++ b/scripts/install-agent.sh @@ -57,6 +57,10 @@ NAME="$(hostname -s 2>/dev/null || hostname 2>/dev/null || echo fiber-agent)" LABELS="os=linux" CONCURRENCY="2" USE_DOCKER="false" +# Never set by a flag: the agent's own default is the safe one (--docker refuses a step +# without `image:`). Only carried over from an existing agent.env, so an upgrade does not +# undo a choice the operator made there. +ALLOW_HOST_SHELL="" BIN_DIR="/usr/local/bin" ETC_DIR="/etc/fiber" STATE_DIR="/var/lib/fiber" @@ -75,7 +79,10 @@ Usage: install-agent.sh --api-url --token [options] --name NAME Agent display name [default: hostname] --labels LIST Comma-separated labels [default: os=linux] --concurrency N Max parallel steps [default: 2] - --docker Enable Docker steps (needs docker + group membership) + --docker Enable Docker steps (needs docker + group membership). The agent + then runs steps in containers only: one without `image:` is + refused unless FIBER_AGENT_ALLOW_HOST_SHELL=true is added to + /etc/fiber/agent.env. --version VER Release tag to install [default: latest] --tarball PATH Install from a local release tarball instead of downloading --check-only Resolve, download and verify a release, then exit (no root). @@ -527,8 +534,11 @@ fi PREV_VERSION="unknown" # Re-running is the upgrade path: keep settings that were not passed this time. if [[ -f "$ETC_DIR/agent.env" ]]; then + # From the file only, never from whatever environment this script was started in. + unset FIBER_AGENT_ALLOW_HOST_SHELL # shellcheck disable=SC1091 . "$ETC_DIR/agent.env" + ALLOW_HOST_SHELL="${FIBER_AGENT_ALLOW_HOST_SHELL:-}" API_URL="${API_URL:-${FIBER_API_URL:-}}" TOKEN="${TOKEN:-${FIBER_AGENT_TOKEN:-}}" [[ "$NAME" == "$(hostname -s 2>/dev/null || echo "$NAME")" ]] && NAME="${FIBER_AGENT_NAME:-$NAME}" @@ -595,6 +605,10 @@ RUST_LOG=info,fiber_agent=info FIBER_AGENT_INSTALLED_VERSION=$INSTALLED_FROM FIBER_AGENT_INSTALLED_SHA256=$BIN_SHA256 ENV +if [[ -n "$ALLOW_HOST_SHELL" ]]; then + echo "FIBER_AGENT_ALLOW_HOST_SHELL=$ALLOW_HOST_SHELL" >> "$ETC_DIR/agent.env" + echo "keeping FIBER_AGENT_ALLOW_HOST_SHELL=$ALLOW_HOST_SHELL from the previous agent.env" +fi chown root:fiber "$ETC_DIR/agent.env" chmod 0640 "$ETC_DIR/agent.env" umask 022 diff --git a/scripts/smoke_runs.py b/scripts/smoke_runs.py index 51c87d5..96d62b4 100644 --- a/scripts/smoke_runs.py +++ b/scripts/smoke_runs.py @@ -5,7 +5,9 @@ Needs an agent already online with `os=linux` (CI's smoke-host starts one; locally see docs/development.md), and `psql` for the one check that has to break a stored definition the API would refuse to save. The fork-PR check starts its own project-scoped agent from -`target/debug/fiber-agent` and stops it by PID afterwards. +`target/debug/fiber-agent` and stops it by PID afterwards — once as a shell agent, which +must refuse the run, and once as a Docker agent (when `docker info` works; it pulls +`alpine:3`), which must run it in a container with no secrets. """ from __future__ import annotations @@ -224,10 +226,13 @@ def main() -> None: req("POST", f"/api/projects/{pid}/secrets", token, {"key": "SMOKE_SECRET", "value": "s3cret"}) pr_secret = "smoke-runs-pr" req("PUT", f"/api/projects/{pid}/webhooks/github", token, {"secret": pr_secret}) + # `image:` because a fork's pull request runs in a container or not at all. A shell + # agent ignores it, so the same-repository run below still runs on the host. create_pipeline(token, pid, "pr", { "name": "pr", "on": {"pull_request": {"branches": ["main"]}}, "steps": [{"id": "probe", "name": "probe", "needs": [], "labels": labels, + "image": "alpine:3", "run": '[ -n "$SMOKE_SECRET" ] && echo has-secret || echo no-secret'}], }) # Same repository: trusted, so any agent runs it and the secret is there. This is @@ -249,23 +254,59 @@ def main() -> None: code, scoped = req("POST", "/api/agents", token, {"name": "smoke-runs-scoped", "labels": labels, "concurrency": 1, "project_id": pid}) check("create a project-scoped agent", code == 201, scoped) - env = dict(os.environ, FIBER_AGENT_TOKEN=scoped["token"], FIBER_API_URL=WS, - FIBER_AGENT_USE_DOCKER="false", FIBER_AGENT_LABELS=",".join(labels), - FIBER_AGENT_NAME="smoke-runs-scoped", - FIBER_AGENT_WORKSPACE_DIR=os.path.join(tempfile.gettempdir(), "fiber-smoke-runs-ws")) - agent = subprocess.Popen([os.path.join(ROOT, "target/debug/fiber-agent")], env=env, - stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - try: - fork = wait_run(token, fork_run) - probe = steps_by_id(fork)["probe"] - check("the project's own agent runs it, without the project's secrets", - fork["run"]["status"] == "succeeded" - and any("no-secret" in l["data"] for l in log_lines(token, probe["id"])), - [l["data"] for l in log_lines(token, probe["id"])]) - finally: + + def scoped_agent(use_docker: bool) -> subprocess.Popen: + env = dict(os.environ, FIBER_AGENT_TOKEN=scoped["token"], FIBER_API_URL=WS, + FIBER_AGENT_USE_DOCKER="true" if use_docker else "false", + FIBER_AGENT_LABELS=",".join(labels), FIBER_AGENT_NAME="smoke-runs-scoped", + FIBER_AGENT_WORKSPACE_DIR=os.path.join(tempfile.gettempdir(), "fiber-smoke-runs-ws")) + # Whatever the caller's shell exports, this agent takes the default for its mode. + env.pop("FIBER_AGENT_ALLOW_HOST_SHELL", None) + return subprocess.Popen([os.path.join(ROOT, "target/debug/fiber-agent")], env=env, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + + def stop(agent: subprocess.Popen) -> None: # By PID, never by name: a pattern can match the shell that started this. agent.send_signal(signal.SIGTERM) agent.wait(timeout=30) + + try: + # A shell agent has nowhere to run a fork's code but its own account, so it + # refuses — even though the step names an image, which a shell agent ignores. + agent = scoped_agent(use_docker=False) + try: + fork = wait_run(token, fork_run) + finally: + stop(agent) + probe = steps_by_id(fork)["probe"] + check("the project's shell agent refuses to run a fork's PR on its host", + fork["run"]["status"] == "failed" + and "fork pull request steps only run in containers" in (probe.get("error") or ""), + (fork["run"]["status"], probe.get("error"))) + + # In a container it runs, and without the project's secrets. Needs an engine, + # which CI's smoke host has; skipped out loud where there is none. + try: + have_docker = subprocess.run(["docker", "info"], stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, timeout=30).returncode == 0 + except (OSError, subprocess.TimeoutExpired): + have_docker = False + if have_docker: + fork_run = pull_request(pid, pr_secret, "mallory/fork", 43)["started"][0] + agent = scoped_agent(use_docker=True) + try: + # Longer than the default: the first run pulls the image. + fork = wait_run(token, fork_run, timeout=240) + finally: + stop(agent) + probe = steps_by_id(fork)["probe"] + check("the project's docker agent runs it in a container, without the project's secrets", + fork["run"]["status"] == "succeeded" + and any("no-secret" in l["data"] for l in log_lines(token, probe["id"])), + [l["data"] for l in log_lines(token, probe["id"])]) + else: + print("SKIP the project's docker agent runs a fork's PR in a container: no docker engine here") + finally: req("DELETE", f"/api/agents/{scoped['agent']['id']}", token) # ---- one durable fiber runs to completion ------------------------------------ From 779da31d82561d873d99897326c5aa96f8100b8f Mon Sep 17 00:00:00 2001 From: Blake Bauman Date: Wed, 11 Nov 2026 15:42:13 -0700 Subject: [PATCH 2/2] Offer a fork's pull request only to an agent that will refuse host shell for it An agent below protocol revision 3 ignores `untrusted` on the offer and would run the fork's step on its host. The scheduler now keeps the agent's Hello revision and leaves untrusted steps out of the candidates for a project agent below UNTRUSTED_OFFER_PROTOCOL: filtered in the query, before the scan limit and before any lease, so the steps stay queued for an agent that qualifies and the trusted steps behind them are still offered. The session warns once when such an agent connects. The plain Compose agent gains the executor=shell label, because docker=true on the Docker agent only lets an image step opt in: a step matches any agent holding every label it asks for. The docs say that, that the agent-docker pin needs an agent image of this release, and that a refused step with retries may land on a different agent. Co-Authored-By: Claude Fable 5.2 --- CHANGELOG.md | 26 ++++++++--- crates/fiber-api/src/ws.rs | 15 ++++++- crates/fiber-core/src/store.rs | 9 ++++ crates/fiber-proto/src/lib.rs | 16 ++++--- crates/fiber-scheduler/src/lib.rs | 72 ++++++++++++++++++++++++++++--- deploy/.env.example | 7 ++- deploy/docker-compose.yml | 16 +++++-- docs/agents.md | 52 ++++++++++++++++++---- docs/configuration.md | 4 +- docs/triggers.md | 15 ++++--- 10 files changed, 194 insertions(+), 38 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fef245a..1dc7f18 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,8 +17,10 @@ runs. **A fork's pull request now runs in containers or fails** — on a shell agent it fails, every step, and no setting changes that. Wire: `Offer` gains `untrusted` (defaulted, so 0.6.7 agents and servers keep working with -the new ones) and `PROTOCOL_VERSION` is now `3`. The fork-PR control is the agent's: -upgrade the agents, not only the API. +the new ones) and `PROTOCOL_VERSION` is now `3`. The fork-PR control is carried out +by the agent, so the server offers a fork pull request's steps only to an agent that +declared revision 3: **until a project agent at this release is online, those steps wait +in `queued`.** Upgrade the agents, not only the API. ### Security @@ -35,7 +37,11 @@ upgrade the agents, not only the API. - A refused step fails before anything is cloned or restored, with the remedy in its log and its error: ``this agent runs steps in containers only: add `image:` to the step, or set FIBER_AGENT_ALLOW_HOST_SHELL=true on the agent``. - - The `agent-docker` Compose profile pins it to `false`. + - The `agent-docker` Compose profile pins it to `false`. The pin only takes effect when + the agent image is this release or later: the profile builds `FROM` the published + `fiber-agent:${FIBER_VERSION}`, and a 0.6.7 binary ignores the variable silently. + Rebuild the service, then confirm in its startup log a line beginning `steps run in + containers only`. - An agent installed with `install-agent.sh --docker` gets the refusing default with no extra flag. A `FIBER_AGENT_ALLOW_HOST_SHELL` line added to `/etc/fiber/agent.env` is kept across re-runs of the installer. @@ -47,7 +53,10 @@ upgrade the agents, not only the API. `image:` on every step and a project agent with Docker on; on a shell agent it fails with `fork pull request steps only run in containers, and this is a shell agent …`. That is the intended outcome: a shell agent has nowhere to run the code but its own - account. An agent older than this release ignores the flag. + account. An agent older than this release does not know the flag, so the server does + not offer it such a step: a project agent whose `Hello` declares a protocol revision + below 3 is passed over, the step stays `queued` rather than failing, and the server + warns once when that agent connects. The agent keeps taking the project's own runs. [triggers](docs/triggers.md#pull-requests-from-forks). - **Seven RustSec advisories in the lockfile, all fixed by upgrades.** `rustls` 0.23.46 (RUSTSEC-2026-0346 to -0349), `rustls-webpki` 0.103.16 (RUSTSEC-2026-0382: certificate @@ -64,9 +73,12 @@ upgrade the agents, not only the API. workflow runs `cargo deny check` before it builds anything. - The release workflow passes the tag to its version check through the environment rather than interpolating it into the script. -- **The `agent-docker` Compose profile registers with `docker=true`** on top of - `FIBER_AGENT_LABELS`, so a step can ask for it with `labels: [docker=true]`. The plain - `agent` profile is unchanged. +- **The Compose agents carry a label each: `docker=true` on the `agent-docker` profile, + `executor=shell` on the plain `agent` profile,** both appended to `FIBER_AGENT_LABELS`. + An `image:` step asks for `docker=true` to reach the Docker agent. A step without + `image:` asks for `executor=shell` to stay off it — `docker=true` alone does not keep + it away, since a step matches any agent that has every label it asks for. + [agents](docs/agents.md#steps-that-would-run-on-the-host). - **`smoke-runs` covers the refusal.** Its fork-PR check now expects a shell agent to refuse the run, then — where a Docker engine is available — runs the same pull request in a container to check it gets no secrets. diff --git a/crates/fiber-api/src/ws.rs b/crates/fiber-api/src/ws.rs index d26b276..bf7d675 100644 --- a/crates/fiber-api/src/ws.rs +++ b/crates/fiber-api/src/ws.rs @@ -991,9 +991,22 @@ async fn handle_agent( break; }; let project_id = agent_row.project_id; + // Once per session, where an operator will see it; the scheduler's own + // note about each fill is at debug. + if project_id.is_some() + && !fiber_scheduler::takes_untrusted_steps(project_id, protocol_version) + { + warn!( + %agent_id, %name, protocol_version, + required = fiber_proto::UNTRUSTED_OFFER_PROTOCOL, + "this project agent predates the untrusted-offer protocol and would run \ + a fork's pull request on its host; it will not be offered those steps, \ + which stay queued until an upgraded agent is online in the project's pool" + ); + } state .scheduler - .register_agent(agent_id, labels, concurrency, project_id) + .register_agent(agent_id, labels, concurrency, project_id, protocol_version) .await; // Re-bind connection under confirmed agent_id (same as token) state diff --git a/crates/fiber-core/src/store.rs b/crates/fiber-core/src/store.rs index f61d94a..f76e80e 100644 --- a/crates/fiber-core/src/store.rs +++ b/crates/fiber-core/src/store.rs @@ -2353,10 +2353,17 @@ impl Store { /// steps whose labels the agent satisfies (`labels <@ agent labels`; an empty /// requirement is contained in anything), so a long run of steps for some *other* /// kind of agent cannot push this agent's work past the scan limit. + /// + /// `include_untrusted` is false for a project agent too old to honour the offer's + /// `untrusted` flag (it would run the fork's step on its host). Filtered here, in the + /// query, for the same scan-limit reason: untrusted steps this agent may not take + /// must not fill the window ahead of the trusted ones it may. Nothing is leased and + /// nothing fails — the steps stay queued for an agent that qualifies. pub async fn list_queued_steps_for_pool( &self, agent_project_id: Option, agent_labels: &[String], + include_untrusted: bool, ) -> Result> { if let Some(pid) = agent_project_id { Ok(sqlx::query_as::<_, StepRun>(AssertSqlSafe(format!( @@ -2366,11 +2373,13 @@ impl Store { WHERE s.status = 'queued' AND r.project_id = $1 AND (s.not_before IS NULL OR s.not_before <= NOW()) AND s.labels <@ $2 + AND ($3 OR NOT r.untrusted) ORDER BY s.queued_at, s.id LIMIT {QUEUE_SCAN_LIMIT}" ))) .bind(pid) .bind(json!(agent_labels)) + .bind(include_untrusted) .fetch_all(&self.pool) .await?) } else { diff --git a/crates/fiber-proto/src/lib.rs b/crates/fiber-proto/src/lib.rs index bcd8cbc..084d8db 100644 --- a/crates/fiber-proto/src/lib.rs +++ b/crates/fiber-proto/src/lib.rs @@ -22,11 +22,16 @@ pub mod validate; /// - `3` — the agent reads `untrusted` on [`ServerMessage::Offer`] and never runs such a /// step on its host shell: it runs in a container or fails the step. It also refuses a /// step with no `image` when it is a Docker agent, unless its operator set -/// `FIBER_AGENT_ALLOW_HOST_SHELL=true`. Nothing gates on this yet either; it is here so -/// a server can tell an agent that enforces this from one that would run a fork's -/// pull request on the host. +/// `FIBER_AGENT_ALLOW_HOST_SHELL=true`. The server gates on this one: see +/// [`UNTRUSTED_OFFER_PROTOCOL`]. pub const PROTOCOL_VERSION: u32 = 3; +/// The revision from which an agent honours `untrusted` on [`ServerMessage::Offer`]. The +/// server offers a step of an untrusted run only to an agent whose `Hello` declared this +/// or later: an older one would ignore the flag and run a fork's pull request on its +/// host, so for it the step stays queued until an agent that qualifies is online. +pub const UNTRUSTED_OFFER_PROTOCOL: u32 = 3; + #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[serde(rename_all = "snake_case")] pub enum StepStatus { @@ -414,8 +419,9 @@ pub enum ServerMessage { /// agent that reads this never runs such a step on its host shell, whatever /// `FIBER_AGENT_ALLOW_HOST_SHELL` says: it runs in a container or fails. Absent /// from older servers, which the agent reads as the old contract — a trusted - /// run. An agent below revision 3 ignores it and runs the step as before, so the - /// server may rely on it only for an agent whose `Hello` said 3 or later. + /// run. An agent below [`UNTRUSTED_OFFER_PROTOCOL`] would ignore it and run the + /// step as before, so the server never sends such an agent an offer with this + /// set: the step waits in the queue for one that declared that revision. #[serde(default)] untrusted: bool, }, diff --git a/crates/fiber-scheduler/src/lib.rs b/crates/fiber-scheduler/src/lib.rs index 7fd8d01..ad84b7b 100644 --- a/crates/fiber-scheduler/src/lib.rs +++ b/crates/fiber-scheduler/src/lib.rs @@ -109,6 +109,9 @@ pub struct AgentPresence { pub reserved: u32, /// `None` = global pool. pub project_id: Option, + /// `Hello.protocol_version`; `0` from an agent older than the field. Decides + /// whether the agent is offered untrusted steps ([`takes_untrusted_steps`]). + pub protocol_version: u32, } impl AgentPresence { @@ -358,6 +361,7 @@ impl Scheduler { labels: Vec, concurrency: u32, project_id: Option, + protocol_version: u32, ) { // Nothing about the agent's running steps is kept here: a lease outlives the // session that took it, so every offer counts the rows the database shows @@ -376,6 +380,7 @@ impl Scheduler { concurrency: concurrency.max(1), reserved, project_id, + protocol_version, }, ); } @@ -803,7 +808,7 @@ impl Scheduler { ) -> Result> { // Fail closed without presence: a socket that never sent Hello (or whose // presence was reclaimed) gets nothing rather than the global pool. - let (agent_labels, concurrency, others_reserved) = { + let (agent_labels, concurrency, others_reserved, protocol) = { let mut agents = self.agents.write().await; match agents.get_mut(&agent_id) { Some(a) => { @@ -811,7 +816,7 @@ impl Scheduler { if !a.try_reserve_slot() { return Ok(None); } - (a.labels.clone(), a.concurrency, others) + (a.labels.clone(), a.concurrency, others, a.protocol_version) } None => { debug!(%agent_id, "no presence for agent; not offering"); @@ -820,7 +825,14 @@ impl Scheduler { } }; let leased = self - .lease_within_slots(agent_id, agent_labels, concurrency, others_reserved, skip) + .lease_within_slots( + agent_id, + agent_labels, + concurrency, + others_reserved, + protocol, + skip, + ) .await; // Leased or not, the reservation is done: a leased step is now `running` in // the database and counted from there. @@ -837,19 +849,22 @@ impl Scheduler { agent_labels: Vec, concurrency: u32, others_reserved: u32, + protocol: u32, skip: &[Uuid], ) -> Result> { let db_running = self.store.count_running_steps_for_agent(agent_id).await?; if !slots_available(concurrency, db_running, others_reserved) { return Ok(None); } - self.try_lease_for(agent_id, agent_labels, skip).await + self.try_lease_for(agent_id, agent_labels, protocol, skip) + .await } async fn try_lease_for( &self, agent_id: Uuid, agent_labels: Vec, + protocol: u32, skip: &[Uuid], ) -> Result> { // Pool scope is authoritative from the DB row, never from in-memory state. @@ -858,9 +873,20 @@ impl Scheduler { None => return Ok(None), }; + // Decided before the candidates are read, so an untrusted step this agent may + // not take is never leased, and never occupies a place in the scan window. + let include_untrusted = takes_untrusted_steps(agent_project_id, protocol); + if agent_project_id.is_some() && !include_untrusted { + // Debug: this is every fill of every such agent. The line an operator sees + // is the warning the session logs once, at `Hello`. + debug!( + %agent_id, protocol, + "agent predates the untrusted-offer protocol; fork pull request steps stay queued for another agent" + ); + } let queued = self .store - .list_queued_steps_for_pool(agent_project_id, &agent_labels) + .list_queued_steps_for_pool(agent_project_id, &agent_labels, include_untrusted) .await?; for step in queued { if skip.contains(&step.id) { @@ -1335,6 +1361,20 @@ pub fn disconnect_policy(hello: Option, goodbye: bool) -> DisconnectPolicy } } +/// Whether an agent may be offered steps of an untrusted run (a fork's pull request). +/// +/// Two conditions, both required. The agent is bound to a project — the query scopes it +/// to that project's runs, and the global pool never sees an untrusted step. And its +/// `Hello` declared [`fiber_proto::UNTRUSTED_OFFER_PROTOCOL`] or later (`hello` is `0` +/// from an agent older than the field): an agent below it ignores `untrusted` on the +/// offer and would run the fork's step on its host shell. +/// +/// False means ineligible, not failed: the step stays queued for an agent that +/// qualifies, which matters when an old and a new agent serve the same project. +pub fn takes_untrusted_steps(agent_project_id: Option, hello: u32) -> bool { + agent_project_id.is_some() && hello >= fiber_proto::UNTRUSTED_OFFER_PROTOCOL +} + /// Whether a message's `attempt` (echoed from the offer; `None` from older agents) /// refers to the attempt the row is on. A `step_run_id` is stable across attempts, so /// this is what keeps a line, artifact or completion that an agent held through a @@ -1420,6 +1460,7 @@ mod tests { concurrency, reserved, project_id: None, + protocol_version: fiber_proto::PROTOCOL_VERSION, } } @@ -1683,6 +1724,27 @@ mod tests { ); } + #[test] + fn only_a_current_project_agent_is_offered_untrusted_steps() { + let project = Some(Uuid::nil()); + let gate = fiber_proto::UNTRUSTED_OFFER_PROTOCOL; + // A project agent that honours `untrusted` on the offer. + assert!(takes_untrusted_steps(project, gate)); + assert!(takes_untrusted_steps(project, gate + 1)); + assert!(takes_untrusted_steps( + project, + fiber_proto::PROTOCOL_VERSION + )); + // One revision short, and an agent older than `protocol_version` itself (0): + // both would run a fork's step on the host, so neither is offered one. + assert!(!takes_untrusted_steps(project, gate - 1)); + assert!(!takes_untrusted_steps(project, 0)); + // The global pool never is, however new the agent. + assert!(!takes_untrusted_steps(None, gate)); + assert!(!takes_untrusted_steps(None, u32::MAX)); + assert!(!takes_untrusted_steps(None, 0)); + } + #[test] fn a_session_that_never_said_hello_touches_no_leases() { // A second socket for the same token that closes before Hello: the agent's diff --git a/deploy/.env.example b/deploy/.env.example index ecf4b47..06e6b09 100644 --- a/deploy/.env.example +++ b/deploy/.env.example @@ -122,7 +122,12 @@ FIBER_AGENT_CONCURRENCY=2 # For pipelines with `image:` steps; uses the token above, and the labels above plus # `docker=true`. It runs steps in containers only (FIBER_AGENT_ALLOW_HOST_SHELL is pinned # to false in the Compose file, not read from here): a step without `image:` is refused, -# because on this agent it would run as root beside the Docker socket. Bound at the same +# because on this agent it would run as root beside the Docker socket. That needs +# FIBER_VERSION at this release or later: the image is built FROM the published agent, and +# a 0.6.7 binary ignores the variable silently. Check the agent's startup log for a line +# beginning "steps run in containers only". `docker=true` lets an `image:` step ask for +# this agent; a step without `image:` stays off it by asking for `executor=shell`, which +# the plain agent's labels end with. Bound at the same # path inside the agent and on the Docker host. On Docker Desktop use a path inside its # VM, e.g. `docker volume create fiber_docker_ws` then # /var/lib/docker/volumes/fiber_docker_ws/_data (docs/agents.md). diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml index 6efa515..6799406 100644 --- a/deploy/docker-compose.yml +++ b/deploy/docker-compose.yml @@ -281,7 +281,9 @@ services: FIBER_API_URL: ws://fiber-api:8080 FIBER_AGENT_TOKEN: ${FIBER_AGENT_TOKEN:-} FIBER_AGENT_NAME: ${FIBER_AGENT_NAME:-compose} - FIBER_AGENT_LABELS: ${FIBER_AGENT_LABELS:-os=linux} + # `executor=shell` is what a step without `image:` asks for to stay off the + # agent-docker profile's agent, which has every other label this one does. + FIBER_AGENT_LABELS: ${FIBER_AGENT_LABELS:-os=linux},executor=shell FIBER_AGENT_CONCURRENCY: ${FIBER_AGENT_CONCURRENCY:-2} RUST_LOG: ${RUST_LOG:-info,fiber_agent=info} # `image:` steps need the docker CLI and the socket, which this image deliberately @@ -299,12 +301,20 @@ services: # Optional worker for pipelines with `image:` steps: # `docker compose --profile agent-docker up -d fiber-agent-docker`. Same token and labels - # as fiber-agent (run one or the other), plus `docker=true` so a step can ask for it. - # It gets the Docker socket — root on the host. + # as fiber-agent (run one or the other), plus `docker=true`. It gets the Docker socket — + # root on the host. # # Which is why FIBER_AGENT_ALLOW_HOST_SHELL is pinned to false here rather than read # from .env: a step without `image:` would run in THIS container, as root, next to that # socket. Such a step is refused; give it an `image:` or run it on the plain agent. + # The pin needs an agent binary that knows the variable: the image is built FROM + # fiber-agent:${FIBER_VERSION}, and 0.6.7 or older ignores it silently. Confirm with + # the startup log line "steps run in containers only". + # + # `docker=true` lets an `image:` step ask for this agent. It does NOT keep a step + # without `image:` away: a step matches any agent holding every label it asks for, and + # this agent holds all of the plain agent's. An imageless step opts out by asking for + # `executor=shell`, which only the plain agent has. # # The workspace is bound at the SAME absolute path inside this container and on the # Docker host. Steps run as `docker run -v /:/workspace`, and the daemon diff --git a/docs/agents.md b/docs/agents.md index 211b905..34cc8e8 100644 --- a/docs/agents.md +++ b/docs/agents.md @@ -6,7 +6,7 @@ An agent's identity is the token it connected with. The server binds `agent_id` A lease belongs to the agent, not to the socket. A step keeps running through a lost connection — an API restart, a proxy timeout, a network blip — and the agent renews its leases on its first heartbeat back; the server judges every late message by the row (is the step still `running` under this agent, on the attempt the message names?), never by which session it arrived on. Every `Offer` carries the attempt number and the agent echoes it on each log batch, artifact, and completion — and on the HTTP artifact routes, as `X-Fiber-Attempt` on the upload and an `attempt` field in the presign and complete bodies ([api](./api.md#agent-endpoints)) — so output an agent held through a reclaim can never be filed under, overwrite, or close the attempt that replaced it; a message whose attempt is not the row's is dropped with a warning, and an upload for one is refused with `401`. A log line the agent re-sends after an aborted write can appear twice: `log_lines` is append-only with no dedupe. See [Lifecycle](#lifecycle) for the limits. -`Hello` carries `protocol_version` (`fiber_proto::PROTOCOL_VERSION`, currently **3**; absent from older agents, read as `0`). The server logs it and uses it to tell an agent that keeps its steps across sessions (1 and up) from one that cancels them on any close (0), and a later server may refuse a revision it no longer supports. It is not yet stored on the agent row. `Welcome` carries the **server's** revision the same way, and the agent reads its absence as "older than `log_batch`" and sends one `log_chunk` per line to that server — so a rolling deploy in either order keeps a build's log, though **upgrade the API before the agents** remains the supported direction. Revision 2 adds batched log messages; see [Log path](#log-path). Revision 3 means the agent reads `untrusted` on an `Offer` and never runs a fork pull request's step on its host shell, and that a Docker agent refuses a step without `image:` unless told otherwise; see [Executor](#executor). `untrusted` is absent from an older server, which the agent reads as a trusted run, and an older agent ignores it — nothing gates on the revision yet. +`Hello` carries `protocol_version` (`fiber_proto::PROTOCOL_VERSION`, currently **3**; absent from older agents, read as `0`). The server logs it and uses it to tell an agent that keeps its steps across sessions (1 and up) from one that cancels them on any close (0), and a later server may refuse a revision it no longer supports. It is not yet stored on the agent row. `Welcome` carries the **server's** revision the same way, and the agent reads its absence as "older than `log_batch`" and sends one `log_chunk` per line to that server — so a rolling deploy in either order keeps a build's log, though **upgrade the API before the agents** remains the supported direction. Revision 2 adds batched log messages; see [Log path](#log-path). Revision 3 means the agent reads `untrusted` on an `Offer` and never runs a fork pull request's step on its host shell, and that a Docker agent refuses a step without `image:` unless told otherwise; see [Executor](#executor). `untrusted` is absent from an older server, which the agent reads as a trusted run. An older agent would ignore it, so the server gates on the revision (`fiber_proto::UNTRUSTED_OFFER_PROTOCOL`): a step of an untrusted run is offered only to a project agent whose `Hello` declared 3 or later, and otherwise stays `queued` — it is not leased and does not fail — until such an agent (0.6.8+) is online in the project's pool. The server warns once per session when a project agent below that revision connects. ## Pools @@ -281,8 +281,19 @@ published agent plus the Docker CLI locally (`deploy/agent-docker.Dockerfile`), engine's socket, and runs as root, because the socket is root on the host anyway. For that reason it runs steps **in containers only**: `FIBER_AGENT_ALLOW_HOST_SHELL` is pinned to `false` in the Compose file, so a step without `image:` is refused rather than run as -root inside the agent's container, next to the socket. It registers with the labels of -the plain agent plus `docker=true`, so a step can ask for it with `labels: [docker=true]`: +root inside the agent's container, next to the socket. + +**The pin only works on an agent that knows the variable.** The profile's image is built +`FROM` the published `fiber-agent:${FIBER_VERSION}`, and a 0.6.7 or older binary ignores +`FIBER_AGENT_ALLOW_HOST_SHELL` without a word and runs imageless steps on its host as +before. Set `FIBER_VERSION` to this release or later, rebuild the service, and confirm it +took effect: the agent's startup log (`docker compose … logs fiber-agent-docker`) has a +line beginning `steps run in containers only`. No such line means the old binary. + +It registers with `FIBER_AGENT_LABELS` plus `docker=true`, and the plain `agent` profile +with `FIBER_AGENT_LABELS` plus `executor=shell`. See +[Steps that would run on the host](#steps-that-would-run-on-the-host) for what those two +labels do and do not route. ```bash echo "FIBER_AGENT_DOCKER_WORKSPACE=/var/lib/fiber/workspaces" >> deploy/.env @@ -457,11 +468,16 @@ could then run nothing. A fork's pull request is never run on the host shell, and the setting does not change that: its author is not someone the operator vouched for. Its steps need an `image:` and a -project agent with Docker on. On a shell agent they fail, by design — see +project agent with Docker on. On a shell agent they fail, by design. The server offers +them only to a project agent at 0.6.8 or later (protocol revision 3), since an older one +would not know to refuse; until one is online they wait in `queued` — see [triggers](./triggers.md#pull-requests-from-forks). -A refused step fails, and a retry will fail the same way. Its log and its `error` say -which case it was and what to change; nothing is cloned or restored first: +A refused step fails, and a retry on the same agent fails the same way. With `retries: N` +a refused step is leased again up to N times, each to whichever matching agent has a free +slot — so in a pool that mixes agents it may be refused N more times, or land on an agent +that accepts it and pass. Its log and its `error` say which case it was and what to +change; nothing is cloned or restored first: ```text this agent runs steps in containers only: add `image:` to the step, or set FIBER_AGENT_ALLOW_HOST_SHELL=true on the agent @@ -469,9 +485,27 @@ fork pull request steps only run in containers: this run builds code from outsid fork pull request steps only run in containers, and this is a shell agent (FIBER_AGENT_USE_DOCKER=false) … ``` -To keep steps without `image:` away from a Docker agent in the first place, give the two -kinds of agent different labels and ask for one in the step's `labels:` — the Compose -`agent-docker` profile adds `docker=true` for this. +Labels decide which agent a step reaches, and a step matches an agent that has **every** +label the step asks for — extra labels on the agent do not exclude it. So a label that +only the Docker agent carries (`docker=true`, which the Compose `agent-docker` profile +adds) lets an `image:` step opt **in** to it; it does not keep a step without `image:` +away, because that step asks for nothing the Docker agent lacks. To keep it away, the +shell agent needs a label of its own that the Docker agent does not have, and the step +has to ask for it. The Compose `agent` profile adds `executor=shell` for this: + +```yaml +steps: + build: + image: rust:1-bookworm + labels: [os=linux, docker=true] # only a Docker agent + notify: + run: ./scripts/notify.sh + labels: [os=linux, executor=shell] # only a shell agent +``` + +A step that asks for neither can be leased by either kind of agent, and fails if the +Docker agent gets it first. Agents you start yourself carry only the labels you give them +in `FIBER_AGENT_LABELS`. Releases up to 0.6.7 ran a step without `image:` on a Docker agent's host without being asked, and ran a fork's pull request on the host shell like any other run. diff --git a/docs/configuration.md b/docs/configuration.md index 21805ad..336a469 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -94,7 +94,7 @@ file passes the variable through as an empty string and each reader falls back. | `FIBER_METRICS_TOKEN` | empty | Bearer token for `/metrics`; empty = off | | `OTEL_EXPORTER_OTLP_ENDPOINT` | empty | OTLP collector base URL. Use this name in Compose, not `FIBER_OTEL_ENDPOINT`: the API reads the alias only when this one is *absent*, and Compose always defines what it passes | | `RUST_LOG` | `info,fiber_api=info` | API log filter | -| `FIBER_AGENT_TOKEN` / `_NAME` / `_LABELS` / `_CONCURRENCY` | — / `compose` / `os=linux` / `2` | The optional `agent` profile | +| `FIBER_AGENT_TOKEN` / `_NAME` / `_LABELS` / `_CONCURRENCY` | — / `compose` / `os=linux` / `2` | The optional `agent` profile. Compose appends `executor=shell` to the labels there, and `docker=true` on the `agent-docker` profile | ## fiber-agent @@ -121,7 +121,7 @@ Installed by `scripts/install-agent.sh` into `/etc/fiber/agent.env` (root-owned, in Compose they come from `deploy/.env` (`FIBER_AGENT_TOKEN`, `FIBER_AGENT_NAME`, `FIBER_AGENT_LABELS`, `FIBER_AGENT_CONCURRENCY`, `FIBER_AGENT_USE_DOCKER`). The Compose `agent-docker` profile pins `FIBER_AGENT_ALLOW_HOST_SHELL=false` and appends `docker=true` -to the labels; the installer writes `FIBER_AGENT_ALLOW_HOST_SHELL` only to carry over a +to the labels, and the `agent` profile appends `executor=shell`; the installer writes `FIBER_AGENT_ALLOW_HOST_SHELL` only to carry over a line already in `agent.env`. The installer also writes `FIBER_AGENT_INSTALLED_VERSION` and `FIBER_AGENT_INSTALLED_SHA256` there so a re-run can say what it is replacing; no crate reads them, and they are not deployment configuration. diff --git a/docs/triggers.md b/docs/triggers.md index 31ff566..432fe0a 100644 --- a/docs/triggers.md +++ b/docs/triggers.md @@ -92,8 +92,9 @@ and its secrets belong to the base repository. Fiber marks such a run **untruste Provenance that cannot be determined from the payload counts as untrusted, and a retry of an untrusted run stays untrusted. 2. **Project-dedicated agents only.** Untrusted steps are never offered to the global - pool — only to an agent bound to that project. If the project has no such agent, the - steps stay queued. + pool — only to an agent bound to that project, and only to one new enough to apply + the third control (0.6.8 or later). If the project has no such agent, the steps stay + queued. 3. **Containers only.** The offer tells the agent the run is untrusted, and the agent never runs such a step on its host shell: a step with an `image:`, on an agent with Docker on, runs in its container; anything else **fails** before a line of the fork's @@ -112,9 +113,13 @@ the first fork PR arrives: - A step without `image:` fails the same way on a Docker agent, even one that allows host shell for the project's own runs. -The third control is the agent's, so it holds only on agents that have it: an agent older -than protocol revision 3 ignores the flag and runs the step as before. Upgrade the agents -that serve projects with fork pull requests. +The third control is carried out by the agent, and an agent older than 0.6.8 (protocol +revision below 3) does not know the flag — it would run the step on its host. So the +server does not offer it one: **a fork pull request's steps wait in `queued` until an +agent at 0.6.8 or later is online in the project's pool.** Nothing fails and nothing is +leased in the meantime; an older agent in the same pool keeps taking the project's own +runs, and the server logs one warning when such an agent connects. If a fork's pull +request sits queued with a project agent online, check that agent's version. **These reduce the blast radius; they do not sandbox the code.** A container shares the host's kernel, has the network unless `FIBER_AGENT_DOCKER_NETWORK=none`, and writes to a