[#11] Add controlled network access for builds

docs/ROADMAP.md

@@ -2,7 +2,7 @@
 
 1. Finish the authentication slice. Done: Keycloak login, `member`/`lead`/`admin` realm roles, subject-based project ownership, role and private-project authorization tests, and configurable branding, and a Flyway baseline migration with Hibernate schema validation. Remaining: integration tests against a real Keycloak and server configuration.
 2. Repository access. Done: JGit fetch of public HTTPS repositories on `github.com` and `gitlab.com`, private access through environment-variable credential references (admins only), and the commit SHA recorded per build, including after a force push. Remaining: private credentials stored using a secrets provider, and project membership and repository permissions.
-3. Manual builds. Done: a single in-process runner using Docker or Podman with no network, CPU/memory/PID limits, a timeout and stored logs; builds left running by a restart are marked failed. Persistent FIFO queue with stored request details, atomic claims, configurable concurrency and queue limits, and restart recovery. Remaining: cancellation, container cleanup after a crash, and a separately isolated Linux worker using disposable rootless containers with a restricted network and no container-engine socket or host credentials exposed. Define these before enabling untrusted builds.
+3. Manual builds. Done: a single in-process runner using Docker or Podman with no network for steps by default and, with Podman, HTTPS access to admin-allowed hosts through a per-build egress proxy that always denies internal and link-local addresses, CPU/memory/PID limits, a timeout and stored logs; builds left running by a restart are marked failed. Persistent FIFO queue with stored request details, atomic claims, configurable concurrency and queue limits, and restart recovery. Remaining: cancellation, container cleanup after a crash, and a separately isolated Linux worker using disposable rootless containers with a restricted network and no container-engine socket or host credentials exposed. Define these before enabling untrusted builds.
 4. Multi-user deployment: TLS, secure Keycloak settings and secrets, database backups, audit records, access control, rate limits, concurrency controls and tested upgrade/migration procedures.
 5. CI configuration, webhooks, artifacts and additional worker platforms.
 

docs/SECURITY.md

@@ -70,11 +70,34 @@ Controls that are in place
 * Only HTTPS repositories on an allowed host are accepted for builds and branch discovery; redirects, embedded credentials, query strings, fragments and non-default ports are rejected. See [Repository addresses](#repository-addresses).
 * Branch discovery for private projects is not attempted because credentials are not stored on the project; the endpoint returns an empty list with reason `private_repository`.
 * Branch names are validated as Git references. Builds may only use images an admin has allowed; image names are validated against a restrictive pattern.
-* The branch is fetched afresh with JGit at depth 1 to read its workflow file and commit ID. The build then builds an image from the workflow: it installs `git`, `make`, `podman` and `podman-compose` and the workflow's `dependencies` as root, with network access, clones the branch at that commit as `65534:65534`, and runs the steps as `65534:65534` without network. The installs and the clone have network access.
+* The branch is fetched afresh with JGit at depth 1 to read its workflow file and commit ID. The build then builds an image from the workflow: it installs `git`, `make`, `podman` and `podman-compose` and the workflow's `dependencies` as root, with network access, clones the branch at that commit as `65534:65534`, and runs the steps as `65534:65534` without network, except through the egress proxy (see [Network for build steps](#network-for-build-steps)). The installs and the clone have network access.
 * The image build runs with `no-new-privileges`, 1 GiB of memory and one CPU under Podman; Docker applies neither limit. The image is removed when the build ends.
 * Builds run up to the concurrency limit (default 1, at most 16 when set by an admin); further requests wait in a bounded queue. Each build times out after ten minutes. Output is stored in full, up to the `BUILD_LOG_MAX_BYTES` safety limit (default 1 GiB); output beyond it is discarded and the build is marked truncated.
 * Builds left in the `RUNNING` state by a server restart are marked `FAILED` at startup. Startup also removes leftover `cisys-build-*` containers and images and `cisys-*` checkout directories from an unclean stop; a cleanup failure is logged and does not stop startup.
 
+### Network for build steps
+
+Steps run with `--network=none`. An admin can allow destinations under **Settings → Network**; an empty list (the default) means steps have no network at all.
+
+How it is enforced
+
+* With a non-empty list, the backend starts one `EgressProxy` per build on a Unix socket in the build's private temporary directory (`0700`). The socket's own directory is `0711`, so the step user can reach the socket but not list it. Podman mounts that directory read-only into every `RUN` (`--volume …:/run/cisys:ro,Z`). The step keeps `--network=none`; `socat` forwards `127.0.0.1:3128` in the step to the socket. Bypassing the proxy variables therefore gains nothing: there is no other route out.
+* The proxy accepts HTTP `CONNECT host:port` only. The port must be in `cisys.build.network.ports` (`BUILD_NETWORK_PORTS`, default `443`). The host must match an entry: an exact name, `*.suffix` for subdomains, or a public IP or CIDR range.
+* Protected destinations are always denied, whatever the list says: loopback, private, CGNAT, link-local (including `169.254.169.254`), multicast, documentation and reserved ranges, IPv4-mapped, IPv4-compatible, NAT64, 6to4 and Teredo forms, every address of the backend host's interfaces, and the hosts and addresses of the datasource, Keycloak and `quarkus.http.host`. Entries that are or overlap protected ranges are rejected when saved.
+* The proxy resolves the host once, checks every address it resolves to, and connects to the checked address, so a DNS change cannot redirect an approved tunnel.
+* Fail closed: an unresolvable host, a protected address, a refused or timed-out connection, or any other request is denied and logged in the build log as `==> Network denied: host:port (reason)`. A stored list that no longer validates, a proxy that cannot start, or internal service hosts that cannot be resolved fail the build. A build never falls back to ordinary networking.
+* The proxy limits concurrent connections (64), the time to send the request (10 s), the connect time (10 s) and idle tunnels (5 min). It is closed, and its socket and directory removed, on every exit path of the build.
+
+Docker and Podman
+
+* Tested on Podman 4.9 and Docker 29.9 with BuildKit (buildx 0.38), on 2026-11-14. With rootless Podman, the mounted socket worked from `RUN --network=none` as `65534:65534`, and so did several concurrent connections, while direct connections failed with *Network is unreachable*.
+* Docker's builder can forward a socket only through `--ssh`, which accepts the connection but passes no data unless it speaks the SSH agent protocol. Docker builds cannot mount host directories. With Docker, a build with a non-empty list therefore fails with `Network access is not supported with docker`.
+
+Residual risk
+
+* The proxy sees only the host and port, not the paths or the content inside TLS. Any allowed host can receive whatever a step sends it, so an allowed host is a possible exfiltration channel; a wildcard such as `*.example.com` widens that to every subdomain. Allow only registries you trust, and prefer exact names.
+* Installing the tools and dependencies and cloning the branch still have ordinary network access, as before.
+
 ### Repository addresses
 
 * Only HTTPS repositories on `cisys.git.allowed-hosts` are accepted; the default is `github.com` and `gitlab.com`.

docs/manual/en/03-roles.md

@@ -14,7 +14,7 @@ The **Lead** role is for people who run projects. Leads can do everything a memb
 
 ## Admin
 
-The **Admin** role manages the installation. Admins can see every project including private projects of other users, start and pin builds, use credential references for private Git repositories, manage the build images, change the branding, and change the settings such as build retention and the build queue.
+The **Admin** role manages the installation. Admins can see every project including private projects of other users, start and pin builds, use credential references for private Git repositories, manage the build images, change the branding, and change the settings such as build retention, the build queue and the hosts build steps may reach.
 
 ## Access summary
 
@@ -30,7 +30,7 @@ The **Admin** role manages the installation. Admins can see every project includ
 | Use a Git credential reference      | no     | no   | yes   |
 | Manage build images                 | no     | no   | yes   |
 | Change branding                     | no     | no   | yes   |
-| Change settings (retention, queue)  | no     | no   | yes   |
+| Change settings (retention, queue, network) | no | no | yes |
 
 "View" always means projects the user may access. The address of a project or build that does not exist or is not visible to the user opens the front page instead, so a private project is not revealed; the same applies to any unknown address.
 

docs/manual/en/05-builds.md

@@ -19,7 +19,7 @@ Members can view build history and logs for the projects they can access. Privat
 
 1. The branch is fetched afresh with JGit at depth 1, so a force push is reflected in the next build, and its commit ID is recorded.
 2. The branch's `.cisys/workflows/ci.yml` is read and stored with the build, then validated. A branch without the file, an invalid file, or a `container` image that is not on the build host or that an admin has not allowed on the **Images** page fails the build with a message in the log.
-3. The workflow becomes a Containerfile. Podman builds it with `no-new-privileges`, 1 GiB of memory, one CPU and no layer cache. The image installs `git`, `make`, `podman` and `podman-compose` if they are missing and the workflow's `dependencies`, clones the branch at the recorded commit, and runs the steps as `65534:65534` without network. See **Workflows**.
+3. The workflow becomes a Containerfile. Podman builds it with `no-new-privileges`, 1 GiB of memory, one CPU and no layer cache. The image installs `git`, `make`, `podman` and `podman-compose` if they are missing and the workflow's `dependencies`, clones the branch at the recorded commit, and runs the steps as `65534:65534` without network. When admins allow hosts under **Settings** → **Network**, steps reach those hosts over HTTPS through the egress proxy and nothing else (Podman only). See **Workflows**.
 4. The status is `PASSED` when the image build succeeds, otherwise `FAILED` with the exit code of the failing step. A build that exceeds ten minutes is stopped and reported as timed out. The image is removed afterwards.
 
 Build requests are stored as `QUEUED` and claimed in request order. By default one build runs at a time. Set `BUILD_CONCURRENCY` (`cisys.build.concurrency`) to change the number of concurrent builds and `BUILD_QUEUE_LIMIT` (`cisys.build.queue-limit`, default 100) to limit waiting requests. Both limits must be positive. Admins can change both under **Settings** → **Queue** (1–16 concurrent builds, 1–1000 queued builds); saved values replace the configured ones. A full queue returns HTTP 429 with a clear error. Requests keep their repository URL, branch and credential reference; credentials themselves are read from the environment when execution begins.
@@ -85,7 +85,7 @@ An image that is allowed but no longer present locally is listed separately, whe
 
 The runner is always on. It uses Podman when it is installed and Docker otherwise.
 
-**Trust boundary:** the runner allows trusted leads and admins to execute code through the local container daemon. Docker socket access grants host-level control to the backend. These limits are not a hostile-code sandbox. Use a dedicated machine, disk quotas and trusted repositories, and do not expose this setup to the Internet. Repository checkout sizes are not bounded. Installing the build tools and cloning the branch happen with network access, as root and as `65534:65534` respectively; the workflow's steps run without network.
+**Trust boundary:** the runner allows trusted leads and admins to execute code through the local container daemon. Docker socket access grants host-level control to the backend. These limits are not a hostile-code sandbox. Use a dedicated machine, disk quotas and trusted repositories, and do not expose this setup to the Internet. Repository checkout sizes are not bounded. Installing the build tools and cloning the branch happen with network access, as root and as `65534:65534` respectively; the workflow's steps run without network, or reach the allowed hosts through the egress proxy only. An allowed host can receive anything a step sends it; allow only registries you trust.
 
 ## Verification status