[#32] Bound the size of repository checkouts for builds

docs/SECURITY.md

@@ -32,6 +32,7 @@ The Keycloak admin console (<http://localhost:8180/admin>) uses `admin` / `admin
 | `PUT /api/branding`                       | no                        | no                   | yes   |
 | `GET`/`PUT /api/settings/retention`       | no                        | no                   | yes   |
 | `GET`/`PUT /api/settings/queue`           | no                        | no                   | yes   |
+| `GET`/`PUT /api/settings/checkout`        | no                        | no                   | yes   |
 | `GET /api/images` (allowed images)        | yes                       | yes                  | yes   |
 | `PUT /api/images`                         | no                        | no                   | yes   |
 | `GET/POST/DELETE /api/images/local`       | no                        | no                   | yes   |
@@ -70,7 +71,8 @@ Controls that are in place
 * Only HTTPS repositories on an allowed host are accepted for builds and branch discovery; redirects, embedded credentials, query strings, fragments and non-default ports are rejected. See [Repository addresses](#repository-addresses).
 * Branch discovery for private projects is not attempted because credentials are not stored on the project; the endpoint returns an empty list with reason `private_repository`.
 * Branch names are validated as Git references. Builds may only use images an admin has allowed; image names are validated against a restrictive pattern.
-* The branch is fetched afresh with JGit at depth 1 to read its workflow file and commit ID. The build then builds an image from the workflow: it installs `git`, `make`, `podman` and `podman-compose` and the workflow's `dependencies` as root, with network access, clones the branch at that commit as `65534:65534`, and runs the steps as `65534:65534` without network. The installs and the clone have network access.
+* The branch is fetched afresh with JGit at depth 1 and its commit ID is recorded. The build then builds an image from the workflow: it installs `git`, `make`, `podman` and `podman-compose` and the workflow's `dependencies` as root, with network access, copies that checkout into the image as `65534:65534`, and runs the steps as `65534:65534` without network. Only the installs have network access; the image build does not fetch the repository again.
+* A checkout is limited to `BUILD_MAX_CHECKOUT_MIB` (default 1024 MiB) and `BUILD_MAX_CHECKOUT_FILES` (default 100000 files); admins can change both under Settings. The received pack is capped while it is fetched, the files are measured as they will be written, after line ending conversion, before checkout, and the written working tree is measured again after checkout. A checkout over a limit fails the build with `Build failed: Repository checkout exceeds the limit of …` and its directory is removed.
 * The image build runs with `no-new-privileges`, 1 GiB of memory and one CPU under Podman; Docker applies neither limit. The image is removed when the build ends.
 * Builds run up to the concurrency limit (default 1, at most 16 when set by an admin); further requests wait in a bounded queue. Each build times out after ten minutes. Output is stored in full, up to the `BUILD_LOG_MAX_BYTES` safety limit (default 1 GiB); output beyond it is discarded and the build is marked truncated.
 * Builds left in the `RUNNING` state by a server restart are marked `FAILED` at startup. Startup also removes leftover `cisys-build-*` containers and images and `cisys-*` checkout directories from an unclean stop; a cleanup failure is logged and does not stop startup.
@@ -84,14 +86,13 @@ Controls that are in place
 
 Known gaps
 
-* Repository checkout size is not bounded.
 * Runner and Podman operation on Windows/WSL has not been verified.
 * Credentials are not scoped to individual projects; only admins may reference one.
 * DNS rebinding between the address check and JGit's own connection is not fully prevented, because JGit resolves the host again when it connects.
 
 ### Git credentials
 
-For private repositories set an environment variable named `CISYS_GIT_<NAME>` to a read-only provider token in the backend environment before starting it, then enter the variable name in the build form as an admin. The value is never stored in the database. The image build receives it as a build secret mounted for the clone step only; it is not stored in the image and later steps cannot read it. Never place secrets in workflow files or build output; each build stores its workflow file.
+For private repositories set an environment variable named `CISYS_GIT_<NAME>` to a read-only provider token in the backend environment before starting it, then enter the variable name in the build form as an admin. The value is never stored in the database. Only the backend's JGit fetch uses it; the image build does not receive it, so it is not in the image and steps cannot read it. Never place secrets in workflow files or build output; each build stores its workflow file.
 
 ## Public access limits
 

docs/entity-diagram.md

@@ -46,6 +46,8 @@ erDiagram
         INTEGER retention_max_builds
         INTEGER max_concurrent_builds
         INTEGER max_queued_builds
+        INTEGER max_checkout_mib
+        INTEGER max_checkout_files
     }
 
     PROJECT ||--o{ BUILD : has

docs/manual/en/03-roles.md

@@ -14,7 +14,7 @@ The **Lead** role is for people who run projects. Leads can do everything a memb
 
 ## Admin
 
-The **Admin** role manages the installation. Admins can see every project including private projects of other users, start and pin builds, use credential references for private Git repositories, manage the build images, change the branding, and change the settings such as build retention and the build queue.
+The **Admin** role manages the installation. Admins can see every project including private projects of other users, start and pin builds, use credential references for private Git repositories, manage the build images, change the branding, and change the settings such as build retention, the build queue and the checkout limits.
 
 ## Access summary
 
@@ -30,7 +30,7 @@ The **Admin** role manages the installation. Admins can see every project includ
 | Use a Git credential reference      | no     | no   | yes   |
 | Manage build images                 | no     | no   | yes   |
 | Change branding                     | no     | no   | yes   |
-| Change settings (retention, queue)  | no     | no   | yes   |
+| Change settings                     | no     | no   | yes   |
 
 "View" always means projects the user may access. The address of a project or build that does not exist or is not visible to the user opens the front page instead, so a private project is not revealed; the same applies to any unknown address.
 

docs/manual/en/05-builds.md

@@ -17,13 +17,15 @@ Members can view build history and logs for the projects they can access. Privat
 
 ## How a build runs
 
-1. The branch is fetched afresh with JGit at depth 1, so a force push is reflected in the next build, and its commit ID is recorded.
+1. The branch is fetched afresh with JGit at depth 1, so a force push is reflected in the next build, and its commit ID is recorded. The checkout must stay within the size and file limits.
 2. The branch's `.cisys/workflows/ci.yml` is read and stored with the build, then validated. A branch without the file, an invalid file, or a `container` image that is not on the build host or that an admin has not allowed on the **Images** page fails the build with a message in the log.
-3. The workflow becomes a Containerfile. Podman builds it with `no-new-privileges`, 1 GiB of memory, one CPU and no layer cache. The image installs `git`, `make`, `podman` and `podman-compose` if they are missing and the workflow's `dependencies`, clones the branch at the recorded commit, and runs the steps as `65534:65534` without network. See **Workflows**.
+3. The workflow becomes a Containerfile. Podman builds it with `no-new-privileges`, 1 GiB of memory, one CPU and no layer cache. The image installs `git`, `make`, `podman` and `podman-compose` if they are missing and the workflow's `dependencies`, copies the checkout from step 1, and runs the steps as `65534:65534` without network. See **Workflows**.
 4. The status is `PASSED` when the image build succeeds, otherwise `FAILED` with the exit code of the failing step. A build that exceeds ten minutes is stopped and reported as timed out. The image is removed afterwards.
 
 Build requests are stored as `QUEUED` and claimed in request order. By default one build runs at a time. Set `BUILD_CONCURRENCY` (`cisys.build.concurrency`) to change the number of concurrent builds and `BUILD_QUEUE_LIMIT` (`cisys.build.queue-limit`, default 100) to limit waiting requests. Both limits must be positive. Admins can change both under **Settings** → **Queue** (1–16 concurrent builds, 1–1000 queued builds); saved values replace the configured ones. A full queue returns HTTP 429 with a clear error. Requests keep their repository URL, branch and credential reference; credentials themselves are read from the environment when execution begins.
 
+A checkout is limited to `BUILD_MAX_CHECKOUT_MIB` (`cisys.build.max-checkout-mib`, default 1024) MiB and `BUILD_MAX_CHECKOUT_FILES` (`cisys.build.max-checkout-files`, default 100000) files. Both limits must be positive. Admins can change both under **Settings** → **Checkout** (1–102400 MiB, 1–1000000 files); saved values replace the configured ones. The fetched data is checked during the fetch, the files are measured as they will be written, after line ending conversion such as `eol=crlf` in `.gitattributes`, before checkout, and the working tree is measured again after checkout. The image copies this checkout and does not fetch the branch again. A repository over a limit fails the build with `Build failed: Repository checkout exceeds the limit of N MiB` (or `N files`) in the log, and the checkout directory is removed.
+
 If saving a completed result fails, the runner retains it in memory and retries on each dispatcher tick without rerunning the build. Its slot stays reserved until the result is saved.
 
 On restart, interrupted `RUNNING` builds become `FAILED` and retain their partial logs. `QUEUED` builds remain queued and run when a slot is available. Startup also removes leftover `cisys-build-*` containers and images and `cisys-*` checkout directories from an unclean stop; a cleanup failure is logged and does not stop startup. Queued and running builds are protected from retention cleanup and prevent project deletion. This runner supports one backend instance; startup recovery assumes the previous instance has stopped.
@@ -67,7 +69,7 @@ Only `github.com` and `gitlab.com` HTTPS URLs are accepted for builds. Redirects
 
 Set an environment variable whose name starts with `CISYS_GIT_`, for example `CISYS_GIT_PROJECT`, to a read-only provider token in the backend environment before starting it. Then, as an admin, enter that variable name in the build form.
 
-* The value is never stored in the project database. The image build receives it as a build secret for the clone step only; it is not stored in the image and later steps cannot read it.
+* The value is never stored in the project database. Only the backend's fetch uses it; the image build does not receive it, so it is not stored in the image and steps cannot read it.
 * Only admins can use credential references until credentials are scoped to individual projects.
 * Never include secrets in workflow files or build output.
 

docs/manual/en/08-workflows.md

@@ -131,7 +131,7 @@ The build finds out what the container is and installs with its package manager:
 * `value` is one package: letters, digits and `+ . _ : = ~ @ -`, starting with a letter or digit, at most 200 characters. Options such as `-y`, spaces and shell characters are rejected.
 * `name` is required, at most 200 characters, and appears once in the list.
 * Packages are installed as root, with network access. The steps themselves still run as `65534:65534` without network.
-* All dependencies are installed together, in one step, before the branch is cloned, so every job and step can use them. `dependencies` on a job or a step is rejected.
+* All dependencies are installed together, in one step, before the checkout is copied in, so every job and step can use them. `dependencies` on a job or a step is rejected.
 * A package that does not exist, or a container without a supported package manager, fails the build with `==> Could not install …` or `==> Cannot install dependencies …`.
 * Dependencies are always up to date. The package manager reads fresh package metadata on every build, never what the image cached, and installs the newest version. A package the image already has is upgraded. The log shows the version each one ended up at, for example `==> Installed: jq 1.7.1-6+deb13u4`. To stay on one version, give it in `value`.
 * Packages are downloaded on every build. A `container` image that already has them starts faster.
@@ -173,7 +173,7 @@ A build turns the workflow into a Containerfile and builds an image from it with
 1. **Read.** The backend fetches the branch and records its commit ID. It reads `.cisys/workflows/ci.yml` and stores the text with the build, then validates it.
 2. **Base.** `container` is looked up among the local images, and the image starts `FROM` its ID. Nothing is pulled.
 3. **Tools and dependencies.** `git`, `make`, `podman` and `podman-compose` are installed as root if the image lacks them, then the workflow's `dependencies`. This step has network access.
-4. **Checkout.** As the unprivileged user `65534:65534`, the branch is cloned into `/workspace` at depth 1. The build then checks that the clone is at the recorded commit; if the branch moved in the meantime, the build fails. The working tree is **on the branch being tested**, with its `.git` directory, so `git status`, `git log -1` and `git rev-parse HEAD` work. `HOME` is `/tmp`.
+4. **Checkout.** The checkout from step 1 is copied into `/workspace`, owned by the unprivileged user `65534:65534`, so the image builds exactly the recorded commit, within the checkout limits. The working tree is **on the branch being tested**, with its `.git` directory, so `git status`, `git log -1` and `git rev-parse HEAD` work. `HOME` is `/tmp`.
 5. **Steps.** The jobs and steps run in order as `65534:65534`, in `/workspace`, **without network**, with the workflow's, the job's and the step's `environment` variables set. The log marks every job and step with `==> Job:` and `==> Step:` lines.
 6. **Result.** The first failing step stops the build. The log shows `==> Step failed:` with the step name and exit code, and the build records that exit code and the status `FAILED`. When every step succeeds, the build is `PASSED`.
 7. **Cleanup.** The image is removed when the build ends. The log, the workflow text, the commit ID and the base image digest are kept.
@@ -194,7 +194,7 @@ Steps cannot reach the network. A step that downloads dependencies, such as a `m
 
 ## Private repositories
 
-For a private repository, an admin starts the build with a credential reference. The image build reads the token from the backend's environment and hands it to the clone step only, as a build secret. It is not written into a layer, the Containerfile, the build arguments or the log, and later steps cannot read it. See **Private Git repositories** in the **Builds** chapter.
+For a private repository, an admin starts the build with a credential reference. The backend reads the token from its environment for its own fetch only. The image build does not receive it, so it is not written into a layer, the Containerfile, the build arguments or the log, and steps cannot read it. See **Private Git repositories** in the **Builds** chapter.
 
 Never put secrets in a workflow file, because each build stores the file and shows it on the build page.
 

docs/manual/en/72-architecture.md

@@ -71,6 +71,8 @@ erDiagram
         INTEGER retention_max_builds
         INTEGER max_concurrent_builds
         INTEGER max_queued_builds
+        INTEGER max_checkout_mib
+        INTEGER max_checkout_files
     }
 
     PROJECT ||--o{ BUILD : has