Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/entity-diagram.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ erDiagram
INTEGER exit_code
BOOLEAN log_truncated
BOOLEAN pinned
BOOLEAN cancel_requested
OID log_oid
TIMESTAMP started_at
TIMESTAMP finished_at
Expand Down Expand Up @@ -56,6 +57,6 @@ erDiagram
* `PROJECT.owner_subject` and `BUILD.requested_by` hold the Keycloak subject (`sub`) of the user.
* `BUILD.project_id` is a plain column, not a foreign key constraint.
* `BUILD.logoid` (type `lo`, from the `lo` extension) references the build log, stored as one PostgreSQL large object. It is not mapped in the `Build` entity. The `lo_manage` trigger unlinks the large object when the build is deleted, so no logs are orphaned.
* `BUILD.status` is `RUNNING`, `PASSED` or `FAILED`.
* `BUILD.status` is `QUEUED`, `RUNNING`, `PASSED`, `FAILED` or `CANCELLED`. `BUILD.cancel_requested` is set on a running build until the runner has stopped it.
* `BRANDING` and `SETTINGS` are singleton rows with `id = 1`. Until `SETTINGS` is first saved, its defaults apply.
* Users, roles and credentials live in Keycloak and are not stored in the cisys database.
5 changes: 3 additions & 2 deletions docs/manual/en/03-roles.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ The **Member** role is the standard read-only role. Members can view public proj

## Lead

The **Lead** role is for people who run projects. Leads can do everything a member can, create projects, start manual builds on projects they can see, and pin builds of the projects they created.
The **Lead** role is for people who run projects. Leads can do everything a member can, create projects, start manual builds on projects they can see, cancel the builds they started, and pin and cancel builds of the projects they created.

## Admin

The **Admin** role manages the installation. Admins can see every project including private projects of other users, start and pin builds, use credential references for private Git repositories, manage the build images, change the branding, and change the settings such as build retention, the build queue, the hosts build steps may reach and the checkout limits.
The **Admin** role manages the installation. Admins can see every project including private projects of other users, start, pin and cancel builds, use credential references for private Git repositories, manage the build images, change the branding, and change the settings such as build retention, the build queue, the hosts build steps may reach and the checkout limits.

## Access summary

Expand All @@ -27,6 +27,7 @@ The **Admin** role manages the installation. Admins can see every project includ
| View build history and logs | yes | yes | yes |
| Start a manual build | no | yes | yes |
| Pin a build | no | own | yes |
| Cancel a build | no | own | yes |
| Use a Git credential reference | no | no | yes |
| Manage build images | no | no | yes |
| Change branding | no | no | yes |
Expand Down
12 changes: 8 additions & 4 deletions docs/manual/en/05-builds.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,19 @@ A checkout is limited to `BUILD_MAX_CHECKOUT_MIB` (`cisys.build.max-checkout-mib

If saving a completed result fails, the runner retains it in memory and retries on each dispatcher tick without rerunning the build. Its slot stays reserved until the result is saved.

On restart, interrupted `RUNNING` builds become `FAILED` and retain their partial logs. `QUEUED` builds remain queued and run when a slot is available. Startup also removes leftover `cisys-build-*` containers and images and `cisys-*` checkout directories from an unclean stop; a cleanup failure is logged and does not stop startup. Queued and running builds are protected from retention cleanup and prevent project deletion. This runner supports one backend instance; startup recovery assumes the previous instance has stopped.
On restart, interrupted `RUNNING` builds become `FAILED`, or `CANCELLED` when a cancellation was requested, and retain their partial logs. `QUEUED` builds remain queued and run when a slot is available. Startup also removes leftover `cisys-build-*` containers and images and `cisys-*` checkout directories from an unclean stop; a cleanup failure is logged and does not stop startup. Queued and running builds are protected from retention cleanup and prevent project deletion. This runner supports one backend instance; startup recovery assumes the previous instance has stopped.

## Cancel a build

Admins, the project's creator and whoever started a build can cancel it while it is `QUEUED` or `RUNNING` with the cancel button on its build page. A queued build becomes `CANCELLED` at once and never runs. For a running build, the runner stops the fetch or the image build within about a second, forcibly after ten more seconds, removes the image and records `CANCELLED` with `Build cancelled` at the end of the log. A cancellation that arrives while the build is finishing still wins. Cancelling a finished build returns HTTP 409.

## Workflow file

The workflow file format, how the image is built, and its limits are described in the **Workflows** chapter.

## What a build records

Each build stores the requested branch, the text of the workflow file it read (shown on the build page), the full name of the base image it found, the credential reference name if one was used (shown to admins only), the base image digest when the container runtime reports one, the commit ID, the exit code of the image build, which is that of the failing step (none when the build timed out or never ran), who started it, and when it started and finished; the duration is derived from these times.
Each build stores the requested branch, the text of the workflow file it read (shown on the build page), the full name of the base image it found, the credential reference name if one was used (shown to admins only), the base image digest when the container runtime reports one, the commit ID, the exit code of the image build, which is that of the failing step (none when the build timed out, was cancelled or never ran), who started it, and when it started and finished; the duration is derived from these times.

## Build logs

Expand All @@ -54,10 +58,10 @@ The project's creator and admins can **Pin** a build on its build page, and **Un

## Build retention

Every hour, and a minute after startup, cisys deletes old builds of each project together with their logs. A finished, unpinned build is deleted when either
Every hour, and a minute after startup, cisys deletes old builds of each project together with their logs. A finished or cancelled, unpinned build is deleted when either

* it finished more than the maximum age ago (default 30 days), or
* the project has more newer finished, unpinned builds than the maximum count (default 50).
* the project has more newer finished or cancelled, unpinned builds than the maximum count (default 50).

Admins change both limits under **Settings** → **Retention**; `0` turns a limit off. Running and pinned builds are never deleted, and pinned builds do not count towards the limit. Deleted builds cannot be restored, so pin a build to keep it. `BUILD_RETENTION_EVERY` (default `1h`) changes how often the cleanup runs.

Expand Down
3 changes: 2 additions & 1 deletion docs/manual/en/72-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The sources are in `src/backend/main/java/ai/mnemosyne/cisys`.
* The data model uses JPA entities backed by [Hibernate ORM Panache](https://quarkus.io/guides/hibernate-orm-panache): `Project`, `Build`, `Branding` and `Settings`.
* HTTP endpoints use JAX-RS via [Quarkus REST](https://quarkus.io/guides/rest): `ProjectResource` (`/api/session`, `/api/projects`), `BuildResource` (`/api/projects/{projectId}/builds`), `BrandingResource` (`/api/branding`) and `SettingsResource` (`/api/settings`).
* `BuildLogs` stores each build log as one PostgreSQL large object, using JDBC and the pgjdbc `LargeObjectManager` directly rather than Hibernate. Output is appended while the build runs and streamed back from `/api/projects/{projectId}/builds/{buildId}/log`, about 1 MB per short transaction, so a log is never held in memory whole. cisys supports PostgreSQL only.
* `BuildRunner` fetches repositories with [JGit](https://www.eclipse.org/jgit/) and runs builds through the Docker or Podman command line. It claims `QUEUED` builds in request order, up to the concurrency limit set under Settings or in configuration.
* `BuildRunner` fetches repositories with [JGit](https://www.eclipse.org/jgit/) and runs builds through the Docker or Podman command line. It claims `QUEUED` builds in request order, up to the concurrency limit set under Settings or in configuration. Each dispatcher tick also stops the running builds whose cancellation was requested.
* `EgressProxy` is a per-build HTTPS `CONNECT` proxy on a Unix socket. Podman mounts the socket directory into each step, which still runs with `--network=none`; `socat` in the step forwards `127.0.0.1:3128` to it. The proxy allows only the hosts set under Settings → Network (`NetworkAllowlist`) on the configured ports, and never protected addresses.
* `BuildRetention` is a [scheduled](https://quarkus.io/guides/scheduler) job that deletes each project's expired, unpinned builds in batches with one SQL statement per batch; the `lo_manage` trigger removes their logs.
* Authentication uses [Quarkus OIDC](https://quarkus.io/guides/security-oidc-bearer-token-authentication) in `service` mode with Keycloak realm roles.
Expand Down Expand Up @@ -51,6 +51,7 @@ erDiagram
INTEGER exit_code
BOOLEAN log_truncated
BOOLEAN pinned
BOOLEAN cancel_requested
OID log_oid
TIMESTAMP started_at
TIMESTAMP finished_at
Expand Down
7 changes: 6 additions & 1 deletion src/backend/main/java/ai/mnemosyne/cisys/Build.java
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ public enum Status {
QUEUED,
RUNNING,
PASSED,
FAILED
FAILED,
CANCELLED
}

@Enumerated(EnumType.STRING)
Expand Down Expand Up @@ -60,6 +61,10 @@ public enum Status {
@Column(nullable = false)
public boolean pinned;

/** Set on a running build; the runner stops it and records it as cancelled. */
@Column(nullable = false)
public boolean cancelRequested;

public Instant startedAt;
public Instant finishedAt;
}
21 changes: 21 additions & 0 deletions src/backend/main/java/ai/mnemosyne/cisys/BuildResource.java
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ public class BuildResource {
@Inject BuildRunner runner;
@Inject BuildLogs logs;
@Inject RepositoryUrlValidator repositoryUrls;
@Inject ProjectAccess access;

/** The image and the commands come from the branch's {@value Workflow#PATH}. */
public record RunRequest(String branch, String credentialRef) {}
Expand Down Expand Up @@ -158,6 +159,26 @@ private BuildResponse setPinned(Long id, Long buildId, boolean pinned) {
return response(b, true);
}

/** Admins, the project's owner and whoever requested the build may cancel it. */
@POST
@Path("/{buildId}/cancel")
public BuildResponse cancel(@PathParam("projectId") Long id, @PathParam("buildId") Long buildId) {
Project p = visible(id);
Build b = Build.findById(buildId);
if (b == null || !id.equals(b.projectId)) throw new NotFoundException();
if (!access.mayCancel(p, b)) throw new ForbiddenException();
if (!runner.cancel(buildId))
throw new WebApplicationException(
jakarta.ws.rs.core.Response.status(409)
.type(MediaType.APPLICATION_JSON)
.entity(java.util.Map.of("error", "The build has already finished."))
.build());
return response(
io.quarkus.narayana.jta.QuarkusTransaction.requiringNew()
.call(() -> Build.<Build>findById(buildId)),
true);
}

@POST
public BuildResponse start(@PathParam("projectId") Long id, RunRequest input) {
Project p = visible(id);
Expand Down
4 changes: 2 additions & 2 deletions src/backend/main/java/ai/mnemosyne/cisys/BuildRetention.java
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@ public class BuildRetention {

private static final String DELETE =
"""
delete from build where status in ('PASSED', 'FAILED') and not pinned and id in (
delete from build where status in ('PASSED', 'FAILED', 'CANCELLED') and not pinned and id in (
select id from (
select id, coalesce(finishedat, startedat) as at, row_number() over (
partition by projectid order by coalesce(finishedat, startedat) desc nulls last, id desc) as n
from build where status in ('PASSED', 'FAILED') and not pinned) eligible
from build where status in ('PASSED', 'FAILED', 'CANCELLED') and not pinned) eligible
where n > :maxBuilds or at < :cutoff
limit :batch)
""";
Expand Down
Loading
Loading