[#31] Limit anonymous log reads and rate limit public endpoints

docs/BUILDING.md

@@ -111,6 +111,10 @@ If you prefer not to use Compose, run PostgreSQL and Keycloak yourself and point
 | `KEYCLOAK_ADMIN_CLIENT_ID` | `cisys-backend` | Keycloak service account used to sign users out |
 | `KEYCLOAK_ADMIN_CLIENT_SECRET` | `cisys` | Secret of that service account |
 | `STATS_CACHE_SECONDS` | `30` | How long the public front-page statistics are cached on the server |
+| `PUBLIC_LOG_MAX_BYTES` | `2097152` | Maximum log bytes returned to an anonymous request |
+| `PUBLIC_RATE_LIMIT_REQUESTS` | `120` | Anonymous requests per client per window; `0` disables the limiter |
+| `PUBLIC_RATE_LIMIT_WINDOW_SECONDS` | `60` | Length of the anonymous rate-limit window |
+| `PUBLIC_RATE_LIMIT_MAX_CLIENTS` | `10000` | Upper bound on tracked client addresses |
 | `DB_URL`            | `jdbc:postgresql://localhost:5432/cisys` | JDBC URL                                             |
 | `DB_USER`           | `cisys`                                  | Database user                                        |
 | `DB_PASSWORD`       | `cisys`                                  | Database password                                    |

docs/SECURITY.md

@@ -93,6 +93,16 @@ Known gaps
 
 For private repositories set an environment variable named `CISYS_GIT_<NAME>` to a read-only provider token in the backend environment before starting it, then enter the variable name in the build form as an admin. The value is never stored in the database. The image build receives it as a build secret mounted for the clone step only; it is not stored in the image and later steps cannot read it. Never place secrets in workflow files or build output; each build stores its workflow file.
 
+## Public access limits
+
+The public project page, its builds, logs and branches, and the public statistics are readable without signing in.
+
+* An anonymous log request returns at most `cisys.public.log-max-bytes` (default 2 MiB) in one response. `offset`, `limit` and `tail` select a range; the `X-Log-Size`, `X-Log-Offset` and `X-Log-Length` headers let a client continue where the response ended. The UI pages through the limit for **Download log**; when a page is rate limited it waits for `Retry-After` and continues from the same offset, so a large anonymous download completes across windows.
+* Signed-in visitors are not capped on this endpoint, so the build page keeps the complete log and the full download works as before. This assumes accounts are trusted; if registration is open, an account can be used to bypass the anonymous limits.
+* Anonymous requests to `/api/project-pages/**` (including branch discovery, which runs `git ls-remote`) and `/api/stats/public` are counted per client address in a fixed window: `cisys.public.rate-limit.requests` per `cisys.public.rate-limit.window-seconds` (default 120 per 60 s). Exceeding it returns `429` with `Retry-After`. Signed-in callers are not counted.
+* The counter lives in this process: it resets on restart, is not shared between instances, and tracks at most `cisys.public.rate-limit.max-clients` addresses (`0` removes the bound). When that many live windows exist, one window is evicted and the new visitor admitted, so a flood of addresses cannot lock new visitors out, and purging and eviction stay constant-time per request.
+* Behind a reverse proxy, set `quarkus.http.proxy.proxy-address-forwarding=true` and `quarkus.http.proxy.enable-forwarded-host=true` and have the proxy send `Forwarded` or `X-Forwarded-For`. Otherwise every visitor shares the proxy's address and therefore one bucket.
+
 ## TLS
 
 The application is not configured for TLS. Terminate TLS in a reverse proxy, or configure the [Quarkus TLS registry](https://quarkus.io/guides/tls-registry-reference), before any shared deployment. Keycloak must also be moved off `start-dev` and onto HTTPS in that case.

docs/manual/en/05-builds.md

@@ -42,7 +42,9 @@ The complete output of the build, standard output and standard error together, i
 
 * `BUILD_LOG_MAX_BYTES` (default 1 GiB) is a safety limit per build, not a normal cap. Output beyond it is read and discarded, a marker line is added, and the build is marked as truncated. The limit is exact in bytes, so if it falls inside a multi-byte character, the last character shown may be incomplete.
 * A build is also marked as truncated if the log could not be stored completely. Logs from before this version were capped at 120 KB; those that reached the cap are marked as truncated.
-* The build page shows the first 2 MiB of a log. **Download log** saves the full log; in Chromium-based browsers it is written straight to the chosen file, in other browsers it is held in browser memory first. The backend never holds a whole log in memory.
+* The build page shows the first 2 MiB of a log. For a running build, **Refresh** appends only the new output instead of downloading the log from the start; once 2 MiB are shown, it switches to the latest 2 MiB so new output stays visible. **Download log** saves the full log; in Chromium-based browsers it is written straight to the chosen file, in other browsers it is held in browser memory first. The backend never holds a whole log in memory.
+* A visitor who is not signed in can read at most `PUBLIC_LOG_MAX_BYTES` (2 MiB by default) of a log in one response. The page requests the log in chunks and retries a rate-limited page after `Retry-After` from the same offset, so **Download log** can save a log larger than the limit, even across rate-limit windows. Signed-in users, including the owner of a private project, are not capped.
+* Public pages pause their refresh while the tab is hidden and refresh less often when no build is running. Anonymous requests to a public page, its branches, its logs and the public statistics are also rate limited; a client that sends too many requests is told to wait and retry.
 
 ## Pinned builds
 

src/backend/main/java/ai/mnemosyne/cisys/BuildLogs.java

@@ -87,17 +87,70 @@ public Throwable appendLine(long buildId, String text) {
     }
 
     public void streamTo(long buildId, OutputStream out) throws IOException {
+        streamTo(buildId, out, 0, -1);
+    }
+
+    /**
+     * Streams the stored bytes from {@code offset}, at most {@code limit} of them. A negative
+     * limit means no limit. Used by the log endpoints to support partial and tail reads.
+     */
+    public void streamTo(long buildId, OutputStream out, long offset, long limit) throws IOException {
+        if (offset < 0) throw new IllegalArgumentException("offset must not be negative");
+        if (limit == 0) return;
         Long oid = logOid(buildId);
         if (oid == null) return;
-        for (long offset = 0; ; ) {
-            byte[] part = read(oid, offset, READ_BYTES);
-            if (part.length > 0) {
-                out.write(part);
-                out.flush();
-                offset += part.length;
+        long written = 0;
+        for (long at = offset; limit < 0 || written < limit; ) {
+            int want = (int) Math.min(READ_BYTES, limit < 0 ? READ_BYTES : limit - written);
+            byte[] part = read(oid, at, want);
+            if (part.length == 0) return;
+            out.write(part);
+            out.flush();
+            at += part.length;
+            written += part.length;
+        }
+    }
+
+    /** Total number of bytes stored for a build, or zero when it has no log yet. */
+    public long size(long buildId) {
+        Long oid = logOid(buildId);
+        if (oid == null) return 0;
+        return QuarkusTransaction.requiringNew().call(() -> {
+            try (Connection c = dataSource.getConnection()) {
+                LargeObject lo = largeObjects(c).open(oid, LargeObjectManager.READ);
+                try {
+                    return lo.size64();
+                } finally {
+                    lo.close();
+                }
             }
-            if (part.length < READ_BYTES) return;
+        });
+    }
+
+    /** A resolved byte range of one log. */
+    public record Range(long offset, long length) {}
+
+    /**
+     * Resolves the optional {@code offset}, {@code limit} and {@code tail} query values against
+     * the stored size. {@code tail} wins over {@code offset}; {@code maxBytes} clamps the result
+     * when it is not negative. A null limit means "everything that remains". A {@code tail} larger
+     * than {@code maxBytes} is reduced to the maximum, so the caller still gets the end of the log.
+     */
+    public static Range range(long size, Long offset, Long limit, Long tail, long maxBytes) {
+        if (offset != null && offset < 0) throw new IllegalArgumentException("offset must not be negative");
+        if (limit != null && limit < 0) throw new IllegalArgumentException("limit must not be negative");
+        if (tail != null && tail < 0) throw new IllegalArgumentException("tail must not be negative");
+        long start;
+        if (tail != null) {
+            long wanted = maxBytes >= 0 ? Math.min(tail, maxBytes) : tail;
+            start = Math.max(0, size - wanted);
+        } else {
+            start = offset == null ? 0 : offset;
         }
+        long available = Math.max(0, size - start);
+        long length = limit == null ? available : Math.min(limit, available);
+        if (maxBytes >= 0) length = Math.min(length, maxBytes);
+        return new Range(start, length);
     }
 
     /** For tests. */

src/backend/main/java/ai/mnemosyne/cisys/BuildResource.java

@@ -14,7 +14,7 @@
 import jakarta.transaction.Transactional;
 import jakarta.ws.rs.*;
 import jakarta.ws.rs.core.MediaType;
-import jakarta.ws.rs.core.StreamingOutput;
+import jakarta.ws.rs.core.Response;
 import org.eclipse.microprofile.jwt.JsonWebToken;
 import java.time.Instant;
 import java.util.List;
@@ -57,10 +57,17 @@ private Project visible(Long id) {
  @GET @Path("/{buildId}") public BuildResponse detail(@PathParam("projectId") Long id,@PathParam("buildId") Long buildId) {
   visible(id); Build b=Build.findById(buildId);if(b==null || !id.equals(b.projectId))throw new NotFoundException();return response(b,true);
  }
- /** A running build returns what is stored so far. */
- @GET @Path("/{buildId}/log") @Produces("text/plain; charset=UTF-8") public StreamingOutput log(@PathParam("projectId") Long id,@PathParam("buildId") Long buildId) {
+ /**
+  * A running build returns what is stored so far. Authenticated callers get the whole log by
+  * default; {@code offset}, {@code limit} and {@code tail} allow partial reads without a cap.
+  */
+ @GET @Path("/{buildId}/log") @Produces("text/plain; charset=UTF-8") public Response log(@PathParam("projectId") Long id,@PathParam("buildId") Long buildId,
+     @QueryParam("offset") Long offset,@QueryParam("limit") Long limit,@QueryParam("tail") Long tail) {
   detail(id,buildId);
-  return out->logs.streamTo(buildId,out);
+  long size=logs.size(buildId);
+  BuildLogs.Range range;
+  try{range=BuildLogs.range(size,offset,limit,tail,-1);}catch(IllegalArgumentException e){throw new BadRequestException(e.getMessage());}
+  return LogResponses.stream(logs,buildId,size,range);
  }
  @PUT @Path("/{buildId}/pin") @Transactional public BuildResponse pin(@PathParam("projectId") Long id,@PathParam("buildId") Long buildId) {
   return setPinned(id,buildId,true);

src/backend/main/java/ai/mnemosyne/cisys/LogResponses.java

@@ -0,0 +1,30 @@
+/*
+ * Eclipse Public License - v 2.0
+ *
+ *   THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
+ *   PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION
+ *   OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
+ */
+
+package ai.mnemosyne.cisys;
+
+import jakarta.ws.rs.core.Response;
+import jakarta.ws.rs.core.StreamingOutput;
+
+/** The response shape shared by the authenticated and public build-log endpoints. */
+final class LogResponses {
+    private LogResponses() {}
+
+    /**
+     * Streams the resolved range as plain text. The headers let a client continue from the end of
+     * this response without downloading the whole log again.
+     */
+    static Response stream(BuildLogs logs, long buildId, long size, BuildLogs.Range range) {
+        return Response.ok((StreamingOutput) out -> logs.streamTo(buildId, out, range.offset(), range.length()))
+                .type("text/plain; charset=UTF-8")
+                .header("X-Log-Size", size)
+                .header("X-Log-Offset", range.offset())
+                .header("X-Log-Length", range.length())
+                .build();
+    }
+}

src/backend/main/java/ai/mnemosyne/cisys/ProjectBranchResource.java

@@ -29,6 +29,7 @@
 @Path("/api/project-pages/{name}/branches")
 @Produces(MediaType.APPLICATION_JSON)
 @PermitAll
+@PublicRateLimited
 public class ProjectBranchResource {
     @Inject ProjectAccess access;
     @Inject GitRepositoryService repositories;

src/backend/main/java/ai/mnemosyne/cisys/ProjectPageResource.java

@@ -8,18 +8,21 @@
 
 package ai.mnemosyne.cisys;
 
+import io.quarkus.security.identity.SecurityIdentity;
 import jakarta.annotation.security.PermitAll;
 import jakarta.inject.Inject;
+import jakarta.ws.rs.BadRequestException;
 import jakarta.ws.rs.GET;
 import jakarta.ws.rs.NotFoundException;
 import jakarta.ws.rs.Path;
 import jakarta.ws.rs.PathParam;
 import jakarta.ws.rs.Produces;
 import jakarta.ws.rs.QueryParam;
 import jakarta.ws.rs.core.MediaType;
-import jakarta.ws.rs.core.StreamingOutput;
+import jakarta.ws.rs.core.Response;
 import java.time.Instant;
 import java.util.List;
+import org.eclipse.microprofile.config.inject.ConfigProperty;
 
 /**
  * The page of a single project, addressed by its name (for example /cisys/ in the UI).
@@ -30,9 +33,13 @@
 @Path("/api/project-pages/{name}")
 @Produces(MediaType.APPLICATION_JSON)
 @PermitAll
+@PublicRateLimited
 public class ProjectPageResource {
     @Inject ProjectAccess access;
     @Inject BuildLogs logs;
+    @Inject SecurityIdentity identity;
+    /** Anonymous callers may read at most this many bytes of one log in one response. */
+    @ConfigProperty(name = "cisys.public.log-max-bytes", defaultValue = "2097152") long publicLogMaxBytes;
 
     public record ProjectInfo(Long id, String name, String repositoryUrl, boolean privateRepository) {}
     /** Leaves out who requested the build and any credential reference, as the page may be anonymous. */
@@ -72,13 +79,30 @@ private static PageBuild pageBuild(Build b, boolean canPin, boolean withWorkflow
                 withWorkflow ? b.workflow : null);
     }
 
+    /**
+     * Streams a public build log. Anonymous callers are capped at {@code cisys.public.log-max-bytes}
+     * per response; use {@code offset}, {@code limit} or {@code tail} to page through a larger log.
+     * The headers expose the total size and the returned range so a client can continue.
+     */
     @GET
     @Path("/builds/{buildId}/log")
     @Produces("text/plain; charset=UTF-8")
-    public StreamingOutput log(@PathParam("name") String name, @PathParam("buildId") Long buildId) {
+    public Response log(@PathParam("name") String name, @PathParam("buildId") Long buildId,
+                        @QueryParam("offset") Long offset, @QueryParam("limit") Long limit,
+                        @QueryParam("tail") Long tail) {
         Project p = access.visible(name);
         Build b = Build.findById(buildId);
         if (b == null || !p.id.equals(b.projectId)) throw new NotFoundException();
-        return out -> logs.streamTo(buildId, out);
+        long size = logs.size(buildId);
+        // The size and range are a snapshot; a running build may append while this response streams.
+        // Signed-in visitors, including the owner on a private project, still get the whole log.
+        long maxBytes = identity.isAnonymous() ? publicLogMaxBytes : -1;
+        BuildLogs.Range range;
+        try {
+            range = BuildLogs.range(size, offset, limit, tail, maxBytes);
+        } catch (IllegalArgumentException e) {
+            throw new BadRequestException(e.getMessage());
+        }
+        return LogResponses.stream(logs, buildId, size, range);
     }
 }

src/backend/main/java/ai/mnemosyne/cisys/PublicRateLimitFilter.java

@@ -0,0 +1,59 @@
+/*
+ * Eclipse Public License - v 2.0
+ *
+ *   THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
+ *   PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION
+ *   OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
+ */
+
+package ai.mnemosyne.cisys;
+
+import io.quarkus.security.identity.SecurityIdentity;
+import io.vertx.core.http.HttpServerRequest;
+import jakarta.annotation.Priority;
+import jakarta.inject.Inject;
+import jakarta.ws.rs.Priorities;
+import jakarta.ws.rs.container.ContainerRequestContext;
+import jakarta.ws.rs.container.ContainerRequestFilter;
+import jakarta.ws.rs.core.Context;
+import jakarta.ws.rs.core.MediaType;
+import jakarta.ws.rs.core.Response;
+import jakarta.ws.rs.ext.Provider;
+import java.util.Map;
+
+/**
+ * Applies {@link PublicRateLimiter} to the endpoints marked with {@link PublicRateLimited}.
+ * Signed-in callers are never counted, so the authenticated build pages keep working during a
+ * burst of anonymous traffic. The client is identified by the socket address; behind a reverse
+ * proxy configure Quarkus to use the forwarded address, otherwise every visitor shares one bucket.
+ */
+@Provider
+@PublicRateLimited
+@Priority(Priorities.AUTHORIZATION)
+public class PublicRateLimitFilter implements ContainerRequestFilter {
+    @Inject SecurityIdentity identity;
+    @Inject PublicRateLimiter limiter;
+    @Context HttpServerRequest request;
+
+    @Override
+    public void filter(ContainerRequestContext context) {
+        // CORS preflight requests are cheap and must not consume a visitor's budget.
+        if (context.getMethod().equals("OPTIONS")) return;
+        if (!identity.isAnonymous()) return;
+        long retryAfter = limiter.acquire(clientAddress(), System.currentTimeMillis());
+        if (retryAfter == 0) return;
+        context.abortWith(Response.status(429)
+                .type(MediaType.APPLICATION_JSON)
+                .header("Retry-After", retryAfter)
+                .entity(Map.of("error", "Too many requests. Try again later."))
+                .build());
+    }
+
+    private String clientAddress() {
+        try {
+            return request.remoteAddress() == null ? "unknown" : request.remoteAddress().hostAddress();
+        } catch (RuntimeException e) {
+            return "unknown";
+        }
+    }
+}

src/backend/main/java/ai/mnemosyne/cisys/PublicRateLimited.java

@@ -0,0 +1,21 @@
+/*
+ * Eclipse Public License - v 2.0
+ *
+ *   THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE
+ *   PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION
+ *   OF THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT.
+ */
+
+package ai.mnemosyne.cisys;
+
+import jakarta.ws.rs.NameBinding;
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/** Marks the anonymous endpoints that {@link PublicRateLimitFilter} rate limits. */
+@NameBinding
+@Retention(RetentionPolicy.RUNTIME)
+@Target({ElementType.TYPE, ElementType.METHOD})
+public @interface PublicRateLimited {}