From a6275a3dc50154cf3ac39b0012e52f4b81c315de Mon Sep 17 00:00:00 2001 From: Matthias Linhuber Date: Tue, 29 Sep 2026 10:10:17 +0200 Subject: [PATCH] feat(eduide): show a maintenance page instead of Envoy's raw error, and keep deploys from needing it While the landing page had no ready pod, visitors got Envoy's plain-text "no healthy upstream". A BackendTrafficPolicy on landing-route now answers 502/503/504 with files/maintenance.html, served by Envoy from a ConfigMap. The page must not contain "%": Envoy reads the body as a format string and otherwise drops the override while the policy still reports Accepted. Deploys now keep a ready pod: maxUnavailable 0 and a 10s preStop drain for the landing page and REST service, and a tcpSocket readiness probe for the service, which had none. landingPage.replicas and service.replicas above 1 add a PodDisruptionBudget each (podDisruptionBudget.enabled) and spread over nodes. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ci.yml | 3 + CHANGELOG.md | 6 + charts/eduide/Chart.yaml | 2 +- charts/eduide/README.md | 8 +- charts/eduide/files/maintenance.html | 37 ++++++ charts/eduide/templates/_helpers.tpl | 26 +++++ charts/eduide/templates/landing-page.yaml | 11 +- charts/eduide/templates/maintenance-page.yaml | 44 +++++++ .../templates/pod-disruption-budgets.yaml | 30 +++++ charts/eduide/templates/service.yaml | 21 +++- charts/eduide/values.yaml | 22 ++++ docs/charts.md | 33 ++++++ scripts/test-maintenance-page.sh | 107 ++++++++++++++++++ 13 files changed, 346 insertions(+), 4 deletions(-) create mode 100644 charts/eduide/files/maintenance.html create mode 100644 charts/eduide/templates/maintenance-page.yaml create mode 100644 charts/eduide/templates/pod-disruption-budgets.yaml create mode 100755 scripts/test-maintenance-page.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index ca27c9a..5c67795 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -46,6 +46,9 @@ jobs: - name: App definitions, landing page and preloading agree run: ./scripts/test-app-consistency.sh + - name: Maintenance page and rolling updates + run: ./scripts/test-maintenance-page.sh + - name: Chart version must be bumped when a chart changes if: github.event_name == 'pull_request' run: | diff --git a/CHANGELOG.md b/CHANGELOG.md index f4f6640..4c632ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,11 @@ # Changelog +## [eduide 2.4.0] - 2026-09-28 + +- [eduide] Serve a static maintenance page instead of Envoy's raw `no healthy upstream` when the landing page is down (`maintenancePage.enabled`, default on) +- [eduide] Add a readiness probe to the REST service, and a `preStop` drain and `maxUnavailable: 0` to the landing page and REST service, so deploys keep a ready pod +- [eduide] Add `landingPage.replicas` and `service.replicas`, with a PodDisruptionBudget (`podDisruptionBudget.enabled`) and node spreading when above 1 + ## [1.2.0] - estimated between 2025-11 and 2026-05 ## [1.1.2] - 2025-09-26 diff --git a/charts/eduide/Chart.yaml b/charts/eduide/Chart.yaml index b5dc686..4fc9cbd 100644 --- a/charts/eduide/Chart.yaml +++ b/charts/eduide/Chart.yaml @@ -15,7 +15,7 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 2.3.0 +version: 2.4.0 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to # follow Semantic Versioning. They should reflect the version the application is using. diff --git a/charts/eduide/README.md b/charts/eduide/README.md index 8ed371f..c98653b 100644 --- a/charts/eduide/README.md +++ b/charts/eduide/README.md @@ -1,6 +1,6 @@ # eduide -![Version: 2.3.0](https://img.shields.io/badge/Version-2.3.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.3.0](https://img.shields.io/badge/AppVersion-1.3.0-informational?style=flat-square) +![Version: 2.4.0](https://img.shields.io/badge/Version-2.4.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.3.0](https://img.shields.io/badge/AppVersion-1.3.0-informational?style=flat-square) EduIDE tenant release: operator, REST service, landing page and routes for one environment. Requires eduide-cluster to be installed on the cluster first. @@ -94,8 +94,11 @@ environment. Requires eduide-cluster to be installed on the cluster first. | landingPage.privacy.dataProtectionOfficer.email | string | `""` | Where the data protection officer is reached, e.g. "dpo@example.edu". | | landingPage.privacy.dataProtectionOfficer.name | string | `""` | Optional name. Most institutions publish only the address. | | landingPage.privacy.scientificUse | bool | `false` | State that anonymised usage data may also be used for scientific research. Leave off unless this installation has a legal basis for it: this is a processing purpose, not a cosmetic string. | +| landingPage.replicas | int | `1` | Number of landing page pods. More than one keeps the landing page up when a node goes away, and turns on its PodDisruptionBudget. | | landingPage.sentry | object | (see details below) | Values related to Sentry on the landing page. | | landingPage.sentry.enable | bool | `false` | Set SENTRY_ENABLE=true in the landing page deployment. Off by default: the DSN is compiled into the published images and points at TUM's Sentry, so enabling this outside TUM sends your hostnames and namespace names there. | +| maintenancePage | object | (see details below) | Static page shown instead of Envoy's raw error when the landing page is down | +| maintenancePage.enabled | bool | `true` | Answer 502/503/504 on the landing route with files/maintenance.html ("EduIDE is currently unavailable") instead of Envoy's plain-text error. Covers the landing page only - the REST service and sessions keep their own errors. Needs Envoy Gateway (BackendTrafficPolicy responseOverride). | | monitor | object | (see details below) | Values to influence the monitor initialization on the operator | | monitor.activityTracker | object | (see details below) | Values to influence the activityTracker module | | monitor.activityTracker.enable | bool | `true` | Should the activityTracker module be enabled | @@ -132,6 +135,8 @@ environment. Requires eduide-cluster to be installed on the cluster first. | operator.storageClassName | string | `"default"` | The name of the storage class for persistent volume claims for workspaces. This storage class must be present on the cluster. Most cloud providers offer a default storage class without additional configuration. | | operator.wondershaperImage | string | `"theiacloud/theia-cloud-wondershaper:1.2.0-next"` | If bandwidthLimiter is set to WONDERSHAPER or K8SANNOTATIONANDWONDERSHAPER this image will be used for the wondershaper init container | | operatorrole.name | string | `"operator-api-access"` | | +| podDisruptionBudget | object | (see details below) | PodDisruptionBudgets for the landing page and REST service | +| podDisruptionBudget.enabled | bool | `true` | Render a PodDisruptionBudget (minAvailable: 1) for every Deployment above that runs more than one replica. Turn this off on single-node clusters: there a drain can never reschedule the evicted replica, so it waits on the last one forever. | | preloading | object | (see details below) | Values to configure preloading of images on Kubernetes nodes. | | preloading.deriveFromApps | bool | `true` | Set to false to preload only preloading.images and nothing derived. | | preloading.enable | bool | `true` | Is image preloading enabled. | @@ -149,6 +154,7 @@ environment. Requires eduide-cluster to be installed on the cluster first. | service.imagePullSecret | string | `nil` | Optional: the image pull secret | | service.port | int | `8081` | service port (default: 8081) | | service.protocol | string | `"https"` | protocol of the REST-API | +| service.replicas | int | `1` | Number of REST service pods. The service keeps no state of its own, so more than one is safe; it also turns on its PodDisruptionBudget. | | service.sentry | object | (see details below) | Values related to Sentry on the service. | | service.sentry.enable | bool | `false` | Set SENTRY_ENABLE=true in the service deployment. Off by default: the DSN is compiled into the published images and points at TUM's Sentry, so enabling this outside TUM sends your hostnames and namespace names there. | | servicerole.name | string | `"service-api-access"` | | diff --git a/charts/eduide/files/maintenance.html b/charts/eduide/files/maintenance.html new file mode 100644 index 0000000..3d827ff --- /dev/null +++ b/charts/eduide/files/maintenance.html @@ -0,0 +1,37 @@ + + + + + + + +EduIDE - currently unavailable + + + +
+ +

EduIDE is currently unavailable

+

We're updating EduIDE and will be back in a few minutes. This page reloads automatically.

+

EduIDE wird gerade aktualisiert und ist in wenigen Minuten wieder da.

+
+ + diff --git a/charts/eduide/templates/_helpers.tpl b/charts/eduide/templates/_helpers.tpl index 1275b0a..3e926c7 100644 --- a/charts/eduide/templates/_helpers.tpl +++ b/charts/eduide/templates/_helpers.tpl @@ -116,3 +116,29 @@ disagree with what the installation offers. {{- end -}} {{- $out | toJson -}} {{- end -}} + +{{/* +Keep serving for a few seconds after the pod is marked for deletion. Envoy only +learns that the endpoint is gone after the Pod is deleted, so without this it +keeps sending requests to a process that is already shutting down. +*/}} +{{- define "eduide.preStopDrain" -}} +preStop: + exec: + command: ["sleep", "10"] +{{- end }} + +{{/* +Prefer putting the replicas of a Deployment on different nodes, so one node +going away does not take all of them. Soft, so a single-node cluster still +schedules every replica. +*/}} +{{- define "eduide.spreadAcrossNodes" -}} +topologySpreadConstraints: + - maxSkew: 1 + topologyKey: kubernetes.io/hostname + whenUnsatisfiable: ScheduleAnyway + labelSelector: + matchLabels: + app: {{ . }} +{{- end }} diff --git a/charts/eduide/templates/landing-page.yaml b/charts/eduide/templates/landing-page.yaml index bd08b37..87aa7da 100644 --- a/charts/eduide/templates/landing-page.yaml +++ b/charts/eduide/templates/landing-page.yaml @@ -7,7 +7,13 @@ metadata: app: landing-page namespace: {{ .Release.Namespace }} spec: - replicas: 1 + replicas: {{ .Values.landingPage.replicas }} + {{- /* Start the new pod before stopping the old one, so a deploy never leaves the route without a ready pod. */}} + strategy: + type: RollingUpdate + rollingUpdate: + maxUnavailable: 0 + maxSurge: 1 selector: matchLabels: app: landing-page @@ -20,6 +26,7 @@ spec: checksum/config: {{ include (print $.Template.BasePath "/landing-page-config-map.yaml") . | sha256sum }} spec: automountServiceAccountToken: false + {{- include "eduide.spreadAcrossNodes" "landing-page" | nindent 6 }} containers: - name: landing-page-container image: {{ tpl (.Values.landingPage.image | toString) . }} @@ -37,6 +44,8 @@ spec: port: 80 initialDelaySeconds: 5 periodSeconds: 5 + lifecycle: + {{- include "eduide.preStopDrain" . | nindent 12 }} volumeMounts: - name: landing-page-config mountPath: /usr/share/nginx/html/config.js diff --git a/charts/eduide/templates/maintenance-page.yaml b/charts/eduide/templates/maintenance-page.yaml new file mode 100644 index 0000000..b6acd31 --- /dev/null +++ b/charts/eduide/templates/maintenance-page.yaml @@ -0,0 +1,44 @@ +{{- /* +The page Envoy serves instead of its raw "no healthy upstream" when the landing +page has no ready pod - during a deploy, or on purpose when the maintenance +workflow in EduIDE-deployment scales it to 0. Envoy serves it itself, so it +must not load anything from the landing page. The status code stays 5xx. +*/}} +{{- if and .Values.maintenancePage.enabled .Values.landingPage.enabled .Values.gateway.enabled .Values.gateway.routes.enabled }} +apiVersion: v1 +kind: ConfigMap +metadata: + name: maintenance-page + namespace: {{ .Release.Namespace }} +data: + response.body: | +{{ .Files.Get "files/maintenance.html" | indent 4 }} +--- +apiVersion: gateway.envoyproxy.io/v1alpha1 +kind: BackendTrafficPolicy +metadata: + name: landing-maintenance-page + namespace: {{ .Release.Namespace }} +spec: + targetRefs: + - group: gateway.networking.k8s.io + kind: HTTPRoute + name: landing-route + responseOverride: + - match: + statusCodes: + - type: Value + value: 502 + - type: Value + value: 503 + - type: Value + value: 504 + response: + contentType: text/html; charset=utf-8 + body: + type: ValueRef + valueRef: + group: "" + kind: ConfigMap + name: maintenance-page +{{- end }} diff --git a/charts/eduide/templates/pod-disruption-budgets.yaml b/charts/eduide/templates/pod-disruption-budgets.yaml new file mode 100644 index 0000000..a3fc9e4 --- /dev/null +++ b/charts/eduide/templates/pod-disruption-budgets.yaml @@ -0,0 +1,30 @@ +{{- /* +Only with more than one replica: a PDB over a single pod blocks every drain. +Single-node clusters must turn these off, because a drain there can never +reschedule the evicted replica and so waits on the second one forever. +*/}} +{{- if and .Values.podDisruptionBudget.enabled .Values.landingPage.enabled (gt (int .Values.landingPage.replicas) 1) }} +apiVersion: policy/v1 +kind: PodDisruptionBudget +metadata: + name: landing-page-pdb + namespace: {{ .Release.Namespace }} +spec: + minAvailable: 1 + selector: + matchLabels: + app: landing-page +{{- end }} +{{- if and .Values.podDisruptionBudget.enabled (gt (int .Values.service.replicas) 1) }} +--- +apiVersion: policy/v1 +kind: PodDisruptionBudget +metadata: + name: service-pdb + namespace: {{ .Release.Namespace }} +spec: + minAvailable: 1 + selector: + matchLabels: + app: service +{{- end }} diff --git a/charts/eduide/templates/service.yaml b/charts/eduide/templates/service.yaml index ad400d0..94098c1 100644 --- a/charts/eduide/templates/service.yaml +++ b/charts/eduide/templates/service.yaml @@ -6,7 +6,13 @@ metadata: app: service namespace: {{ .Release.Namespace }} spec: - replicas: 1 + replicas: {{ .Values.service.replicas }} + {{- /* Start the new pod before stopping the old one, so a deploy never leaves the route without a ready pod. */}} + strategy: + type: RollingUpdate + rollingUpdate: + maxUnavailable: 0 + maxSurge: 1 selector: matchLabels: app: service @@ -21,6 +27,7 @@ spec: # we need to be able to create and read sessions automountServiceAccountToken: true serviceAccountName: service-api-service-account + {{- include "eduide.spreadAcrossNodes" "service" | nindent 6 }} containers: - name: service-container image: {{ tpl (.Values.service.image | toString) . }} @@ -29,6 +36,18 @@ spec: - name: http containerPort: {{ tpl (.Values.service.port | toString) . }} protocol: TCP + {{- /* + The service has no health endpoint. An open port means Quarkus is up, + which is what matters: without this the pod counted as ready the moment + the container started and the old one was stopped too early. + */}} + readinessProbe: + tcpSocket: + port: http + initialDelaySeconds: 5 + periodSeconds: 5 + lifecycle: + {{- include "eduide.preStopDrain" . | nindent 10 }} envFrom: - configMapRef: name: service-config diff --git a/charts/eduide/values.yaml b/charts/eduide/values.yaml index de21f67..fde6e48 100644 --- a/charts/eduide/values.yaml +++ b/charts/eduide/values.yaml @@ -111,6 +111,9 @@ hosts: landingPage: # -- Whether the landing page shall be enabled enabled: true + # -- Number of landing page pods. More than one keeps the landing page up + # when a node goes away, and turns on its PodDisruptionBudget. + replicas: 1 # -- the landing page image to use. Templated, so the tag follows # versions.landingPage unless the whole string is overridden. image: '{{ .Values.imageRegistry }}/eduide-landing-page:{{ .Values.versions.landingPage }}' @@ -447,8 +450,27 @@ service: enable: false # -- service port (default: 8081) port: 8081 + # -- Number of REST service pods. The service keeps no state of its own, so + # more than one is safe; it also turns on its PodDisruptionBudget. + replicas: 1 # -- protocol of the REST-API protocol: https +# -- Static page shown instead of Envoy's raw error when the landing page is down +# @default -- (see details below) +maintenancePage: + # -- Answer 502/503/504 on the landing route with files/maintenance.html + # ("EduIDE is currently unavailable") instead of Envoy's plain-text error. + # Covers the landing page only - the REST service and sessions keep their + # own errors. Needs Envoy Gateway (BackendTrafficPolicy responseOverride). + enabled: true +# -- PodDisruptionBudgets for the landing page and REST service +# @default -- (see details below) +podDisruptionBudget: + # -- Render a PodDisruptionBudget (minAvailable: 1) for every Deployment + # above that runs more than one replica. Turn this off on single-node + # clusters: there a drain can never reschedule the evicted replica, so it + # waits on the last one forever. + enabled: true # -- Gateway API configuration (Envoy Gateway by default) gateway: # -- Master switch for Gateway API resources. diff --git a/docs/charts.md b/docs/charts.md index 376e660..8bbf47a 100644 --- a/docs/charts.md +++ b/docs/charts.md @@ -78,6 +78,39 @@ One install means one namespace, so prefixing buys no collision protection anyway. Standard `app.kubernetes.io/*` labels give the same grouping without the rename, and labels are additive on upgrade. +## Maintenance page and rolling updates + +While the landing page has no ready pod, Envoy would answer with a bare +`no healthy upstream`. The `eduide` chart ships a `BackendTrafficPolicy` on +`landing-route` that replaces every 502/503/504 there with +`charts/eduide/files/maintenance.html` ("EduIDE is currently unavailable"). +Envoy serves the page from the `maintenance-page` ConfigMap itself, so it works +with the landing page down and loads nothing from anywhere else. The status +code stays 5xx. It needs Envoy Gateway; turn it off with +`maintenancePage.enabled: false` elsewhere. + +Keep `%` out of the page. Envoy reads the body as a format string, rejects +the override because of it and still reports the policy as Accepted, so +visitors get an empty 503. `scripts/test-maintenance-page.sh` checks for it. + +It covers the landing page only. The REST service and sessions keep their own +errors, and nothing helps while Envoy itself is down. + +To show the page on purpose, scale `landing-page-deployment` to 0 - the +`maintenance.yml` workflow in EduIDE-deployment does exactly that. The next +deploy scales it back. + +So that deploys rarely show it: + +- the landing page and REST service start the new pod before stopping the old + one (`maxUnavailable: 0`), and the REST service is only ready once its port + is open +- both sleep 10s in `preStop`, so Envoy stops sending traffic before the + process exits +- `landingPage.replicas` / `service.replicas` above 1 add a + PodDisruptionBudget each (`podDisruptionBudget.enabled`, off on single-node + clusters, where it would block every drain) and prefer different nodes + ## Checking a change ```bash diff --git a/scripts/test-maintenance-page.sh b/scripts/test-maintenance-page.sh new file mode 100755 index 0000000..13adc42 --- /dev/null +++ b/scripts/test-maintenance-page.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# The maintenance page and the settings that keep deploys from showing it. +# +# While the landing page has no ready pod, Envoy answers with a bare +# "no healthy upstream". The chart replaces that with files/maintenance.html, +# and makes rolling updates keep a ready pod around so it is rarely needed. +# None of this is visible until something goes down, so assert it here. + +set -uo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +CHART="$ROOT/charts/eduide" +PAGE="$CHART/files/maintenance.html" +FAILED=0 + +ok() { printf ' PASS %s\n' "$1"; } +bad() { printf ' FAIL %s\n' "$1"; [[ -n "${2:-}" ]] && printf ' %s\n' "$2"; FAILED=1; } + +"$(dirname "${BASH_SOURCE[0]}")/resolve-deps.sh" "$CHART" >/dev/null || { + echo "could not resolve chart dependencies" >&2; exit 1; } + +render() { + helm template t "$CHART" --set skipPreflight=true --set demoApplication.install=false \ + --set keycloak.allowUnauthenticated=true "$@" 2>/dev/null +} + +# expect