Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,20 @@ not ours to move.
node; under `--wait` it would time out and `--atomic` would then roll back a
healthy deploy.

**A release tag is `vX.Y.Z`; the image it publishes is `X.Y.Z`.** The shared
build workflow normalises with `${RELEASE_TAG#v}`, but an `image-tag` override
is used verbatim - so a caller passing `github.event.release.tag_name` defeats
the normalisation and publishes `v1.2.0`, which no chart can consume. Releasing
`1.2.0` without the `v` happens to produce the right image and fails the Tag
format check, which is how this stayed hidden. Fixed in EduIDE-Cloud and
EduIDE-Landing-Page; check any repo that has not been.

**helm-diff must stay pinned.** Its current release declares `platformHooks`,
which the pinned helm v3.16.3 cannot parse, so the plugin fails to load. Both
the deploy and the bootstrap preview end in `|| true`, so the diff renders
nothing at all and the step still passes - every "Pending change" summary was
silently empty until it was pinned to v3.9.11.

## Two `lookup` calls make rendering nondeterministic

The `eduide` chart preserves live `minInstances`/`maxInstances` on
Expand Down
73 changes: 60 additions & 13 deletions docs/cluster-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ that reports itself healthy and serves nothing.
| Gateway API CRDs, Envoy Gateway, cert-manager, storage | **you**, once per cluster |
| A GatewayClass whose load balancer address matches DNS | **you** |
| The ACME `ClusterIssuer` | `Bootstrap cluster`, from `spec.acmeEmail` |
| The GatewayClass and its EnvoyProxy | `Bootstrap cluster`, from `spec.gatewayClass` and `spec.envoyProxy` |
| Redirects from hostnames you used to serve | `Bootstrap cluster`, from `spec.redirects` |
| The webview wildcard certificate | **you**, see [tum-certificates.md](tum-certificates.md) |
| DNS records | **you** (RBG) |
| Keycloak client and redirect URIs | **you**, see [keycloak-setup.md](keycloak-setup.md) |
Expand Down Expand Up @@ -83,7 +85,7 @@ Envoy Gateway can be configured to **merge gateways**: every `Gateway` using a
GatewayClass shares one Envoy deployment and therefore one external address.
`tum-student` is configured that way today:

```
```text
GatewayClass envoy
-> parametersRef: EnvoyProxy envoy-gateway-system/artemis-envoy-proxy
mergeGateways: true
Expand All @@ -92,7 +94,7 @@ GatewayClass envoy

and the EduIDE hostnames point somewhere else:

```
```text
eduide.student.k8s.aet.cit.tum.de -> k8s-stud-lb3 -> 131.159.88.14 (pool lb3)
```

Expand All @@ -112,9 +114,11 @@ own `EnvoyProxy` pinned to pool `lb3`, and set `gatewayClassName` in
`clusters/tum-student.yaml` to match. EduIDE then has its own data plane on
`131.159.88.14`, which is what DNS already says.

The `eduide-cluster` chart can create both objects (`gatewayClass.create` and
`envoyProxy.create`), but **`bootstrap-cluster.yml` does not pass either**, so
today option (b) means creating them by hand or extending the workflow. See
`bootstrap-cluster.yml` passes both from the cluster manifest, so option (b) is
`spec.gatewayClass.create: true` plus a `spec.envoyProxy` block. `envoyProxy.name`
is the name of the EnvoyProxy resource itself; the MetalLB pool goes inside it,
under `spec.provider.kubernetes.envoyService.annotations`. That is exactly what
`tum-production` does. See
[envoy-gateway-setup.md](envoy-gateway-setup.md) for the MetalLB details.

`clusters/tum-production.yaml` carries `spec.loadBalancerIP: 131.159.88.82`.
Expand Down Expand Up @@ -174,7 +178,7 @@ certificate supplied through `wildcardTLSSecret`; see

Four records per environment, all pointing at the address from step 2:

```
```text
<landing> the landing page
service.<landing> the REST service
instance.<landing> session ingress
Expand Down Expand Up @@ -203,7 +207,7 @@ Full instructions, including what is missing today, are in

## Step 6: bootstrap

```
```text
Actions -> Bootstrap cluster
cluster: tum-student
chart_version: 2.1.0
Expand Down Expand Up @@ -274,7 +278,7 @@ invalid certificate. Nothing appeared in any log.

## Step 8: deploy an environment

```
```text
Actions -> Deploy (dispatch) -> environment: test1, dry_run: true
```

Expand Down Expand Up @@ -317,12 +321,55 @@ EduIDE somewhere unexpected.
`runner` is per cluster because deploying is not building: a deploy has to reach
the API server, and the clusters may differ in how they are reachable.

## Known gaps
## When a certificate will not issue

Three things this repository describes but does not yet apply. All three are
listed above in context; collected here so they are not missed:
Two failures look identical from outside - the listener never programs - and
have different causes.

**No challenges outstanding, order `errored`.** Let's Encrypt occasionally fails
to finalize an order that has already validated:

```text
Failed to finalize Order: 404 urn:ietf:params:acme:error:malformed:
Certificate not found
```

Confirm it is that failure before doing anything, by reading down the chain the
`Certificate` owns - its conditions alone do not tell you:

```bash
kubectl -n eduide-system describe certificate <name>
kubectl -n eduide-system get certificaterequest,order,challenge
```

cert-manager retries on its own, but only after an exponential backoff that
starts at an hour. Clear the backoff to retry immediately (`--subresource`
needs kubectl v1.24 or newer):

```bash
kubectl -n eduide-system patch certificate <name> --type=merge --subresource=status \
-p '{"status":{"lastFailureTime":null,"failedIssuanceAttempts":null}}'
```

It issued on the first retry when this happened during the `tum-production`
bootstrap, so one clean retry is the expected outcome and no configuration
change is called for. If the retry fails too, the transient explanation is
wrong: go back over the `CertificateRequest`, `Order`, `Challenge` and
`ClusterIssuer`, and the cert-manager controller logs, before changing anything.

**Challenges pending, staying pending.** That is a real problem, and the error
on the `Challenge` says which:

- **HTTP 404** - something answered, but nothing routes the solver path. The
hostname has no listener on the Gateway, or the listener exists on a Gateway
the solver's HTTPRoute does not attach to.
- **DNS, connection refused or timeout** - nothing answered at all. The name
does not resolve, or it resolves somewhere that is not this Gateway.

See step 3.

## Known gaps

| Gap | Consequence |
|---|---|
| `bootstrap-cluster.yml` passes no `gatewayClass` or `envoyProxy` | A cluster needing its own data plane address must have it created by hand (step 2) |
| `spec.loadBalancerIP` is in the schema and in `tum-production.yaml` | Nothing reads it. It records intent only (step 2) |
| `spec.loadBalancerIP` is in the schema and in `tum-production.yaml` | Nothing reads it. `spec.envoyProxy` is what actually pins the address (step 2) |
39 changes: 36 additions & 3 deletions docs/envoy-gateway-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ If the GatewayClass is configured with `mergeGateways: true`, **every** Gateway
using that class shares one Envoy deployment and one external address. On
`tum-student` today:

```
```text
GatewayClass envoy
-> EnvoyProxy envoy-gateway-system/artemis-envoy-proxy
mergeGateways: true
Expand Down Expand Up @@ -96,8 +96,10 @@ envoyProxy:
type: LoadBalancer
```

**`bootstrap-cluster.yml` does not pass either block.** Until it does, create
them by hand or add the values to the workflow. `spec.loadBalancerIP` in
**`bootstrap-cluster.yml` passes both blocks straight through** from
`clusters/<name>.yaml`, so the manifest above is the whole configuration - do
not create a GatewayClass or an EnvoyProxy by hand as well, or bootstrap and
the hand-made copy will fight over the same names. `spec.loadBalancerIP` in
`clusters/tum-production.yaml` records the intended address but is read by
nothing.

Expand Down Expand Up @@ -136,6 +138,37 @@ dig +short eduide.student.k8s.aet.cit.tum.de A
The Gateway's address and the DNS answer must be the same. A service's
`clusterIP` is internal and never what DNS should point at.

## Redirecting a hostname you used to serve

A renamed installation leaves its old hostname in bookmarks, course pages and
Artemis exercise configurations. `spec.redirects` in the cluster manifest keeps
it answering and 301s it to the new name, path and query intact:

```yaml
redirects:
- name: legacy-landing
from: theia.artemis.cit.tum.de
to: eduide.artemis.cit.tum.de
tlsSecretName: legacy-landing-cert-secret
```

Bootstrap gives it an HTTPS listener and adds its hostname to the certificate
for that secret, so it renews like any other name. It also gets an ACME `:80`
listener, but **only when the cluster sets `spec.acmeHttp: true`** - on a
cluster that issues its certificates some other way there is no such listener,
and nothing should be written that assumes one. Check it with headers, not a
browser:

```bash
curl -sI https://theia.artemis.cit.tum.de/foo?bar=1 | grep -i '^location'
# location: https://eduide.artemis.cit.tum.de/foo?bar=1
```

A 302 would work but should not be used. It means *temporarily* moved, so
clients and bookmarks keep the old hostname and keep sending traffic to it. A
301 says the move is permanent, which is both true here and what lets clients
and search engines update what they have stored.

## TLS

Each listener terminates with a Secret named per cluster in
Expand Down
Loading