diff --git a/.trivyignore.yaml b/.trivyignore.yaml index 5b26958b3a..773252a408 100644 --- a/.trivyignore.yaml +++ b/.trivyignore.yaml @@ -541,6 +541,16 @@ misconfigurations: grants get alone, scoped by resourceNames to the six substrate-* CA Secrets cert-manager issues, so the identity cannot list or watch the namespace's Secrets, cannot read any other Secret, and cannot create, change or delete one. + - id: KSV-0113 + paths: + - k8s/providers/hetzner/apps/customer-identity/role.yaml + statement: >- + This is a namespaced Role in the customer-identity namespace, bound to the + customer-identity/customer-identity-setup ServiceAccount of the ZITADEL setup CronJob. Its one + rule grants get and patch alone, scoped by resourceNames to the single + customer-dex-zitadel-client Secret the CronJob fills with the client ZITADEL issues, so the + identity cannot list or watch the namespace's Secrets, cannot read any other Secret, and cannot + create or delete one. # KSV-0020 "runs with UID <= 10000" on the vault-config Job — 2 findings, both LOW. # # UID 100 is the identity the openbao image itself DEFINES — measured against the pinned diff --git a/docs/dr/crypto-custody.md b/docs/dr/crypto-custody.md index 17a833985b..8cc54bd4c9 100644 --- a/docs/dr/crypto-custody.md +++ b/docs/dr/crypto-custody.md @@ -25,6 +25,8 @@ and [openbao.md](openbao.md) (OpenBao-specific DR). | **Hetzner Cloud token** | provision / destroy nodes; reconfigure Cloud LB and firewall | destroy infrastructure; pivot via the LB | mint a new one in the Hetzner console; rotate `HCLOUD_TOKEN` | | **Cloudflare API token** | manage DNS for the platform domain | redirect prod traffic; mint Origin CA certs | mint a new one in the Cloudflare dashboard | | **Actual Budget E2EE password** | client-side key to encrypt/decrypt an Actual budget file | decrypt that budget's financial data | the budget is **permanently unrecoverable** — the server never held the key | +| **ZITADEL master key** | encrypt the customer identity provider's signing keys and stored secrets (`customer-identity` namespace) | with a database copy, decrypt those keys and secrets — rotate them in ZITADEL | restored `customer-identity-db` backups cannot be used; customer accounts are recreated | +| **ZITADEL setup user key** | sign the setup CronJob's tokens as ZITADEL's `zitadel-setup` system API user (instance owner) | administer the customer identity provider — delete `customer-identity/zitadel-setup-key` so cert-manager issues a new key | nothing to recover: cert-manager issues a new key and ZITADEL trusts the new certificate | The rest of this document walks each row in detail. @@ -316,6 +318,38 @@ strongest reason to keep the OpenBao entry backed up. --- +## ZITADEL master key + +ZITADEL (customer sign-in for the business-site portal) encrypts its signing keys and stored +secrets with a 32-character master key. The key is generated once in-cluster and written to +OpenBao `apps/customer-identity/zitadel` (property `masterkey`) by a create-only PushSecret +(`k8s/bases/infrastructure/vault-seed/push-secret-push-customer-identity-zitadel-masterkey.yaml`). +OpenBao is its custody of record: an existing value is never replaced, and +`customer-identity/zitadel-masterkey` is read back from it. + +- **Restore order:** restore OpenBao first ([openbao.md](openbao.md)), then `customer-identity-db` + from its R2 backups. A fresh OpenBao would generate a new key that cannot read the restored database. +- **After a database restore:** the restored database can hold an older secret for the customer + Dex's client than `customer-identity/customer-dex-zitadel-client`. Nothing has to be done by + hand: the setup CronJob proves the stored secret against ZITADEL on every run and issues a new one + within 15 minutes when ZITADEL refuses it. +- **Rotation:** not supported by ZITADEL for an existing database. Treat the key as permanent. + +## ZITADEL setup user key + +The setup CronJob (`k8s/providers/hetzner/apps/customer-identity/cron-job-setup.yaml`) reaches +ZITADEL's APIs as the system API user `zitadel-setup`, whose only role is `IAM_OWNER` on the instance. +ZITADEL trusts the public key in cert-manager's certificate `customer-identity/zitadel-setup-key`. +It mounts only `tls.crt`; only the CronJob's signer container mounts `tls.key`, and no Role grants +reading the Secret. + +- **Custody:** the cluster only. The key is not backed up and needs no backup: a rebuild issues a new + one, and ZITADEL trusts whatever certificate it mounts. +- **Rotation:** automatic. cert-manager issues a new key 30 days before the 90-day certificate expires, + and Reloader restarts ZITADEL to read it. To rotate at once, delete the Secret. + +--- + ## Cross-references - [`runbook.md`](runbook.md) — operational scenarios (rotation steps, full diff --git a/k8s/bases/infrastructure/vault-config/job.yaml b/k8s/bases/infrastructure/vault-config/job.yaml index dfb403d15d..ab7119eed6 100644 --- a/k8s/bases/infrastructure/vault-config/job.yaml +++ b/k8s/bases/infrastructure/vault-config/job.yaml @@ -906,6 +906,16 @@ spec: } POLICY + # Customer sign-in (apps/customer-identity, business-site#302) + # reads ZITADEL's master key, and later the customer Dex's + # ZITADEL client, through the shared `external-secrets` + # ClusterSecretStore role. + bao policy write app-customer-identity-readonly - <<'POLICY' + path "secret/data/apps/customer-identity/*" { + capabilities = ["read"] + } + POLICY + # actual-budget reads its Enable Banking bank-sync credential # (application-id + secret-key) from OpenBao via the shared # `external-secrets` ClusterSecretStore role (apps/actual-budget/ @@ -1067,6 +1077,16 @@ spec: path "secret/data/apps/umami/*" { capabilities = ["create", "update", "read"] } + # ZITADEL's master key, generated once by its create-only + # PushSecret (vault-seed), and the customer Dex's ZITADEL + # client, which the customer-identity setup CronJob issues and + # a PushSecret copies here. Exact keys only. + path "secret/data/apps/customer-identity/zitadel" { + capabilities = ["create", "update", "read"] + } + path "secret/data/apps/customer-identity/dex-zitadel-client" { + capabilities = ["create", "update", "read"] + } path "secret/data/apps/actual-budget/*" { capabilities = ["create", "update", "read"] } @@ -1128,6 +1148,12 @@ spec: path "secret/metadata/apps/umami/*" { capabilities = ["create", "update", "read"] } + path "secret/metadata/apps/customer-identity/zitadel" { + capabilities = ["create", "update", "read"] + } + path "secret/metadata/apps/customer-identity/dex-zitadel-client" { + capabilities = ["create", "update", "read"] + } path "secret/metadata/apps/actual-budget/*" { capabilities = ["create", "update", "read"] } @@ -1163,7 +1189,7 @@ spec: bao write auth/kubernetes/role/external-secrets \ bound_service_account_names=external-secrets \ bound_service_account_namespaces=external-secrets \ - policies=infra-backup-readonly,infra-dns-readonly,infra-monitoring-readonly,infra-hcloud-readonly,infra-tls-readonly,infra-ghcr-readonly,app-fleetdm-readonly,app-wedding-readonly,app-umami-readonly,app-kagent-readonly,app-actual-budget-readonly,app-ascoachingogvaner-readonly,infra-oidc-readonly,vault-seed-write \ + policies=infra-backup-readonly,infra-dns-readonly,infra-monitoring-readonly,infra-hcloud-readonly,infra-tls-readonly,infra-ghcr-readonly,app-fleetdm-readonly,app-wedding-readonly,app-umami-readonly,app-kagent-readonly,app-customer-identity-readonly,app-actual-budget-readonly,app-ascoachingogvaner-readonly,infra-oidc-readonly,vault-seed-write \ ttl=1h # Dedicated role for the fleetdm VaultDynamicSecret generator. diff --git a/k8s/bases/infrastructure/vault-seed/external-secret-generated-customer-identity-zitadel-masterkey.yaml b/k8s/bases/infrastructure/vault-seed/external-secret-generated-customer-identity-zitadel-masterkey.yaml new file mode 100644 index 0000000000..e88ed4b151 --- /dev/null +++ b/k8s/bases/infrastructure/vault-seed/external-secret-generated-customer-identity-zitadel-masterkey.yaml @@ -0,0 +1,17 @@ +--- +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: generated-customer-identity-zitadel-masterkey + namespace: openbao +spec: + refreshInterval: "0" + target: + name: generated-customer-identity-zitadel-masterkey + creationPolicy: Owner + dataFrom: + - sourceRef: + generatorRef: + apiVersion: generators.external-secrets.io/v1alpha1 + kind: Password + name: customer-identity-zitadel-masterkey diff --git a/k8s/bases/infrastructure/vault-seed/kustomization.yaml b/k8s/bases/infrastructure/vault-seed/kustomization.yaml index e924834de9..2b03611eed 100644 --- a/k8s/bases/infrastructure/vault-seed/kustomization.yaml +++ b/k8s/bases/infrastructure/vault-seed/kustomization.yaml @@ -26,6 +26,9 @@ resources: - push-secret-push-umami-admin-password.yaml - external-secret-generated-umami-ascoachingogvaner-password.yaml - push-secret-push-umami-ascoachingogvaner-password.yaml + - password-customer-identity-zitadel-masterkey.yaml + - external-secret-generated-customer-identity-zitadel-masterkey.yaml + - push-secret-push-customer-identity-zitadel-masterkey.yaml - push-secret-seed-r2-credentials.yaml - push-secret-seed-wedding-db-backup-r2.yaml - push-secret-seed-github-app.yaml diff --git a/k8s/bases/infrastructure/vault-seed/password-customer-identity-zitadel-masterkey.yaml b/k8s/bases/infrastructure/vault-seed/password-customer-identity-zitadel-masterkey.yaml new file mode 100644 index 0000000000..57bb59f9ff --- /dev/null +++ b/k8s/bases/infrastructure/vault-seed/password-customer-identity-zitadel-masterkey.yaml @@ -0,0 +1,13 @@ +--- +# ZITADEL requires exactly 32 characters. +apiVersion: generators.external-secrets.io/v1alpha1 +kind: Password +metadata: + name: customer-identity-zitadel-masterkey + namespace: openbao +spec: + length: 32 + digits: 8 + symbols: 0 + noUpper: false + allowRepeat: true diff --git a/k8s/bases/infrastructure/vault-seed/push-secret-push-customer-identity-zitadel-masterkey.yaml b/k8s/bases/infrastructure/vault-seed/push-secret-push-customer-identity-zitadel-masterkey.yaml new file mode 100644 index 0000000000..e01abdc28e --- /dev/null +++ b/k8s/bases/infrastructure/vault-seed/push-secret-push-customer-identity-zitadel-masterkey.yaml @@ -0,0 +1,25 @@ +--- +# ZITADEL's master key (apps/customer-identity/external-secret-masterkey.yaml). +# Create-only: once OpenBao holds a key, a regenerated in-cluster value never +# replaces it, so a rebuilt cluster that restores OpenBao keeps the key its +# restored database was encrypted with (docs/dr/crypto-custody.md). +apiVersion: external-secrets.io/v1alpha1 +kind: PushSecret +metadata: + name: push-customer-identity-zitadel-masterkey + namespace: openbao +spec: + refreshInterval: 1h + updatePolicy: IfNotExists + secretStoreRefs: + - name: openbao + kind: ClusterSecretStore + selector: + secret: + name: generated-customer-identity-zitadel-masterkey + data: + - match: + secretKey: password + remoteRef: + remoteKey: apps/customer-identity/zitadel + property: masterkey diff --git a/k8s/providers/hetzner/apps/customer-identity/certificate-setup-key.yaml b/k8s/providers/hetzner/apps/customer-identity/certificate-setup-key.yaml new file mode 100644 index 0000000000..724b98878a --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/certificate-setup-key.yaml @@ -0,0 +1,29 @@ +# The key pair of ZITADEL's `zitadel-setup` system API user (helm-release.yaml +# SystemAPIUsers). ZITADEL mounts only tls.crt; only the setup CronJob's +# signer mounts tls.key (cron-job-setup.yaml), and no Role lets anyone read +# the Secret through the API. +# +# cert-manager renews it with a new key 30 days before it expires. Reloader +# then restarts ZITADEL so it reads the new certificate; a setup run during +# that restart fails and the next one succeeds. +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: zitadel-setup-key + namespace: customer-identity +spec: + secretName: zitadel-setup-key + commonName: zitadel-setup + duration: 2160h + renewBefore: 720h + isCA: false + usages: + - digital signature + privateKey: + algorithm: RSA + size: 2048 + encoding: PKCS8 + rotationPolicy: Always + issuerRef: + name: customer-identity-setup + kind: Issuer diff --git a/k8s/providers/hetzner/apps/customer-identity/certificate.yaml b/k8s/providers/hetzner/apps/customer-identity/certificate.yaml new file mode 100644 index 0000000000..b6a61884ab --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/certificate.yaml @@ -0,0 +1,18 @@ +# DNS-01 certificate for the customer identity names: ZITADEL +# (accounts.devantler.tech) and the portal's customer Dex +# (login.devantler.tech, next change). The Gateway listeners read it through +# reference-grant.yaml. +apiVersion: cert-manager.io/v1 +kind: Certificate +metadata: + name: customer-identity + namespace: customer-identity +spec: + secretName: customer-identity-tls + dnsNames: + - accounts.devantler.tech + - login.devantler.tech + issuerRef: + group: cert-manager.io + kind: ClusterIssuer + name: letsencrypt-prod diff --git a/k8s/providers/hetzner/apps/customer-identity/cilium-network-policy-coroot.yaml b/k8s/providers/hetzner/apps/customer-identity/cilium-network-policy-coroot.yaml new file mode 100644 index 0000000000..e080ec653e --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/cilium-network-policy-coroot.yaml @@ -0,0 +1,24 @@ +# Let Coroot's cluster-agent (observability namespace) reach the +# customer-identity-db instances on 5432 for its Postgres integration +# (cluster.yaml). Additive: it admits only the cluster-agent, and only to the +# database pods. +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: allow-coroot-postgres-scrape + namespace: customer-identity +spec: + endpointSelector: + matchLabels: + cnpg.io/cluster: customer-identity-db + ingress: + - fromEndpoints: + - matchLabels: + k8s:io.kubernetes.pod.namespace: observability + app.kubernetes.io/managed-by: coroot-operator + app.kubernetes.io/part-of: coroot + app.kubernetes.io/component: coroot-cluster-agent + toPorts: + - ports: + - port: "5432" + protocol: TCP diff --git a/k8s/providers/hetzner/apps/customer-identity/cilium-network-policy.yaml b/k8s/providers/hetzner/apps/customer-identity/cilium-network-policy.yaml new file mode 100644 index 0000000000..6c97224538 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/cilium-network-policy.yaml @@ -0,0 +1,62 @@ +# ZITADEL and its database. Only the Gateway reaches ZITADEL (8080); only +# ZITADEL, the other instance and the CNPG operator reach the database. +# Egress: the namespace itself, the API server (CNPG instance manager), R2 +# for backups, and the business site's SES SMTP endpoint for ZITADEL's +# invitation and recovery mail (platform#5400). +apiVersion: cilium.io/v2 +kind: CiliumNetworkPolicy +metadata: + name: allow-customer-identity + namespace: customer-identity +spec: + endpointSelector: {} + ingress: + - fromEntities: + - ingress + toPorts: + - ports: + - port: "8080" + protocol: TCP + - fromEndpoints: + - matchLabels: + k8s:io.kubernetes.pod.namespace: customer-identity + - fromEndpoints: + - matchLabels: + k8s:io.kubernetes.pod.namespace: cnpg-system + toPorts: + - ports: + - port: "8000" + protocol: TCP + - port: "5432" + protocol: TCP + egress: + - toEndpoints: + - matchLabels: + k8s:io.kubernetes.pod.namespace: customer-identity + - toEntities: + - kube-apiserver + - toFQDNs: + - matchPattern: "*.r2.cloudflarestorage.com" + toPorts: + - ports: + - port: "443" + protocol: TCP + - toFQDNs: + - matchName: email-smtp.eu-central-1.amazonaws.com + toPorts: + - ports: + - port: "587" + protocol: TCP + - toEndpoints: + - matchLabels: + k8s:io.kubernetes.pod.namespace: kube-system + k8s-app: kube-dns + toPorts: + - ports: + - port: "53" + protocol: UDP + - port: "53" + protocol: TCP + rules: + dns: + - matchPattern: "*" diff --git a/k8s/providers/hetzner/apps/customer-identity/cluster.yaml b/k8s/providers/hetzner/apps/customer-identity/cluster.yaml new file mode 100644 index 0000000000..184391e907 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/cluster.yaml @@ -0,0 +1,105 @@ +# ZITADEL's PostgreSQL store: every customer account, credential, MFA factor +# and session. Not rebuildable, so it is backed up off-cluster to R2 +# (object-store.yaml, scheduled-backup.yaml) and Flux never prunes it. +# +# CNPG creates the `zitadel` database and its owner login with the generated +# password (external-secret-db-zitadel.yaml). ZITADEL connects as that owner +# and creates its own schemas (`zitadel init zitadel`, then `zitadel setup`), +# so no ZITADEL step needs a superuser (helm-release.yaml). +apiVersion: postgresql.cnpg.io/v1 +kind: Cluster +metadata: + name: customer-identity-db + namespace: customer-identity + annotations: + kustomize.toolkit.fluxcd.io/prune: disabled +spec: + # renovate: datasource=docker depName=ghcr.io/cloudnative-pg/postgresql + imageName: ghcr.io/cloudnative-pg/postgresql:18.7-system-trixie + instances: 2 + # The platform's storage tier for database pods (as every production + # database declares). CNPG ignores a priorityClassName change, so the + # matching env var is what rolls the instances when it changes. + priorityClassName: hcloud-volume-workload + env: + - name: PLATFORM_DATABASE_PRIORITY_CLASS + value: hcloud-volume-workload + # Hetzner Cloud volumes, as OpenBao and Coroot use, not Longhorn: CNPG + # already keeps a second copy, so Longhorn's three replicas per instance + # would triple storage the shared pool cannot spare (the first rollout's + # volume was refused for insufficient Longhorn storage). An hcloud volume is + # not force-detached when its node leaves, so the instances stay on the + # static baseline workers (require-baseline-workers-for-hcloud-volumes). + # Temporary: every platform volume finishes on Longhorn V2, and this + # cluster's two claims move there with the database migration (#5842). + storage: + size: 10Gi + storageClass: hcloud + affinity: + nodeSelector: + platform.devantler.tech/baseline-worker: "true" + # Coroot's Postgres integration, as every platform database has. CNPG + # copies these annotations onto the instance pods, where Coroot's + # cluster-agent discovers them; it logs in as the coroot_monitor role below, + # and only it may reach the instances for this (cilium-network-policy-coroot.yaml). + inheritedMetadata: + annotations: + coroot.com/postgres-scrape: "true" + coroot.com/postgres-scrape-port: "5432" + coroot.com/postgres-scrape-credentials-secret-name: customer-identity-db-coroot-monitor + coroot.com/postgres-scrape-credentials-secret-username-key: username + coroot.com/postgres-scrape-credentials-secret-password-key: password + coroot.com/postgres-scrape-param-sslmode: require + # Same quorum replication as the platform's other databases. While + # synchronous replication is active, every commit waits for the standby, so + # a promoted standby holds every committed transaction. With + # dataDurability: preferred, CNPG counts only healthy standbys, so writes + # continue when the standby is unavailable (down or unhealthy), and commits + # made then exist only on the primary until the standby or the WAL archive + # receives them. This is not RPO=0. + postgresql: + synchronous: + method: any + number: 1 + dataDurability: preferred + # Any pg_stat_statements.* parameter makes CNPG preload the extension and + # create it in every database, which feeds Coroot's per-query view. + parameters: + pg_stat_statements.max: "10000" + pg_stat_statements.track: all + bootstrap: + initdb: + database: zitadel + owner: zitadel + secret: + name: customer-identity-db-zitadel + postInitApplicationSQL: + - REVOKE CREATE ON SCHEMA public FROM PUBLIC + managed: + roles: + # Keeps the owner's password equal to its Secret after bootstrap. + - name: zitadel + ensure: present + login: true + passwordSecret: + name: customer-identity-db-zitadel + # Coroot's read-only monitoring login: pg_monitor only, no access to + # ZITADEL's data. + - name: coroot_monitor + ensure: present + login: true + inRoles: + - pg_monitor + passwordSecret: + name: customer-identity-db-coroot-monitor + resources: + requests: + cpu: 50m + memory: 256Mi + limits: + memory: 512Mi + plugins: + - name: barman-cloud.cloudnative-pg.io + isWALArchiver: true + parameters: + barmanObjectName: customer-identity-db diff --git a/k8s/providers/hetzner/apps/customer-identity/cron-job-setup.yaml b/k8s/providers/hetzner/apps/customer-identity/cron-job-setup.yaml new file mode 100644 index 0000000000..d6ce4565a5 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/cron-job-setup.yaml @@ -0,0 +1,510 @@ +# Declarative ZITADEL setup for the customer sign-in (business-site#302). +# Every 15 minutes it reconciles, through ZITADEL's own APIs: +# - the "Customer sign-in" project and its "Customer Dex" client, exactly as the +# business-site preparer's CreateApplication template declares it (code grant +# only, the exact Dex callback, Login V1). It restores drifted settings; the +# two lists the client does not use (additional origins, post-logout URIs) +# are set only at creation, because ZITADEL cannot clear a list by update. +# Every run proves the stored secret against ZITADEL's token endpoint, and +# when it is missing, the placeholder or refused (for example after a +# database restore), it issues a new one into secret-dex-zitadel-client.yaml; +# - the SES SMTP login: when the business site's credential (OpenBao +# apps/business-site/smtp) changes, it updates ZITADEL's provider; +# - the administrator's one-time password: until the maintainer's first +# sign-in replaces it, it stays equal to zitadel-first-admin, so the +# commissioning step works even after that Secret is regenerated. Once he has +# chosen his own password it is never touched again. +# A run that finds nothing to change writes nothing. A duplicate, a deactivated +# client, a missing Secret, an API error or a refused write stops the run +# before it changes anything further; the next run starts again. +# +# Credentials stay apart: +# - The signer (init container) alone mounts the private key of ZITADEL's +# `zitadel-setup` system API user (certificate-setup-key.yaml). It writes a +# 10-minute token to a memory volume and exits. +# - The setup container alone gets a Kubernetes token, projected to +# /run/kube-api, which may only read and update the client Secret (role.yaml). +# It reads the SMTP login (zitadel-smtp) and the one-time password +# (zitadel-first-admin) as mounted files, never as environment variables. +# - Neither prints a secret. +# +# The maintainer's own first sign-in (choosing his password, enrolling MFA) is +# not part of this. +apiVersion: batch/v1 +kind: CronJob +metadata: + name: customer-identity-setup + namespace: customer-identity + annotations: + # The scripts' own ${...} expansions are bash, not Flux variables. + kustomize.toolkit.fluxcd.io/substitute: disabled +spec: + schedule: "*/15 * * * *" + concurrencyPolicy: Forbid + startingDeadlineSeconds: 300 + successfulJobsHistoryLimit: 1 + failedJobsHistoryLimit: 3 + jobTemplate: + spec: + backoffLimit: 0 + activeDeadlineSeconds: 300 + ttlSecondsAfterFinished: 86400 + template: + metadata: + labels: + app.kubernetes.io/name: customer-identity-setup + spec: + serviceAccountName: customer-identity-setup + automountServiceAccountToken: false + restartPolicy: Never + securityContext: + fsGroupChangePolicy: OnRootMismatch + runAsNonRoot: true + runAsUser: 65532 + runAsGroup: 65532 + fsGroup: 65532 + seccompProfile: + type: RuntimeDefault + initContainers: + - name: sign-token + image: docker.io/alpine/openssl:3.5.8@sha256:3f25da71f70eba788067daac3f3df03bd1de7a7c52ed89fa93b94ad2c92d986b + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 65532 + runAsGroup: 65532 + capabilities: + drop: + - ALL + env: + - name: ZITADEL_AUDIENCE + value: https://accounts.devantler.tech + - name: ZITADEL_SYSTEM_USER + value: zitadel-setup + volumeMounts: + - name: setup-key + mountPath: /run/setup-key + readOnly: true + - name: setup-token + mountPath: /run/setup-token + resources: + requests: + cpu: 10m + memory: 16Mi + limits: + cpu: 100m + memory: 32Mi + command: + - /bin/sh + - -c + - |- + #!/bin/sh + # Signs a short-lived JWT for ZITADEL's `zitadel-setup` system API user with + # the key cert-manager keeps in zitadel-setup-key. Only this init container + # mounts the private key; it writes the token to a memory-backed volume that + # the setup container reads. The token is never printed. + set -eu + key=/run/setup-key/tls.key + out=/run/setup-token/token + audience="${ZITADEL_AUDIENCE:?}" + user="${ZITADEL_SYSTEM_USER:?}" + lifetime="${TOKEN_LIFETIME_SECONDS:-600}" + + b64url() { openssl base64 -A | tr '+/' '-_' | tr -d '='; } + + [ -s "$key" ] || { echo "sign-token: missing signing key" >&2; exit 1; } + now=$(date +%s) + header=$(printf '{"alg":"RS256","typ":"JWT"}' | b64url) + payload=$(printf '{"iss":"%s","sub":"%s","aud":"%s","iat":%d,"exp":%d}' \ + "$user" "$user" "$audience" "$now" "$((now + lifetime))" | b64url) + signature=$(printf '%s.%s' "$header" "$payload" | openssl dgst -sha256 -sign "$key" -binary | b64url) + umask 077 + printf '%s.%s.%s' "$header" "$payload" "$signature" > "$out" + echo "sign-token: wrote a ${lifetime}s token for ${user}" + containers: + - name: setup + image: docker.io/alpine/k8s:1.37.2@sha256:dcb7185e112a453aa72f426754849a97357727ef8f40168fb208c8cf59952042 + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 65532 + runAsGroup: 65532 + capabilities: + drop: + - ALL + env: + - name: HOME + value: /tmp + - name: ZITADEL_URL + value: http://zitadel.customer-identity.svc:8080 + - name: ZITADEL_HOST + value: accounts.devantler.tech + - name: TOKEN_FILE + value: /run/setup-token/token + - name: ADMIN_EMAIL + value: ned@devantler.tech + volumeMounts: + - name: setup-token + mountPath: /run/setup-token + readOnly: true + - name: tmp + mountPath: /tmp + - name: smtp + mountPath: /run/smtp + readOnly: true + - name: first-admin + mountPath: /run/first-admin + readOnly: true + - name: kube-api + mountPath: /run/kube-api + readOnly: true + resources: + requests: + cpu: 10m + memory: 32Mi + limits: + cpu: 200m + memory: 128Mi + command: + - /bin/bash + - -c + - |- + #!/usr/bin/env bash + # Reconciles ZITADEL's customer Dex client and its SES SMTP login from the + # state declared here. Every run is idempotent: it changes nothing when ZITADEL + # and the client Secret already match. Any unexpected state (a duplicate, a + # deactivated client, a missing Secret, an API error) stops the run before it + # writes anything further. No secret value is ever printed. + set -euo pipefail + umask 077 + + : "${ZITADEL_URL:?}" "${ZITADEL_HOST:?}" "${TOKEN_FILE:?}" "${ADMIN_EMAIL:?}" + # Secrets arrive as files, never as environment variables. + smtp_dir=${SMTP_DIR:-/run/smtp} + admin_password_file=${ADMIN_PASSWORD_FILE:-/run/first-admin/password} + kube_api_dir=${KUBE_API_DIR:-/run/kube-api} + namespace=customer-identity + client_secret_name=customer-dex-zitadel-client + # Fingerprints of what this script last applied, kept as annotations on the + # one Secret it may write, so a run with nothing new changes nothing. + fingerprint_annotation=platform.devantler.tech/smtp-credential-sha256 + admin_annotation=platform.devantler.tech/first-admin-password-sha256 + project_name="Customer sign-in" + app_name="Customer Dex" + dex_callback=https://login.devantler.tech/callback + smtp_host=email-smtp.eu-central-1.amazonaws.com:587 + smtp_sender=no-reply@devantler.tech + smtp_sender_name="Devantler Tech" + smtp_description="Business site sender (Amazon SES)" + placeholder=PLACEHOLDER + + # The business-site preparer's CreateApplication template (server/internal/ + # customeridentity/config.go): one confidential web client, code grant only, + # the exact Dex callback, Login V1. + create_oidc=$(jq -nc --arg cb "$dex_callback" '{ + redirectUris: [$cb], responseTypes: ["OIDC_RESPONSE_TYPE_CODE"], + grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE"], applicationType: "OIDC_APP_TYPE_WEB", + authMethodType: "OIDC_AUTH_METHOD_TYPE_BASIC", version: "OIDC_VERSION_1_0", developmentMode: false, + accessTokenRoleAssertion: false, idTokenRoleAssertion: false, idTokenUserinfoAssertion: false, + additionalOrigins: [], postLogoutRedirectUris: [], loginVersion: {loginV1: {}}}') + # The settings kept equal to that template on every run. ZITADEL cannot clear a + # list through an update (it reads an empty list as "unchanged"), so the two + # lists the client does not use, additional origins and post-logout redirect + # URIs, are set only at creation and not managed afterwards. + desired_oidc=$(jq -c 'del(.additionalOrigins, .postLogoutRedirectUris)' <<<"$create_oidc") + # ZITADEL omits fields that hold their default value, so compare normalised. + normalise_oidc='{ + redirectUris: (.redirectUris // []), responseTypes: (.responseTypes // ["OIDC_RESPONSE_TYPE_CODE"]), + grantTypes: (.grantTypes // ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE"]), + applicationType: (.applicationType // "OIDC_APP_TYPE_WEB"), + authMethodType: (.authMethodType // "OIDC_AUTH_METHOD_TYPE_BASIC"), + version: (.version // "OIDC_VERSION_1_0"), developmentMode: (.developmentMode // false), + accessTokenRoleAssertion: (.accessTokenRoleAssertion // false), + idTokenRoleAssertion: (.idTokenRoleAssertion // false), + idTokenUserinfoAssertion: (.idTokenUserinfoAssertion // false), + loginVersion: (.loginVersion // {loginV1: {}})}' + + log() { printf 'setup: %s\n' "$*"; } + fail() { printf 'setup: FAILED: %s\n' "$*" >&2; exit 1; } + + work=$(mktemp -d) + trap 'rm -rf "$work"' EXIT + read_secret() { [[ -s $1 ]] || fail "missing $2"; printf '%s' "$(<"$1")"; } + [[ -s $TOKEN_FILE ]] || fail "no signed token at $TOKEN_FILE" + printf 'Authorization: Bearer %s\n' "$(<"$TOKEN_FILE")" > "$work/auth" + smtp_user=$(read_secret "$smtp_dir/username" "SMTP username") + smtp_password=$(read_secret "$smtp_dir/password" "SMTP password") + admin_password=$(read_secret "$admin_password_file" "administrator one-time password") + + # The Kubernetes token is projected outside the default path, so kubectl gets + # a kubeconfig that reads it from there. + if [[ -z ${KUBECONFIG:-} ]]; then + cat > "$work/kubeconfig" </dev/null || echo 'unreadable error')" + fi + cat "$work/response" + } + connect() { api POST "/$1" "${2-}"; } + + # The client Secret is created empty by Flux and only ever patched here. + secret_json() { + kubectl -n "$namespace" get secret "$client_secret_name" -o json || + fail "cannot read Secret $client_secret_name" + } + patch_secret() { + printf '%s' "$1" > "$work/patch" + kubectl -n "$namespace" patch secret "$client_secret_name" --type merge --patch-file "$work/patch" >/dev/null || + fail "cannot update Secret $client_secret_name" + } + # client_authenticates ID SECRET: succeeds when ZITADEL accepts this client + # secret. A token request with a code that cannot exist fails either at client + # authentication (invalid_client: the secret is wrong, for example after a + # database restore) or after it, at the code (the secret is right). Any other + # answer stops the run. + client_authenticates() { + local status error + CLIENT_SECRET=$2 jq -rn --arg id "$1" '"Authorization: Basic " + ("\($id | @uri):\(env.CLIENT_SECRET | @uri)" | @base64)' \ + > "$work/client-auth" + status=$(curl -sS -m 20 -o "$work/token" -w '%{http_code}' -X POST -H "@$work/client-auth" \ + -H "Host: $ZITADEL_HOST" --data-urlencode grant_type=authorization_code \ + --data-urlencode code=customer-identity-setup-probe --data-urlencode "redirect_uri=$dex_callback" \ + "$ZITADEL_URL/oauth/v2/token") || fail "token endpoint: no answer from ZITADEL" + error=$(jq -r '.error // ""' "$work/token" 2>/dev/null || true) + case $error in + invalid_client) return 1 ;; + invalid_grant | invalid_request) return 0 ;; + *) fail "token endpoint: unexpected answer (HTTP $status${error:+, $error})" ;; + esac + } + store_client() { + local id=$1 attempt + [[ -n $id && -n $CLIENT_SECRET ]] || fail "ZITADEL returned an empty client" + patch_secret "$(jq -nc --arg id "$id" \ + '{data: {"client-id": ($id | @base64), "client-secret": (env.CLIENT_SECRET | @base64)}}')" + [[ $(secret_json | jq -r '.data["client-id"] // "" | @base64d') == "$id" ]] || + fail "Secret $client_secret_name did not keep the new client" + for attempt in 1 2 3 4 5; do + client_authenticates "$id" "$CLIENT_SECRET" && return 0 + sleep 2 + done + fail "ZITADEL does not accept the client secret it issued" + } + + for attempt in $(seq 1 20); do + curl -sf -m 5 -o /dev/null "$ZITADEL_URL/debug/ready" && break + [[ $attempt == 20 ]] && fail "ZITADEL is not ready" + sleep 6 + done + + # Stop before ZITADEL issues anything if there is nowhere to keep it. + secret_json >/dev/null + [[ $(kubectl -n "$namespace" auth can-i patch "secrets/$client_secret_name" 2>/dev/null) == yes ]] || + fail "may not update Secret $client_secret_name" + + changes=() + org_id=$(api GET /admin/v1/orgs/default | jq -er '.org.id') || fail "no default organization" + + # --- Project --------------------------------------------------------------- + projects=$(connect zitadel.project.v2.ProjectService/ListProjects '{"pagination":{"limit":1000}}') + [[ $(jq -r '.pagination.totalResult // "0"' <<<"$projects") -le $(jq '.projects // [] | length' <<<"$projects") ]] || + fail "project list is incomplete" + project_ids=$(jq -r --arg n "$project_name" --arg o "$org_id" \ + '.projects // [] | map(select(.name == $n and .organizationId == $o)) | .[].projectId' <<<"$projects") + case $(grep -c . <<<"$project_ids" || true) in + 0) + project_id=$(connect zitadel.project.v2.ProjectService/CreateProject \ + "$(jq -nc --arg o "$org_id" --arg n "$project_name" '{organizationId: $o, name: $n}')" | jq -er '.projectId') + changes+=("created project") + ;; + 1) project_id=$project_ids ;; + *) fail "more than one project named '$project_name'" ;; + esac + + # --- Application ------------------------------------------------------------- + apps=$(connect zitadel.application.v2.ApplicationService/ListApplications \ + "$(jq -nc --arg p "$project_id" '{pagination: {limit: 1000}, filters: [{projectIdFilter: {projectId: $p}}]}')") + [[ $(jq -r '.pagination.totalResult // "0"' <<<"$apps") -le $(jq '.applications // [] | length' <<<"$apps") ]] || + fail "application list is incomplete" + matches=$(jq -c --arg n "$app_name" '[.applications // [] | .[] | select(.name == $n)]' <<<"$apps") + case $(jq length <<<"$matches") in + 0) + connect zitadel.application.v2.ApplicationService/CreateApplication \ + "$(jq -nc --arg p "$project_id" --arg n "$app_name" --argjson c "$create_oidc" \ + '{projectId: $p, name: $n, oidcConfiguration: $c}')" > "$work/created" + app_id=$(jq -er '.applicationId' "$work/created") + client_id=$(jq -er '.oidcConfiguration.clientId' "$work/created") + CLIENT_SECRET=$(jq -er '.oidcConfiguration.clientSecret' "$work/created") store_client "$client_id" + changes+=("created client") + ;; + 1) + app=$(jq -c '.[0]' <<<"$matches") + app_id=$(jq -er '.applicationId' <<<"$app") + client_id=$(jq -er '.oidcConfiguration.clientId' <<<"$app") + [[ $(jq -r '.state' <<<"$app") == APPLICATION_STATE_ACTIVE ]] || + fail "client '$app_name' is not active; reactivate or remove it on purpose" + want=$(jq -cS "$normalise_oidc" <<<"$desired_oidc") + if [[ $(jq -cS ".oidcConfiguration | $normalise_oidc" <<<"$app") != "$want" ]]; then + connect zitadel.application.v2.ApplicationService/UpdateApplication \ + "$(jq -nc --arg p "$project_id" --arg a "$app_id" --argjson c "$desired_oidc" \ + '{projectId: $p, applicationId: $a, oidcConfiguration: $c}')" >/dev/null + [[ $(connect zitadel.application.v2.ApplicationService/ListApplications \ + "$(jq -nc --arg p "$project_id" '{pagination: {limit: 1000}, filters: [{projectIdFilter: {projectId: $p}}]}')" | + jq -cS --arg a "$app_id" ".applications[] | select(.applicationId == \$a) | .oidcConfiguration | $normalise_oidc") == "$want" ]] || + fail "client '$app_name' still differs from its declared settings after the update" + changes+=("restored client settings") + fi + stored=$(secret_json) + stored_id=$(jq -r '.data["client-id"] // "" | @base64d' <<<"$stored") + stored_secret=$(jq -r '.data["client-secret"] // "" | @base64d' <<<"$stored") + reissue= + if [[ $stored_id != "$client_id" || $stored_secret =~ ^($placeholder)?$ ]]; then + reissue="none stored" + elif ! client_authenticates "$client_id" "$stored_secret"; then + reissue="stored one refused" + fi + if [[ -n $reissue ]]; then + connect zitadel.application.v2.ApplicationService/GenerateClientSecret \ + "$(jq -nc --arg p "$project_id" --arg a "$app_id" '{projectId: $p, applicationId: $a}')" > "$work/generated" + CLIENT_SECRET=$(jq -er '.clientSecret' "$work/generated") store_client "$client_id" + changes+=("issued client secret ($reissue)") + fi + ;; + *) fail "more than one client named '$app_name'" ;; + esac + + # --- SMTP -------------------------------------------------------------------- + fingerprint=$(printf '%s\n%s' "$smtp_user" "$smtp_password" | sha256sum | cut -d' ' -f1) + providers=$(api POST /admin/v1/smtp/_search) + provider=$(jq -c --arg h "$smtp_host" --arg s "$smtp_sender" \ + '[.result // [] | .[] | select(.host == $h and .senderAddress == $s)]' <<<"$providers") + [[ $(jq length <<<"$provider") == 1 ]] || fail "expected exactly one SES provider for $smtp_sender" + provider=$(jq -c '.[0]' <<<"$provider") + smtp_id=$(jq -er '.id' <<<"$provider") + recorded=$(secret_json | jq -r --arg a "$fingerprint_annotation" '.metadata.annotations[$a] // ""') + if [[ $recorded != "$fingerprint" || $(jq -r '.user' <<<"$provider") != "$smtp_user" || + $(jq -r '.tls // false' <<<"$provider") != true || $(jq -r '.senderName // ""' <<<"$provider") != "$smtp_sender_name" ]]; then + api PUT "/admin/v1/smtp/$smtp_id" "$(jq -nc --arg s "$smtp_sender" --arg n "$smtp_sender_name" \ + --arg h "$smtp_host" --arg u "$smtp_user" --arg d "$smtp_description" \ + '{senderAddress: $s, senderName: $n, tls: true, host: $h, user: $u, description: $d}')" >/dev/null + api PUT "/admin/v1/smtp/$smtp_id/password" "$(SMTP_PASSWORD=$smtp_password jq -nc '{password: env.SMTP_PASSWORD}')" >/dev/null + patch_secret "$(jq -nc --arg a "$fingerprint_annotation" --arg f "$fingerprint" '{metadata: {annotations: {($a): $f}}}')" + changes+=("updated SMTP login") + fi + if [[ $(api POST /admin/v1/smtp/_search | jq -r --arg i "$smtp_id" '.result[] | select(.id == $i) | .state') != SMTP_CONFIG_ACTIVE ]]; then + api POST "/admin/v1/smtp/$smtp_id/_activate" >/dev/null + changes+=("activated SMTP provider") + fi + + # --- The administrator's one-time password ------------------------------------- + # ZITADEL set it once, when it created the instance, from zitadel-first-admin. + # Until the maintainer's first sign-in replaces it, keep it equal to that Secret + # (which a rebuild regenerates). Once he has chosen his own password, ZITADEL + # no longer requires a change and this never touches it again. + admins=$(connect zitadel.user.v2.UserService/ListUsers "$(jq -nc --arg e "$ADMIN_EMAIL" \ + '{queries: [{emailQuery: {emailAddress: $e, method: "TEXT_QUERY_METHOD_EQUALS"}}, {typeQuery: {type: "TYPE_HUMAN"}}]}')") + [[ $(jq '.result // [] | length' <<<"$admins") == 1 ]] || fail "expected exactly one administrator with $ADMIN_EMAIL" + admin_id=$(jq -er '.result[0].userId' <<<"$admins") + admin_state=own + if [[ $(jq -r '.result[0].human.passwordChangeRequired // false' <<<"$admins") == true ]]; then + admin_state=one-time + admin_fingerprint=$(printf '%s' "$admin_password" | sha256sum | cut -d' ' -f1) + if [[ $(secret_json | jq -r --arg a "$admin_annotation" '.metadata.annotations[$a] // ""') != "$admin_fingerprint" ]]; then + connect zitadel.user.v2.UserService/SetPassword "$(ADMIN_PASSWORD=$admin_password jq -nc --arg u "$admin_id" \ + '{userId: $u, newPassword: {password: env.ADMIN_PASSWORD, changeRequired: true}}')" >/dev/null + patch_secret "$(jq -nc --arg a "$admin_annotation" --arg f "$admin_fingerprint" '{metadata: {annotations: {($a): $f}}}')" + changes+=("set the administrator's one-time password") + fi + fi + + summary="project=$project_id client=$client_id application=$app_id smtp=$smtp_id administrator=$admin_state" + if ((${#changes[@]})); then + log "OK $summary changes: $(IFS=,; echo "${changes[*]}")" + else + log "OK $summary no changes" + fi + volumes: + - name: setup-key + secret: + secretName: zitadel-setup-key + defaultMode: 0440 + items: + - key: tls.key + path: tls.key + - name: setup-token + emptyDir: + medium: Memory + sizeLimit: 1Mi + # Holds ZITADEL's answers, which can carry the new client secret, + # so it never touches a disk. + - name: tmp + emptyDir: + medium: Memory + sizeLimit: 16Mi + - name: kube-api + projected: + defaultMode: 0440 + sources: + - serviceAccountToken: + path: token + expirationSeconds: 3600 + - configMap: + name: kube-root-ca.crt + items: + - key: ca.crt + path: ca.crt + - downwardAPI: + items: + - path: namespace + fieldRef: + fieldPath: metadata.namespace + - name: smtp + secret: + secretName: zitadel-smtp + defaultMode: 0440 + items: + - key: username + path: username + - key: password + path: password + - name: first-admin + secret: + secretName: zitadel-first-admin + defaultMode: 0440 + items: + - key: password + path: password diff --git a/k8s/providers/hetzner/apps/customer-identity/external-secret-db-backup.yaml b/k8s/providers/hetzner/apps/customer-identity/external-secret-db-backup.yaml new file mode 100644 index 0000000000..55ec1111f2 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/external-secret-db-backup.yaml @@ -0,0 +1,31 @@ +# R2 credentials for customer-identity-db's Barman Cloud backups +# (object-store.yaml). The shared backup key from OpenBao +# infrastructure/backup/r2, the same one Velero and umami-db use; CNPG reads +# s3Credentials only from the Cluster's own namespace. +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: customer-identity-db-backup-r2 + namespace: customer-identity +spec: + refreshInterval: 1h + secretStoreRef: + name: openbao + kind: ClusterSecretStore + target: + name: customer-identity-db-backup-r2 + creationPolicy: Owner + template: + data: + ACCESS_KEY_ID: "{{ .ACCESS_KEY_ID }}" + SECRET_ACCESS_KEY: "{{ .SECRET_ACCESS_KEY }}" + REGION: "auto" + data: + - secretKey: ACCESS_KEY_ID + remoteRef: + key: infrastructure/backup/r2 + property: access_key_id + - secretKey: SECRET_ACCESS_KEY + remoteRef: + key: infrastructure/backup/r2 + property: secret_access_key diff --git a/k8s/providers/hetzner/apps/customer-identity/external-secret-db-coroot-monitor.yaml b/k8s/providers/hetzner/apps/customer-identity/external-secret-db-coroot-monitor.yaml new file mode 100644 index 0000000000..2912543f01 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/external-secret-db-coroot-monitor.yaml @@ -0,0 +1,26 @@ +# Coroot's read-only monitoring login for customer-identity-db (pg_monitor only, +# no access to ZITADEL's data). Read only by the scrape annotations in cluster.yaml. +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: customer-identity-db-coroot-monitor + namespace: customer-identity +spec: + refreshInterval: "0" + target: + name: customer-identity-db-coroot-monitor + creationPolicy: Owner + template: + type: kubernetes.io/basic-auth + metadata: + labels: + cnpg.io/reload: "true" + data: + username: coroot_monitor + password: "{{ .password }}" + dataFrom: + - sourceRef: + generatorRef: + apiVersion: generators.external-secrets.io/v1alpha1 + kind: Password + name: customer-identity-db-coroot-monitor diff --git a/k8s/providers/hetzner/apps/customer-identity/external-secret-db-zitadel.yaml b/k8s/providers/hetzner/apps/customer-identity/external-secret-db-zitadel.yaml new file mode 100644 index 0000000000..bfdd640e41 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/external-secret-db-zitadel.yaml @@ -0,0 +1,30 @@ +# The `zitadel` database owner login. Generated once in-cluster: CNPG creates +# the role and the database with it (cluster.yaml), and ZITADEL reads the same +# password (helm-release.yaml). ZITADEL's state lives in this database and its +# off-cluster backup, so a regenerated password is only a rotation. +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: customer-identity-db-zitadel + namespace: customer-identity +spec: + refreshInterval: "0" + target: + name: customer-identity-db-zitadel + creationPolicy: Owner + template: + engineVersion: v2 + # CNPG reads a role's password from a basic-auth Secret. + type: kubernetes.io/basic-auth + metadata: + labels: + cnpg.io/reload: "true" + data: + username: zitadel + password: "{{ .password }}" + dataFrom: + - sourceRef: + generatorRef: + apiVersion: generators.external-secrets.io/v1alpha1 + kind: Password + name: customer-identity-db-zitadel diff --git a/k8s/providers/hetzner/apps/customer-identity/external-secret-first-admin.yaml b/k8s/providers/hetzner/apps/customer-identity/external-secret-first-admin.yaml new file mode 100644 index 0000000000..9a4a186c77 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/external-secret-first-admin.yaml @@ -0,0 +1,21 @@ +# One-time initial password for the instance's first administrator, the +# maintainer (helm-release.yaml FirstInstance). ZITADEL uses it only when it +# creates the instance and forces a change at the first sign-in, where MFA +# enrolment is also required. Read it once with an administrator kubeconfig: +# kubectl -n customer-identity get secret zitadel-first-admin -o jsonpath='{.data.password}' | base64 -d +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: zitadel-first-admin + namespace: customer-identity +spec: + refreshInterval: "0" + target: + name: zitadel-first-admin + creationPolicy: Owner + dataFrom: + - sourceRef: + generatorRef: + apiVersion: generators.external-secrets.io/v1alpha1 + kind: Password + name: zitadel-first-admin diff --git a/k8s/providers/hetzner/apps/customer-identity/external-secret-masterkey.yaml b/k8s/providers/hetzner/apps/customer-identity/external-secret-masterkey.yaml new file mode 100644 index 0000000000..c49370695b --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/external-secret-masterkey.yaml @@ -0,0 +1,22 @@ +# ZITADEL's master key: it encrypts the signing keys and secrets stored in the +# database, so the database is unusable without it. Generated once into OpenBao +# by infrastructure/vault-seed (create-only), which makes OpenBao its custody +# of record: a rebuilt cluster restores OpenBao first and gets the same key. +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: zitadel-masterkey + namespace: customer-identity +spec: + refreshInterval: 1h + secretStoreRef: + name: openbao + kind: ClusterSecretStore + target: + name: zitadel-masterkey + creationPolicy: Owner + data: + - secretKey: masterkey + remoteRef: + key: apps/customer-identity/zitadel + property: masterkey diff --git a/k8s/providers/hetzner/apps/customer-identity/external-secret-smtp.yaml b/k8s/providers/hetzner/apps/customer-identity/external-secret-smtp.yaml new file mode 100644 index 0000000000..27ae97d258 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/external-secret-smtp.yaml @@ -0,0 +1,26 @@ +# ZITADEL's mail login: the business site's send-only Amazon SES credential +# (platform#5400), read from OpenBao apps/business-site/smtp through the shared +# store, so no one types it into ZITADEL's console. ZITADEL copies it into its +# own SMTP provider when it creates the instance (helm-release.yaml). +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: zitadel-smtp + namespace: customer-identity +spec: + refreshInterval: 1h + secretStoreRef: + name: openbao + kind: ClusterSecretStore + target: + name: zitadel-smtp + creationPolicy: Owner + data: + - secretKey: username + remoteRef: + key: apps/business-site/smtp + property: username + - secretKey: password + remoteRef: + key: apps/business-site/smtp + property: password diff --git a/k8s/providers/hetzner/apps/customer-identity/helm-release.yaml b/k8s/providers/hetzner/apps/customer-identity/helm-release.yaml new file mode 100644 index 0000000000..103f037790 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/helm-release.yaml @@ -0,0 +1,282 @@ +# Self-hosted ZITADEL for invited business-site customers (business-site#302, +# ADR 0021). The official upstream image, unmodified and pinned by digest +# (business-site ADR 0031): upstream does not sign it, so admission treats it +# as a third-party image. +# +# Shape: +# - Login V1, embedded in the API server (ADR 0021). The chart's separate +# Login V2 service and its API credential stay off. +# - No superuser: CNPG creates the database and its owner (cluster.yaml), and +# `zitadel init zitadel` and `zitadel setup` run as that owner. +# - No machine administrator: the chart's default machine user and the +# kubectl containers and secret-writing Role that export its key are +# dropped. The only bootstrap identity is one human administrator (the +# maintainer), with a generated one-time password that must be changed at +# the first sign-in, where the login policy also forces MFA. +# - One system API user, `zitadel-setup`, declared in SystemAPIUsers below, +# for the setup CronJob (cron-job-setup.yaml). ZITADEL trusts the public key +# in cert-manager's certificate (certificate-setup-key.yaml); only that +# CronJob's signer mounts the private key. Its only role is IAM_OWNER, so it +# can manage this instance but not ZITADEL's system API (no SYSTEM_OWNER). +# - TLS ends at the platform Gateway, as for every platform app; the Gateway +# reaches ZITADEL over h2c inside the WireGuard-encrypted cluster network. +# +# The instance policies come from the business-site preparer +# (`go run ./cmd/customer-identity-config --prepare`, server/CUSTOMER-IDENTITY.md): +# no self-registration or public organisation creation, no external or +# passwordless login, MFA always, 14-character passwords, short-lived tokens. +# ZITADEL applies DefaultInstance and FirstInstance only when it creates the +# instance; later changes go through the console or its API. +# +# Mail is declared: when ZITADEL creates the instance it adds and activates +# one SMTP provider from DefaultInstance.SMTPConfiguration below, the business +# site's send-only Amazon SES sender (platform#5400): STARTTLS to +# email-smtp.eu-central-1.amazonaws.com:587 as no-reply@devantler.tech, with +# the credential from OpenBao apps/business-site/smtp (external-secret-smtp.yaml). +# ZITADEL then keeps the provider in its database: like every DefaultInstance +# setting it is not re-applied to an existing instance, so the setup CronJob +# applies a rotated SES credential through ZITADEL's admin API. +apiVersion: helm.toolkit.fluxcd.io/v2 +kind: HelmRelease +metadata: + name: zitadel + namespace: customer-identity + labels: + helm.toolkit.fluxcd.io/remediation: enabled +spec: + interval: 10m0s + timeout: 10m + chart: + spec: + chart: zitadel + version: 10.5.0 + sourceRef: + kind: HelmRepository + name: zitadel + values: + replicaCount: 2 + # ZITADEL reads its Secrets at startup. CNPG renews the database CA + # before it expires (every 90 days); restarting on that change keeps the + # verified database TLS working. + annotations: + reloader.stakater.com/auto: "true" + image: + repository: ghcr.io/zitadel/zitadel + tag: v4.20.3@sha256:1b30fea7ed88826dc0a98dacee4e2f2c8b05201fd26f5e4cb5386615b29f7638 + login: + enabled: false + # Only the `zitadel` phase of `zitadel init` (its schemas and tables), + # which runs as the database owner. The database, role and grant phases + # need a superuser and are CNPG's job here (cluster.yaml). + initJob: + command: zitadel + activeDeadlineSeconds: 600 + resources: + requests: + cpu: 100m + memory: 128Mi + limits: + cpu: "1" + memory: 256Mi + # Its post-delete hook only removes the machine-key Secrets this release + # never creates. + cleanupJob: + enabled: false + setupJob: + activeDeadlineSeconds: 600 + resources: + requests: + cpu: 100m + memory: 256Mi + limits: + cpu: "1" + memory: 512Mi + resources: + requests: + cpu: 50m + memory: 192Mi + limits: + cpu: "1" + memory: 512Mi + pdb: + enabled: true + maxUnavailable: 1 + podSecurityContext: + runAsNonRoot: true + runAsUser: 65532 + runAsGroup: 65532 + fsGroup: 65532 + seccompProfile: + type: RuntimeDefault + securityContext: + runAsNonRoot: true + runAsUser: 65532 + runAsGroup: 65532 + readOnlyRootFilesystem: true + privileged: false + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + seccompProfile: + type: RuntimeDefault + # Only the CA certificate: the CNPG CA Secret also holds the CA's private + # key, which the chart's dbSslCaCrtSecret option would mount as well. + extraVolumes: + - name: db-ca + secret: + secretName: customer-identity-db-ca + defaultMode: 0440 + items: + - key: ca.crt + path: ca.crt + # The setup user's certificate only, never its private key. Optional, + # so ZITADEL starts before cert-manager has issued it; ZITADEL reads it + # when the setup CronJob first signs in. + - name: setup-user + secret: + secretName: zitadel-setup-key + defaultMode: 0440 + optional: true + items: + - key: tls.crt + path: zitadel-setup.crt + extraVolumeMounts: + - name: db-ca + mountPath: /db-ca + readOnly: true + - name: setup-user + mountPath: /system-users + readOnly: true + # Rendered into the setup job and the server alike. + env: + - name: ZITADEL_DATABASE_POSTGRES_USER_PASSWORD + valueFrom: + secretKeyRef: + name: customer-identity-db-zitadel + key: password + - name: ZITADEL_DATABASE_POSTGRES_USER_SSL_ROOTCERT + value: /db-ca/ca.crt + - name: ZITADEL_FIRSTINSTANCE_ORG_HUMAN_PASSWORD + valueFrom: + secretKeyRef: + name: zitadel-first-admin + key: password + - name: ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_PLAINAUTH_USER + valueFrom: + secretKeyRef: + name: zitadel-smtp + key: username + - name: ZITADEL_DEFAULTINSTANCE_SMTPCONFIGURATION_SMTP_PLAINAUTH_PASSWORD + valueFrom: + secretKeyRef: + name: zitadel-smtp + key: password + zitadel: + masterkeySecretName: zitadel-masterkey + configmapConfig: + ExternalDomain: accounts.devantler.tech + ExternalPort: 443 + ExternalSecure: true + TLS: + Enabled: false + # The setup CronJob's identity (see above). Tokens must be signed by + # the certificate's key, name this user as issuer and subject, and be + # addressed to https://accounts.devantler.tech. + SystemAPIUsers: + zitadel-setup: + Path: /system-users/zitadel-setup.crt + Memberships: + - MemberType: System + Roles: + - IAM_OWNER + Database: + Postgres: + Host: customer-identity-db-rw.customer-identity.svc + Port: 5432 + Database: zitadel + MaxOpenConns: 10 + MaxIdleConns: 5 + User: + Username: zitadel + SSL: + Mode: verify-full + FirstInstance: + InstanceName: Devantler Tech + DefaultLanguage: en + Org: + Name: Devantler Tech + # Drops the chart's default machine administrator (see above). + Machine: null + Human: + UserName: ned + FirstName: Nikolai + LastName: Damm + # The maintainer's own published company address. Marked + # verified so the first sign-in does not wait for mail, which + # cannot be sent until the SMTP provider is added. Customers are + # never pre-verified (ADR 0021). + Email: + Address: ned@devantler.tech + Verified: true + PreferredLanguage: en + PasswordChangeRequired: true + DefaultInstance: + Features: + LoginV2: + Required: false + LockoutPolicy: + MaxOTPAttempts: 5 + MaxPasswordAttempts: 5 + ShouldShowLockoutFailure: false + LoginPolicy: + AllowDomainDiscovery: false + AllowExternalIDP: false + AllowRegister: false + AllowUsernamePassword: true + DefaultRedirectURI: https://portal.devantler.tech/portal + DisableLoginWithPhone: true + ExternalLoginCheckLifetime: 15m + ForceMFA: true + ForceMFALocalOnly: false + HidePasswordReset: false + IgnoreUnknownUsernames: true + MfaInitSkipLifetime: 0s + MultiFactorCheckLifetime: 15m + PasswordCheckLifetime: 15m + PasswordlessType: 0 + SecondFactorCheckLifetime: 15m + OIDCSettings: + AccessTokenLifetime: 5m + IdTokenLifetime: 5m + RefreshTokenExpiration: 8h + RefreshTokenIdleExpiration: 15m + PasswordComplexityPolicy: + MinLength: 14 + Restrictions: + DisallowPublicOrgRegistration: true + # TLS: true makes ZITADEL upgrade the port-587 session with STARTTLS + # (certificate checked for the host) and never send in plain text. + # The empty PlainAuth keys must stay: ZITADEL reads an environment + # variable only for a key its configuration already has, and the + # SMTP.User/Password keys in its sample defaults are not read by v4. + SMTPConfiguration: + Description: Business site sender (Amazon SES) + SMTP: + Host: email-smtp.eu-central-1.amazonaws.com:587 + PlainAuth: + User: "" + Password: "" + TLS: true + From: no-reply@devantler.tech + FromName: Devantler Tech + # The Devantler Tech look (near-black with #39ff14). The logo, font + # and watermark are set in the console (Branding). + LabelPolicy: + PrimaryColor: "#1f7a0f" + BackgroundColor: "#fafafa" + FontColor: "#111111" + PrimaryColorDark: "#39ff14" + BackgroundColorDark: "#0b0b0b" + FontColorDark: "#ededed" + HideLoginNameSuffix: true diff --git a/k8s/providers/hetzner/apps/customer-identity/helm-repository.yaml b/k8s/providers/hetzner/apps/customer-identity/helm-repository.yaml new file mode 100644 index 0000000000..050427d2ed --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/helm-repository.yaml @@ -0,0 +1,9 @@ +apiVersion: source.toolkit.fluxcd.io/v1 +kind: HelmRepository +metadata: + name: zitadel + namespace: customer-identity +spec: + # ZITADEL's official chart repository. + url: https://charts.zitadel.com + interval: 60m diff --git a/k8s/providers/hetzner/apps/customer-identity/http-route.yaml b/k8s/providers/hetzner/apps/customer-identity/http-route.yaml new file mode 100644 index 0000000000..b0b6775643 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/http-route.yaml @@ -0,0 +1,28 @@ +# accounts.devantler.tech: ZITADEL's sign-in, invitation, recovery and MFA pages +# and its console. A customer-facing name on the company domain, because +# customers see it whenever they sign in; once the instance exists it is +# ZITADEL's fixed external domain. The listener and certificate are +# infrastructure/patches/add-devantler-tech-listeners.yaml and certificate.yaml; +# external-dns creates the record from this route. +apiVersion: gateway.networking.k8s.io/v1 +kind: HTTPRoute +metadata: + name: zitadel + namespace: customer-identity +spec: + parentRefs: + - name: platform + namespace: kube-system + sectionName: https-accounts-devantler-tech + hostnames: + - accounts.devantler.tech + rules: + - filters: + - type: ResponseHeaderModifier + responseHeaderModifier: + set: + - name: Strict-Transport-Security + value: max-age=63072000; includeSubDomains; preload + backendRefs: + - name: zitadel + port: 8080 diff --git a/k8s/providers/hetzner/apps/customer-identity/issuer.yaml b/k8s/providers/hetzner/apps/customer-identity/issuer.yaml new file mode 100644 index 0000000000..96c1d142f1 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/issuer.yaml @@ -0,0 +1,10 @@ +# Signs the setup user's certificate (certificate-setup-key.yaml) with its +# own key. ZITADEL trusts that one certificate's public key directly, so no CA +# is involved. +apiVersion: cert-manager.io/v1 +kind: Issuer +metadata: + name: customer-identity-setup + namespace: customer-identity +spec: + selfSigned: {} diff --git a/k8s/providers/hetzner/apps/customer-identity/kustomization.yaml b/k8s/providers/hetzner/apps/customer-identity/kustomization.yaml new file mode 100644 index 0000000000..9e9a3dac58 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/kustomization.yaml @@ -0,0 +1,38 @@ +# Customer sign-in for the business-site portal (business-site#302): ZITADEL +# with its own CNPG database. Production-only, like the business-site tenant +# it serves. The portal stays switched off until #302's checks pass. +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +resources: + - namespace.yaml + - password-db-zitadel.yaml + - external-secret-db-zitadel.yaml + # Coroot's Postgres integration for customer-identity-db. + - password-db-coroot-monitor.yaml + - external-secret-db-coroot-monitor.yaml + - cilium-network-policy-coroot.yaml + - external-secret-db-backup.yaml + - object-store.yaml + - cluster.yaml + - scheduled-backup.yaml + - external-secret-masterkey.yaml + - external-secret-smtp.yaml + - password-first-admin.yaml + - external-secret-first-admin.yaml + - helm-repository.yaml + - helm-release.yaml + - cilium-network-policy.yaml + - certificate.yaml + - reference-grant.yaml + - http-route.yaml + # The declarative setup: ZITADEL's system API user's key, the CronJob that + # reconciles the customer Dex client and the SES login, and the one Secret + # it writes. + - issuer.yaml + - certificate-setup-key.yaml + - service-account.yaml + - role.yaml + - role-binding.yaml + - secret-dex-zitadel-client.yaml + - push-secret-dex-zitadel-client.yaml + - cron-job-setup.yaml diff --git a/k8s/providers/hetzner/apps/customer-identity/namespace.yaml b/k8s/providers/hetzner/apps/customer-identity/namespace.yaml new file mode 100644 index 0000000000..4f2ca5872a --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/namespace.yaml @@ -0,0 +1,11 @@ +# Customer sign-in for the business-site portal (devantler-tech/business-site#302, +# ADR 0021): self-hosted ZITADEL now, and the portal's own customer Dex next. +# Kept apart from the platform's infrastructure identity (the `dex` namespace): +# separate issuers, clients, secrets, administration and stores. +apiVersion: v1 +kind: Namespace +metadata: + name: customer-identity + labels: + pod-security.kubernetes.io/enforce: restricted + pod-security.kubernetes.io/enforce-version: latest diff --git a/k8s/providers/hetzner/apps/customer-identity/object-store.yaml b/k8s/providers/hetzner/apps/customer-identity/object-store.yaml new file mode 100644 index 0000000000..6b201bdbe5 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/object-store.yaml @@ -0,0 +1,27 @@ +# Off-cluster backups for customer-identity-db: continuous WAL archiving plus the +# daily base backup in scheduled-backup.yaml, so customer accounts can be +# restored to any point in the last 30 days. Same R2 bucket and layout as umami-db. +apiVersion: barmancloud.cnpg.io/v1 +kind: ObjectStore +metadata: + name: customer-identity-db + namespace: customer-identity +spec: + retentionPolicy: 30d + configuration: + destinationPath: s3://${r2_bucket}/cnpg/customer-identity-db + endpointURL: ${r2_endpoint} + s3Credentials: + accessKeyId: + name: customer-identity-db-backup-r2 + key: ACCESS_KEY_ID + secretAccessKey: + name: customer-identity-db-backup-r2 + key: SECRET_ACCESS_KEY + region: + name: customer-identity-db-backup-r2 + key: REGION + wal: + compression: gzip + data: + compression: gzip diff --git a/k8s/providers/hetzner/apps/customer-identity/password-db-coroot-monitor.yaml b/k8s/providers/hetzner/apps/customer-identity/password-db-coroot-monitor.yaml new file mode 100644 index 0000000000..b8efe0ceca --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/password-db-coroot-monitor.yaml @@ -0,0 +1,12 @@ +--- +apiVersion: generators.external-secrets.io/v1alpha1 +kind: Password +metadata: + name: customer-identity-db-coroot-monitor + namespace: customer-identity +spec: + length: 32 + digits: 8 + symbols: 0 + noUpper: false + allowRepeat: true diff --git a/k8s/providers/hetzner/apps/customer-identity/password-db-zitadel.yaml b/k8s/providers/hetzner/apps/customer-identity/password-db-zitadel.yaml new file mode 100644 index 0000000000..0ffc0c244a --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/password-db-zitadel.yaml @@ -0,0 +1,12 @@ +--- +apiVersion: generators.external-secrets.io/v1alpha1 +kind: Password +metadata: + name: customer-identity-db-zitadel + namespace: customer-identity +spec: + length: 32 + digits: 8 + symbols: 0 + noUpper: false + allowRepeat: true diff --git a/k8s/providers/hetzner/apps/customer-identity/password-first-admin.yaml b/k8s/providers/hetzner/apps/customer-identity/password-first-admin.yaml new file mode 100644 index 0000000000..c9704884c1 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/password-first-admin.yaml @@ -0,0 +1,15 @@ +--- +# Meets ZITADEL's password policy (helm-release.yaml: at least 14 characters +# with upper and lower case, a digit and a symbol). +apiVersion: generators.external-secrets.io/v1alpha1 +kind: Password +metadata: + name: zitadel-first-admin + namespace: customer-identity +spec: + length: 32 + digits: 6 + symbols: 4 + symbolCharacters: "-_.!" + noUpper: false + allowRepeat: true diff --git a/k8s/providers/hetzner/apps/customer-identity/push-secret-dex-zitadel-client.yaml b/k8s/providers/hetzner/apps/customer-identity/push-secret-dex-zitadel-client.yaml new file mode 100644 index 0000000000..5ed3b5872c --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/push-secret-dex-zitadel-client.yaml @@ -0,0 +1,31 @@ +# Keeps OpenBao apps/customer-identity/dex-zitadel-client equal to the client +# the setup CronJob wrote (secret-dex-zitadel-client.yaml), as `client-id` and +# `client-secret`. Platform-applied, so it may use the shared store, which +# holds an exact-key grant for this one record (vault-config). Nothing reads +# the copy back: after a rebuild the CronJob issues a new secret. +apiVersion: external-secrets.io/v1alpha1 +kind: PushSecret +metadata: + name: customer-dex-zitadel-client + namespace: customer-identity +spec: + updatePolicy: Replace + deletionPolicy: None + refreshInterval: 10m + secretStoreRefs: + - name: openbao + kind: ClusterSecretStore + selector: + secret: + name: customer-dex-zitadel-client + data: + - match: + secretKey: client-id + remoteRef: + remoteKey: apps/customer-identity/dex-zitadel-client + property: client-id + - match: + secretKey: client-secret + remoteRef: + remoteKey: apps/customer-identity/dex-zitadel-client + property: client-secret diff --git a/k8s/providers/hetzner/apps/customer-identity/reference-grant.yaml b/k8s/providers/hetzner/apps/customer-identity/reference-grant.yaml new file mode 100644 index 0000000000..738a25f3da --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/reference-grant.yaml @@ -0,0 +1,16 @@ +# Lets the shared platform Gateway (kube-system) terminate TLS for the customer +# identity listeners with certificate.yaml's Secret, and nothing else. +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: ReferenceGrant +metadata: + name: platform-gateway-customer-identity-tls + namespace: customer-identity +spec: + from: + - group: gateway.networking.k8s.io + kind: Gateway + namespace: kube-system + to: + - group: "" + kind: Secret + name: customer-identity-tls diff --git a/k8s/providers/hetzner/apps/customer-identity/role-binding.yaml b/k8s/providers/hetzner/apps/customer-identity/role-binding.yaml new file mode 100644 index 0000000000..7128e77322 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/role-binding.yaml @@ -0,0 +1,13 @@ +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: customer-identity-setup + namespace: customer-identity +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: customer-identity-setup +subjects: + - kind: ServiceAccount + name: customer-identity-setup + namespace: customer-identity diff --git a/k8s/providers/hetzner/apps/customer-identity/role.yaml b/k8s/providers/hetzner/apps/customer-identity/role.yaml new file mode 100644 index 0000000000..ce42e8a708 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/role.yaml @@ -0,0 +1,19 @@ +# The setup CronJob may read and update one Secret: the customer Dex's ZITADEL +# client (secret-dex-zitadel-client.yaml). It cannot create, list or delete +# Secrets, and it has no access to its own signing key, which reaches it only +# as a mounted volume. +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: customer-identity-setup + namespace: customer-identity +rules: + - apiGroups: + - "" + resources: + - secrets + resourceNames: + - customer-dex-zitadel-client + verbs: + - get + - patch diff --git a/k8s/providers/hetzner/apps/customer-identity/scheduled-backup.yaml b/k8s/providers/hetzner/apps/customer-identity/scheduled-backup.yaml new file mode 100644 index 0000000000..f6f06e5c07 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/scheduled-backup.yaml @@ -0,0 +1,17 @@ +# Daily base backup of customer-identity-db to R2 (object-store.yaml). The first +# runs as soon as the cluster exists. +apiVersion: postgresql.cnpg.io/v1 +kind: ScheduledBackup +metadata: + name: customer-identity-db-daily + namespace: customer-identity +spec: + # CNPG cron is 6-field (seconds first): 03:30 UTC daily, after umami-db. + schedule: "0 30 3 * * *" + immediate: true + backupOwnerReference: self + method: plugin + pluginConfiguration: + name: barman-cloud.cloudnative-pg.io + cluster: + name: customer-identity-db diff --git a/k8s/providers/hetzner/apps/customer-identity/secret-dex-zitadel-client.yaml b/k8s/providers/hetzner/apps/customer-identity/secret-dex-zitadel-client.yaml new file mode 100644 index 0000000000..2434f667ff --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/secret-dex-zitadel-client.yaml @@ -0,0 +1,23 @@ +# The customer Dex's confidential client in ZITADEL (connector +# `customer-zitadel`, business-site#302). The setup CronJob +# (cron-job-setup.yaml) creates the client in ZITADEL and writes its ID and +# secret here, for the portal's customer Dex to read. A PushSecret keeps a copy +# in OpenBao (push-secret-dex-zitadel-client.yaml). +# +# Flux only creates this placeholder (ssa: IfNotPresent) and never overwrites +# the values the CronJob writes. While it holds the placeholder, nothing can +# sign in through the client. +# +# Not a real secret (literally "PLACEHOLDER"), so it is committed in plaintext, +# like the Velero placeholder. +apiVersion: v1 +kind: Secret +metadata: + name: customer-dex-zitadel-client + namespace: customer-identity + annotations: + kustomize.toolkit.fluxcd.io/ssa: IfNotPresent +type: Opaque +stringData: + client-id: PLACEHOLDER + client-secret: PLACEHOLDER # checkov:skip=CKV_SECRET_6:literal placeholder; the setup CronJob writes the real value diff --git a/k8s/providers/hetzner/apps/customer-identity/service-account.yaml b/k8s/providers/hetzner/apps/customer-identity/service-account.yaml new file mode 100644 index 0000000000..cd54d0e9c4 --- /dev/null +++ b/k8s/providers/hetzner/apps/customer-identity/service-account.yaml @@ -0,0 +1,8 @@ +# The setup CronJob's identity in Kubernetes (cron-job-setup.yaml). No token is +# mounted by default; the CronJob projects one into its setup container only. +apiVersion: v1 +kind: ServiceAccount +metadata: + name: customer-identity-setup + namespace: customer-identity +automountServiceAccountToken: false diff --git a/k8s/providers/hetzner/apps/kustomization.yaml b/k8s/providers/hetzner/apps/kustomization.yaml index b3f71d2a0c..4ad405de11 100644 --- a/k8s/providers/hetzner/apps/kustomization.yaml +++ b/k8s/providers/hetzner/apps/kustomization.yaml @@ -10,6 +10,8 @@ resources: - khoist-documentation/ # Fetch/verify the exact reviewed publication only; no tenant application. - business-site/ + # Customer sign-in for the business-site portal (ZITADEL, business-site#302). + - customer-identity/ # Prod-only tenant: the UniFi network managed declaratively through Crossplane # Managed Resources (providers/hetzner/apps/unifi/). Lives in the hetzner apps # overlay because its provider and the controller API it reaches are diff --git a/k8s/providers/hetzner/infrastructure/flux-notifications/alert.yaml b/k8s/providers/hetzner/infrastructure/flux-notifications/alert.yaml index a16fb412e3..59f54422a2 100644 --- a/k8s/providers/hetzner/infrastructure/flux-notifications/alert.yaml +++ b/k8s/providers/hetzner/infrastructure/flux-notifications/alert.yaml @@ -118,6 +118,9 @@ spec: - kind: HelmRelease name: "*" namespace: crossplane-system + - kind: HelmRelease + name: "*" + namespace: customer-identity - kind: HelmRelease name: "*" namespace: data-product-controller diff --git a/k8s/providers/hetzner/infrastructure/patches/add-devantler-tech-listeners.yaml b/k8s/providers/hetzner/infrastructure/patches/add-devantler-tech-listeners.yaml index 0982301794..e3f470cf01 100644 --- a/k8s/providers/hetzner/infrastructure/patches/add-devantler-tech-listeners.yaml +++ b/k8s/providers/hetzner/infrastructure/patches/add-devantler-tech-listeners.yaml @@ -81,3 +81,26 @@ kinds: - group: gateway.networking.k8s.io kind: HTTPRoute +# ZITADEL's customer sign-in name (apps/customer-identity/http-route.yaml), with +# the customer-identity namespace's own certificate +# (apps/customer-identity/certificate.yaml, read through its reference-grant.yaml). +- op: add + path: /spec/listeners/- + value: + name: https-accounts-devantler-tech + hostname: accounts.devantler.tech + port: 443 + protocol: HTTPS + tls: + mode: Terminate + certificateRefs: + - kind: Secret + group: "" + namespace: customer-identity + name: customer-identity-tls + allowedRoutes: + namespaces: + from: All + kinds: + - group: gateway.networking.k8s.io + kind: HTTPRoute diff --git a/scripts/tests/test-trivyignore-first-party-rbac-boundary.sh b/scripts/tests/test-trivyignore-first-party-rbac-boundary.sh index 347cf7aa76..d7a44a2b71 100755 --- a/scripts/tests/test-trivyignore-first-party-rbac-boundary.sh +++ b/scripts/tests/test-trivyignore-first-party-rbac-boundary.sh @@ -102,6 +102,7 @@ readonly reviewed_pairs=( "KSV-0113:$vault_config_role" # Premises asserted in section 6 below. "KSV-0113:k8s/bases/infrastructure/substrate/role.yaml" + "KSV-0113:k8s/providers/hetzner/apps/customer-identity/role.yaml" "KSV-0048:k8s/providers/hetzner/infrastructure/controllers/longhorn/role-kyverno-mutate-csi-sidecars.yaml" "KSV-0048:k8s/providers/hetzner/infrastructure/controllers/longhorn/role-kyverno-mutate-longhorn-csi-plugin.yaml" "KSV-0048:k8s/providers/hetzner/infrastructure/controllers/longhorn/role-kyverno-mutate-longhorn-engine-image.yaml" @@ -384,6 +385,8 @@ assert_single_rule_role KSV-0049 'k8s/bases/infrastructure/controllers/prune-pro '/configmaps' 'prune-protected-orphan-alert-state' 'get patch' assert_single_rule_role KSV-0113 'k8s/bases/infrastructure/substrate/role.yaml' ate-system '/secrets' \ 'substrate-actor-id-ca,substrate-actor-id-jwt,substrate-egress-mitm-ca,substrate-service-dns-ca,substrate-pod-identity-ca,substrate-postgres-ca' 'get' +assert_single_rule_role KSV-0113 'k8s/providers/hetzner/apps/customer-identity/role.yaml' customer-identity '/secrets' \ + 'customer-dex-zitadel-client' 'get patch' readonly war_role='k8s/providers/hetzner/apps/world-at-ruin/role-operator.yaml' assert_single_rule_role KSV-0053 "$war_role" world-at-ruin '/pods/portforward,pods/exec' '' 'create' diff --git a/scripts/tests/test-trivyignore-vendored-operator-boundary.sh b/scripts/tests/test-trivyignore-vendored-operator-boundary.sh index 429ebdbbb2..f027a28ef1 100755 --- a/scripts/tests/test-trivyignore-vendored-operator-boundary.sh +++ b/scripts/tests/test-trivyignore-vendored-operator-boundary.sh @@ -62,6 +62,7 @@ readonly first_party_pairs=( 'KSV-0048:k8s/bases/infrastructure/resource-graph-definitions/webapp/cluster-role.yaml' 'KSV-0113:k8s/bases/infrastructure/vault-config/role.yaml' 'KSV-0113:k8s/bases/infrastructure/substrate/role.yaml' + 'KSV-0113:k8s/providers/hetzner/apps/customer-identity/role.yaml' 'KSV-0048:k8s/providers/hetzner/infrastructure/controllers/longhorn/role-kyverno-mutate-csi-sidecars.yaml' 'KSV-0048:k8s/providers/hetzner/infrastructure/controllers/longhorn/role-kyverno-mutate-longhorn-csi-plugin.yaml' 'KSV-0048:k8s/providers/hetzner/infrastructure/controllers/longhorn/role-kyverno-mutate-longhorn-engine-image.yaml' diff --git a/scripts/validate-eks-ci-role-policy/approved-surface.txt b/scripts/validate-eks-ci-role-policy/approved-surface.txt index 81fe16a525..6abc29f6dc 100644 --- a/scripts/validate-eks-ci-role-policy/approved-surface.txt +++ b/scripts/validate-eks-ci-role-policy/approved-surface.txt @@ -60,6 +60,8 @@ helm.toolkit.fluxcd.io/v2|HelmRelease|cnpg-system|plugin-barman-cloud 74682cb45a helm.toolkit.fluxcd.io/v2|HelmRelease|crossplane-system|crossplane fb717f7c9adb21b0f772e6a84554900e568a2675c91d375d1bb6ee76c2a0b1f3 +helm.toolkit.fluxcd.io/v2|HelmRelease|customer-identity|zitadel f6fb7a8b2d084beb33aeb0a08b1b82eeeac495cbbe498b65154a57c8c353a905 + helm.toolkit.fluxcd.io/v2|HelmRelease|dex|dex e2cbe6e6b5ace19ebded8d0e75f508277dd450c079ed51062d13649e148904b9 helm.toolkit.fluxcd.io/v2|HelmRelease|external-dns|external-dns 107eaf8ad3d74abff5c6b1c6d7ca1b49ac75829fe425672f81cdc1e799457511 @@ -308,6 +310,8 @@ rbac.authorization.k8s.io/v1|RoleBinding|ate-system|substrate-ca-reader b8b1991b rbac.authorization.k8s.io/v1|RoleBinding|aws|aws-managed-resources 876e05322100f78b490b870ba8733b26daa0f93b65b8fdd77d559e9de94588c1 +rbac.authorization.k8s.io/v1|RoleBinding|customer-identity|customer-identity-setup fab367a0961fa37ea2870ac2bd77f7840cedeadc9f589ce7adc72ab758c71377 + rbac.authorization.k8s.io/v1|RoleBinding|data-product-controller|data-product-readiness-observer 00bcf69a4402d05bf5af23e273f9d3c5d7e7ebeed8c1778017a9f702d261360e rbac.authorization.k8s.io/v1|RoleBinding|data-product-controller|khoist-documentation 2690828917afd76647b314b9009a0401578780b33809e8d3a1f04a1b16244565 @@ -364,6 +368,8 @@ rbac.authorization.k8s.io/v1|Role|ate-system|substrate-ca-reader 153cc02bc210a1f rbac.authorization.k8s.io/v1|Role|aws|aws-managed-resources 9207b057fb6d4f1492b13663277658e3bd31849a9aa11dd2185ceb0a0afcbfab +rbac.authorization.k8s.io/v1|Role|customer-identity|customer-identity-setup 7f700c5af8b692b4e51b9e5150cf90b7d7e70f0f4b7d215e5b169a147fadfdcd + rbac.authorization.k8s.io/v1|Role|data-product-controller|data-product-readiness-observer 10df039485be13b0344199a50eacc4fbd942f2bb9b831a84b11c024918a3422a rbac.authorization.k8s.io/v1|Role|data-product-controller|khoist-documentation 35e63ab8f23f3de55976023d5c6144ac5a3cd7ed5c0947f1c451d74cf4167bef @@ -418,6 +424,8 @@ source.toolkit.fluxcd.io/v1|HelmRepository|cnpg-system|cloudnative-pg 2ca8952aaa source.toolkit.fluxcd.io/v1|HelmRepository|crossplane-system|crossplane f47ce87b7611344b0d38ff450e6f789a60711bd657a2ff063253c5fcbd30e754 +source.toolkit.fluxcd.io/v1|HelmRepository|customer-identity|zitadel f156b723dae5eac81d6b84453a416739de755378d12a665970576b0ce199b219 + source.toolkit.fluxcd.io/v1|HelmRepository|dex|dex d251b34144fb4d376b47526e24b3520a1cc696159aa72f1a8c06fa67aadbda7f source.toolkit.fluxcd.io/v1|HelmRepository|external-dns|external-dns 81022de05d792a296ac5ce122fd92258c7cd7ac85ce62aa703aa12fdb0ac1326