From ab775f75897103ed26c0c09876d0736568bfc770 Mon Sep 17 00:00:00 2001 From: Matt Robinson Date: Thu, 18 Jun 2026 21:49:47 +0100 Subject: [PATCH] fix(helm): default web.enabled=true; refocus getting-started on Helm deploy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The web frontend is the BFF — the single ingress entry point that proxies all browser, CLI, and listener traffic to the API. Defaulting web.enabled to false (while api and listener defaulted true) meant any install with ingress.enabled=true failed to template ("Ingress is enabled but web.enabled is false"), including the documented basic install. Enable it by default so a stock install renders a working stack. Docs: - Rewrite getting-started.md around deploying the Helm chart onto a Kubernetes cluster (or a single-node k3s VM), not the Tilt local-dev stack: prerequisites, k3s bootstrap, helm install, ingress/DNS access, API token, first workspace + plan/apply. Verified the install command renders against the chart. - Move the Tilt/from-source contributor workflow into a new local-development.md, linked from getting-started and index. - Update index.md Quick Start to the Helm one-liner and add the Local Development nav entry (index + mkdocs). - Correct the web.enabled default row in deployment.md. --- docs/deployment.md | 2 +- docs/getting-started.md | 159 ++++++++++++++++++-------------------- docs/index.md | 20 +++-- docs/local-development.md | 77 ++++++++++++++++++ helm/terrapod/values.yaml | 5 +- mkdocs.yml | 1 + 6 files changed, 174 insertions(+), 90 deletions(-) create mode 100644 docs/local-development.md diff --git a/docs/deployment.md b/docs/deployment.md index 9f89c0fe..15c69f36 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -218,7 +218,7 @@ The chart ships with a `values.schema.json` that validates all values at `helm i | Value | Default | Description | |---|---|---| -| `web.enabled` | `false` | Enable web UI deployment | +| `web.enabled` | `true` | Enable web UI deployment (the BFF ingress entry point) | | `web.replicas` | `2` | Number of web replicas | | `web.image.repository` | `ghcr.io/mattrobinsonsre/terrapod-web` | Web Docker image | | `web.resources.requests.cpu` | `100m` | CPU request | diff --git a/docs/getting-started.md b/docs/getting-started.md index b7dc8946..06438da2 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,6 +1,8 @@ # Getting Started -This guide walks through setting up Terrapod locally, creating your first workspace, and running your first plan and apply against it. +This guide deploys Terrapod onto a Kubernetes cluster with the Helm chart, then creates your first workspace and runs your first plan and apply against it. + +> Terrapod runs **only on Kubernetes** — the runner uses the Jobs API to schedule plan/apply executions. You don't need a managed cloud cluster; a single-node [k3s](https://k3s.io/) VM works. If you instead want to run Terrapod **from source** for development, see [Local Development](local-development.md). > **OpenTofu is the recommended execution backend.** Terrapod supports both [OpenTofu](https://opentofu.org/) (`tofu`) and Terraform (`terraform`) as execution backends. OpenTofu is recommended because it is open-source under the MPL-2.0 license. All CLI examples in this guide use `tofu`, but `terraform` commands work identically. @@ -8,89 +10,86 @@ This guide walks through setting up Terrapod locally, creating your first worksp ## Prerequisites -### Required Software - -| Tool | Purpose | Install | -|---|---|---| -| Docker | Container runtime | [docker.com](https://www.docker.com/) | -| Kubernetes cluster | Local K8s (any of the below) | See below | -| Tilt | Local K8s dev environment | `brew install tilt` | -| mkcert | Local TLS certificates | `brew install mkcert` | -| tofu (recommended) or terraform | Infrastructure CLI | [opentofu.org](https://opentofu.org/) (recommended) or [terraform.io](https://www.terraform.io/) | - -### Supported Local Kubernetes Clusters - -Any of these will work: -- [Rancher Desktop](https://rancherdesktop.io/) -- [Docker Desktop](https://www.docker.com/products/docker-desktop/) (with Kubernetes enabled) -- [minikube](https://minikube.sigs.k8s.io/) -- [kind](https://kind.sigs.k8s.io/) -- [OrbStack](https://orbstack.dev/) -- [colima](https://github.com/abiosoft/colima) - ---- +| Need | Notes | +|---|---| +| A Kubernetes cluster (1.27+) | Any conformant cluster. No cluster? Spin up [k3s](https://k3s.io/) on a single VM (below). | +| Helm 3.x | `brew install helm` | +| PostgreSQL 14+ and Redis 7+ | **External** — the chart does not bundle them. Provide a connection URL for each (a managed service, or run them on the cluster/VM). See [Deployment → Database Setup](deployment.md#database-setup). | +| Storage | Defaults to a PVC-backed filesystem (no extra setup). For S3/Azure/GCS see [Deployment → Storage Backend Setup](deployment.md#storage-backend-setup). | +| `tofu` (recommended) or `terraform` | Infrastructure CLI — [opentofu.org](https://opentofu.org/) or [terraform.io](https://www.terraform.io/). | -## Local Development Setup +### Don't have a cluster? Use k3s -### Step 1: Install mkcert and create local CA +[k3s](https://k3s.io/) is a fully conformant single-binary Kubernetes that runs on one VM (~512 MB RAM) and ships with an ingress controller (Traefik) and local-path storage: ```zsh -brew install mkcert -mkcert -install +curl -sfL https://get.k3s.io | sh - # single-node cluster +export KUBECONFIG=/etc/rancher/k3s/k3s.yaml +curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash ``` -This installs a local Certificate Authority into your system trust store so that `https://terrapod.local` is trusted by your browser and the terraform CLI. +Run PostgreSQL and Redis as managed services or on a separate VM so your data survives cluster recreation. + +--- -### Step 2: Add hosts entry +## Deploy Terrapod -```zsh -sudo sh -c 'echo "127.0.0.1 terrapod.local" >> /etc/hosts' -``` +Terrapod is published to GitHub Container Registry as an OCI Helm chart at `oci://ghcr.io/mattrobinsonsre/terrapod`. -### Step 3: Start Terrapod +Pick a hostname and install: ```zsh -make dev +export TERRAPOD_HOST=terrapod.example.com + +helm install terrapod oci://ghcr.io/mattrobinsonsre/terrapod \ + --namespace terrapod --create-namespace \ + --set ingress.enabled=true \ + --set ingress.hostname="$TERRAPOD_HOST" \ + --set ingress.className=traefik \ + --set postgresql.url="postgresql+asyncpg://terrapod:PASSWORD@PGHOST:5432/terrapod" \ + --set redis.url="redis://REDISHOST:6379" \ + --set bootstrap.adminEmail="admin@example.com" \ + --set bootstrap.adminPassword="change-me-now" ``` -This runs `tilt up --port 10352` which: +What the defaults give you: filesystem storage on a PVC, local password auth enabled, a pre-install migrations job, and a post-install bootstrap job that creates the admin user from `bootstrap.adminEmail` / `bootstrap.adminPassword`. Set `ingress.className` to your cluster's ingress controller (`traefik` on k3s, often `nginx` on managed clusters). Pin a version with `--version X.Y.Z`. -1. Creates the `terrapod` namespace -2. Generates TLS certificates via mkcert -3. Deploys PostgreSQL and Redis in-cluster -4. Builds the API and web Docker images -5. Runs database migrations (Alembic) -6. Bootstraps the initial admin user -7. Deploys the API server, web UI, and runner listener +> For production, put the admin password in a Kubernetes Secret (`bootstrap.existingSecret`) rather than on the command line, terminate TLS at the ingress (e.g. cert-manager), and set `api.config.external_url` so outbound links (VCS status, notifications) are correct. The full value reference — storage backends, external DB, SSO, scaling, TLS, runners — is in [Deployment](deployment.md), and [Production Checklist](production-checklist.md) covers hardening. -Open the Tilt UI at http://localhost:10352 to monitor the deployment. +Watch it come up: -### Step 4: Access Terrapod +```zsh +kubectl -n terrapod get pods -w +``` -Open https://terrapod.local in your browser. +### Access -![Login](images/login.png) +Point your hostname's DNS at the ingress controller's external IP (on k3s, the VM's IP — Traefik listens on ports 80/443). For a quick local test you can add a hosts entry instead: -The bootstrap job creates an initial admin user. The default credentials are `admin` / `admin` (configured in `values-local.yaml`). Check the Tilt logs for the `terrapod-bootstrap-1` resource to confirm. +```zsh +sudo sh -c 'echo " terrapod.example.com" >> /etc/hosts' +``` + +Then open `https://terrapod.example.com` and log in with the bootstrap admin email + password. -### Step 5: Get an API Token +> TLS: the ingress expects a certificate by default (`ingress.tls=true`). For a quick HTTP-only evaluation, add `--set ingress.tls=false`; for real TLS see [Deployment → Ingress](deployment.md#ingress). +> +> No DNS available at all? Port-forward the web service for a quick look: `kubectl -n terrapod port-forward svc/terrapod-web 8080:3000`, then open `http://localhost:8080`. -After logging in to the web UI, navigate to **Settings > API Tokens** and create a new token. Copy the token value -- it is only shown once. +### Get an API token -![API Tokens](images/api-tokens.png) +In the web UI, go to **Settings → API Tokens**, create a token, and copy the value (shown once): ```zsh export TERRAPOD_TOKEN="" ``` -Alternatively, use `terraform login`: +Or let the CLI mint one for you via the browser login flow: ```zsh -terraform login terrapod.local +tofu login terrapod.example.com ``` -This opens a browser window, authenticates via the configured identity provider, and stores the token automatically. - --- ## Creating Your First Workspace @@ -98,7 +97,7 @@ This opens a browser window, authenticates via the configured identity provider, ### Via the API ```zsh -curl -s -X POST https://terrapod.local/api/v2/organizations/default/workspaces \ +curl -s -X POST https://terrapod.example.com/api/v2/organizations/default/workspaces \ -H "Authorization: Bearer $TERRAPOD_TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d '{ @@ -134,7 +133,7 @@ Create a configuration that uses Terrapod as its backend: # main.tf terraform { cloud { - hostname = "terrapod.local" + hostname = "terrapod.example.com" organization = "default" workspaces { @@ -165,12 +164,14 @@ If you'd rather select a workspace at run time (typical when a single repo backs ```hcl workspaces { - tags = ["network"] # any workspace whose labels include "network" - # or, for an exact match: - tags = { repo = "aws-infra" } + tags = ["network"] # any workspace with the label key "network" + # or, for an exact key:value match: + tags = ["repo:aws-infra"] # workspaces whose `repo` label is `aws-infra` } ``` +> OpenTofu only accepts the **set-of-strings** `tags` form, so use the `key:value` string above for an exact match. The Terraform 1.10+ **map** form (`tags = { repo = "aws-infra" }`) is rejected by `tofu` at config validation with `set of string required`. + Then pick a specific workspace per invocation: ```zsh @@ -182,7 +183,7 @@ tofu workspace select network-staging If you have not already run `tofu login`: ```zsh -tofu login terrapod.local +tofu login terrapod.example.com ``` Then initialize: @@ -239,7 +240,7 @@ Agent workspaces support two workflows depending on whether VCS is connected: 1. Set the workspace execution mode to `agent`: ```zsh -curl -X PATCH https://terrapod.local/api/v2/workspaces/ws-{id} \ +curl -X PATCH https://terrapod.example.com/api/v2/workspaces/ws-{id} \ -H "Authorization: Bearer $TERRAPOD_TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d '{ @@ -256,7 +257,7 @@ curl -X PATCH https://terrapod.local/api/v2/workspaces/ws-{id} \ ```zsh # Create configuration version -CV_RESPONSE=$(curl -s -X POST https://terrapod.local/api/v2/workspaces/ws-{id}/configuration-versions \ +CV_RESPONSE=$(curl -s -X POST https://terrapod.example.com/api/v2/workspaces/ws-{id}/configuration-versions \ -H "Authorization: Bearer $TERRAPOD_TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d '{"data": {"type": "configuration-versions", "attributes": {"auto-queue-runs": true}}}') @@ -274,7 +275,7 @@ curl -X PUT "$UPLOAD_URL" \ 3. A run is automatically queued (plan-only). Monitor it: ```zsh -curl -s https://terrapod.local/api/v2/workspaces/ws-{id}/runs \ +curl -s https://terrapod.example.com/api/v2/workspaces/ws-{id}/runs \ -H "Authorization: Bearer $TERRAPOD_TOKEN" | jq '.data[0].attributes.status' ``` @@ -289,7 +290,7 @@ Variables can be set per-workspace via the API or web UI. ### Terraform Variables ```zsh -curl -X POST https://terrapod.local/api/v2/workspaces/ws-{id}/vars \ +curl -X POST https://terrapod.example.com/api/v2/workspaces/ws-{id}/vars \ -H "Authorization: Bearer $TERRAPOD_TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d '{ @@ -309,7 +310,7 @@ curl -X POST https://terrapod.local/api/v2/workspaces/ws-{id}/vars \ ### Environment Variables ```zsh -curl -X POST https://terrapod.local/api/v2/workspaces/ws-{id}/vars \ +curl -X POST https://terrapod.example.com/api/v2/workspaces/ws-{id}/vars \ -H "Authorization: Bearer $TERRAPOD_TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d '{ @@ -330,7 +331,7 @@ curl -X POST https://terrapod.local/api/v2/workspaces/ws-{id}/vars \ Set `"sensitive": true` for secrets. The value is protected by database encryption-at-rest and never returned in API responses: ```zsh -curl -X POST https://terrapod.local/api/v2/workspaces/ws-{id}/vars \ +curl -X POST https://terrapod.example.com/api/v2/workspaces/ws-{id}/vars \ -H "Authorization: Bearer $TERRAPOD_TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d '{ @@ -363,7 +364,7 @@ For managing variables across multiple workspaces, see variable sets (admin-only 1. Create the module: ```zsh -curl -X POST https://terrapod.local/api/terrapod/v1/registry-modules \ +curl -X POST https://terrapod.example.com/api/terrapod/v1/registry-modules \ -H "Authorization: Bearer $TERRAPOD_TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d '{ @@ -380,7 +381,7 @@ curl -X POST https://terrapod.local/api/terrapod/v1/registry-modules \ 2. Create a version: ```zsh -curl -X POST https://terrapod.local/api/terrapod/v1/registry-modules/private/default/vpc/aws/versions \ +curl -X POST https://terrapod.example.com/api/terrapod/v1/registry-modules/private/default/vpc/aws/versions \ -H "Authorization: Bearer $TERRAPOD_TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d '{ @@ -399,7 +400,7 @@ curl -X POST https://terrapod.local/api/terrapod/v1/registry-modules/private/def ```hcl module "vpc" { - source = "terrapod.local/default/vpc/aws" + source = "terrapod.example.com/default/vpc/aws" version = "1.0.0" } ``` @@ -419,25 +420,19 @@ See [vcs-integration.md](vcs-integration.md) for detailed setup instructions. --- -## Running Tests and Linting - -All tests run in Docker -- no local Python or Node.js install needed: +## Uninstall ```zsh -make test # Run pytest with LocalStack for S3 testing -make lint # Run ruff check + format check + mypy -make test-down # Tear down test containers +helm uninstall terrapod --namespace terrapod ``` ---- - -## Stopping the Dev Environment - -```zsh -make dev-down -``` +PersistentVolumeClaims (filesystem storage, ephemeral runner scratch) are not +removed by `helm uninstall` — delete them explicitly if you want to reclaim the +storage, and drop the namespace with `kubectl delete namespace terrapod`. -This stops all Tilt-managed resources and cleans up the namespace. +> Building and testing Terrapod from source — Tilt, live-reload, `make test` — +> is covered in [Local Development](local-development.md). You don't need any of +> that to run Terrapod. --- diff --git a/docs/index.md b/docs/index.md index 26be7035..771fd8ff 100644 --- a/docs/index.md +++ b/docs/index.md @@ -41,14 +41,21 @@ Terrapod is **not** a fork of Terraform or OpenTofu. It orchestrates them. ## Quick Start +Deploy onto any Kubernetes cluster (or a single-node [k3s](https://k3s.io/) VM) with the Helm chart: + ```zsh -# Prerequisites: Docker, local K8s cluster, Tilt, mkcert -brew install mkcert && mkcert -install -sudo sh -c 'echo "127.0.0.1 terrapod.local" >> /etc/hosts' -make dev +helm install terrapod oci://ghcr.io/mattrobinsonsre/terrapod \ + --namespace terrapod --create-namespace \ + --set ingress.enabled=true \ + --set ingress.hostname="terrapod.example.com" \ + --set ingress.className=traefik \ + --set postgresql.url="postgresql+asyncpg://terrapod:PASSWORD@PGHOST:5432/terrapod" \ + --set redis.url="redis://REDISHOST:6379" \ + --set bootstrap.adminEmail="admin@example.com" \ + --set bootstrap.adminPassword="change-me-now" ``` -Open `https://terrapod.local` in your browser. See [Getting Started](getting-started.md) for the full walkthrough. +PostgreSQL and Redis are external (not bundled). See [Getting Started](getting-started.md) for the full walkthrough, or [Local Development](local-development.md) if you want to run Terrapod from source. --- @@ -80,7 +87,8 @@ See [Architecture](architecture.md) for the full breakdown. | Guide | Description | |---|---| -| [Getting Started](getting-started.md) | Local development setup, first workspace, first plan/apply | +| [Getting Started](getting-started.md) | Deploy the Helm chart on Kubernetes (or k3s), first workspace, first plan/apply | +| [Local Development](local-development.md) | Run Terrapod from source with Tilt (contributors only) | | [Architecture](architecture.md) | System components, storage, runners, auth flows | | [Authentication](authentication.md) | Local auth, OIDC, SAML, terraform login, API tokens, scoped service tokens (bound/detached) + offboarding idle guard | | [RBAC](rbac.md) | Permission model, label-based access control, custom roles | diff --git a/docs/local-development.md b/docs/local-development.md new file mode 100644 index 00000000..33d4d3ff --- /dev/null +++ b/docs/local-development.md @@ -0,0 +1,77 @@ +# Local Development + +This page is for **contributors** who want to run Terrapod from source on a +local Kubernetes cluster. If you just want to **deploy and use** Terrapod, see +[Getting Started](getting-started.md) — you do not need any of this. + +Terrapod's local dev stack is driven by [Tilt](https://tilt.dev/): it builds the +images, deploys the Helm chart against a local cluster, runs the database +migrations, bootstraps an admin user, and live-syncs source changes into the +running pods. + +## Prerequisites + +| Tool | Purpose | Install | +|---|---|---| +| Docker | Container runtime | [docker.com](https://www.docker.com/) | +| A local Kubernetes cluster | Run the stack | Rancher Desktop, Docker Desktop (K8s enabled), minikube, kind, OrbStack, or colima | +| Tilt | Local dev orchestration | `brew install tilt` | +| mkcert | Local TLS certificate | `brew install mkcert` | +| tofu (recommended) or terraform | Exercise the CLI flows | [opentofu.org](https://opentofu.org/) | + +The local stack is deliberately pinned to its own namespace (`terrapod`), Tilt +port (`10352`), and hostname (`terrapod.local`) so it can run alongside other +local projects. + +## Setup + +### 1. Local CA + hosts entry + +```zsh +brew install mkcert && mkcert -install +sudo sh -c 'echo "127.0.0.1 terrapod.local" >> /etc/hosts' +``` + +`mkcert -install` adds a local CA to your system trust store so +`https://terrapod.local` is trusted by your browser and the terraform/tofu CLI. + +### 2. Start the stack + +```zsh +make dev # runs `tilt up --port 10352` +``` + +This creates the `terrapod` namespace, generates the TLS cert, deploys +PostgreSQL and Redis in-cluster, builds the API/web images, runs the Alembic +migrations, bootstraps the admin user, and deploys the API, web UI, and runner +listener. Watch progress in the Tilt UI at . + +Tear it down with `make dev-down`. + +### 3. Access + +Open . The bootstrap job creates an admin user — the +default local credentials are `admin` / `admin` (set in +`helm/terrapod/values-local.yaml`). Check the `terrapod-bootstrap-1` resource in +the Tilt UI to confirm it ran. + +From there the workflow is the same as a real deployment — see +[Getting Started](getting-started.md) for creating a workspace and running your +first plan/apply (substitute `terrapod.local` for the hostname). + +## Day-to-day + +- **Live reload** — `tilt up` live-syncs `services/terrapod` and `web/src` into + the running pods; the API auto-reloads (uvicorn) and the web hot-reloads + (Next.js). If a change doesn't take, force it: + `tilt trigger --port 10352 terrapod-api` (or `terrapod-web`). +- **Migrations** — after adding an Alembic revision, trigger the migration job: + `tilt trigger --port 10352 terrapod-migrations-1`. +- **Tests & lint** (containerised — no local Python needed): + `make test`, `make lint`. Tear down test containers with `make test-down`. +- **Never** use `kubectl cp`, `kubectl apply`, or `docker build` against + Tilt-managed resources — it corrupts Tilt's state. Let Tilt manage its + resources; use `tilt trigger` to force a rebuild. + +For the architecture and conventions behind the stack, see +[Architecture](architecture.md) and the repository `CLAUDE.md`. diff --git a/helm/terrapod/values.yaml b/helm/terrapod/values.yaml index 3a0a10b4..c83e8224 100644 --- a/helm/terrapod/values.yaml +++ b/helm/terrapod/values.yaml @@ -546,8 +546,11 @@ api: annotations: {} # ── Web UI (Next.js) ────────────────────────────────────────────────────────── +# The web frontend is the BFF (Backend-For-Frontend): all browser, CLI, and +# listener traffic enters through it and is proxied to the API. The Ingress +# routes here, so it is enabled by default alongside the API and listener. web: - enabled: false + enabled: true replicas: 2 diff --git a/mkdocs.yml b/mkdocs.yml index 788ec07e..dd5cfa32 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -50,6 +50,7 @@ markdown_extensions: nav: - Home: index.md - Getting Started: getting-started.md + - Local Development: local-development.md - Architecture: architecture.md - Features: - VCS Integration: vcs-integration.md