From 106e6c84dd00065bdbace992131c8c86005b11cb Mon Sep 17 00:00:00 2001 From: Benjamin Knofe-Vider Date: Tue, 28 Jul 2026 15:13:15 +0100 Subject: [PATCH] feat(admin): provision warehouses from the console via the shared API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Operators can now provision a managed warehouse from the admin console instead of hand-rolling a curl against the control plane. The console does NOT get its own provisioning implementation: the new page is a form over the public provisioning API, posting to POST /api/v1/orgs/:id/provision — the exact route, handler, validation, transaction and analytics event the PostHog backend calls. A warehouse an operator creates is therefore identical to one a user creates by construction, not by convention. Frontend (controlplane/admin/ui): - New /orgs/provision page: org id, database name, team id + optional schema override, metadata store (cnpg-shard | external RDS), data store (new per-org bucket | existing bucket), with a live request preview of the literal call it will send, a database-name uniqueness pre-flight, a warning when the org already has a live warehouse (409), and a type-the-org-id confirmation. The once-only root password is surfaced with copy affordances plus a live warehouse/status poll. - Body construction and the validation mirror live in lib/provision.ts so they are unit-testable: the request must match the PostHog backend's. Optional fields are omitted rather than sent empty, and ducklake.enabled is hard-coded true (the server rejects false). - Reset-root-password action on the org page — the recovery path when a provision response is lost, since only the bcrypt hash is stored. - Entry points from the Orgs list and from an org with no warehouse. Backend: - auditActionFor now maps /provision, /deprovision and /reset-password to warehouse.provision / warehouse.deprovision / warehouse.reset_password. They previously fell through to the generic org.create, making a real infrastructure action indistinguishable from an org-config write in the console's audit view. - Topology tripwires pin the "one implementation" invariant from both sides: TestProvisioningAPIRouteTopology pins the shared route set, and TestAdminAPIRegistersNoProvisioningRoutes pins the absence of an admin-side twin (gin only panics on an exactly-matching duplicate, so a near-miss path would fork the contract silently). Tests: provision.test.ts (validation + body construction), ProvisionWarehouse.test.tsx (the page posts exactly that body, gates on the server's rules, admin-gated), audit_test.go, the two topology tests, and admin_provision_parity in the mw-dev e2e harness — which replays the console's body against the provisioned org (409, not a shape 400), pins the ducklake:false rejection, and asserts the warehouse.provision / warehouse.deprovision audit actions. Docs: CLAUDE.md, controlplane/admin/README.md, docs/design/admin-ui.md, tests/mw-dev/README.md. --- CLAUDE.md | 22 + controlplane/admin/README.md | 50 +- controlplane/admin/audit.go | 12 + controlplane/admin/audit_test.go | 9 + controlplane/admin/provision_parity_test.go | 43 ++ controlplane/admin/ui/src/App.tsx | 4 + .../admin/ui/src/components/Copyable.tsx | 31 ++ controlplane/admin/ui/src/hooks/useApi.ts | 63 +++ controlplane/admin/ui/src/lib/api.ts | 24 + controlplane/admin/ui/src/lib/audit.ts | 5 + .../admin/ui/src/lib/provision.test.ts | 166 ++++++ controlplane/admin/ui/src/lib/provision.ts | 169 +++++++ controlplane/admin/ui/src/pages/OrgDetail.tsx | 104 +++- controlplane/admin/ui/src/pages/Orgs.tsx | 33 +- .../ui/src/pages/ProvisionWarehouse.test.tsx | 150 ++++++ .../admin/ui/src/pages/ProvisionWarehouse.tsx | 476 ++++++++++++++++++ controlplane/admin/ui/src/types/api.ts | 93 ++++ .../provisioning/api_topology_test.go | 50 ++ docs/design/admin-ui.md | 10 +- tests/mw-dev/README.md | 16 + tests/mw-dev/e2e/harness.sh | 67 +++ 21 files changed, 1579 insertions(+), 18 deletions(-) create mode 100644 controlplane/admin/provision_parity_test.go create mode 100644 controlplane/admin/ui/src/components/Copyable.tsx create mode 100644 controlplane/admin/ui/src/lib/provision.test.ts create mode 100644 controlplane/admin/ui/src/lib/provision.ts create mode 100644 controlplane/admin/ui/src/pages/ProvisionWarehouse.test.tsx create mode 100644 controlplane/admin/ui/src/pages/ProvisionWarehouse.tsx create mode 100644 controlplane/provisioning/api_topology_test.go diff --git a/CLAUDE.md b/CLAUDE.md index 2d081365..247ebd8a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -342,6 +342,28 @@ impersonation, audit log; sliceable by org + user). Design + decisions: `RoleGate` requires admin for all mutating verbs + the audit GET. `AuditMiddleware` records every mutation. Keep new mutating routes under this gate; never add a write path that bypasses RoleGate/audit. +- **Warehouse provisioning has exactly ONE implementation.** The console's + Provision-warehouse page (`ui/src/pages/ProvisionWarehouse.tsx`, route + `/orgs/provision`) posts to `POST /api/v1/orgs/:id/provision` — the route + `controlplane/provisioning.RegisterAPI` mounts on the SAME audited `/api/v1` + group (`multitenant.go`), i.e. the exact endpoint the PostHog backend + (Django) calls. Same handler, validation, transaction and analytics event ⇒ + an operator-provisioned warehouse is identical to a user-provisioned one by + construction. NEVER add an admin-local provisioning handler (gin only panics + on an exactly-matching duplicate, so a near-miss path would fork the contract + silently): `provisioning.TestProvisioningAPIRouteTopology` pins the shared + route set and `admin.TestAdminAPIRegistersNoProvisioningRoutes` pins the + absence of a twin. Same for the siblings the console drives — + `warehouse/status`, `database-name/check`, `reset-password`, `deprovision`. + The client-side validation mirror (`ui/src/lib/provision.ts`) is a courtesy: + LOOSER than the server is safe, STRICTER blocks a body the PostHog backend + may legitimately send. Audit actions for these are `warehouse.provision` / + `warehouse.deprovision` / `warehouse.reset_password` (`auditActionFor`; keep + `ui/src/lib/audit.ts` labels in sync). Touching any of this → update + `controlplane/admin/provision_parity_test.go`, `audit_test.go`, + `controlplane/provisioning/api_topology_test.go`, + `ui/src/lib/provision.test.ts`, `ui/src/pages/ProvisionWarehouse.test.tsx`, + AND the `admin_provision_parity` assertion in `tests/mw-dev/e2e/harness.sh`. - **Impersonation is a real session** (`impersonate.go` + `admin_providers.go`): it reuses `SessionManager.CreateSessionWithProtocol` (workers trust the CP — no password) and **always** `DestroySession` in a defer. Admin-only, every diff --git a/controlplane/admin/README.md b/controlplane/admin/README.md index c8e4b353..3623888c 100644 --- a/controlplane/admin/README.md +++ b/controlplane/admin/README.md @@ -78,6 +78,44 @@ Added for the console: | `POST /api/v1/operators` | admin | add/update an operator (`{email, role}`; last-admin demotion → 409) | | `DELETE /api/v1/operators/:email` | admin | remove an operator (removing the last admin → 409) | +### Warehouse provisioning — one implementation, two callers + +The console's **Provision warehouse** page (`ui/src/pages/ProvisionWarehouse.tsx`, +route `/orgs/provision`) has **no provisioning backend of its own**. It posts to +`POST /api/v1/orgs/:id/provision` — the route +`controlplane/provisioning.RegisterAPI` mounts on *this* router group (see +`multitenant.go`), i.e. the exact endpoint, handler, validation, transaction and +analytics event the PostHog backend (Django) calls. A warehouse an operator +creates from the console is therefore identical to one a user creates, by +construction rather than by convention. + +Consequences to preserve: + +- **Never add an admin-local provisioning handler.** Two implementations would + drift, and the drift would only show up as differently-shaped tenants in + production. `provisioning.TestProvisioningAPIRouteTopology` pins the shared + route set and `admin.TestAdminAPIRegistersNoProvisioningRoutes` pins the + absence of an admin-side twin (gin only panics on an exactly-matching + duplicate, so a near-miss path would fork silently). +- Because those routes live on the audited admin group, an operator's provision + and the PostHog backend's provision write the same audit action — they differ + only by actor (`internal-secret` vs the SSO email). The actions are + `warehouse.provision` / `warehouse.deprovision` / `warehouse.reset_password` + (`auditActionFor`; keep `ui/src/lib/audit.ts` labels in sync). +- The client-side validation mirror (`ui/src/lib/provision.ts`) is a courtesy + only. Keeping it LOOSER than the server is safe (the error moves to submit + time); making it STRICTER blocks a body the PostHog backend may legitimately + send, which is exactly the divergence this surface exists to prevent. +- The form hard-codes `ducklake.enabled=true` (the server rejects `false`) and + omits absent optional fields rather than sending empty strings, so the request + is byte-identical to the backend's for the same intent. + +The console also drives the sibling lifecycle endpoints on the same shared API: +`GET /orgs/:id/warehouse/status` (the post-provision poll), `GET +/database-name/check` (uniqueness pre-flight), `POST /orgs/:id/reset-password` +(root rotation — the recovery path when a provision response is lost, since only +the bcrypt hash is stored) and `POST /orgs/:id/deprovision`. + ### Cross-CP live-state aggregation (`live_aggregate.go` + `controlplane/live_aggregator.go`) Live session/query state is **in-memory per CP** — each replica only knows the @@ -148,9 +186,11 @@ embedded, under Vite, or under the Go devserver. **Backend:** `authz_test.go` (SSO role mapping, RoleGate, SQL classifier), `dashboard_test.go` (TokenSet / break-glass login / cookie), `api_test.go` + -`api_postgres_test.go` (CRUD), `models_api_test.go` (redaction). e2e: the -`admin_*` / `impersonation_*` / `models_explorer_api` assertions in -`tests/e2e-mw-dev/harness.sh`. +`api_postgres_test.go` (CRUD), `models_api_test.go` (redaction), +`audit_test.go` (action mapping, incl. the warehouse lifecycle actions), +`provision_parity_test.go` (no admin-side provisioning twin). e2e: the +`admin_*` (incl. `admin_provision_parity`) / `impersonation_*` / +`models_explorer_api` assertions in `tests/mw-dev/e2e/harness.sh`. **Frontend** (`ui/`, Vitest + Testing Library — `just ui-test`, CI job `ui-tests`): the dashboard's data-derivation logic has shipped wrong more than @@ -161,3 +201,7 @@ of inline JSX. `src/lib/fleet.test.ts` pins the worker-fleet/load math `src/pages/Overview.test.tsx` renders the page with mocked hooks and asserts the Workers card + leak warning. New derivation/display logic on a page **must** get a `*.test.ts(x)` here — keep computed values out of the JSX so they're testable. +The provisioning form follows the same rule for a stronger reason: the REQUEST it +builds has to match the PostHog backend's, so the body construction and the +validation mirror live in `src/lib/provision.ts` (`provision.test.ts`) and +`src/pages/ProvisionWarehouse.test.tsx` asserts the page posts exactly that body. diff --git a/controlplane/admin/audit.go b/controlplane/admin/audit.go index ef93c0fd..59bb1f78 100644 --- a/controlplane/admin/audit.go +++ b/controlplane/admin/audit.go @@ -198,6 +198,18 @@ func auditActionFor(method, path string) string { case hasSeg(segs, "secrets"): // /orgs/:id/users/:username/secrets/:name. return "secret." + verb + case last == "provision": + // POST /orgs/:id/provision — the ONE warehouse-creation path, + // shared verbatim by the PostHog backend and the admin console + // (provisioning/api.go). Distinct from "warehouse.update" (a + // config-row edit) because it creates real infrastructure. + return "warehouse.provision" + case last == "deprovision": + // POST /orgs/:id/deprovision — asynchronous duckling teardown. + return "warehouse.deprovision" + case last == "reset-password": + // POST /orgs/:id/reset-password — rotates the org's root login. + return "warehouse.reset_password" case hasSeg(segs, "warehouse"): // PUT /warehouse and PATCH /warehouse/pinning both map here. return "warehouse." + verb diff --git a/controlplane/admin/audit_test.go b/controlplane/admin/audit_test.go index 0a52a15f..0c46f389 100644 --- a/controlplane/admin/audit_test.go +++ b/controlplane/admin/audit_test.go @@ -32,6 +32,15 @@ func TestAuditActionFor(t *testing.T) { {"org team update", http.MethodPut, "/api/v1/orgs/acme/teams/7", "team.update"}, {"org team delete", http.MethodDelete, "/api/v1/orgs/acme/teams/7", "team.delete"}, {"warehouse pinning patch", http.MethodPatch, "/api/v1/orgs/acme/warehouse/pinning", "warehouse.update"}, + // The provisioning API's mutating routes are mounted on the SAME audited + // admin group, so an operator provisioning from the console and the + // PostHog backend provisioning over the internal secret write the same + // action code — distinguishable only by actor. Before these cases they + // all fell through to the generic "org.create". + {"warehouse provision", http.MethodPost, "/api/v1/orgs/acme/provision", "warehouse.provision"}, + {"warehouse provision no prefix", http.MethodPost, "/orgs/:id/provision", "warehouse.provision"}, + {"warehouse deprovision", http.MethodPost, "/api/v1/orgs/acme/deprovision", "warehouse.deprovision"}, + {"warehouse reset password", http.MethodPost, "/api/v1/orgs/acme/reset-password", "warehouse.reset_password"}, {"org create", http.MethodPost, "/api/v1/orgs", "org.create"}, {"org update", http.MethodPut, "/api/v1/orgs/acme", "org.update"}, {"org delete", http.MethodDelete, "/api/v1/orgs/acme", "org.delete"}, diff --git a/controlplane/admin/provision_parity_test.go b/controlplane/admin/provision_parity_test.go new file mode 100644 index 00000000..863f86dc --- /dev/null +++ b/controlplane/admin/provision_parity_test.go @@ -0,0 +1,43 @@ +//go:build kubernetes + +package admin + +import ( + "strings" + "testing" +) + +// TestAdminAPIRegistersNoProvisioningRoutes is the other half of the "one +// provisioning path" tripwire (see +// provisioning.TestProvisioningAPIRouteTopology). +// +// The admin console provisions warehouses by calling the SAME endpoints the +// PostHog backend calls — POST /orgs/:id/provision, POST /orgs/:id/deprovision, +// POST /orgs/:id/reset-password, GET /orgs/:id/warehouse/status, GET +// /database-name/check — which the provisioning package registers on this very +// router group (see controlplane/multitenant.go). That shared registration is +// what guarantees a console-provisioned warehouse is identical to a +// user-provisioned one: same validation, same defaults, same transaction, same +// analytics event. +// +// If the admin package ever grows its own provisioning handler, gin would +// panic on the duplicate route — but only for an exactly-matching path. A +// near-miss (e.g. /orgs/:id/provision-warehouse) would silently fork the +// contract, so assert the absence explicitly. +func TestAdminAPIRegistersNoProvisioningRoutes(t *testing.T) { + r := newTestAPIRouter(&fakeAPIStore{}) + + // Substrings that would indicate an admin-local warehouse lifecycle + // implementation. "warehouse" alone is fine — PUT /orgs/:id/warehouse and + // PATCH /orgs/:id/warehouse/pinning edit the CONFIG ROW of an + // already-provisioned warehouse; they do not create or destroy infra. + forbidden := []string{"provision", "reset-password", "database-name", "warehouse/status"} + + for _, ri := range r.Routes() { + for _, f := range forbidden { + if strings.Contains(ri.Path, f) { + t.Fatalf("admin API registered %s %s: warehouse provisioning must stay in controlplane/provisioning so the console and the PostHog backend share one implementation", ri.Method, ri.Path) + } + } + } +} diff --git a/controlplane/admin/ui/src/App.tsx b/controlplane/admin/ui/src/App.tsx index 490594e3..f97b38fb 100644 --- a/controlplane/admin/ui/src/App.tsx +++ b/controlplane/admin/ui/src/App.tsx @@ -6,6 +6,7 @@ import { NotFound } from "@/pages/NotFound"; import { Overview } from "@/pages/Overview"; import { Orgs } from "@/pages/Orgs"; import { OrgDetail } from "@/pages/OrgDetail"; +import { ProvisionWarehouse } from "@/pages/ProvisionWarehouse"; import { OrgTeams } from "@/pages/OrgTeams"; import { ReshardForm } from "@/pages/ReshardForm"; import { Reshards } from "@/pages/Reshards"; @@ -34,6 +35,9 @@ export default function App() { } /> } /> + {/* Static segment before the :id route so "provision" is never read as + an org id. */} + } /> } /> } /> } /> diff --git a/controlplane/admin/ui/src/components/Copyable.tsx b/controlplane/admin/ui/src/components/Copyable.tsx new file mode 100644 index 00000000..b4b35831 --- /dev/null +++ b/controlplane/admin/ui/src/components/Copyable.tsx @@ -0,0 +1,31 @@ +import { useState } from "react"; +import { Check, Copy } from "lucide-react"; + +// A labelled value with a copy-to-clipboard button. Used for once-only +// credentials (the provision response, a password reset) where re-reading the +// value is impossible, so copying has to be one click and obviously available. +export function Copyable({ label, value }: { label: string; value: string }) { + const [copied, setCopied] = useState(false); + return ( +
+ {label} + + {value} + + +
+ ); +} diff --git a/controlplane/admin/ui/src/hooks/useApi.ts b/controlplane/admin/ui/src/hooks/useApi.ts index ccdf4abe..67e86f5f 100644 --- a/controlplane/admin/ui/src/hooks/useApi.ts +++ b/controlplane/admin/ui/src/hooks/useApi.ts @@ -19,6 +19,7 @@ import type { ClusterStatus, ClusterSummary, CreateUserBody, + DatabaseNameCheck, DucklingDriftResponse, DucklingMetadataResponse, ErrorEntry, @@ -39,6 +40,7 @@ import type { OrgUser, OrgUserSecret, PromRangeResponse, + ProvisionBody, QueryDetail, ReshardLogEntry, ReshardOperation, @@ -47,6 +49,7 @@ import type { SessionStatus, StartReshardBody, UpdateUserBody, + WarehouseStatus, WorkerStatus, } from "@/types/api"; @@ -174,6 +177,66 @@ export function useDeprovisionWarehouse(id: string) { }); } +// ---- warehouse provisioning (the PostHog-backend endpoints) ---- + +// POST /orgs/:id/provision — the SAME call the PostHog backend makes. Returns +// 202 plus the root password, which is readable exactly once, so the caller +// must surface it before navigating away. +export function useProvisionWarehouse() { + const qc = useQueryClient(); + return useMutation({ + mutationFn: (v: { org: string; body: ProvisionBody }) => api.provisionWarehouse(v.org, v.body), + onSuccess: (_res, v) => { + qc.invalidateQueries({ queryKey: ["orgs"] }); + qc.invalidateQueries({ queryKey: ["orgs", v.org, "warehouse"] }); + qc.invalidateQueries({ queryKey: ["org-teams"] }); + }, + }); +} + +// GET /orgs/:id/warehouse/status — the lifecycle view the PostHog backend +// polls. 404 (no warehouse row yet) resolves to null rather than erroring. +// `enabled` lets the provision page start polling only once it has submitted. +export function useWarehouseStatus(id: string | undefined, enabled = true) { + return useQuery({ + queryKey: ["orgs", id, "warehouse-status"], + queryFn: () => tolerate404(null)(api.getWarehouseStatus(id!)), + enabled: !!id && enabled, + refetchInterval: POLL.normal, + }); +} + +// GET /database-name/check — debounced global uniqueness pre-flight for the +// provision form. The server is authoritative (it 409s a taken name at +// provision time); this only gives the operator the answer before submit. +export function useDatabaseNameAvailable(name: string) { + const debounced = useDebounced(name.trim(), 350); + return useQuery({ + queryKey: ["database-name-check", debounced], + queryFn: () => tolerate404(null)(api.checkDatabaseName(debounced)), + enabled: debounced !== "", + staleTime: 10_000, + }); +} + +// POST /orgs/:id/reset-password — rotates the org's root login. The recovery +// path when a provision response was lost; the new plaintext is returned once. +export function useResetWarehousePassword(id: string) { + return useMutation({ + mutationFn: () => api.resetWarehousePassword(id), + }); +} + +// useDebounced returns `value` after it has stopped changing for `ms`. +function useDebounced(value: T, ms: number): T { + const [settled, setSettled] = useState(value); + useEffect(() => { + const t = setTimeout(() => setSettled(value), ms); + return () => clearTimeout(t); + }, [value, ms]); + return settled; +} + // ---- org teams ---- // All teams across every org (the Org teams nav page). 404-tolerant so a diff --git a/controlplane/admin/ui/src/lib/api.ts b/controlplane/admin/ui/src/lib/api.ts index 1f69b5a1..6dc47ee2 100644 --- a/controlplane/admin/ui/src/lib/api.ts +++ b/controlplane/admin/ui/src/lib/api.ts @@ -11,6 +11,7 @@ import type { ClusterSummary, CreateUserBody, CPInstance, + DatabaseNameCheck, DucklingDriftResponse, DucklingMetadataResponse, ErrorEntry, @@ -32,16 +33,20 @@ import type { OrgUser, OrgUserSecret, PromRangeResponse, + ProvisionBody, + ProvisionResult, QueryDetail, QueryResult, ReshardLogEntry, ReshardOperation, ReshardTargetsResponse, + ResetPasswordResult, RunningQuery, SessionStatus, StartReshardBody, UpdateUserBody, UserKillResult, + WarehouseStatus, WorkerStatus, } from "@/types/api"; @@ -137,6 +142,25 @@ export const api = { getWarehouse: (id: string) => get(`/orgs/${enc(id)}/warehouse`), updateWarehouse: (id: string, body: Partial) => put(`/orgs/${enc(id)}/warehouse`, body), + // ---- warehouse lifecycle: the PostHog-backend endpoints, verbatim ---- + // + // provisioning/api.go registers these on this same /api/v1 group, so the + // console and the PostHog backend hit ONE implementation. Never reimplement + // any of them behind an admin-only route: an operator-provisioned warehouse + // must be indistinguishable from a user-provisioned one. + + // Starts asynchronous provisioning (202). The response carries the root + // password, readable exactly once — it is never stored in plaintext. + provisionWarehouse: (id: string, body: ProvisionBody) => + post(`/orgs/${enc(id)}/provision`, body), + // Lifecycle view the PostHog backend polls; 404 until a warehouse row exists. + getWarehouseStatus: (id: string) => get(`/orgs/${enc(id)}/warehouse/status`), + // Global database-name uniqueness pre-flight (the provision form's check). + checkDatabaseName: (name: string) => get("/database-name/check", { name }), + // Rotates the org's root password; returns the new plaintext once. 409s + // unless the warehouse is ready. + resetWarehousePassword: (id: string) => + post(`/orgs/${enc(id)}/reset-password`, {}), // Kicks off asynchronous teardown of the org's duckling (202). 409s when the // warehouse state isn't ready/failed/provisioning. deprovisionWarehouse: (id: string) => diff --git a/controlplane/admin/ui/src/lib/audit.ts b/controlplane/admin/ui/src/lib/audit.ts index bd873dda..7cb17406 100644 --- a/controlplane/admin/ui/src/lib/audit.ts +++ b/controlplane/admin/ui/src/lib/audit.ts @@ -19,6 +19,11 @@ const ACTION_LABELS: Record = { "warehouse.create": "Created warehouse config", "warehouse.update": "Updated warehouse config", "warehouse.delete": "Deleted warehouse config", + // Lifecycle actions on the shared provisioning API — real infrastructure, + // as opposed to the config-row edits above. + "warehouse.provision": "Provisioned warehouse", + "warehouse.deprovision": "Deprovisioned warehouse", + "warehouse.reset_password": "Reset warehouse root password", "team.create": "Created org team", "team.update": "Updated org team", diff --git a/controlplane/admin/ui/src/lib/provision.test.ts b/controlplane/admin/ui/src/lib/provision.test.ts new file mode 100644 index 00000000..f756b222 --- /dev/null +++ b/controlplane/admin/ui/src/lib/provision.test.ts @@ -0,0 +1,166 @@ +import { describe, expect, it } from "vitest"; +import { + buildProvisionBody, + DEFAULT_PROVISION_FORM, + MAX_SLUG_ORG_ID_LENGTH, + validateOrgId, + validateProvisionForm, + validateSchemaName, + type ProvisionForm, +} from "./provision"; + +const form = (over: Partial = {}): ProvisionForm => ({ + ...DEFAULT_PROVISION_FORM, + orgId: "acme", + databaseName: "acme", + teamId: "42", + ...over, +}); + +describe("validateOrgId", () => { + it("accepts DNS-1123 slugs and canonical UUIDs", () => { + expect(validateOrgId("acme")).toBeNull(); + expect(validateOrgId("acme-prod-1")).toBeNull(); + // 36 chars — over the slug cap, but allowed because the bucket name + // compacts a UUID's hyphens (configstore.DucklingBucketName). + expect(validateOrgId("0192f3c4-5d6e-7f80-9123-456789abcdef")).toBeNull(); + }); + + it("rejects non-DNS-1123 shapes", () => { + expect(validateOrgId("")).toMatch(/required/); + expect(validateOrgId("Acme")).toMatch(/DNS-1123/); + expect(validateOrgId("acme_prod")).toMatch(/DNS-1123/); + expect(validateOrgId("-acme")).toMatch(/DNS-1123/); + expect(validateOrgId("acme-")).toMatch(/DNS-1123/); + }); + + it("caps non-UUID slugs at the bucket-name budget", () => { + expect(validateOrgId("a".repeat(MAX_SLUG_ORG_ID_LENGTH))).toBeNull(); + expect(validateOrgId("a".repeat(MAX_SLUG_ORG_ID_LENGTH + 1))).toMatch(/at most 35/); + }); +}); + +describe("validateSchemaName", () => { + it("treats empty as unset (the server derives team_)", () => { + expect(validateSchemaName("")).toBeNull(); + }); + + it("mirrors configstore.ValidateOrgTeamSchemaName", () => { + expect(validateSchemaName("team_42")).toBeNull(); + expect(validateSchemaName("_private")).toBeNull(); + expect(validateSchemaName("42team")).toMatch(/lowercase identifier/); + expect(validateSchemaName("Team")).toMatch(/lowercase identifier/); + expect(validateSchemaName("a".repeat(64))).toMatch(/at most 63/); + }); +}); + +describe("validateProvisionForm", () => { + it("passes the default cnpg-shard shape", () => { + expect(validateProvisionForm(form(), false)).toEqual({}); + }); + + it("requires team_id only when the provision creates a new org", () => { + expect(validateProvisionForm(form({ teamId: "" }), false).teamId).toMatch(/team_id is required/); + expect(validateProvisionForm(form({ teamId: "" }), true).teamId).toBeUndefined(); + }); + + it("rejects a non-positive-integer team id", () => { + expect(validateProvisionForm(form({ teamId: "0" }), false).teamId).toMatch(/positive/); + expect(validateProvisionForm(form({ teamId: "-3" }), false).teamId).toMatch(/positive/); + expect(validateProvisionForm(form({ teamId: "1.5" }), false).teamId).toMatch(/positive/); + }); + + it("requires database_name", () => { + expect(validateProvisionForm(form({ databaseName: " " }), false).databaseName).toMatch(/required/); + }); + + it("requires team_id for a schema_name override", () => { + const errs = validateProvisionForm(form({ teamId: "", schemaName: "legacy" }), true); + expect(errs.schemaName).toMatch(/requires team_id/); + }); + + it("requires endpoint + secret for an external metadata store", () => { + const errs = validateProvisionForm(form({ metadataType: "external" }), false); + expect(errs.externalEndpoint).toMatch(/endpoint/); + expect(errs.externalSecret).toMatch(/password_aws_secret/); + }); + + it("requires bucket_name for an external data store", () => { + const errs = validateProvisionForm(form({ dataStoreType: "external" }), false); + expect(errs.bucketName).toMatch(/bucket_name/); + }); +}); + +describe("buildProvisionBody", () => { + it("emits the standard PostHog-backend body for the defaults", () => { + expect(buildProvisionBody(form())).toEqual({ + database_name: "acme", + team_id: 42, + metadata_store: { type: "cnpg-shard" }, + data_store: { type: "s3bucket" }, + ducklake: { enabled: true }, + }); + }); + + it("omits optional fields instead of sending them empty", () => { + const body = buildProvisionBody(form({ teamId: "", schemaName: "" })); + expect("team_id" in body).toBe(false); + expect("schema_name" in body).toBe(false); + }); + + it("always requests DuckLake (the server rejects false)", () => { + expect(buildProvisionBody(form()).ducklake).toEqual({ enabled: true }); + }); + + it("carries the external metadata store block, defaulted user/database included", () => { + const body = buildProvisionBody( + form({ + metadataType: "external", + externalEndpoint: " db.example.internal ", + externalSecret: " duckling-acme-rds-password ", + }), + ); + expect(body.metadata_store).toEqual({ + type: "external", + external: { + endpoint: "db.example.internal", + password_aws_secret: "duckling-acme-rds-password", + user: "postgres", + database: "postgres", + }, + }); + }); + + it("omits user/database when cleared so the XRD default applies", () => { + const body = buildProvisionBody( + form({ + metadataType: "external", + externalEndpoint: "db.example.internal", + externalSecret: "duckling-acme-rds-password", + externalUser: "", + externalDatabase: "", + }), + ); + expect(body.metadata_store.external).toEqual({ + endpoint: "db.example.internal", + password_aws_secret: "duckling-acme-rds-password", + }); + }); + + it("carries an external data store with its optional region", () => { + expect(buildProvisionBody(form({ dataStoreType: "external", bucketName: "existing-bucket" })).data_store).toEqual({ + type: "external", + bucket_name: "existing-bucket", + }); + expect( + buildProvisionBody(form({ dataStoreType: "external", bucketName: "existing-bucket", region: "us-east-1" })) + .data_store, + ).toEqual({ type: "external", bucket_name: "existing-bucket", region: "us-east-1" }); + }); + + it("trims whitespace out of every identifier", () => { + const body = buildProvisionBody(form({ orgId: " acme ", databaseName: " acme_db ", schemaName: " team_42 " })); + expect(body.database_name).toBe("acme_db"); + expect(body.schema_name).toBe("team_42"); + }); +}); diff --git a/controlplane/admin/ui/src/lib/provision.ts b/controlplane/admin/ui/src/lib/provision.ts new file mode 100644 index 00000000..6c2deea9 --- /dev/null +++ b/controlplane/admin/ui/src/lib/provision.ts @@ -0,0 +1,169 @@ +// Client-side mirror of the warehouse provisioning contract. +// +// The console posts to POST /api/v1/orgs/:id/provision — the exact endpoint the +// PostHog backend (Django) posts to (controlplane/provisioning/api.go). The +// server is and stays authoritative: everything here is a pre-submit courtesy +// so an operator sees "org id must be a DNS-1123 label" while typing instead of +// as a 400 afterwards. Keep these rules in lockstep with api.go — a rule that +// drifts LOOSER just moves the error to submit time (safe); a rule that drifts +// STRICTER silently blocks a body the PostHog backend is allowed to send, which +// is the divergence this whole surface exists to prevent. + +import type { ProvisionBody, ProvisionDataStore, ProvisionMetadataStore } from "@/types/api"; + +// Mirrors provisioning/api.go: ducklingOrgIDPattern (a single DNS-1123 label) +// and canonicalDucklingUUIDPattern (the UUID-shaped org ids PostHog sends). +const ORG_ID_RE = /^[a-z0-9]([a-z0-9-]*[a-z0-9])?$/; +const ORG_ID_UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/; +// maxDucklingSlugOrgIDLength: a non-UUID org id must leave room for the +// derived S3 bucket name (63-char cap minus the managed-warehouse suffix). +export const MAX_SLUG_ORG_ID_LENGTH = 35; + +// Mirrors configstore.ValidateOrgTeamSchemaName. +const SCHEMA_NAME_RE = /^[a-z_][a-z0-9_]*$/; +const MAX_SCHEMA_NAME_LENGTH = 63; + +export function validateOrgId(orgID: string): string | null { + const v = orgID.trim(); + if (v === "") return "org id is required"; + if (!ORG_ID_RE.test(v)) { + return "org id must be a DNS-1123 label (lowercase alphanumerics and hyphens, starting and ending alphanumeric)"; + } + if (!ORG_ID_UUID_RE.test(v) && v.length > MAX_SLUG_ORG_ID_LENGTH) { + return `org id must be a canonical UUID or a slug of at most ${MAX_SLUG_ORG_ID_LENGTH} characters`; + } + return null; +} + +export function validateSchemaName(name: string): string | null { + const v = name.trim(); + if (v === "") return null; // optional — the server derives "team_" + if (v.length > MAX_SCHEMA_NAME_LENGTH) { + return `schema_name must be at most ${MAX_SCHEMA_NAME_LENGTH} characters`; + } + if (!SCHEMA_NAME_RE.test(v)) { + return "schema_name must be a lowercase identifier: [a-z0-9_], not starting with a digit"; + } + return null; +} + +// The form's editable state. Kept as strings (what inputs hold) and narrowed +// into a ProvisionBody by buildProvisionBody. +export interface ProvisionForm { + orgId: string; + databaseName: string; + teamId: string; + schemaName: string; + metadataType: "cnpg-shard" | "external"; + externalEndpoint: string; + externalSecret: string; + externalUser: string; + externalDatabase: string; + dataStoreType: "s3bucket" | "external"; + bucketName: string; + region: string; +} + +// The defaults the PostHog backend sends for a standard managed warehouse: +// a cnpg shard for the DuckLake catalog, a fresh control-plane-named per-org +// S3 bucket, DuckLake on. An operator provisioning from the console starts +// from exactly that shape. +export const DEFAULT_PROVISION_FORM: ProvisionForm = { + orgId: "", + databaseName: "", + teamId: "", + schemaName: "", + metadataType: "cnpg-shard", + externalEndpoint: "", + externalSecret: "", + externalUser: "postgres", + externalDatabase: "postgres", + dataStoreType: "s3bucket", + bucketName: "", + region: "", +}; + +// validateProvisionForm returns field-keyed messages for everything the server +// would reject. `orgExists` mirrors the one server-side rule the client cannot +// evaluate alone: team_id is REQUIRED when the provision creates a NEW org +// (ErrProvisionTeamRequired → 400) and optional when re-provisioning an +// existing one (the stored teams are preserved, never wiped). +export function validateProvisionForm( + f: ProvisionForm, + orgExists: boolean, +): Partial> { + const errs: Partial> = {}; + + const orgErr = validateOrgId(f.orgId); + if (orgErr) errs.orgId = orgErr; + + if (f.databaseName.trim() === "") errs.databaseName = "database_name is required"; + + const teamId = f.teamId.trim(); + if (teamId === "") { + if (!orgExists) { + errs.teamId = "team_id is required when provisioning a warehouse for a new org"; + } + } else if (!/^\d+$/.test(teamId) || Number(teamId) <= 0) { + errs.teamId = "team_id must be a positive PostHog team id"; + } + + const schemaErr = validateSchemaName(f.schemaName); + if (schemaErr) errs.schemaName = schemaErr; + if (f.schemaName.trim() !== "" && teamId === "") errs.schemaName = "schema_name requires team_id"; + + if (f.metadataType === "external") { + if (f.externalEndpoint.trim() === "") + errs.externalEndpoint = "endpoint is required for an external metadata store"; + if (f.externalSecret.trim() === "") { + errs.externalSecret = "password_aws_secret is required for an external metadata store"; + } + } + + if (f.dataStoreType === "external" && f.bucketName.trim() === "") { + errs.bucketName = "bucket_name is required for an external data store"; + } + + return errs; +} + +// buildProvisionBody narrows the form into the wire body. Optional fields are +// OMITTED rather than sent empty, so the request is byte-identical to what the +// PostHog backend sends for the same intent — the server's defaults (and the +// XRD's) apply to absent fields, never to an empty string. +export function buildProvisionBody(f: ProvisionForm): ProvisionBody { + const metadata_store: ProvisionMetadataStore = + f.metadataType === "external" + ? { + type: "external", + external: { + endpoint: f.externalEndpoint.trim(), + password_aws_secret: f.externalSecret.trim(), + ...(f.externalUser.trim() !== "" ? { user: f.externalUser.trim() } : {}), + ...(f.externalDatabase.trim() !== "" ? { database: f.externalDatabase.trim() } : {}), + }, + } + : { type: "cnpg-shard" }; + + const data_store: ProvisionDataStore = + f.dataStoreType === "external" + ? { + type: "external", + bucket_name: f.bucketName.trim(), + ...(f.region.trim() !== "" ? { region: f.region.trim() } : {}), + } + : { type: "s3bucket" }; + + const teamId = f.teamId.trim(); + return { + database_name: f.databaseName.trim(), + ...(teamId !== "" ? { team_id: Number(teamId) } : {}), + ...(f.schemaName.trim() !== "" ? { schema_name: f.schemaName.trim() } : {}), + metadata_store, + data_store, + // Never operator-settable: the server rejects `false` outright (a warehouse + // without a catalog has nothing to attach), so exposing it as a toggle + // would only offer a guaranteed 400. + ducklake: { enabled: true }, + }; +} diff --git a/controlplane/admin/ui/src/pages/OrgDetail.tsx b/controlplane/admin/ui/src/pages/OrgDetail.tsx index 57bc5b49..d2e1e763 100644 --- a/controlplane/admin/ui/src/pages/OrgDetail.tsx +++ b/controlplane/admin/ui/src/pages/OrgDetail.tsx @@ -1,6 +1,18 @@ import { useEffect, useState } from "react"; import { Link, useNavigate, useParams } from "react-router-dom"; -import { AlertTriangle, ArrowLeft, Database, Layers, Pencil, Plus, Save, Trash2, Warehouse } from "lucide-react"; +import { + AlertTriangle, + ArrowLeft, + Database, + KeyRound, + Layers, + Pencil, + Plus, + Rocket, + Save, + Trash2, + Warehouse, +} from "lucide-react"; import { PageBody, PageHeader } from "@/components/AppShell"; import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; import { Button } from "@/components/ui/button"; @@ -30,10 +42,12 @@ import { useOrg, useOrgReshards, useOrgTeams, + useResetWarehousePassword, useUpdateOrg, useUpdateWarehouse, useWarehouse, } from "@/hooks/useApi"; +import { Copyable } from "@/components/Copyable"; import { BackfillBadge, CreateTeamDialog, @@ -465,9 +479,16 @@ function WarehousePanel({ {loading ? ( ) : notFound || !data ? ( -

- No managed warehouse provisioned for this org. -

+
+

No managed warehouse provisioned for this org.

+ + + +
) : ( <> {/* Read-only provisioning states */} @@ -543,6 +564,10 @@ function WarehousePanel({ + {/* Root credential rotation — the recovery path when a provision + response was lost. Same endpoint the PostHog backend calls. */} + {data.state === "ready" && } + {/* Teardown + reshard */} {canDeprovision && (
@@ -616,6 +641,77 @@ function WarehousePanel({ ); } +// ResetRootPassword rotates the org's root login via POST +// /orgs/:id/reset-password — the same endpoint the PostHog backend calls, and +// the only way to recover from a lost provision response (duckgres stores only +// the bcrypt hash). The new plaintext is returned once and rendered here; it is +// never persisted client-side, so leaving the page loses it. +function ResetRootPassword({ orgId }: { orgId: string }) { + const reset = useResetWarehousePassword(orgId); + const [confirmOpen, setConfirmOpen] = useState(false); + const [issued, setIssued] = useState<{ username: string; password: string } | null>(null); + const [err, setErr] = useState(null); + + const run = async () => { + setErr(null); + try { + setIssued(await reset.mutateAsync()); + } catch (e) { + setErr(e instanceof Error ? e.message : "Reset failed"); + } + setConfirmOpen(false); + }; + + return ( +
+

Root credentials

+
+ + + + + Rotates the org's root login. Existing clients using the old + password stop authenticating. + +
+ {err &&

{err}

} + {issued && ( +
+

+ Shown once — duckgres stores only the bcrypt hash. Copy it now. +

+
+ + +
+
+ )} + + setConfirmOpen(o)}> + + + Reset the root password for "{orgId}"? + + The current root password stops working immediately and cannot be recovered. Any client + still using it will fail to authenticate until it is updated. + + + + + + + + +
+ ); +} + // ReshardHistory lists the org's reshard operations with links to each // operation's live overview/log page. Hidden while the org has none. function ReshardHistory({ orgId }: { orgId: string }) { diff --git a/controlplane/admin/ui/src/pages/Orgs.tsx b/controlplane/admin/ui/src/pages/Orgs.tsx index 7bf21a28..1f326b2d 100644 --- a/controlplane/admin/ui/src/pages/Orgs.tsx +++ b/controlplane/admin/ui/src/pages/Orgs.tsx @@ -1,9 +1,11 @@ import { useMemo, useState } from "react"; -import { useNavigate } from "react-router-dom"; +import { Link, useNavigate } from "react-router-dom"; import { type ColumnDef } from "@tanstack/react-table"; -import { AlertTriangle, Building2, Search } from "lucide-react"; +import { AlertTriangle, Building2, Rocket, Search } from "lucide-react"; import { PageBody, PageHeader } from "@/components/AppShell"; +import { AdminGate } from "@/components/AdminOnly"; import { DataTable } from "@/components/DataTable"; +import { Button } from "@/components/ui/button"; import { Input } from "@/components/ui/input"; import { Badge } from "@/components/ui/badge"; import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; @@ -135,14 +137,25 @@ export function Orgs() { title="Organizations" description="Tenants and their per-org limits. Click a row to edit config and warehouse." actions={ -
- - setFilter(e.target.value)} - placeholder="Filter orgs…" - className="w-64 pl-8" - /> +
+
+ + setFilter(e.target.value)} + placeholder="Filter orgs…" + className="w-64 pl-8" + /> +
+ {/* Provisioning goes through the same public API the PostHog + backend uses — see pages/ProvisionWarehouse.tsx. */} + + +
} /> diff --git a/controlplane/admin/ui/src/pages/ProvisionWarehouse.test.tsx b/controlplane/admin/ui/src/pages/ProvisionWarehouse.test.tsx new file mode 100644 index 00000000..12c5f7f0 --- /dev/null +++ b/controlplane/admin/ui/src/pages/ProvisionWarehouse.test.tsx @@ -0,0 +1,150 @@ +import { describe, expect, it, vi, beforeEach } from "vitest"; +import { fireEvent, render, screen } from "@testing-library/react"; +import { MemoryRouter } from "react-router-dom"; +import { TooltipProvider } from "@/components/ui/tooltip"; +import type { Org } from "@/types/api"; + +// Mock the data hooks so the form renders against a controlled org list and a +// spy-able provision mutation — the point of these tests is the REQUEST the +// page builds, since that request is what has to match the PostHog backend's. +const hooks = vi.hoisted(() => ({ + useOrgs: vi.fn(), + useProvisionWarehouse: vi.fn(), + useDatabaseNameAvailable: vi.fn(), + useWarehouseStatus: vi.fn(), +})); +vi.mock("@/hooks/useApi", () => hooks); + +const identity = vi.hoisted(() => ({ useIdentity: vi.fn() })); +vi.mock("@/components/IdentityProvider", () => identity); + +import { ProvisionWarehouse } from "./ProvisionWarehouse"; + +const ok = (data: T) => ({ data, isSuccess: true, isLoading: false, isError: false, refetch: vi.fn() }); + +const org = (name: string, warehouseState?: string): Org => + ({ + name, + database_name: name, + warehouse: warehouseState ? { state: warehouseState } : null, + }) as unknown as Org; + +function renderPage() { + render( + // AdminGate wraps a viewer's disabled control in a Tooltip, which needs the + // provider the real app mounts in main.tsx. + + + + + , + ); +} + +function type(label: RegExp, value: string) { + fireEvent.change(screen.getByLabelText(label), { target: { value } }); +} + +describe("ProvisionWarehouse page", () => { + let mutateAsync: ReturnType; + + beforeEach(() => { + vi.clearAllMocks(); + identity.useIdentity.mockReturnValue({ isAdmin: true, me: { email: "a@posthog.com", role: "admin", source: "sso" } }); + mutateAsync = vi.fn().mockResolvedValue({ + status: "provisioning started", + org: "acme", + username: "root", + password: "s3cr3t", + bucket: "posthog-duckling-acme-mw-dev-us", + }); + hooks.useProvisionWarehouse.mockReturnValue({ mutateAsync, isPending: false }); + hooks.useOrgs.mockReturnValue(ok([])); + hooks.useDatabaseNameAvailable.mockReturnValue(ok(null)); + hooks.useWarehouseStatus.mockReturnValue(ok(null)); + }); + + it("posts the standard PostHog-backend body to the shared provisioning endpoint", async () => { + renderPage(); + type(/org id/i, "acme"); + type(/database name/i, "acme_db"); + type(/team id/i, "42"); + + fireEvent.click(screen.getByRole("button", { name: /provision warehouse/i })); + // Destructive-action confirmation: the org id must be typed back. + type(/type the org id to confirm/i, "acme"); + fireEvent.click(screen.getByRole("button", { name: /^provision$/i })); + + expect(mutateAsync).toHaveBeenCalledWith({ + org: "acme", + body: { + database_name: "acme_db", + team_id: 42, + metadata_store: { type: "cnpg-shard" }, + data_store: { type: "s3bucket" }, + ducklake: { enabled: true }, + }, + }); + }); + + it("shows the literal request so the shared-endpoint claim is inspectable", () => { + renderPage(); + type(/org id/i, "acme"); + type(/database name/i, "acme_db"); + type(/team id/i, "42"); + + expect(screen.getByText(/POST \/api\/v1\/orgs\/acme\/provision/)).toBeInTheDocument(); + }); + + it("blocks submit until the form matches the server's rules", () => { + renderPage(); + // An upper-case org id is not a DNS-1123 label, and team_id is required for + // a new org — both are server 400s, surfaced before submit. + type(/org id/i, "ACME"); + type(/database name/i, "acme_db"); + + expect(screen.getByText(/DNS-1123/)).toBeInTheDocument(); + expect(screen.getByText(/team_id is required/)).toBeInTheDocument(); + expect(screen.getByRole("button", { name: /provision warehouse/i })).toBeDisabled(); + }); + + it("warns that an org with a live warehouse would be refused", () => { + hooks.useOrgs.mockReturnValue(ok([org("acme", "ready")])); + renderPage(); + type(/org id/i, "acme"); + + expect(screen.getByText(/deprovision it first/i)).toBeInTheDocument(); + }); + + it("flags a database name already taken by another org", () => { + hooks.useDatabaseNameAvailable.mockReturnValue(ok({ name: "taken_db", available: false })); + renderPage(); + type(/database name/i, "taken_db"); + + expect(screen.getByText(/already in use by another org/i)).toBeInTheDocument(); + }); + + it("surfaces the once-only root password after a successful provision", async () => { + renderPage(); + type(/org id/i, "acme"); + type(/database name/i, "acme_db"); + type(/team id/i, "42"); + fireEvent.click(screen.getByRole("button", { name: /provision warehouse/i })); + type(/type the org id to confirm/i, "acme"); + fireEvent.click(screen.getByRole("button", { name: /^provision$/i })); + + expect(await screen.findByText("s3cr3t")).toBeInTheDocument(); + expect(screen.getByText(/shown/i)).toBeInTheDocument(); + expect(screen.getByText("posthog-duckling-acme-mw-dev-us")).toBeInTheDocument(); + }); + + it("hides the provision affordance from viewers", () => { + identity.useIdentity.mockReturnValue({ isAdmin: false, me: { email: "v@posthog.com", role: "viewer", source: "sso" } }); + renderPage(); + type(/org id/i, "acme"); + type(/database name/i, "acme_db"); + type(/team id/i, "42"); + + expect(screen.getByRole("button", { name: /provision warehouse/i })).toBeDisabled(); + }); +}); diff --git a/controlplane/admin/ui/src/pages/ProvisionWarehouse.tsx b/controlplane/admin/ui/src/pages/ProvisionWarehouse.tsx new file mode 100644 index 00000000..177f4efc --- /dev/null +++ b/controlplane/admin/ui/src/pages/ProvisionWarehouse.tsx @@ -0,0 +1,476 @@ +import { useMemo, useState } from "react"; +import { Link, useNavigate } from "react-router-dom"; +import { AlertTriangle, ArrowLeft, Loader2, Rocket } from "lucide-react"; +import { PageBody, PageHeader } from "@/components/AppShell"; +import { Copyable } from "@/components/Copyable"; +import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card"; +import { Button } from "@/components/ui/button"; +import { Input } from "@/components/ui/input"; +import { Label } from "@/components/ui/label"; +import { Badge } from "@/components/ui/badge"; +import { StateBadge } from "@/components/StateBadge"; +import { AdminGate } from "@/components/AdminOnly"; +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, +} from "@/components/ui/dialog"; +import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from "@/components/ui/select"; +import { + buildProvisionBody, + DEFAULT_PROVISION_FORM, + validateProvisionForm, + type ProvisionForm, +} from "@/lib/provision"; +import { useDatabaseNameAvailable, useOrgs, useProvisionWarehouse, useWarehouseStatus } from "@/hooks/useApi"; +import type { ProvisionResult } from "@/types/api"; + +// Provision a managed warehouse from the console. +// +// This page is a FORM OVER THE PUBLIC PROVISIONING API — it POSTs to +// /api/v1/orgs/:id/provision, the exact endpoint (and handler, validation, +// transaction and analytics event) the PostHog backend calls when a customer +// provisions a warehouse. There is deliberately no operator-only provisioning +// path: a warehouse created here is indistinguishable from one a user created. +// The request preview below shows the literal call so that equivalence is +// visible, not just asserted. +export function ProvisionWarehouse() { + const navigate = useNavigate(); + const orgs = useOrgs(); + const provision = useProvisionWarehouse(); + + const [form, setForm] = useState(DEFAULT_PROVISION_FORM); + const [confirmOpen, setConfirmOpen] = useState(false); + const [confirmText, setConfirmText] = useState(""); + const [err, setErr] = useState(null); + const [result, setResult] = useState(null); + + const set = (k: K, v: ProvisionForm[K]) => + setForm((f) => ({ ...f, [k]: v })); + + const orgId = form.orgId.trim(); + // Re-provisioning an EXISTING org is legal (the server keeps its teams and + // only 409s while the warehouse row is non-terminal) and relaxes the team_id + // requirement — so the check drives validation, not just the warning below. + const existingOrg = useMemo(() => (orgs.data ?? []).find((o) => o.name === orgId), [orgs.data, orgId]); + const orgExists = existingOrg != null; + const liveWarehouse = existingOrg?.warehouse != null && existingOrg.warehouse.state !== "deleted"; + + const errs = validateProvisionForm(form, orgExists); + const valid = Object.keys(errs).length === 0; + const body = buildProvisionBody(form); + + const dbCheck = useDatabaseNameAvailable(form.databaseName); + const dbTaken = dbCheck.data != null && !dbCheck.data.available; + + const run = async () => { + setErr(null); + try { + const res = await provision.mutateAsync({ org: orgId, body }); + setResult(res); + setConfirmOpen(false); + setConfirmText(""); + } catch (e) { + setErr(e instanceof Error ? e.message : "provisioning failed"); + setConfirmOpen(false); + setConfirmText(""); + } + }; + + if (result) { + return ( + navigate(`/orgs/${encodeURIComponent(orgId)}`)} + /> + ); + } + + return ( + <> + + + Back to orgs + + + } + /> + + + + New managed warehouse + + +

+ This form posts to POST /api/v1/orgs/:id/provision — the same + endpoint, handler and validation the PostHog backend calls. Provisioning runs asynchronously; + the warehouse is not usable until it reports ready. +

+ + + set("orgId", e.target.value)} + placeholder="PostHog organization UUID, or a short slug" + className="font-mono text-xs" + /> + + {orgExists && ( + + {liveWarehouse ? ( + <> + Org {orgId} already has a warehouse in state{" "} + {existingOrg?.warehouse?.state}. Provisioning is + refused (409) unless it is failed or{" "} + deleted — deprovision it first. + + ) : ( + <> + Org {orgId} already exists; this re-provisions its + warehouse. Its existing teams are kept, and team_id{" "} + becomes optional. + + )} + + )} + + + set("databaseName", e.target.value)} + placeholder="the dbname clients connect to" + className="font-mono text-xs" + /> + + {form.databaseName.trim() !== "" && dbCheck.data != null && ( + + {dbTaken ? ( + <> + Database name {dbCheck.data.name} is already in use by + another org — the provision would be rejected with 409. + + ) : ( + <> + Database name {dbCheck.data.name} is available. + + )} + + )} + +
+ + set("teamId", e.target.value)} + placeholder="PostHog Team.id" + className="font-mono text-xs" + /> + + + set("schemaName", e.target.value)} + placeholder="defaults to team_" + className="font-mono text-xs" + /> + +
+ +
+ + +

+ Where the DuckLake catalog lives. The cnpg composition picks the active shard itself. +

+
+ + {form.metadataType === "external" && ( +
+ + set("externalEndpoint", e.target.value)} + placeholder="the RDS host" + className="font-mono text-xs" + /> + + + set("externalSecret", e.target.value)} + placeholder="Secrets Manager secret NAME" + className="font-mono text-xs" + /> + + + set("externalUser", e.target.value)} + className="font-mono text-xs" + /> + + + set("externalDatabase", e.target.value)} + className="font-mono text-xs" + /> + +
+ )} + +
+ + +
+ + {form.dataStoreType === "external" && ( +
+ + set("bucketName", e.target.value)} + className="font-mono text-xs" + /> + + + set("region", e.target.value)} + placeholder="composition default" + className="font-mono text-xs" + /> + +
+ )} + +
+ ducklake enabled + + Always on — the server rejects a warehouse without a catalog. + +
+ + {/* The literal request. Makes the "same call as the PostHog backend" + claim inspectable, and doubles as a copy/paste curl body when an + operator would rather run it by hand. */} +
+ + 
+                {`POST /api/v1/orgs/${orgId || ":id"}/provision\n${JSON.stringify(body, null, 2)}`}
+
+
+ + {err && ( +

+ + {err} +

+ )} + +
+ + + + {!valid && ( + Fix the highlighted fields first. + )} +
+ + + + + (o ? setConfirmOpen(true) : setConfirmOpen(false))}> + + + Provision a warehouse for "{orgId}"? + + This creates real infrastructure — a Duckling CR, an S3 bucket, a metadata database and an IAM + role — and bills the org for it. The root password is returned once and cannot be read back + afterwards. + + + + setConfirmText(e.target.value)} + placeholder={orgId} + className="font-mono text-xs" + /> + + + + + + + + + ); +} + +// ProvisionStarted is the post-202 view: the once-only root password, the +// control-plane-owned bucket name, and the live lifecycle poll. The password is +// deliberately not persisted anywhere in the client — navigating away loses it, +// and the recovery path is the reset-password action on the org page. +function ProvisionStarted({ + org, + result, + onDone, +}: { + org: string; + result: ProvisionResult; + onDone: () => void; +}) { + const status = useWarehouseStatus(org); + const state = status.data?.state; + const done = state === "ready" || state === "failed"; + + return ( + <> + + Open org + + } + /> + + + + + {done ? null : } + Root credentials + + + + + The password below is shown once. Duckgres stores only its bcrypt hash, so it + cannot be retrieved later — copy it now. If it is lost, rotate it with "Reset root password" on + the org page once the warehouse is ready. + +
+ + + {result.bucket && } + +
+ +
+

+ Provisioning state +

+
+ + {status.data?.status_message && ( + + {status.data.status_message} + + )} +
+

+ Polled from GET /orgs/{org}/warehouse/status — the same + lifecycle view the PostHog backend polls. A cold provision typically takes several minutes. +

+
+ +
+ +
+ + + + + ); +} + +// Field binds its Label to the control via htmlFor/id — not decoration: +// screen readers (and the page tests) resolve an input by its label text. +function Field({ + id, + label, + error, + children, +}: { + id: string; + label: string; + error?: string; + children: React.ReactNode; +}) { + return ( +
+ + {children} + {error &&

{error}

} +
+ ); +} + +function Note({ kind, children }: { kind: "info" | "warn"; children: React.ReactNode }) { + return ( +

+ {kind === "warn" && } + {children} +

+ ); +} diff --git a/controlplane/admin/ui/src/types/api.ts b/controlplane/admin/ui/src/types/api.ts index 67d16cfe..8aa69b0a 100644 --- a/controlplane/admin/ui/src/types/api.ts +++ b/controlplane/admin/ui/src/types/api.ts @@ -240,6 +240,99 @@ export interface ManagedWarehouse { updated_at: string; } +// ---- Warehouse provisioning (the PostHog-backend contract) ---- +// +// These mirror controlplane/provisioning/api.go EXACTLY. The console posts the +// same body to the same endpoint the PostHog backend (Django) posts to, so an +// operator-provisioned warehouse and a user-provisioned one are identical by +// construction. Do not add console-only fields here — the server would ignore +// them, which is precisely the divergence this shared contract prevents. + +export interface ProvisionMetadataStore { + // "cnpg-shard": the composition picks the active shard and provisions a + // per-tenant role+database on it. "external": a pre-existing Postgres. + type: "cnpg-shard" | "external"; + external?: { + // RDS host. + endpoint: string; + // AWS Secrets Manager secret NAME holding the password (never the value). + password_aws_secret: string; + // Both default to "postgres" at the XRD when omitted. + user?: string; + database?: string; + }; +} + +export interface ProvisionDataStore { + // "s3bucket" provisions a fresh per-org bucket (the control plane names it); + // "external" reuses an existing bucket and then requires bucket_name. + type: "s3bucket" | "external"; + bucket_name?: string; + region?: string; +} + +export interface ProvisionBody { + database_name: string; + // The org's first PostHog team. REQUIRED when the provision creates a NEW + // org (the server 400s otherwise — a warehouse cannot exist without a team). + team_id?: number; + // Optional override of the conventional "team_" warehouse schema for + // that first team row; requires team_id. + schema_name?: string; + metadata_store: ProvisionMetadataStore; + data_store?: ProvisionDataStore; + // Must be true — a warehouse without a catalog has nothing to attach. + ducklake: { enabled: boolean }; +} + +// 202 response of POST /orgs/:id/provision. `password` is the freshly generated +// root plaintext and is returned HERE ONLY — it is never stored (only its +// bcrypt hash is) and cannot be read back. `bucket` is the authoritative +// control-plane-owned bucket name (absent for an external data store, or when +// CP-side naming is disabled). +export interface ProvisionResult { + status: string; + org: string; + username: string; + password: string; + bucket?: string; +} + +// GET /orgs/:id/warehouse/status — the same lifecycle view the PostHog backend +// polls after provisioning. `connection` appears only once the state is ready +// and never carries a password. +export interface WarehouseStatus { + org_id: string; + state: WarehouseState; + status_message: string; + s3_state: WarehouseState; + metadata_store_state: WarehouseState; + identity_state: WarehouseState; + secrets_state: WarehouseState; + ready_at?: string | null; + failed_at?: string | null; + connection?: { + host: string; + port: number; + database: string; + username: string; + } | null; + bucket?: string; +} + +// GET /database-name/check?name=… — global uniqueness pre-flight for the form. +export interface DatabaseNameCheck { + name: string; + available: boolean; +} + +// POST /orgs/:id/reset-password — rotates the org's root login and returns the +// new plaintext once. The recovery path when a provision response was lost. +export interface ResetPasswordResult { + username: string; + password: string; +} + // ---- Duckling drift (admin-only) ---- // One mismatch between a managed warehouse row and its Duckling custom resource. diff --git a/controlplane/provisioning/api_topology_test.go b/controlplane/provisioning/api_topology_test.go new file mode 100644 index 00000000..3e6dbb34 --- /dev/null +++ b/controlplane/provisioning/api_topology_test.go @@ -0,0 +1,50 @@ +package provisioning + +import ( + "sort" + "strings" + "testing" + + "github.com/gin-gonic/gin" +) + +// TestProvisioningAPIRouteTopology pins the exact route set RegisterAPI mounts. +// +// This is the "one provisioning path" tripwire. The PostHog backend (Django) +// and the admin console both create warehouses by POSTing to +// /api/v1/orgs/:id/provision — the SAME gin route, the same handler, the same +// validation, the same transaction. That is what makes an operator-provisioned +// warehouse byte-for-byte equivalent to a user-provisioned one; there is +// deliberately no second, console-only provisioning implementation to drift +// from this one (see TestAdminAPIRegistersNoProvisioningRoutes on the admin +// side, which pins the absence). +// +// A change here means the public provisioning contract moved: update the +// PostHog backend client, the admin console's api.ts, and the e2e harness in +// the same PR. +func TestProvisioningAPIRouteTopology(t *testing.T) { + gin.SetMode(gin.TestMode) + r := gin.New() + RegisterAPI(r.Group("/api/v1"), newFakeStore(), "") + + want := []string{ + "DELETE /api/v1/orgs/:id/teams/:team_id", + "GET /api/v1/database-name/check", + "GET /api/v1/orgs/:id/teams", + "GET /api/v1/orgs/:id/warehouse/status", + "POST /api/v1/orgs/:id/deprovision", + "POST /api/v1/orgs/:id/provision", + "POST /api/v1/orgs/:id/reset-password", + "POST /api/v1/orgs/:id/teams", + } + + var got []string + for _, ri := range r.Routes() { + got = append(got, ri.Method+" "+ri.Path) + } + sort.Strings(got) + + if strings.Join(got, "\n") != strings.Join(want, "\n") { + t.Fatalf("provisioning route topology changed:\ngot:\n%s\n\nwant:\n%s", strings.Join(got, "\n"), strings.Join(want, "\n")) + } +} diff --git a/docs/design/admin-ui.md b/docs/design/admin-ui.md index 3916ee6e..c8579563 100644 --- a/docs/design/admin-ui.md +++ b/docs/design/admin-ui.md @@ -107,7 +107,15 @@ for live views), TanStack Table (dense sortable tables), Recharts (trends). Page 1. **Overview** — fleet vCPU/mem, worker counts by lifecycle state, hot-idle + queue depth, leader/replica health, cluster-wide query rate + error %. 2. **Organizations** — list + detail; edit org config (max_workers, connections, default - worker cpu/mem/ttl, hot-idle floor, hostname alias); managed-warehouse view/edit. + worker cpu/mem/ttl, hot-idle floor, hostname alias); managed-warehouse view/edit; + **provision a warehouse** (`/orgs/provision`), deprovision, and rotate the root + password. The provisioning page is a form over the PUBLIC provisioning API — it posts + to `POST /api/v1/orgs/:id/provision`, the same route/handler/validation/transaction the + PostHog backend calls, so an operator-created warehouse is identical to a user-created + one. There is deliberately **no operator-only provisioning path**; the form even renders + the literal request it will send. See `controlplane/admin/README.md` ("Warehouse + provisioning — one implementation, two callers") for the invariants and the topology + tripwire tests that pin the absence of a second implementation. 3. **Users** — per-org users CRUD; persistent secrets list/delete. 4. **Live** — running queries / sessions / connections, sliced + filterable by org & user, with progress bars (SessionProgress), a running-query **Duration** column diff --git a/tests/mw-dev/README.md b/tests/mw-dev/README.md index a37ff42b..bef010ec 100644 --- a/tests/mw-dev/README.md +++ b/tests/mw-dev/README.md @@ -141,6 +141,22 @@ client-go: reused hot-idle worker (cnpg lane). - **isolation** — two CNPG-backed tenants see distinct catalogs; a cross-tenant read is denied. +- **admin-console provisioning parity** (`admin_provision_parity`) — the console's + Provision-warehouse form has no provisioning implementation of its own: it posts + to the same `/api/v1/orgs/:id/provision` route the PostHog backend uses. The + console's exact request body (cnpg-shard + s3bucket + `ducklake.enabled=true` + + `team_id`), replayed against the provisioned org, is refused with the + warehouse-exists **409** — proving it passed the shared validation and reached + the shared store rather than tripping a shape 400; `ducklake.enabled=false` is a + 400 (why the form offers no toggle); and the org's real provision is recorded in + the admin audit log as `warehouse.provision` (with the teardown lane asserting + `warehouse.deprovision`), never the generic `org.create` these paths used to + fall through to. The SPA itself can't be driven from the in-cluster Job (no + browser) — the form's body construction and validation mirror are covered by + `ui/src/lib/provision.test.ts` + `ui/src/pages/ProvisionWarehouse.test.tsx`, and + the "no second provisioning path" topology by + `provisioning.TestProvisioningAPIRouteTopology` + + `admin.TestAdminAPIRegistersNoProvisioningRoutes`. - **lifecycle** — deprovision → `warehouse=deleted` → the Crossplane Duckling CR **fully** deletes (`kubectl wait --for=delete`, asserting the finalizer cascade that drops the cnpg role+db completed). Right after the deprovision diff --git a/tests/mw-dev/e2e/harness.sh b/tests/mw-dev/e2e/harness.sh index 0882a962..328a30c8 100755 --- a/tests/mw-dev/e2e/harness.sh +++ b/tests/mw-dev/e2e/harness.sh @@ -1353,6 +1353,62 @@ provision_team_contract() { # org provisioned_team_id log "provision team OK: legacy default_team_id on the admin PUT is accepted and ignored" } +# ---- admin-console provisioning parity ------------------------------------- +# Operators provision warehouses from the admin console +# (ui/src/pages/ProvisionWarehouse.tsx). The console does NOT have its own +# provisioning implementation: it POSTs the same body to the same +# /api/v1/orgs/:id/provision route the PostHog backend uses, so a warehouse an +# operator creates is identical to one a user creates. The SPA can't be driven +# from this in-cluster Job (no browser), so this asserts the two backend facts +# that make the parity real and would break silently if it regressed: +# +# 1. The body the console builds (buildProvisionBody: cnpg-shard metadata, +# s3bucket data store, ducklake enabled, team_id) reaches the real +# handler — replayed against the already-provisioned org it is refused +# with the warehouse-exists 409, NOT a 400 shape/validation error. A +# console-only request shape would 400 here. +# 2. Provisioning is recorded in the admin audit log as +# "warehouse.provision" (the org's own provision, done at the top of this +# run through the very same route). Before the audit mapping existed, +# /provision, /deprovision and /reset-password all fell through to the +# generic "org.create" — indistinguishable from an org-config write in +# the console's audit view. +# +# Also pins the reason the console hard-codes ducklake.enabled=true: sending +# false is a 400, so a toggle would only ever offer a guaranteed failure. +admin_provision_parity() { # org team_id + org="$1"; team="$2" + log "admin console provisioning parity on $org" + + # 1. The console's request body hits the shared handler (409 = warehouse + # exists, i.e. it passed validation and reached the store). + console_body='{"database_name":"'"$org"'","team_id":'"$team"',"metadata_store":{"type":"cnpg-shard"},"data_store":{"type":"s3bucket"},"ducklake":{"enabled":true}}' + code="$(curl -s -o /tmp/parity_out -w '%{http_code}' -X POST -H "$H" -H 'Content-Type: application/json' \ + -d "$console_body" "$API/api/v1/orgs/$org/provision")" + [ "$code" = "409" ] \ + || fail "provision parity: console body -> HTTP $code want 409 (existing warehouse): $(cat /tmp/parity_out)" + grep -q "non-terminal" /tmp/parity_out \ + || fail "provision parity: 409 should name the non-terminal warehouse: $(cat /tmp/parity_out)" + + # ducklake:false is rejected — why the console offers no toggle. + nodl='{"database_name":"'"$org"'","team_id":'"$team"',"metadata_store":{"type":"cnpg-shard"},"data_store":{"type":"s3bucket"},"ducklake":{"enabled":false}}' + code="$(curl -s -o /tmp/parity_nodl -w '%{http_code}' -X POST -H "$H" -H 'Content-Type: application/json' \ + -d "$nodl" "$API/api/v1/orgs/$org/provision")" + [ "$code" = "400" ] || fail "provision parity: ducklake.enabled=false -> HTTP $code want 400: $(cat /tmp/parity_nodl)" + + # 2. The org's real provision is audited under the warehouse.provision action. + audit="$(curl -fsS -H "$H" "$API/api/v1/audit?org=$org&limit=500")" \ + || fail "provision parity: GET /audit?org=$org failed" + echo "$audit" | jq -e --arg o "$org" \ + 'any(.entries[]?; .action=="warehouse.provision" and .method=="POST" and .org==$o and (.path | endswith("/provision")))' >/dev/null \ + || fail "provision parity: no warehouse.provision audit entry for $org: $(echo "$audit" | jq -c '[.entries[]?|{action,method,path}]' | head -c 600)" + # Regression net for the old generic mapping: a /provision path must never be + # recorded as an org write. + echo "$audit" | jq -e 'any(.entries[]?; (.path | endswith("/provision")) and .action != "warehouse.provision") | not' >/dev/null \ + || fail "provision parity: a /provision request was audited under a non-warehouse.provision action" + log "provision parity OK: shared endpoint + warehouse.provision audit on $org" +} + # ---- org teams CRUD (duckgres_org_teams via the provisioning API) ----------- # The PostHog backend manages an org's team rows through # GET/POST /orgs/:id/teams + DELETE /orgs/:id/teams/:team_id. Contract asserted @@ -3007,6 +3063,16 @@ lifecycle_teardown_cnpg() { # org fi [ "$(curl -fsS -H "$H" "$API/api/v1/database-name/check?name=$dbname" | jq -r '.available')" = "true" ] \ || fail "database_name $dbname still squatted after org deletion" + + # The console drives teardown through this same route, so the deprovision + # must read as a warehouse lifecycle action in the audit log rather than + # falling through to the generic org write (see admin_provision_parity). + # Filter by path, not org: the org row is gone by now, but audit rows are + # append-only and keep the org they were recorded against. + curl -fsS -H "$H" "$API/api/v1/audit?limit=500" \ + | jq -e --arg p "/api/v1/orgs/$1/deprovision" \ + 'any(.entries[]?; .path==$p and .action=="warehouse.deprovision" and .method=="POST")' >/dev/null \ + || fail "deprovision of $1 was not audited as warehouse.deprovision" } # ---- cnpg duckling: cnpg-shard metadata + DuckLake ------------------------ @@ -3363,6 +3429,7 @@ main() { # Provisioning/admin contracts are backend-independent; retain their live # coverage on the primary CNPG org. provision_team_contract "$CNPG" "$CNPG_TEAM_ID" + admin_provision_parity "$CNPG" "$CNPG_TEAM_ID" org_teams_crud "$CNPG" "$CNPG_TEAM_ID" discovery_endpoints "$CNPG" "$CNPG_TEAM_ID"