diff --git a/docs/README.md b/docs/README.md index f566bc7..d793ef3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,6 +14,7 @@ design reasoning. - [Add a conversion event](development/how-to/add-a-conversion-event.md) - [Update the repository standard](development/how-to/update-the-repository-standard.md) - [Upgrade consumer repositories](development/how-to/upgrade-consumers.md) +- [Set up the analytics production](operations/how-to/set-up-production.md) - [Verify a deployment](operations/how-to/verify-a-deployment.md) - [Read an analytics report](operations/how-to/read-a-report.md) - [Rotate the Web Analytics token](operations/how-to/rotate-the-cwa-token.md) diff --git a/docs/operations/how-to/set-up-production.md b/docs/operations/how-to/set-up-production.md new file mode 100644 index 0000000..8b4752e --- /dev/null +++ b/docs/operations/how-to/set-up-production.md @@ -0,0 +1,147 @@ +# Set up the analytics production + +This guide sets up everything the analytics repository needs in production, starting from nothing: +the GitHub `production` environment, the two Cloudflare tokens, the collector Worker at +`events.lasvegasfortransit.org` with its secret, and the weekly report. Every step starts by +checking whether it is already done, so you can follow the guide again at any time; a step that is +done is skipped, and nothing is replaced unless a step says so. Replacing a value on purpose is +covered in [Rotate the collector secret](rotate-the-events-secret.md). + +Once this repository moves to repository tooling 0.4.2 or later, a `platform.json` and +`pnpm bootstrap --production` will check and set up most of this for you, and this guide will point +to them. + +## Before you start + +- `pnpm bootstrap` passes on your machine, so the GitHub CLI and Wrangler are signed in. +- You are an admin of `LasVegasForTransit/analytics`, so you can create environments and secrets. +- You have the Super Administrator role on the LVBT Cloudflare account, "Las Vegans for Better + Transit" (ID `2557b5c2e166292ded0f8425b73075e9`), because the tokens below belong to that account + rather than to you. Everything here is created in that account, never in a personal one. + +Never paste a token or secret into a command line or a chat. Each step below pastes it into a prompt +that hides it, or into a GitHub or Cloudflare field. + +## 1. Create the GitHub environment + +The collector's Deploy workflow reads its secrets from an environment named `production`. + +1. Check: open the repository's Settings, then Environments. If `production` is listed, go to + step 2. +2. Click "New environment", type `production`, and click "Configure environment". + +## 2. Store the account ID + +The account ID is not secret, but both workflows read it as a secret, in two places: the +`production` environment (for the Deploy workflow) and the repository itself (for the weekly report, +which runs without an environment). Check first: + +```bash +gh secret list --env production +gh secret list +``` + +For each list that lacks `CLOUDFLARE_ACCOUNT_ID`, run the matching command and paste +`2557b5c2e166292ded0f8425b73075e9` at its prompt: + +```bash +gh secret set CLOUDFLARE_ACCOUNT_ID --env production +gh secret set CLOUDFLARE_ACCOUNT_ID +``` + +## 3. Create the deploy token + +The Deploy workflow uses `CLOUDFLARE_API_TOKEN` in the `production` environment to publish the +collector Worker and attach its custom domain. If `gh secret list --env production` already shows +`CLOUDFLARE_API_TOKEN`, skip this section. + +It is an account API token, so deploys keep working after the person who made it leaves. Wrangler +accepts it because the workflow also sets `CLOUDFLARE_ACCOUNT_ID`. + +1. In the Cloudflare dashboard, choose "Las Vegans for Better Transit" and go to Manage Account, + then "Account API Tokens" + (). Click "Create + Token", then "Create Custom Token". +2. Name it `analytics collector deploy (GitHub Actions)`. +3. Under "Permissions", add these rows: "Account", "Workers Scripts", "Edit" (uploads the Worker and + attaches its custom domain); "Account", "Account Settings", "Read"; "Zone", "Zone", "Read" (finds + the zone for the custom domain); and "Zone", "Workers Routes", "Edit". +4. Under "Zone Resources", choose "Include", then "Specific zone", then `lasvegasfortransit.org`. +5. Leave the expiration empty, so deploys keep working. Click "Continue to summary", then "Create + Token". +6. Run `gh secret set CLOUDFLARE_API_TOKEN --env production` in a terminal and leave it waiting at + its prompt. +7. Copy the token (Cloudflare shows it only once), paste it at that prompt, and press Enter. + +## 4. Create the report token + +The weekly report reads the collector's Analytics Engine data through Cloudflare's SQL API with +`CLOUDFLARE_ANALYTICS_READ_TOKEN`, a repository secret. It can read analytics and nothing else. If +`gh secret list` already shows `CLOUDFLARE_ANALYTICS_READ_TOKEN`, skip this section. + +1. On the same "Account API Tokens" page, click "Create Token", then "Create Custom Token". +2. Name it `analytics weekly report (GitHub Actions)`. +3. Under "Permissions", add one row: "Account", "Account Analytics", "Read". Add nothing else. +4. Leave the expiration empty, click "Continue to summary", then "Create Token". +5. Run `gh secret set CLOUDFLARE_ANALYTICS_READ_TOKEN` and leave it waiting at its prompt. +6. Copy the token, paste it at that prompt, and press Enter. + +## 5. Deploy the collector and its custom domain + +The collector's `apps/collector/wrangler.jsonc` declares `events.lasvegasfortransit.org` as a custom +domain. Deploying creates the domain's DNS record and certificate, and redeploying an unchanged +Worker changes nothing. + +1. Check: open . If it answers, go to section 6. +2. If the Cloudflare dashboard's DNS records for `lasvegasfortransit.org` already have a CNAME named + `events`, delete it first: a custom domain cannot replace an existing CNAME. +3. In the repository's Actions tab, open "Deploy collector" and click "Run workflow" on `main`. It + runs `pnpm check`, deploys, and then checks `/health`. + +## 6. Set the collector secret + +`LVBT_EVENTS_SECRET` lets a website's server prove that a conversion event came from it; browser +events do not use it. It is 32 random bytes written as 64 hexadecimal characters. Setup makes it +without ever showing it, and it never needs to be copied. + +1. Check: + + ```bash + pnpm --filter @lasvegasfortransit/analytics-collector exec wrangler secret list + ``` + + If `LVBT_EVENTS_SECRET` is listed, go to section 7. Do not set it again: a new value would lock + out every server that sends events. + +2. Make it and store it on the collector in one step, so it never appears on screen: + + ```bash + openssl rand -hex 32 | pnpm --filter @lasvegasfortransit/analytics-collector exec wrangler secret put LVBT_EVENTS_SECRET + ``` + +No website sends server events yet, so the collector is the only place that holds the value. When a +website first needs it, follow [Rotate the collector secret](rotate-the-events-secret.md): it makes +one new value and stores it on the collector and on that website together, because a stored value +cannot be read back to copy. + +## 7. Turn on the weekly report + +The weekly report runs every Monday only while the repository variable `LVBT_ANALYTICS_LIVE` is `1`. +It checks that each LVBT site still loads analytics and updates the pinned "LVBT analytics weekly +report" issue. + +1. Check: run `gh variable list`. If `LVBT_ANALYTICS_LIVE` is `1`, go to step 3. +2. Run `gh variable set LVBT_ANALYTICS_LIVE --body 1`. The value is not secret. +3. In the Actions tab, open "Analytics weekly report" and click "Run workflow" once to confirm it + passes. + +## 8. Confirm + +- answers. +- `gh secret list --env production` shows `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_API_TOKEN`. +- `gh secret list` shows `CLOUDFLARE_ACCOUNT_ID` and `CLOUDFLARE_ANALYTICS_READ_TOKEN`. +- `gh variable list` shows `LVBT_ANALYTICS_LIVE` set to `1`. +- The "LVBT analytics weekly report" issue has this week's date. + +If a token ever leaks, roll it on the "Account API Tokens" page, run its `gh secret set` command, +and paste the new token at the prompt before doing anything else. diff --git a/packages/analytics/src/cli/commands.ts b/packages/analytics/src/cli/commands.ts index 8f49f23..b4d1c7a 100644 --- a/packages/analytics/src/cli/commands.ts +++ b/packages/analytics/src/cli/commands.ts @@ -22,7 +22,11 @@ export async function checkCsp(path: string) { return checkHeadersFile(path); } -export async function writeCsp(path: string) { +export interface WriteCspResult { + changed: boolean; +} + +export async function writeCsp(path: string): Promise { const contents = await readFile(path, 'utf8'); const pattern = /^(\s*Content-Security-Policy:\s*)(.+)$/gim; if (!pattern.test(contents)) @@ -31,7 +35,9 @@ export async function writeCsp(path: string) { const updated = contents.replace(pattern, (_line, prefix: string, policy: string) => { return `${prefix}${csp.merge(policy)}`; }); + if (updated === contents) return { changed: false }; await writeFile(path, updated); + return { changed: true }; } export async function writeClient(path: string) { diff --git a/packages/analytics/src/cli/index.ts b/packages/analytics/src/cli/index.ts index e1d1762..d554200 100644 --- a/packages/analytics/src/cli/index.ts +++ b/packages/analytics/src/cli/index.ts @@ -12,6 +12,22 @@ function usage(): never { ); } +async function runCsp(args: string[]) { + const checkPath = value(args, '--check'); + if (checkPath) { + const errors = await checkCsp(checkPath); + if (errors.length > 0) throw new Error(errors.join('\n')); + return; + } + const writePath = value(args, '--write') ?? usage(); + const result = await writeCsp(writePath); + process.stdout.write( + result.changed + ? `Updated the Content-Security-Policy header in ${writePath}.\n` + : `The Content-Security-Policy header in ${writePath} already includes everything analytics needs; no change made.\n`, + ); +} + export async function run(args = process.argv.slice(2)) { const [command, operand] = args; switch (command) { @@ -19,16 +35,9 @@ export async function run(args = process.argv.slice(2)) { if (!args.includes('--markdown')) usage(); process.stdout.write(eventsMarkdown()); return; - case 'csp': { - const checkPath = value(args, '--check'); - if (checkPath) { - const errors = await checkCsp(checkPath); - if (errors.length > 0) throw new Error(errors.join('\n')); - return; - } - await writeCsp(value(args, '--write') ?? usage()); + case 'csp': + await runCsp(args); return; - } case 'client': await writeClient(value(args, '--out') ?? usage()); return; diff --git a/packages/analytics/tests/cli.test.ts b/packages/analytics/tests/cli.test.ts index b749724..f1636fe 100644 --- a/packages/analytics/tests/cli.test.ts +++ b/packages/analytics/tests/cli.test.ts @@ -1,7 +1,13 @@ -import { readFile } from 'node:fs/promises'; -import { resolve } from 'node:path'; +import { mkdtemp, readFile, stat, writeFile } from 'node:fs/promises'; +import os from 'node:os'; +import path, { resolve } from 'node:path'; import { expect, test, vi } from 'vitest'; -import { eventsMarkdown, verifyDeployment, type VerifyPage } from '../src/cli/commands.js'; +import { + eventsMarkdown, + verifyDeployment, + writeCsp, + type VerifyPage, +} from '../src/cli/commands.js'; test('renders the event reference from the runtime allowlist', () => { const markdown = eventsMarkdown(); @@ -178,3 +184,37 @@ test('verifies excluded deployments by observing no analytics requests', async ( verifyDeployment('https://preview.example', 'labs.lasvegasfortransit.org', 'absent', withPage), ).resolves.toBeUndefined(); }); + +test('writes the merged CSP once; a second run leaves the file untouched', async () => { + const directory = await mkdtemp(path.join(os.tmpdir(), 'lvbt-analytics-csp-')); + const file = path.join(directory, '_headers'); + await writeFile(file, "/*\n Content-Security-Policy: default-src 'self'; object-src 'none'\n"); + + const first = await writeCsp(file); + expect(first.changed).toBe(true); + const contentsAfterFirst = await readFile(file, 'utf8'); + expect(contentsAfterFirst).toContain('https://static.cloudflareinsights.com'); + expect(contentsAfterFirst).toContain('https://events.lasvegasfortransit.org'); + const mtimeAfterFirst = (await stat(file)).mtimeMs; + + // A real filesystem's mtime resolution can be coarser than the gap between + // these two writes, so wait past it — otherwise an unwanted second write + // could coincidentally land on the same tick and hide as "unchanged". + await new Promise((resolveTimeout) => setTimeout(resolveTimeout, 20)); + + const second = await writeCsp(file); + expect(second.changed).toBe(false); + const contentsAfterSecond = await readFile(file, 'utf8'); + const mtimeAfterSecond = (await stat(file)).mtimeMs; + + expect(contentsAfterSecond).toBe(contentsAfterFirst); + expect(mtimeAfterSecond).toBe(mtimeAfterFirst); +}); + +test('rejects a headers file with no Content-Security-Policy header', async () => { + const directory = await mkdtemp(path.join(os.tmpdir(), 'lvbt-analytics-csp-')); + const file = path.join(directory, '_headers'); + await writeFile(file, '/*\n X-Content-Type-Options: nosniff\n'); + + await expect(writeCsp(file)).rejects.toThrow('No Content-Security-Policy header was found'); +});