Implement Cedar Park council scraper

apps/scraper/README.md

@@ -4,15 +4,16 @@ Pulls in government content like bills, court cases, and White House content and
 
 ## Active data sources
 
-Only these five are registered and run by `all`:
+Only these six are registered and run by `all`:
 
-| CLI name            | Source and data fetched                                                                    | Stored/used as                                                                                        |
-| ------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------- |
-| `federalregister`   | Federal Register API presidential documents, then each document's body HTML                | `government_content`; AI article/summary and feed-image enrichment                                    |
-| `congress`          | Congress.gov API bill list, detail, CRS summaries, formatted text, and legislative actions | `bill`; powers federal bill content and AI/feed enrichment                                            |
-| `scotus`            | CourtListener opinion clusters, dockets, and sub-opinion text for the Supreme Court        | `court_case`; powers court content and AI/feed enrichment                                             |
-| `scc-cvig`          | Hand-configured Santa Clara County voter-guide PDFs                                        | Candidate statements in `CivicApiCache`; the API matches statements to candidates                     |
-| `ca-sos-statements` | California SOS statewide-office candidate-statement pages                                  | Candidate statements in `CivicApiCache`; the API reads the cache and can fall back to the live source |
+| CLI name             | Source and data fetched                                                                    | Stored/used as                                                                                        |
+| -------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------- |
+| `federalregister`    | Federal Register API presidential documents, then each document's body HTML                | `government_content`; AI article/summary and feed-image enrichment                                    |
+| `congress`           | Congress.gov API bill list, detail, CRS summaries, formatted text, and legislative actions | `bill`; powers federal bill content and AI/feed enrichment                                            |
+| `scotus`             | CourtListener opinion clusters, dockets, and sub-opinion text for the Supreme Court        | `court_case`; powers court content and AI/feed enrichment                                             |
+| `scc-cvig`           | Hand-configured Santa Clara County voter-guide PDFs                                        | Candidate statements in `CivicApiCache`; the API matches statements to candidates                     |
+| `ca-sos-statements`  | California SOS statewide-office candidate-statement pages                                  | Candidate statements in `CivicApiCache`; the API reads the cache and can fall back to the live source |
+| `cedar-park-council` | Cedar Park's CivicEngage City Council page and its official Municode Meetings embed        | Provider-neutral local meetings, documents, agenda items, motions, outcomes, and votes                |
 
 `vote411`, `ca-lao-fiscal`, and `ca-vig-archive` remain under
 `src/scrapers/disabled/` and do not run. Their caches had no application
@@ -124,6 +125,7 @@ CONGRESS_MAX_ITEMS=10 pnpm --filter @acme/scraper run start congress
 | `SCOTUS_MAX_ITEMS`              |      50 | CourtListener opinion clusters                      |
 | `SCC_CVIG_MAX_ITEMS`            |      10 | Voter-guide PDF documents                           |
 | `CA_SOS_MAX_ITEMS`              |       9 | Statewide-office candidate-statement pages          |
+| `CEDAR_PARK_COUNCIL_MAX_ITEMS`  |     100 | Council meetings (after the 12-month cutoff)        |
 | `SCRAPER_MAX_NEW_ITEMS_PER_RUN` |      10 | New records receiving expensive AI/image enrichment |
 
 These are per-run limits, not durable calendar-day quotas. Schedule one run per
@@ -132,6 +134,28 @@ invocation gets a fresh allowance. Source limits cap API/page work;
 `SCRAPER_MAX_NEW_ITEMS_PER_RUN` separately caps expensive enrichment while
 still storing additional raw records for later backfill.
 
+## Cedar Park City Council (`civicengage.ts`)
+
+Cedar Park's public site is CivicEngage, but the City Council records page now
+embeds a keyless Municode Meetings publish page. The adapter follows that
+official embed, keeps a 12-month Council-only window, downloads at most two
+documents concurrently, and deterministically parses PDF text. AI/vision is not
+used. Agendas, packets, minutes, and later document URLs are versioned beneath
+one provider-neutral meeting record; unchanged reruns produce the same natural
+keys and checksums.
+
+```bash
+pnpm --filter @acme/scraper run start cedar-park-council --max-items 1
+```
+
+To add a second CivicEngage jurisdiction using the same kind of official
+Municode embed, add a `CivicEngageJurisdictionConfig` beside
+`cedarParkCouncilSource` with its CivicEngage host/path, IANA timezone,
+Municode `cid`/`ppid`, and governing-body matcher, then instantiate the same
+discovery/parser pipeline. If its records page uses Agenda Center, Legistar, or
+another provider, implement that provider behind the same local-government
+persistence contract instead.
+
 ---
 
 ## Congress bills (`congress.ts`)

apps/scraper/src/scraper-contracts.ts

@@ -1,6 +1,7 @@
 import type { ScraperEnvContract } from "@acme/env";
 
 import { caSosStatementsConfig } from "./scrapers/ca-sos-statements.config.js";
+import { cedarParkCouncilConfig } from "./scrapers/civicengage.config.js";
 import { congressConfig } from "./scrapers/congress.config.js";
 import { federalregisterConfig } from "./scrapers/federalregister.config.js";
 import { sccCvigConfig } from "./scrapers/scc-cvig.config.js";
@@ -12,4 +13,5 @@ export const scraperContracts: readonly ScraperEnvContract[] = [
   scotusConfig,
   sccCvigConfig,
   caSosStatementsConfig,
+  cedarParkCouncilConfig,
 ];

apps/scraper/src/scrapers.ts

@@ -1,5 +1,6 @@
 import type { Scraper } from "./utils/types.js";
 import { caSosStatements } from "./scrapers/ca-sos-statements.js";
+import { cedarParkCouncil } from "./scrapers/civicengage.js";
 import { congress } from "./scrapers/congress.js";
 import { federalregister } from "./scrapers/federalregister.js";
 import { sccCvig } from "./scrapers/scc-cvig.js";
@@ -11,4 +12,5 @@ export const scrapers: readonly Scraper[] = [
   scotus,
   sccCvig,
   caSosStatements,
+  cedarParkCouncil,
 ];

apps/scraper/src/scrapers/civicengage-parser.test.ts

@@ -0,0 +1,115 @@
+import assert from "node:assert/strict";
+import { readFile } from "node:fs/promises";
+import test from "node:test";
+
+import {
+  parseAgendaItems,
+  parseCentralMeetingDate,
+  parseMunicodePublishPage,
+} from "./civicengage-parser.js";
+import { cedarParkCouncilSource } from "./civicengage.config.js";
+
+const fixture = (name: string) =>
+  readFile(new URL(`./fixtures/civicengage/${name}`, import.meta.url), "utf8");
+
+test("parses regular, special, cancelled, and amended meeting records", async () => {
+  const meetings = parseMunicodePublishPage(
+    await fixture("meetings.html"),
+    cedarParkCouncilSource,
+    { now: new Date("2026-03-01T00:00:00Z") },
+  );
+  assert.equal(meetings.length, 4);
+  assert.deepEqual(
+    meetings.map(({ title, meetingType, status }) => ({
+      title,
+      meetingType,
+      status,
+    })),
+    [
+      {
+        title: "City Council Mtg. - CANCELLED",
+        meetingType: "regular",
+        status: "cancelled",
+      },
+      {
+        title: "City Council - Special Called",
+        meetingType: "special",
+        status: "held",
+      },
+      {
+        title: "City Council Mtg.",
+        meetingType: "regular",
+        status: "completed",
+      },
+      {
+        title: "City Council Mtg. - Amended",
+        meetingType: "regular",
+        status: "amended",
+      },
+    ],
+  );
+  assert.match(meetings[2]?.externalId ?? "", /^[a-f0-9]{32}$/);
+  assert.equal(
+    meetings[2]?.documents[0]?.url,
+    "https://mccmeetings.blob.core.usgovcloudapi.net/cptx-pubu/MEET-Agenda-2c1128fc619f48d6bf29cb94897a2d7f.pdf",
+  );
+  assert.equal(
+    meetings[2]?.canonicalUrl,
+    "https://www.cedarparktexas.gov/596/City-Council-Agendas",
+  );
+  assert.deepEqual(
+    meetings[3]?.documents.map(({ isCurrent }) => isCurrent),
+    [true, false],
+  );
+});
+
+test("applies a twelve-month cutoff and parses Central daylight time", async () => {
+  const meetings = parseMunicodePublishPage(
+    await fixture("meetings.html"),
+    cedarParkCouncilSource,
+    {
+      now: new Date("2026-07-21T12:00:00Z"),
+      cutoff: new Date("2025-07-21T12:00:00Z"),
+    },
+  );
+  assert.equal(meetings.length, 3);
+  assert.equal(
+    parseCentralMeetingDate("2/12/2026", "6:00 PM").toISOString(),
+    "2026-02-13T00:00:00.000Z",
+  );
+  assert.equal(
+    parseCentralMeetingDate("7/9/2026", "7:00 PM").toISOString(),
+    "2026-07-10T00:00:00.000Z",
+  );
+});
+
+test("deterministically parses items, motions, outcomes, tallies, and roll calls", async () => {
+  const agenda = await fixture("agenda.txt");
+  const minutes = await fixture("minutes.txt");
+  const sourceUrl = "https://official.example/agenda.pdf";
+  const first = parseAgendaItems(agenda, minutes, sourceUrl);
+  const second = parseAgendaItems(agenda, minutes, sourceUrl);
+  assert.deepEqual(first, second);
+  assert.equal(first.length, 6);
+
+  const ordinance = first.find((item) => item.itemNumber === "E.1");
+  assert.equal(ordinance?.itemType, "ordinance");
+  assert.equal(ordinance?.consent, true);
+  assert.equal(ordinance?.outcome, "approved");
+
+  const resolution = first.find((item) => item.itemNumber === "F.1");
+  assert.equal(
+    resolution?.motion,
+    "Motion to approve Agenda Item F.1 as presented.",
+  );
+  assert.match(resolution?.voteSummary ?? "", /^6-0/);
+  assert.equal(resolution?.outcome, "approved");
+
+  const rollCall = first.find((item) => item.itemNumber === "H.1");
+  assert.equal(rollCall?.votes.length, 6);
+  assert.deepEqual(rollCall?.votes.at(-1), { voterName: "Darby", value: "no" });
+  assert.equal(rollCall?.sourceUrl, sourceUrl);
+
+  const noAction = first.find((item) => item.itemNumber === "H.2");
+  assert.equal(noAction?.outcome, "no_action");
+});