billion-app · ThatXliner · Jul 22, 2026
diff --git a/apps/scraper/README.md b/apps/scraper/README.md
@@ -4,7 +4,7 @@ Pulls in government content like bills, court cases, and White House content and
 
 ## Active data sources
 
-Only these five are registered and run by `all`:
+Only these six are registered and run by `all`:
 
 | CLI name            | Source and data fetched                                                                    | Stored/used as                                                                                        |
 | ------------------- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------- |
@@ -13,6 +13,7 @@ Only these five are registered and run by `all`:
 | `scotus`            | CourtListener opinion clusters, dockets, and sub-opinion text for the Supreme Court        | `court_case`; powers court content and AI/feed enrichment                                             |
 | `scc-cvig`          | Hand-configured Santa Clara County voter-guide PDFs                                        | Candidate statements in `CivicApiCache`; the API matches statements to candidates                     |
 | `ca-sos-statements` | California SOS statewide-office candidate-statement pages                                  | Candidate statements in `CivicApiCache`; the API reads the cache and can fall back to the live source |
+| `texas-legislature` | Texas Legislative Council anonymous FTP: current-session history XML and bulk documents     | State-aware `bill` rows; read through `content.texasBills` and `content.getById`                       |
 
 `vote411`, `ca-lao-fiscal`, and `ca-vig-archive` remain under
 `src/scrapers/disabled/` and do not run. Their caches had no application
@@ -55,6 +56,7 @@ work:
 | `COURTLISTENER_API_KEY`                      | Optional                                | Higher CourtListener limits for `scotus`.                                                                          |
 | `GOOGLE_API_KEY` / `GOOGLE_SEARCH_ENGINE_ID` | Optional pair                           | Google Custom Search article thumbnails.                                                                           |
 | `GOOGLE_GENERATIVE_AI_API_KEY`               | Optional                                | Gemini vision fallback for `scc-cvig` PDF extraction.                                                              |
+| `OPEN_STATES_API_KEY`                        | Optional                                | Adds an exact Open States bill ID when Texas jurisdiction/session/identifier match.                                |
 
 See [the launch environment guide](../../docs/launch.md) for the complete
 per-scraper matrix, provider setup links, defaults, and production guidance.
@@ -124,6 +126,7 @@ CONGRESS_MAX_ITEMS=10 pnpm --filter @acme/scraper run start congress
 | `SCOTUS_MAX_ITEMS`              |      50 | CourtListener opinion clusters                      |
 | `SCC_CVIG_MAX_ITEMS`            |      10 | Voter-guide PDF documents                           |
 | `CA_SOS_MAX_ITEMS`              |       9 | Statewide-office candidate-statement pages          |
+| `TEXAS_LEGISLATURE_MAX_ITEMS`   |     100 | Bills from the latest Texas bulk session            |
 | `SCRAPER_MAX_NEW_ITEMS_PER_RUN` |      10 | New records receiving expensive AI/image enrichment |
 
 These are per-run limits, not durable calendar-day quotas. Schedule one run per
@@ -152,6 +155,34 @@ await scrapeCongress({
 
 ---
 
+## Texas Legislature Online (`texas-legislature.ts`)
+
+Uses only the Texas Legislative Council's anonymous FTP service at
+`ftp.legis.state.tx.us`; it does not crawl interactive TLO bill pages. The job:
+
+- discovers the newest session directory under `/bills` and rejects a stale
+  `TEXAS_LEGISLATURE_SESSION` assertion (official codes look like `89R` or
+  `892`);
+- parses bill-history XML for identity, caption, sponsors, subjects, actions,
+  structured votes when present, and document metadata;
+- downloads bill text, analyses, and fiscal notes from the matching FTP bulk
+  HTML paths and stores their extracted text alongside official HTML/PDF links;
+- optionally stores an exact Open States ID without using Open States as the
+  legislative data source.
+
+Run a small current-session import:
+
+```bash
+pnpm --filter @acme/scraper run start texas-legislature --max-items 10
+```
+
+Apply `packages/db/migrations/add_state_legislation_fields.sql` before the first
+run. The public `content.texasBills` procedure lists only the newest persisted
+Texas session; `content.getById` returns its documents, actions, and votes. This
+work deliberately does not provide historical-session browsing or backfills.
+
+---
+
 ## Court cases (`scotus.ts`)
 
 Uses the [CourtListener API](https://www.courtlistener.com/api/) — free, works without a key. Fetches recent opinions and pulls in the plain-text opinion content for AI article generation.
@@ -179,4 +210,7 @@ All scrapers call into `src/utils/db/operations.ts`. Each time a bill or case is
 - If the **content changed** → regenerates the article
 - If **nothing changed** → backfills any missing AI summary/article/thumbnail fields, otherwise skips AI generation
 
+The Texas importer passes `skipEnrichment` so official source data is persisted
+without AI summaries, generated imagery, or videos.
+
 Set `SCRAPER_FORCE_AI_REGEN=1` to force a full AI refresh even when the record already has AI content.
diff --git a/apps/scraper/package.json b/apps/scraper/package.json
@@ -12,6 +12,7 @@
     "@ai-sdk/google": "^3.0.53",
     "@openrouter/ai-sdk-provider": "^2.10.0",
     "ai": "^6.0.141",
+    "basic-ftp": "^6.0.1",
     "cheerio": "^1.2.0",
     "consola": "^3.4.2",
     "drizzle-orm": "^0.45.2",

diff --git a/apps/scraper/src/scraper-contracts.ts b/apps/scraper/src/scraper-contracts.ts
@@ -5,11 +5,13 @@ import { congressConfig } from "./scrapers/congress.config.js";
 import { federalregisterConfig } from "./scrapers/federalregister.config.js";
 import { sccCvigConfig } from "./scrapers/scc-cvig.config.js";
 import { scotusConfig } from "./scrapers/scotus.config.js";
+import { texasLegislatureConfig } from "./scrapers/texas-legislature.config.js";
 
 export const scraperContracts: readonly ScraperEnvContract[] = [
   federalregisterConfig,
   congressConfig,
   scotusConfig,
   sccCvigConfig,
   caSosStatementsConfig,
+  texasLegislatureConfig,
 ];
diff --git a/apps/scraper/src/scrapers.ts b/apps/scraper/src/scrapers.ts
@@ -4,11 +4,13 @@ import { congress } from "./scrapers/congress.js";
 import { federalregister } from "./scrapers/federalregister.js";
 import { sccCvig } from "./scrapers/scc-cvig.js";
 import { scotus } from "./scrapers/scotus.js";
+import { texasLegislature } from "./scrapers/texas-legislature.js";
 
 export const scrapers: readonly Scraper[] = [
   federalregister,
   congress,
   scotus,
   sccCvig,
   caSosStatements,
+  texasLegislature,
 ];
diff --git a/apps/scraper/src/scrapers/fixtures/texas-hb9-history.xml b/apps/scraper/src/scrapers/fixtures/texas-hb9-history.xml
@@ -0,0 +1,78 @@
+<?xml version="1.0" encoding="utf-8"?>
+<billhistory bill="89(R) HB 9" lastUpdate="6/12/2025 2:00:01 AM" source="Texas Legislature Online website (ftp://ftp.legis.state.tx.us) - Texas Legislative Council">
+  <caption>Relating to an exemption from ad valorem taxation.</caption>
+  <authors>Meyer | Bonnen</authors>
+  <coauthors>Bernal</coauthors>
+  <sponsors>Bettencourt</sponsors>
+  <cosponsors></cosponsors>
+  <subjects>
+    <subject>Taxation--Property-Exemptions (I0793)</subject>
+    <subject>Business &amp; Commerce--General (I0050)</subject>
+  </subjects>
+  <committees>
+    <house name="Ways &amp; Means" status="Out of committee" ayeVotes="9" nayVotes="1" presentNotVotingVotes="0" absentVotes="1" />
+    <senate name="Local Government" status="Out of committee" ayeVotes="6" nayVotes="0" presentNotVotingVotes="0" absentVotes="3" />
+  </committees>
+  <billtext>
+    <docTypes>
+      <bill>
+        <versions>
+          <version>
+            <versionDescription>Introduced</versionDescription>
+            <WebHTMLURL>https://capitol.texas.gov/tlodocs/89R/billtext/html/HB00009I.htm</WebHTMLURL>
+            <WebPDFURL>https://capitol.texas.gov/tlodocs/89R/billtext/pdf/HB00009I.pdf</WebPDFURL>
+            <FTPHTMLURL>ftp://ftp.legis.state.tx.us/bills/89R/billtext/HTML/house_bills/HB00001_HB00099/HB00009I.HTM</FTPHTMLURL>
+            <FTPPDFURL>ftp://ftp.legis.state.tx.us/bills/89R/billtext/PDF/house_bills/HB00001_HB00099/HB00009I.PDF</FTPPDFURL>
+          </version>
+          <version>
+            <versionDescription>Enrolled</versionDescription>
+            <WebHTMLURL>https://capitol.texas.gov/tlodocs/89R/billtext/html/HB00009F.htm</WebHTMLURL>
+            <WebPDFURL>https://capitol.texas.gov/tlodocs/89R/billtext/pdf/HB00009F.pdf</WebPDFURL>
+            <FTPHTMLURL>ftp://ftp.legis.state.tx.us/bills/89R/billtext/HTML/house_bills/HB00001_HB00099/HB00009F.HTM</FTPHTMLURL>
+            <FTPPDFURL>ftp://ftp.legis.state.tx.us/bills/89R/billtext/PDF/house_bills/HB00001_HB00099/HB00009F.PDF</FTPPDFURL>
+          </version>
+        </versions>
+      </bill>
+      <analysis>
+        <versions>
+          <version>
+            <versionDescription>House Committee Report</versionDescription>
+            <WebHTMLURL>https://capitol.texas.gov/tlodocs/89R/analysis/html/HB00009H.htm</WebHTMLURL>
+            <WebPDFURL>https://capitol.texas.gov/tlodocs/89R/analysis/pdf/HB00009H.pdf</WebPDFURL>
+            <FTPHTMLURL>ftp://ftp.legis.state.tx.us/bills/89R/analysis/HTML/house_bills/HB00001_HB00099/HB00009H.HTM</FTPHTMLURL>
+            <FTPPDFURL>ftp://ftp.legis.state.tx.us/bills/89R/analysis/PDF/house_bills/HB00001_HB00099/HB00009H.PDF</FTPPDFURL>
+          </version>
+        </versions>
+      </analysis>
+      <fiscalNote>
+        <versions>
+          <version>
+            <versionDescription>Introduced</versionDescription>
+            <WebHTMLURL>https://capitol.texas.gov/tlodocs/89R/fiscalnotes/html/HB00009I.htm</WebHTMLURL>
+            <WebPDFURL>https://capitol.texas.gov/tlodocs/89R/fiscalnotes/pdf/HB00009I.pdf</WebPDFURL>
+            <FTPHTMLURL>ftp://ftp.legis.state.tx.us/bills/89R/fiscalNotes/HTML/house_bills/HB00001_HB00099/HB00009I.HTM</FTPHTMLURL>
+            <FTPPDFURL>ftp://ftp.legis.state.tx.us/bills/89R/fiscalNotes/PDF/house_bills/HB00001_HB00099/HB00009I.PDF</FTPPDFURL>
+          </version>
+        </versions>
+      </fiscalNote>
+    </docTypes>
+  </billtext>
+  <actions>
+    <action>
+      <date>06/12/2025</date>
+      <actionNumber>E100</actionNumber>
+      <description>Effective immediately</description>
+    </action>
+    <action>
+      <date>11/12/2024</date>
+      <actionNumber>H001</actionNumber>
+      <description>Filed</description>
+    </action>
+    <action>
+      <date>05/19/2025</date>
+      <actionNumber>H630</actionNumber>
+      <description>Record vote</description>
+      <comment>RV#2999</comment>
+    </action>
+  </actions>
+</billhistory>