feat(events-crawler): widen relevance to digital-skills content

BACKLOG.md

@@ -501,39 +501,41 @@ re-scopes it:
 Revisit only if United Way BC publishes a real feed (ICS/RSS/JSON) or server-renders its dates.
 Both are cheap to re-test: `curl https://uwbc.ca/wp-json/wp/v2/event` and check for a date field.
 
-**events-crawler — `relevance.ts` misses digital-skills-for-employment content (evidence only)**
-Recording observations for a future relevance-filter pass. **Not scoped and not proposed** — the
-filter is shared by all 10 live sources, so any change needs its own PR and a full regression
-dry-run across every source, not a fix bolted onto an adapter PR.
-
-The digital-literacy group currently matches `digital literacy`, `computer (basics|skills|help)`,
-`tech (cafe|help|support)`, `device clinic`, `online safety`, `internet basics`. That vocabulary
-misses adjacent content that is plainly on-mission:
-- **NVCL (PR #92):** `MS Office learn and practice: Intro to Excel` and `… Intro to Word` — named
-  by product, not by the category words the filter knows. Both are digital-skills-for-employment
-  sessions of exactly the kind newcomers are pointed at.
-- **NVDPL:** `Open Door Community Hub Drop-In` — already documented as a known drop in
-  `lib/relevance.ts`'s own header, and NVCL runs an identically-named program.
-- **Capilano (PR #94), marginal — recorded as a weak case, not evidence:**
-  `Fall 2026 Term Commences / New Student Orientation`. Arguably useful to an arriving
-  international student, but `relevance.ts` deliberately avoids a bare `orientation` term (its
-  header notes it would catch West Van's "Recording Studio Orientation"), and this title gives
-  nothing else to match on. Capilano's genuinely relevant item — `Fall 2026 New International
-  Student Orientation` — **is** caught, via `international student`. So the filter is working
-  here; this is listed only so the eventual pass sees the borderline shape, not as a second
-  example of the MS Office problem.
-
-Scale of the effect, measured on NVCL's live feed 2026-08-02: its whole 4-month window is 27
-distinct titles, the filter matches **none**, and the source therefore yields **0 rows** with
-`relevanceFilter: true` versus 25 with it off. Most of those 27 (storytimes, teen gaming,
-knitting, Book Bike) are correctly excluded — the point is that the two MS Office sessions are
-not.
+**events-crawler — `relevance.ts` near-misses still open after the 2026-08-08 pass**
+The digital-skills gap recorded here is **closed**: `dryrun.ts` gained `--no-filter --titles`,
+and the filter gained qualified office-suite terms, `tech(nology)? (cafe|help|support)`,
+`laptop help`, and `open door (community|drop)`. Measured over all 845 rows / 395 distinct
+titles of the seven filtered sources' 4-month windows, 9 distinct titles (56 rows) flipped
+DROP → KEEP and nothing else moved. `lib/relevance_test.ts` now locks in the trap set.
+
+**Correction to the NVCL measurement this entry used to carry.** It claimed NVCL's whole
+4-month window was 27 distinct titles matching **none** of the filter, for 0 rows. Re-measured
+2026-08-08 with the walk untruncated: the window is **61 distinct titles**, and the filter
+already kept **10** of them before this pass (an `English Corner` series, via `english corner`).
+The old figure was taken with `MAX_PER_ORG` at 25, which stops the unfiltered walk after about
+two pages — so it only ever saw August's storytimes, not September's settlement programming.
+Anyone re-measuring a source's whole window must raise the cap locally first.
+
+Still open, recorded as evidence rather than scoped work:
+- **`Protect Yourself Online: 2FA Made Simple` / `… Your Digital Footprint` (SFU, 3 rows).**
+  Substantively the `online safety` term's own category, but titled as a program name with no
+  generalizable vocabulary to match on. Three rows did not clear the bar that `open door`
+  cleared at 29 rows across two libraries. Revisit if another source runs a similar series.
+- **`Connect with Confidence: iPhones and iPads` (2 rows), `Introduction to iPad` (1).** Device
+  classes, adjacent to the `device clinic` / `laptop help` family, but library-catalogue and
+  seniors framing rather than newcomer framing. Judgement call, deliberately left out.
+- **Capilano, marginal — a weak case, not evidence.** `Fall 2026 Term Commences / New Student
+  Orientation`. `relevance.ts` deliberately avoids a bare `orientation` (it would catch West
+  Van's "Recording Studio Orientation"), and this title gives nothing else to match on.
+  Capilano's genuinely relevant item — `Fall 2026 New International Student Orientation` — **is**
+  caught, via `international student`. The filter is working here.
 
 Cautions for whoever picks this up: `relevance.ts` matches **titles only**, deliberately — its
 header records that matching descriptions was measured and rejected because library blurbs close
-with "newcomer families welcome" and pull in every storytime. And a naive term like `\bword\b`
-would catch "Crossword" and "Wordplay". Add terms narrowly, and re-run every source's dry-run to
-see what else they let in.
+with "newcomer families welcome" and pull in every storytime. Bare office words are traps: the
+live feeds contain "Excel in Your Studies", "Crossword Club", "Wordplay for Toddlers" and
+"PowerPoint Karaoke Night". Add terms narrowly, add each one's target **and** its near-misses to
+`lib/relevance_test.ts`, and re-capture every source with `--no-filter --titles` to see the flips.
 
 **Client-only tab state elsewhere — same back-navigation bug Community just fixed**
 Found while fixing Community (this PR): tab state held in `useState` never reaches the URL,

supabase/functions/events-crawler/dryrun.ts

@@ -13,6 +13,8 @@
 //   deno run --allow-net --allow-env=PEXELS_API_KEY \
 //     supabase/functions/events-crawler/dryrun.ts --source vpl
 //   …--source vpl --json     # full rows as JSON
+//   …--source vpl --titles   # one KEEP/DROP line per row, no detail
+//   …--source vpl --no-filter --titles   # the source's WHOLE calendar, with verdicts
 //   …--list                  # the registry, with enabled / filtered flags
 //
 // The env grant is scoped to PEXELS_API_KEY because the tier-2 cover lookup reads it.
@@ -22,8 +24,20 @@
 //
 // Disabled sources can be previewed here — that is what makes it possible to review a
 // staged source before anyone flips `enabled`.
+//
+// WHY --no-filter EXISTS. lib/relevance.ts is shared by every filtered source, so a term
+// added for one library changes what all of them ingest. A normal dry run cannot measure
+// that: it shows only the titles that already PASSED, while the whole question is which
+// currently-rejected titles a new term would let in. --no-filter runs the adapter against
+// a copy of the Source with relevanceFilter off, so the output is the source's entire
+// calendar; --titles then prints each one with the verdict the real filter gives it. Diff
+// that before and after a relevance change and every flip is visible.
+//
+// It stays write-free: the override is a local object spread, the registry itself is not
+// mutated, and there is still no Supabase import anywhere in this file.
 
 import { ADAPTERS, makeContext, SOURCES } from './lib/sources.ts';
+import { isSettlementRelevant } from './lib/relevance.ts';
 import type { EventRow } from './lib/types.ts';
 
 function arg(name: string): string | undefined {
@@ -45,6 +59,18 @@ function printRow(row: EventRow, i: number): void {
   console.log(`    desc     ${desc.slice(0, 100)}${desc.length > 100 ? '…' : ''}`);
 }
 
+/**
+ * One line per row: the verdict lib/relevance.ts gives the title, then the date and title.
+ *
+ * `filtered` is the source's REGISTRY setting, not the setting this run used. A settlement
+ * agency is never relevance-filtered, so printing KEEP/DROP beside its events would invite
+ * exactly the wrong conclusion — those rows are ingested either way. Hence `n/a`.
+ */
+function printTitle(row: EventRow, filtered: boolean): void {
+  const verdict = !filtered ? 'n/a ' : isSettlementRelevant(row.title) ? 'KEEP' : 'DROP';
+  console.log(`${verdict}  ${row.event_datetime.slice(0, 10)}  ${row.title}`);
+}
+
 if (Deno.args.includes('--list')) {
   console.log('Sources:');
   for (const s of SOURCES) {
@@ -58,29 +84,39 @@ if (Deno.args.includes('--list')) {
 
 const slug = arg('source');
 if (!slug) {
-  console.error('Usage: dryrun.ts --source <slug> [--json] | --list');
+  console.error('Usage: dryrun.ts --source <slug> [--json|--titles] [--no-filter] | --list');
   Deno.exit(1);
 }
 
-const source = SOURCES.find((s) => s.slug === slug);
-if (!source) {
+const registered = SOURCES.find((s) => s.slug === slug);
+if (!registered) {
   console.error(`Unknown source '${slug}'. Run with --list to see the registry.`);
   Deno.exit(1);
 }
 
+// Copy rather than mutate: SOURCES is the shared registry every other import sees.
+const noFilter = Deno.args.includes('--no-filter');
+const source = noFilter ? { ...registered, relevanceFilter: false } : registered;
+
 const ctx = makeContext();
 
 console.log(
   `DRY RUN — ${source.slug} (${source.kind}), window ${ctx.today} → ${ctx.windowEnd}, ` +
-    `${source.enabled ? 'enabled' : 'DISABLED in production'}. No writes.\n`,
+    `${registered.enabled ? 'enabled' : 'DISABLED in production'}` +
+    `${noFilter ? ', RELEVANCE FILTER OFF for this run' : ''}. No writes.\n`,
 );
 
 const rows = await ADAPTERS[source.kind](source, ctx);
 
 if (Deno.args.includes('--json')) {
   console.log(JSON.stringify(rows, null, 2));
+} else if (Deno.args.includes('--titles')) {
+  rows.forEach((row) => printTitle(row, Boolean(registered.relevanceFilter)));
 } else {
   rows.forEach(printRow);
 }
 
-console.log(`\n${rows.length} row(s) would be inserted. Nothing was written.`);
+// With --no-filter the count is what the source PUBLISHES, not what the crawler would take:
+// production still applies the filter. Say so, so a capture can't be misread as a forecast.
+const verb = noFilter ? 'row(s) in the source calendar (filter off)' : 'row(s) would be inserted';
+console.log(`\n${rows.length} ${verb}. Nothing was written.`);

supabase/functions/events-crawler/lib/relevance.ts

@@ -18,6 +18,14 @@
 //
 // Measured keep-rates on the live feeds when this landed (2026-07-31):
 //   West Van 8/50 · VPL 6/100 · SFU 5/200 · NVDPL 16/100 · Surrey 2/10 (page 1)
+//
+// Re-measured 2026-08-08 over each source's WHOLE 4-month window, before → after the
+// office-suite / technology-help / Open Door terms below (`dryrun.ts --no-filter
+// --titles`, with MAX_PER_ORG raised locally so the walk isn't truncated):
+//   West Van 6/50 → 6 · VPL 22/200 → 22 · SFU 3/111 → 3 · NVDPL 11/82 → 20
+//   Surrey 31/192 → 46 · NVCL 10/200 → 42 · Capilano 1/10 → 1
+// Across all 395 distinct titles, 9 flipped DROP → KEEP and nothing else moved — a term
+// added here is a union alternative, so it can only add keeps, never drop an existing one.
 
 /**
  * Accents are stripped before matching (NFD, then drop combining marks), so a single
@@ -54,8 +62,28 @@ const RELEVANCE_RE = new RegExp(
     // Housing
     'housing|tenant|tenancy|rental|renting|landlord|lease\\b',
     // Digital literacy
-    'digital\\s+literacy|computer\\s+(basics|skills|help)|tech\\s+(cafe|help|support)',
-    'device\\s+clinic|online\\s+safety|internet\\s+basics',
+    // `\b` before tech: without it the alternative matches the tail of "Biotech Help" or
+    // "Fintech Support", both plausible at SFU. It was already missing on the narrower
+    // `tech` this widened.
+    'digital\\s+literacy|computer\\s+(basics|skills|help)|\\btech(nology)?\\s+(cafe|help|support)',
+    'device\\s+clinic|laptop\\s+help|online\\s+safety|internet\\s+basics',
+    // Digital literacy — office suite. Libraries title these by product, not by the
+    // category words above, so the group missed them entirely: NVCL's "MS Office learn
+    // and practice: Intro to Word" and Surrey's office sessions are settlement-adjacent
+    // employment content by any reading. Every office word is QUALIFIED — behind `ms`,
+    // `microsoft`, or `intro to` — because the bare forms are traps: `\bexcel\b` catches
+    // "Excel in Your Studies", `\bword\b` catches "Crossword Club" and "Wordplay for
+    // Toddlers", and `\bpowerpoint\b` catches "PowerPoint Karaoke Night". All four are
+    // real titles from the live feeds. See lib/relevance_test.ts.
+    '\\bms\\s+(office|word|excel|powerpoint|outlook)\\b',
+    'microsoft\\s+(office|word|excel|powerpoint|outlook|365)\\b',
+    'intro(duction)?\\s+to\\s+(excel|word|powerpoint|outlook)\\b|\\bspreadsheet',
+    // Named settlement programs. Kept to programs run under the same name by more than
+    // one source, so this doesn't become a list of one-off event names: the Open Door
+    // community hub is NVCL's and NVDPL's shared newcomer drop-in, 29 occurrences across
+    // the two 4-month windows, and was the single largest miss in the corpus. Qualified
+    // so it cannot catch "Open Door Poetry Reading" or "Open Doors Heritage Tour".
+    'open\\s+door\\s+(community|drop)',
     // Health-system navigation
     'health\\s+card|family\\s+doctor|\\bmsp\\b|medical\\s+insurance',
   ].join('|'),

supabase/functions/events-crawler/lib/relevance_test.ts

@@ -0,0 +1,138 @@
+// Traps for the settlement-relevance filter.
+//
+// WHY THIS FILE EXISTS. `isSettlementRelevant` is a pure function shared by every
+// relevance-filtered source, so a term added for one library silently changes what all of
+// them ingest — and the tempting shorthand for each new term is usually the broken one.
+// The 2026-08-08 round wanted NVCL's "MS Office … Intro to Excel"; the obvious `\bexcel\b`
+// and `\bword\b` also take "Excel in Your Studies", "Crossword Club" and "Wordplay for
+// Toddlers". That hazard was previously recorded in BACKLOG.md prose, which cannot fail a
+// run. Here it can.
+//
+// Every string below is a REAL title from a live feed, captured via
+// `dryrun.ts --source <slug> --no-filter --titles`. Adding a term means adding its target
+// here, and adding any near-miss the corpus shows it brushed against.
+//
+// RUN (from web-app/):
+//   deno test supabase/functions/events-crawler/lib/relevance_test.ts
+
+import { assert } from 'jsr:@std/assert@1';
+import { isSettlementRelevant } from './relevance.ts';
+
+/** Real titles the filter must keep, grouped by the term that earns each one. */
+const KEEP: Record<string, string[]> = {
+  'status & settlement': [
+    'Settlement & Integration Services',
+    'Employment Services for Newcomers',
+    'Newcomer Family Storytime',
+    'International Student Orientation - Fall 2026',
+    'Circletime success: Welcoming newcomer children at circletime',
+  ],
+  language: [
+    'ESL Conversation Practice',
+    'Practice Speaking English: Advanced',
+    'English Conversation Program (55+)',
+    'English Corner (virtual) — September 2026',
+    'Beginner English Corner (in-person) — September 2026',
+  ],
+  employment: [
+    'Canadian Job Search Workshop',
+    'Ask a Career Counsellor: Resume Workshop',
+    // Accent folding is load-bearing here — a plain `resume` term misses "Résumé".
+    'Résumé Clinic with S.U.C.C.E.S.S.',
+  ],
+  'digital literacy': [
+    'Tech Help',
+    'Device Clinic',
+    // Accent folding again: "Café", and the Farsi title carries combining marks.
+    'Tech Café - Capilano (Drop-In)',
+    'Farsi Tech Café / تِکِ کافه - Lynn Valley',
+    // Added 2026-08-08 — `technology` spelled out, 21 rows across NVCL + Surrey.
+    'One on One Technology Help',
+    'Drop-in Technology Help',
+    'Drop-in technology help in Farsi کمک به رفع اشکال تکنولوژی به زبان فارسی',
+    'Windows laptop help',
+    'Mac laptop help',
+  ],
+  'office suite': [
+    'MS Office learn and practice: Intro to Word',
+    'MS Office learn and practice: Intro to Excel',
+    'Microsoft Word Basics',
+    'Intro to Excel',
+    'Spreadsheet Basics',
+  ],
+  'named programs': [
+    'Open door community hub',
+    'Open Door Community Hub Drop-In',
+  ],
+};
+
+/**
+ * Titles the filter must reject. The first block is the near-misses — each one is what a
+ * lazier version of a term above would have caught, so these are the assertions that
+ * actually earn their keep. The rest is ordinary library programming, there to prove the
+ * filter still rejects the bulk of a what's-on calendar.
+ */
+const DROP: Record<string, string[]> = {
+  'near-misses for the office-suite terms': [
+    'Excel in Your Studies: Study Skills',
+    'Crossword Club',
+    'Wordplay for Toddlers',
+    'Word Whips',
+    'Word Games Night',
+    'Sword Fighting Demo',
+    'Keyword Search Basics',
+    'PowerPoint Karaoke Night',
+    'Microsoft Flight Simulator Club',
+  ],
+  'near-misses for tech(nology) and laptop help': [
+    'Tech for tots',
+    'Technology terminology',
+    'Tech Club',
+    'SFU Burnaby Science Alive High School Camps: A Byte of Tech',
+    'Repair Café',
+    // `tech` inside a longer word — the reason the alternative carries a leading `\b`.
+    // Constructed rather than captured: plausible SFU titles, not yet seen in a feed.
+    'Biotech Help',
+    'Fintech Support',
+    'Edtech Café',
+  ],
+  'near-misses for open door': [
+    'Open Door Poetry Reading',
+    'Open Doors Heritage Tour',
+    'Open sewing studio',
+    'Open gaming: Minecraft',
+  ],
+  'near-miss for a bare `orientation`': [
+    // Why `orientation` is deliberately absent — see isSettlementRelevant's doc comment.
+    'Recording Studio Orientation',
+  ],
+  'ordinary library programming': [
+    'Babytime',
+    'Family Storytime',
+    'Teen open space',
+    'Craft \'N\' Yarn',
+    'North Shore knitters',
+    'Book Bike at Lonsdale Quay Market',
+    'LEGO® Block Party',
+    'Simorgh: a self-portrait',
+    'Yuehao Hu | MSc Thesis Defence',
+    'Adult Colouring Club',
+    'Scrabble® Sundays',
+  ],
+};
+
+Deno.test('isSettlementRelevant keeps real settlement titles', () => {
+  for (const [group, titles] of Object.entries(KEEP)) {
+    for (const title of titles) {
+      assert(isSettlementRelevant(title), `[${group}] should KEEP: ${title}`);
+    }
+  }
+});
+
+Deno.test('isSettlementRelevant rejects near-misses and general programming', () => {
+  for (const [group, titles] of Object.entries(DROP)) {
+    for (const title of titles) {
+      assert(!isSettlementRelevant(title), `[${group}] should DROP: ${title}`);
+    }
+  }
+});