diff --git a/BACKLOG.md b/BACKLOG.md index 75c6c6c..a0da3b1 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -501,39 +501,41 @@ re-scopes it: Revisit only if United Way BC publishes a real feed (ICS/RSS/JSON) or server-renders its dates. Both are cheap to re-test: `curl https://uwbc.ca/wp-json/wp/v2/event` and check for a date field. -**events-crawler — `relevance.ts` misses digital-skills-for-employment content (evidence only)** -Recording observations for a future relevance-filter pass. **Not scoped and not proposed** — the -filter is shared by all 10 live sources, so any change needs its own PR and a full regression -dry-run across every source, not a fix bolted onto an adapter PR. - -The digital-literacy group currently matches `digital literacy`, `computer (basics|skills|help)`, -`tech (cafe|help|support)`, `device clinic`, `online safety`, `internet basics`. That vocabulary -misses adjacent content that is plainly on-mission: -- **NVCL (PR #92):** `MS Office learn and practice: Intro to Excel` and `… Intro to Word` — named - by product, not by the category words the filter knows. Both are digital-skills-for-employment - sessions of exactly the kind newcomers are pointed at. -- **NVDPL:** `Open Door Community Hub Drop-In` — already documented as a known drop in - `lib/relevance.ts`'s own header, and NVCL runs an identically-named program. -- **Capilano (PR #94), marginal — recorded as a weak case, not evidence:** - `Fall 2026 Term Commences / New Student Orientation`. Arguably useful to an arriving - international student, but `relevance.ts` deliberately avoids a bare `orientation` term (its - header notes it would catch West Van's "Recording Studio Orientation"), and this title gives - nothing else to match on. Capilano's genuinely relevant item — `Fall 2026 New International - Student Orientation` — **is** caught, via `international student`. So the filter is working - here; this is listed only so the eventual pass sees the borderline shape, not as a second - example of the MS Office problem. - -Scale of the effect, measured on NVCL's live feed 2026-08-02: its whole 4-month window is 27 -distinct titles, the filter matches **none**, and the source therefore yields **0 rows** with -`relevanceFilter: true` versus 25 with it off. Most of those 27 (storytimes, teen gaming, -knitting, Book Bike) are correctly excluded — the point is that the two MS Office sessions are -not. +**events-crawler — `relevance.ts` near-misses still open after the 2026-08-08 pass** +The digital-skills gap recorded here is **closed**: `dryrun.ts` gained `--no-filter --titles`, +and the filter gained qualified office-suite terms, `tech(nology)? (cafe|help|support)`, +`laptop help`, and `open door (community|drop)`. Measured over all 845 rows / 395 distinct +titles of the seven filtered sources' 4-month windows, 9 distinct titles (56 rows) flipped +DROP → KEEP and nothing else moved. `lib/relevance_test.ts` now locks in the trap set. + +**Correction to the NVCL measurement this entry used to carry.** It claimed NVCL's whole +4-month window was 27 distinct titles matching **none** of the filter, for 0 rows. Re-measured +2026-08-08 with the walk untruncated: the window is **61 distinct titles**, and the filter +already kept **10** of them before this pass (an `English Corner` series, via `english corner`). +The old figure was taken with `MAX_PER_ORG` at 25, which stops the unfiltered walk after about +two pages — so it only ever saw August's storytimes, not September's settlement programming. +Anyone re-measuring a source's whole window must raise the cap locally first. + +Still open, recorded as evidence rather than scoped work: +- **`Protect Yourself Online: 2FA Made Simple` / `… Your Digital Footprint` (SFU, 3 rows).** + Substantively the `online safety` term's own category, but titled as a program name with no + generalizable vocabulary to match on. Three rows did not clear the bar that `open door` + cleared at 29 rows across two libraries. Revisit if another source runs a similar series. +- **`Connect with Confidence: iPhones and iPads` (2 rows), `Introduction to iPad` (1).** Device + classes, adjacent to the `device clinic` / `laptop help` family, but library-catalogue and + seniors framing rather than newcomer framing. Judgement call, deliberately left out. +- **Capilano, marginal — a weak case, not evidence.** `Fall 2026 Term Commences / New Student + Orientation`. `relevance.ts` deliberately avoids a bare `orientation` (it would catch West + Van's "Recording Studio Orientation"), and this title gives nothing else to match on. + Capilano's genuinely relevant item — `Fall 2026 New International Student Orientation` — **is** + caught, via `international student`. The filter is working here. Cautions for whoever picks this up: `relevance.ts` matches **titles only**, deliberately — its header records that matching descriptions was measured and rejected because library blurbs close -with "newcomer families welcome" and pull in every storytime. And a naive term like `\bword\b` -would catch "Crossword" and "Wordplay". Add terms narrowly, and re-run every source's dry-run to -see what else they let in. +with "newcomer families welcome" and pull in every storytime. Bare office words are traps: the +live feeds contain "Excel in Your Studies", "Crossword Club", "Wordplay for Toddlers" and +"PowerPoint Karaoke Night". Add terms narrowly, add each one's target **and** its near-misses to +`lib/relevance_test.ts`, and re-capture every source with `--no-filter --titles` to see the flips. **Client-only tab state elsewhere — same back-navigation bug Community just fixed** Found while fixing Community (this PR): tab state held in `useState` never reaches the URL, diff --git a/supabase/functions/events-crawler/dryrun.ts b/supabase/functions/events-crawler/dryrun.ts index f182fe7..99a0603 100644 --- a/supabase/functions/events-crawler/dryrun.ts +++ b/supabase/functions/events-crawler/dryrun.ts @@ -13,6 +13,8 @@ // deno run --allow-net --allow-env=PEXELS_API_KEY \ // supabase/functions/events-crawler/dryrun.ts --source vpl // …--source vpl --json # full rows as JSON +// …--source vpl --titles # one KEEP/DROP line per row, no detail +// …--source vpl --no-filter --titles # the source's WHOLE calendar, with verdicts // …--list # the registry, with enabled / filtered flags // // The env grant is scoped to PEXELS_API_KEY because the tier-2 cover lookup reads it. @@ -22,8 +24,20 @@ // // Disabled sources can be previewed here — that is what makes it possible to review a // staged source before anyone flips `enabled`. +// +// WHY --no-filter EXISTS. lib/relevance.ts is shared by every filtered source, so a term +// added for one library changes what all of them ingest. A normal dry run cannot measure +// that: it shows only the titles that already PASSED, while the whole question is which +// currently-rejected titles a new term would let in. --no-filter runs the adapter against +// a copy of the Source with relevanceFilter off, so the output is the source's entire +// calendar; --titles then prints each one with the verdict the real filter gives it. Diff +// that before and after a relevance change and every flip is visible. +// +// It stays write-free: the override is a local object spread, the registry itself is not +// mutated, and there is still no Supabase import anywhere in this file. import { ADAPTERS, makeContext, SOURCES } from './lib/sources.ts'; +import { isSettlementRelevant } from './lib/relevance.ts'; import type { EventRow } from './lib/types.ts'; function arg(name: string): string | undefined { @@ -45,6 +59,18 @@ function printRow(row: EventRow, i: number): void { console.log(` desc ${desc.slice(0, 100)}${desc.length > 100 ? '…' : ''}`); } +/** + * One line per row: the verdict lib/relevance.ts gives the title, then the date and title. + * + * `filtered` is the source's REGISTRY setting, not the setting this run used. A settlement + * agency is never relevance-filtered, so printing KEEP/DROP beside its events would invite + * exactly the wrong conclusion — those rows are ingested either way. Hence `n/a`. + */ +function printTitle(row: EventRow, filtered: boolean): void { + const verdict = !filtered ? 'n/a ' : isSettlementRelevant(row.title) ? 'KEEP' : 'DROP'; + console.log(`${verdict} ${row.event_datetime.slice(0, 10)} ${row.title}`); +} + if (Deno.args.includes('--list')) { console.log('Sources:'); for (const s of SOURCES) { @@ -58,29 +84,39 @@ if (Deno.args.includes('--list')) { const slug = arg('source'); if (!slug) { - console.error('Usage: dryrun.ts --source [--json] | --list'); + console.error('Usage: dryrun.ts --source [--json|--titles] [--no-filter] | --list'); Deno.exit(1); } -const source = SOURCES.find((s) => s.slug === slug); -if (!source) { +const registered = SOURCES.find((s) => s.slug === slug); +if (!registered) { console.error(`Unknown source '${slug}'. Run with --list to see the registry.`); Deno.exit(1); } +// Copy rather than mutate: SOURCES is the shared registry every other import sees. +const noFilter = Deno.args.includes('--no-filter'); +const source = noFilter ? { ...registered, relevanceFilter: false } : registered; + const ctx = makeContext(); console.log( `DRY RUN — ${source.slug} (${source.kind}), window ${ctx.today} → ${ctx.windowEnd}, ` + - `${source.enabled ? 'enabled' : 'DISABLED in production'}. No writes.\n`, + `${registered.enabled ? 'enabled' : 'DISABLED in production'}` + + `${noFilter ? ', RELEVANCE FILTER OFF for this run' : ''}. No writes.\n`, ); const rows = await ADAPTERS[source.kind](source, ctx); if (Deno.args.includes('--json')) { console.log(JSON.stringify(rows, null, 2)); +} else if (Deno.args.includes('--titles')) { + rows.forEach((row) => printTitle(row, Boolean(registered.relevanceFilter))); } else { rows.forEach(printRow); } -console.log(`\n${rows.length} row(s) would be inserted. Nothing was written.`); +// With --no-filter the count is what the source PUBLISHES, not what the crawler would take: +// production still applies the filter. Say so, so a capture can't be misread as a forecast. +const verb = noFilter ? 'row(s) in the source calendar (filter off)' : 'row(s) would be inserted'; +console.log(`\n${rows.length} ${verb}. Nothing was written.`); diff --git a/supabase/functions/events-crawler/lib/relevance.ts b/supabase/functions/events-crawler/lib/relevance.ts index 5377404..54fa947 100644 --- a/supabase/functions/events-crawler/lib/relevance.ts +++ b/supabase/functions/events-crawler/lib/relevance.ts @@ -18,6 +18,14 @@ // // Measured keep-rates on the live feeds when this landed (2026-07-31): // West Van 8/50 · VPL 6/100 · SFU 5/200 · NVDPL 16/100 · Surrey 2/10 (page 1) +// +// Re-measured 2026-08-08 over each source's WHOLE 4-month window, before → after the +// office-suite / technology-help / Open Door terms below (`dryrun.ts --no-filter +// --titles`, with MAX_PER_ORG raised locally so the walk isn't truncated): +// West Van 6/50 → 6 · VPL 22/200 → 22 · SFU 3/111 → 3 · NVDPL 11/82 → 20 +// Surrey 31/192 → 46 · NVCL 10/200 → 42 · Capilano 1/10 → 1 +// Across all 395 distinct titles, 9 flipped DROP → KEEP and nothing else moved — a term +// added here is a union alternative, so it can only add keeps, never drop an existing one. /** * Accents are stripped before matching (NFD, then drop combining marks), so a single @@ -54,8 +62,28 @@ const RELEVANCE_RE = new RegExp( // Housing 'housing|tenant|tenancy|rental|renting|landlord|lease\\b', // Digital literacy - 'digital\\s+literacy|computer\\s+(basics|skills|help)|tech\\s+(cafe|help|support)', - 'device\\s+clinic|online\\s+safety|internet\\s+basics', + // `\b` before tech: without it the alternative matches the tail of "Biotech Help" or + // "Fintech Support", both plausible at SFU. It was already missing on the narrower + // `tech` this widened. + 'digital\\s+literacy|computer\\s+(basics|skills|help)|\\btech(nology)?\\s+(cafe|help|support)', + 'device\\s+clinic|laptop\\s+help|online\\s+safety|internet\\s+basics', + // Digital literacy — office suite. Libraries title these by product, not by the + // category words above, so the group missed them entirely: NVCL's "MS Office learn + // and practice: Intro to Word" and Surrey's office sessions are settlement-adjacent + // employment content by any reading. Every office word is QUALIFIED — behind `ms`, + // `microsoft`, or `intro to` — because the bare forms are traps: `\bexcel\b` catches + // "Excel in Your Studies", `\bword\b` catches "Crossword Club" and "Wordplay for + // Toddlers", and `\bpowerpoint\b` catches "PowerPoint Karaoke Night". All four are + // real titles from the live feeds. See lib/relevance_test.ts. + '\\bms\\s+(office|word|excel|powerpoint|outlook)\\b', + 'microsoft\\s+(office|word|excel|powerpoint|outlook|365)\\b', + 'intro(duction)?\\s+to\\s+(excel|word|powerpoint|outlook)\\b|\\bspreadsheet', + // Named settlement programs. Kept to programs run under the same name by more than + // one source, so this doesn't become a list of one-off event names: the Open Door + // community hub is NVCL's and NVDPL's shared newcomer drop-in, 29 occurrences across + // the two 4-month windows, and was the single largest miss in the corpus. Qualified + // so it cannot catch "Open Door Poetry Reading" or "Open Doors Heritage Tour". + 'open\\s+door\\s+(community|drop)', // Health-system navigation 'health\\s+card|family\\s+doctor|\\bmsp\\b|medical\\s+insurance', ].join('|'), diff --git a/supabase/functions/events-crawler/lib/relevance_test.ts b/supabase/functions/events-crawler/lib/relevance_test.ts new file mode 100644 index 0000000..e72aa09 --- /dev/null +++ b/supabase/functions/events-crawler/lib/relevance_test.ts @@ -0,0 +1,138 @@ +// Traps for the settlement-relevance filter. +// +// WHY THIS FILE EXISTS. `isSettlementRelevant` is a pure function shared by every +// relevance-filtered source, so a term added for one library silently changes what all of +// them ingest — and the tempting shorthand for each new term is usually the broken one. +// The 2026-08-08 round wanted NVCL's "MS Office … Intro to Excel"; the obvious `\bexcel\b` +// and `\bword\b` also take "Excel in Your Studies", "Crossword Club" and "Wordplay for +// Toddlers". That hazard was previously recorded in BACKLOG.md prose, which cannot fail a +// run. Here it can. +// +// Every string below is a REAL title from a live feed, captured via +// `dryrun.ts --source --no-filter --titles`. Adding a term means adding its target +// here, and adding any near-miss the corpus shows it brushed against. +// +// RUN (from web-app/): +// deno test supabase/functions/events-crawler/lib/relevance_test.ts + +import { assert } from 'jsr:@std/assert@1'; +import { isSettlementRelevant } from './relevance.ts'; + +/** Real titles the filter must keep, grouped by the term that earns each one. */ +const KEEP: Record = { + 'status & settlement': [ + 'Settlement & Integration Services', + 'Employment Services for Newcomers', + 'Newcomer Family Storytime', + 'International Student Orientation - Fall 2026', + 'Circletime success: Welcoming newcomer children at circletime', + ], + language: [ + 'ESL Conversation Practice', + 'Practice Speaking English: Advanced', + 'English Conversation Program (55+)', + 'English Corner (virtual) — September 2026', + 'Beginner English Corner (in-person) — September 2026', + ], + employment: [ + 'Canadian Job Search Workshop', + 'Ask a Career Counsellor: Resume Workshop', + // Accent folding is load-bearing here — a plain `resume` term misses "Résumé". + 'Résumé Clinic with S.U.C.C.E.S.S.', + ], + 'digital literacy': [ + 'Tech Help', + 'Device Clinic', + // Accent folding again: "Café", and the Farsi title carries combining marks. + 'Tech Café - Capilano (Drop-In)', + 'Farsi Tech Café / تِکِ کافه - Lynn Valley', + // Added 2026-08-08 — `technology` spelled out, 21 rows across NVCL + Surrey. + 'One on One Technology Help', + 'Drop-in Technology Help', + 'Drop-in technology help in Farsi کمک به رفع اشکال تکنولوژی به زبان فارسی', + 'Windows laptop help', + 'Mac laptop help', + ], + 'office suite': [ + 'MS Office learn and practice: Intro to Word', + 'MS Office learn and practice: Intro to Excel', + 'Microsoft Word Basics', + 'Intro to Excel', + 'Spreadsheet Basics', + ], + 'named programs': [ + 'Open door community hub', + 'Open Door Community Hub Drop-In', + ], +}; + +/** + * Titles the filter must reject. The first block is the near-misses — each one is what a + * lazier version of a term above would have caught, so these are the assertions that + * actually earn their keep. The rest is ordinary library programming, there to prove the + * filter still rejects the bulk of a what's-on calendar. + */ +const DROP: Record = { + 'near-misses for the office-suite terms': [ + 'Excel in Your Studies: Study Skills', + 'Crossword Club', + 'Wordplay for Toddlers', + 'Word Whips', + 'Word Games Night', + 'Sword Fighting Demo', + 'Keyword Search Basics', + 'PowerPoint Karaoke Night', + 'Microsoft Flight Simulator Club', + ], + 'near-misses for tech(nology) and laptop help': [ + 'Tech for tots', + 'Technology terminology', + 'Tech Club', + 'SFU Burnaby Science Alive High School Camps: A Byte of Tech', + 'Repair Café', + // `tech` inside a longer word — the reason the alternative carries a leading `\b`. + // Constructed rather than captured: plausible SFU titles, not yet seen in a feed. + 'Biotech Help', + 'Fintech Support', + 'Edtech Café', + ], + 'near-misses for open door': [ + 'Open Door Poetry Reading', + 'Open Doors Heritage Tour', + 'Open sewing studio', + 'Open gaming: Minecraft', + ], + 'near-miss for a bare `orientation`': [ + // Why `orientation` is deliberately absent — see isSettlementRelevant's doc comment. + 'Recording Studio Orientation', + ], + 'ordinary library programming': [ + 'Babytime', + 'Family Storytime', + 'Teen open space', + 'Craft \'N\' Yarn', + 'North Shore knitters', + 'Book Bike at Lonsdale Quay Market', + 'LEGO® Block Party', + 'Simorgh: a self-portrait', + 'Yuehao Hu | MSc Thesis Defence', + 'Adult Colouring Club', + 'Scrabble® Sundays', + ], +}; + +Deno.test('isSettlementRelevant keeps real settlement titles', () => { + for (const [group, titles] of Object.entries(KEEP)) { + for (const title of titles) { + assert(isSettlementRelevant(title), `[${group}] should KEEP: ${title}`); + } + } +}); + +Deno.test('isSettlementRelevant rejects near-misses and general programming', () => { + for (const [group, titles] of Object.entries(DROP)) { + for (const title of titles) { + assert(!isSettlementRelevant(title), `[${group}] should DROP: ${title}`); + } + } +});