feat(events-crawler): LiveWhale (SFU) + Communico RSS (NVDPL) adapters - #87
Conversation
Third Phase-2 PR. Two new adapters on the registry from the previous PR, both staged
disabled and both relevance-filtered. Registered in lib/sources.ts alongside the existing
sources, so the dryrun harness picks them up with no separate wiring.
SFU — LiveWhale JSON, not the ICS the plan called for. events.sfu.ca publishes both, but
the JSON feed carries structured fields plus explicit cancelled / online / all-day flags,
where the ICS would need RFC-5545 line unfolding, `\,` unescaping and CRLF handling to
reach less information. Two properties of the feed drove the design, both measured:
* Recurring events repeat the SAME url across every occurrence — one gallery exhibition
alone accounted for 852 of the 1,000 items. Since url becomes external_link, which is
UNIQUE, they collapse to a single row regardless; the only question is which
occurrence survives. Sorting before deduping makes that the soonest one, so the stored
row is the next date a user could actually attend, and one busy series can no longer
consume the whole MAX_PER_ORG cap. Live effect: 28 in-window relevant occurrences
dedupe to 2 real events.
* 927 of 1,000 events are all-day, where date_utc is local midnight. The time carries no
meaning for those, only the date — which is all either render path shows.
The feed is hard-capped at 1,000 items over roughly four weeks; ?max=, /max|N,
/starts_after|, /range| and /days|N were all tested and none widen it. Short of the
4-month window but harmless: the cron is weekly, so events are ingested as the horizon
rolls forward.
NVDPL — Communico RSS, the only readable surface (the calendar is a JS SPA, and event
pages are client-rendered behind Cloudflare). Two limitations are load-bearing:
* The feed carries no location field of any kind, so the source supplies
defaultLocation. Per-branch scoping is impossible anyway: the ?l=<branch> filter the
site's own UI uses is ignored by the RSS endpoint, which returns the identical
system-wide set. Hence NVDPL-wide rather than Lynn Valley.
* The event datetime exists only inside the description ("Date/Time: Thu, 30 Jul 2026,
10:00am - 11:00am"); pubDate is the listing's publish date, often months earlier, and
is not used. All 100 items parse with one pattern; an item that fails to parse is
skipped rather than guessed at.
Those times are local wall-clock with no offset, so they go through a new DST-aware
zonedWallClockToUtc rather than being read as UTC (7-8 hours out) or shifted by a
hardcoded offset (an hour out for half the year). It derives the offset from Intl at the
target instant and re-checks after correcting, so DST transitions resolve correctly.
Verified across seven cases including both 2026 transitions.
Both new adapters carry the two fixes from the previous PR's review: the relevance filter
runs on the full title before truncation, and row mapping uses Promise.allSettled so one
failed cover lookup cannot discard the whole source for a run.
Live dry-run: sfu 2 rows, nvdpl 16 rows (0 unparsable dates), and the five enabled orgs
are unchanged (25/24/14/0/2).
Everything new stays enabled: false. No migration, cron, or DB object touched; nothing
deployed; no writes to shared prod.
Verification:
deno check (incl. dryrun.ts) → clean
npx tsc --noEmit → clean
npx eslint next.config.ts → clean
DST conversion suite → 7/7 pass
dryrun sfu / nvdpl → 2 / 16 rows
migrations/backfills guard → empty
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughChangesEvent adapter expansion
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
@coderabbitai review |
✅ Action performedReview finished.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@supabase/functions/events-crawler/adapters/livewhale.ts`:
- Around line 138-145: Update the candidate sort in the flow using
candidates.sort so startIso values are compared with relational string
comparison rather than localeCompare, preserving ascending chronological
ordering for fixed-format ISO UTC timestamps.
- Around line 103-113: Update the event object construction in the livewhale
adapter to compare the parsed end time from ev.date2_utc with startIso,
assigning endIso only when it is not earlier than the start; otherwise return
null or omit it according to the existing event schema convention. Preserve the
current UTC conversion for valid end dates.
- Around line 80-82: Update toCandidate to reject candidates unless the trimmed
url and non-empty thumbnail are absolute http(s) URLs before calling
resolveCover or creating the row. Apply the same validation to both
external_link and the thumbnail input, preserving the existing null return for
invalid required fields and preventing relative paths such as /events/12345 from
reaching resolveImageUrl.
In `@supabase/functions/events-crawler/lib/dates.ts`:
- Around line 67-74: Update zonedWallClockToUtc around the Date.UTC conversion
to validate year, month, day, hour, and minute before constructing the instant,
rejecting overflowed or otherwise impossible calendar values instead of allowing
Date.UTC normalization. Adjust the DST resolution logic to detect spring-forward
gaps and choose the first valid post-transition local instant, including the
2026-03-08 02:30 America/Vancouver case, and document or enforce this convention
in the function’s JSDoc. Add regression tests covering invalid date overflow and
spring-forward resolution.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 2a163d7f-37ee-470a-968d-892498c6dfab
📒 Files selected for processing (6)
next.config.tssupabase/functions/events-crawler/adapters/communico.tssupabase/functions/events-crawler/adapters/livewhale.tssupabase/functions/events-crawler/lib/dates.tssupabase/functions/events-crawler/lib/sources.tssupabase/functions/events-crawler/lib/types.ts
| const url = typeof ev.url === 'string' ? ev.url.trim() : ''; | ||
| const startIso = toIsoUtc(ev.date_utc); | ||
| if (!title || !url || !startIso) return null; // NOT NULL columns |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Inspect URL normalization helpers and the image allowlist.
fd -t f 'images.ts' supabase/functions/events-crawler --exec cat -n
fd -t f 'next.config.ts' --exec cat -n
rg -n -C3 'resolveImageUrl|new URL\(' supabase/functions/events-crawlerRepository: UnifyCN/web-app
Length of output: 15632
🏁 Script executed:
#!/bin/bash
# Map and inspect the LiveWhale adapter and shared URL guard.
fd -t f 'livewhale.ts' supabase/functions/events-crawler --exec sh -c 'echo "===== $1 ====="; wc -l "$1"; sed -n "1,180p" "$1"' sh {}
fd -t f 'ssrf.ts' supabase/functions/events-crawler/_shared --exec sh -c 'echo "===== $1 ====="; cat -n "$1"' sh {}
rg -n -C4 'liveWhale|events-crawler|enabled|toCandidate|external_link|thumbnail|url' supabase/functions/events-crawler supabaseRepository: UnifyCN/web-app
Length of output: 50372
🏁 Script executed:
#!/bin/bash
set -e
# Inspect the URL guard implementation and any URL parsing behavior currently in the repository.
rg -n -C6 'isPublicHttpUrl|function isPublicHttpUrl|const isPublicHttpUrl|new URL\(|URL protocol|^function ' supabaseRepository: UnifyCN/web-app
Length of output: 50373
🌐 Web query:
SFU LiveWhale public events JSON url field relative path examples
💡 Result:
In LiveWhale, which powers the SFU Events Calendar, event URLs and similar links are typically stored as relative paths to allow for portable, environment-independent linking [1][2][3]. Regarding the public events JSON API: 1. URL Structure: LiveWhale CMS generates dynamic content URLs in a /{id}-{title} format (e.g., /1234-my-event-title) [2]. Within the LiveWhale templating system and calendar components, the href variable explicitly provides these as relative paths (e.g., /event/123456-my-event-title) [3]. 2. JSON API Output: When requesting event data via the LiveWhale JSON API (e.g., /live/json/events), the fields returned—such as those found within the data object—typically include these relative path references for event details [4][5]. 3. Configuration: If you are configuring a LiveWhale widget, the show_calendar_link setting specifically allows for a forward slash (/) representing the main calendar or a relative URL (e.g., /admissions) to link to specific landing pages [1]. If you are developing an integration and need to construct absolute URLs from these relative paths, you should prepend your base domain (e.g., https://events.sfu.ca) to the relative path provided by the API [6][5]. For direct user interaction, such as configuring calendar subscriptions or website widgets, SFU's documentation confirms the use of these paths relative to the domain root (e.g., /live/ical/events/...) [7].
Citations:
- 1: https://support.livewhale.com/docs/widget-and-api-settings/
- 2: https://support.livewhale.com/live/blurbs/custom-urls
- 3: https://support.livewhale.com/docs/calendar-components/
- 4: https://support.livewhale.com/live/blurbs/json-api
- 5: https://support.livewhale.com/calendar-onboarding/widgets-and-api/
- 6: https://support.livewhale.com/live/blurbs/api
- 7: https://www.sfu.ca/communicators-toolkit/events-calendar/using-sfu-events-calendar.html
🏁 Script executed:
#!/bin/bash
# Locate the exact file referenced by the import and inspect the guard implementation without running repository code.
printf 'Files named ssrf.ts:\n'
fd -t f 'ssrf.ts' supabase/functions/events-crawler/_shared || true
file="$(fd -t f 'ssrf.ts' supabase/functions/events-crawler/_shared | head -n 1)"
if [ -n "$file" ]; then
printf '\n===== %s =====\n' "$file"
wc -l "$file"
cat -n "$file"
fiRepository: UnifyCN/web-app
Length of output: 432
Abort on relative LiveWhale links before creating rows.
toCandidate only trims ev.url and stores it as external_link; the same candidate also passes ev.thumbnail directly into resolveCover. SFU LiveWhale can return /events/12345-style paths, and resolveImageUrl treats /events/12345 as a public HTTP URL because isPublicHttpUrl('/events/12345') is true. A subsequent HEAD request resolves that relative value against the crawler runtime origin, not https://events.sfu.ca, so an inbound relative link can write an invalid canonical key. Drop the candidate unless both url and non-empty thumbnail resolve to absolute http(s) URLs, or normalize them before storage.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@supabase/functions/events-crawler/adapters/livewhale.ts` around lines 80 -
82, Update toCandidate to reject candidates unless the trimmed url and non-empty
thumbnail are absolute http(s) URLs before calling resolveCover or creating the
row. Apply the same validation to both external_link and the thumbnail input,
preserving the existing null return for invalid required fields and preventing
relative paths such as /events/12345 from reaching resolveImageUrl.
Three of four findings applied as reported; the fourth was half right and is fixed on its
valid half only.
1. zonedWallClockToUtc rejected invalid dates and resolves DST gaps forward (Major,
correctness). Two real bugs, both confirmed by probe before fixing:
* Date.UTC normalises overflow silently, so "31 Apr 2026" became May 1 and "30 Feb"
became Mar 2. A malformed feed date was therefore stored as a real event on the
wrong day instead of being skipped. Calendar components are now range-checked and
the constructed date is round-tripped, so an impossible date returns null — which
communico already treats as unparsable and counts.
* A time inside the spring-forward gap resolved BACKWARDS: 2026-03-08 02:30 came out
as 01:30 local, before the gap, contradicting this function's own JSDoc. Both
candidate instants are now round-tripped: exactly one matching wins; both matching
means the time is ambiguous (fall-back) and the earlier is taken as the first
occurrence; neither matching means it is in the gap and the later is taken, the
first instant after the transition. JSDoc rewritten to state the convention it
actually implements.
2. LiveWhale url must be an absolute http(s) URL (Major, as reported — but for a different
reason than given). CodeRabbit's stated rationale was that isPublicHttpUrl('/events/1')
returns true and would make resolveImageUrl fetch a relative thumbnail against the
crawler's own origin. That is incorrect: `new URL()` throws without a base, so the guard
fails closed. Verified directly — isPublicHttpUrl('/events/12345') === false, as are
'events/12345' and '//evil.com/x'. There is no SSRF exposure here and a relative
thumbnail simply falls through to the stock image tiers.
The finding is still worth acting on for its other half: LiveWhale documents relative
`url` values, and external_link is the canonical unique key and a rendered link, so a
relative value would be a broken row. Absolute-only now, with the SSRF reasoning
corrected in the code comment so the next reader isn't misled.
3. Drop a LiveWhale end time at or before the start (Trivial). communico and surrey
already did this; livewhale didn't.
4. Sort ISO timestamps with relational operators rather than localeCompare (Trivial).
Applied across all three adapters that sort, not just the one flagged.
Behaviour is unchanged on live data: all nine registered sources return the same row
counts as before (25/24/14/0/2, westvan 5, vpl 25, sfu 2, nvdpl 16).
Verification:
DST/calendar suite expanded 7 → 15 cases, all pass — now covering the spring-forward
gap, the ambiguous fall-back hour, Apr 31 / Feb 30 / month 13 / hour 24 / minute 60
rejection, and the 2028 leap day.
deno check (incl. dryrun.ts) → clean
npx tsc --noEmit / eslint → clean
migrations/backfills guard → empty
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Applied in 1.
2. LiveWhale absolute-URL guard (Major) — applied, but the stated rationale is incorrect. So there's no SSRF exposure — a relative thumbnail is rejected and falls through to the stock image tiers. I applied the guard anyway for the finding's other half, which is valid: LiveWhale documents relative 3. Drop an end time at or before the start (Trivial) — 4. Relational ISO comparison instead of Regression: all nine registered sources return identical row counts (25/24/14/0/2, westvan 5, vpl 25, sfu 2, nvdpl 16). The DST/calendar suite grew 7 → 15 cases, all passing. @coderabbitai review |
|
I will review commit ✏️ Learnings added
✅ Action performedReview finished.
|
|
@coderabbitai full review |
✅ Action performedFull review finished. Your included review limit is currently reached under our Fair Usage Limits Policy. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 37 minutes. |
|
@coderabbitai full review |
✅ Action performedFull review finished. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@supabase/functions/events-crawler/adapters/communico.ts`:
- Line 37: Update the Communico description parsing around DESCRIPTION_RE to
remove a surrounding CDATA wrapper before extracting the date/body and passing
the content to htmlToParagraphs(), ensuring descriptionHtml and stored
descriptions contain no literal CDATA markers. Revise the existing “two entity
layers” comment to document the CDATA layer as well.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 14fb43b5-3c67-4f8f-b95f-89a2ac3b55a1
📒 Files selected for processing (7)
next.config.tssupabase/functions/events-crawler/adapters/bibliocommons.tssupabase/functions/events-crawler/adapters/communico.tssupabase/functions/events-crawler/adapters/livewhale.tssupabase/functions/events-crawler/lib/dates.tssupabase/functions/events-crawler/lib/sources.tssupabase/functions/events-crawler/lib/types.ts
| const ITEM_RE = /<item\b[^>]*>([\s\S]*?)<\/item>/gi; | ||
| const TITLE_RE = /<title>([\s\S]*?)<\/title>/i; | ||
| const LINK_RE = /<link>([\s\S]*?)<\/link>/i; | ||
| const DESCRIPTION_RE = /<description>([\s\S]*?)<\/description>/i; |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== lib/text.ts implementation (clean / decodeEntities / htmlToParagraphs) =="
fd -a 'text\.ts$' supabase/functions/events-crawler/lib | xargs -I{} cat -n {}
echo "== search for existing CDATA handling anywhere in events-crawler =="
rg -n -i 'cdata' supabase/functions/events-crawler -S || echo "no CDATA handling found"
echo "== search for communico adapter tests/fixtures with CDATA =="
fd -a -i 'communico' supabase/functions/events-crawler | xargs -I{} rg -n -i 'cdata|description' {} || trueRepository: UnifyCN/web-app
Length of output: 5689
🌐 Web query:
NVDPL Communico rss /rss feed description CDATA example
💡 Result:
NVDPL (North Vancouver District Public Library) uses Communico, a cloud-based suite for libraries, to manage and display content [1][2]. While there is no specific "NVDPL Communico" technical documentation detailing custom RSS CDATA implementations, Communico provides standard mechanisms for generating RSS event feeds [3]. In RSS 2.0, CDATA sections are the recommended method for including HTML markup or special characters within tags like or <title>, ensuring the XML remains valid and is parsed correctly by aggregators [4][5][6]. Example of CDATA usage in an RSS tag: Join us for our upcoming library event! Registration required.
]]> Key points for CDATA usage in RSS: 1. XML Compliance: The element itself must not be inside the CDATA tag; only the content within it should be [4]. 2. Validation: Using CDATA prevents XML parsers from failing when they encounter HTML characters like <, >, or &, which would otherwise need to be manually escaped (e.g., <, >) [7][5][8]. 3. Aggregator Support: Most modern RSS readers handle CDATA sections natively, making it a standard practice for robust feed generation [4]. If you are encountering issues with feed validation or rendering, ensure your RSS generator is correctly wrapping only the content body in the CDATA section [4].Citations:
- 1: https://www.nvdpl.ca/news
- 2: https://communico.us/home
- 3: https://communicocollege.com/rss-event-feeds
- 4: https://stackoverflow.com/questions/7272857/issuses-with-using-cdata-to-validate-rss-feeds
- 5: https://www.rssboard.org/rss-draft-1
- 6: https://www.rssboard.org/rss-draft-1-09
- 7: https://www.rssboard.org/rss-encoding-examples
- 8: https://core.trac.wordpress.org/ticket/59082
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== communico adapter relevant section =="
sed -n '1,230p' supabase/functions/events-crawler/adapters/communico.ts | cat -n
echo "== fetch live NVDPL RSS /rss item description wrapper =="
tmpdir="$(mktemp -d)"
curl -fsSL -A 'Mozilla/5.0' 'https://nvdpl.libnet.info/rss' -o "$tmpdir/rss.xml" \
|| { python3 - <<'PY'
print("curl failed")
PY
exit 0; }
python3 - <<'PY'
from pathlib import Path
xml = Path('tmpdir rss.xml replaced').read_text()
text = Path('./tmpdir/rss.xml').read_text() if Path('./tmpdir/rss.xml').exists() else ''
root = text.strip()
# Print first <item> and description values with wrappers
import xml.etree.ElementTree as ET
try:
root = ET.fromstring(text)
ns = {'rdf': 'http://purl.org/rss/1.0/resources/rdf#', 'item': 'http://purl.org/rss/1.0/modules/content/'}
# Handle common RSS namespaces if needed by raw text search first
descriptions = []
import re
for i,m in enumerate(re.finditer(r'<item\b.*?>', root, re.S)):
start = m.start()
item_match = re.search(r'<item\b.*?</item>', root[start:], re.S)
if not item_match:
continue
item = item_match.group(0)
m2 = re.search(r'<description\b[^>]*>([\s\S]*?)</description>', item, re.I)
if m2:
desc = m2.group(1)
short = desc[:300]
descriptions.append((i+1, short.startswith('<![CDATA['), desc.startswith('<![CDATA[['), short))
if i >= 9:
break
for idx, is_cdata_like, is_exact_cdata, short in descriptions:
print(f"item {idx}: cdata-like={is_cdata_like} exact-cdata={is_exact_cdata} text={short!r}")
except Exception as e:
print(f"parse failed: {type(e).__name__}: {e}")
PYRepository: UnifyCN/web-app
Length of output: 11204
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== fetch live NVDPL RSS /rss item description wrapper =="
tmpdir="$(mktemp -d)"
curl -fsSL -A 'Mozilla/5.0' 'https://nvdpl.libnet.info/rss' -o "$tmpdir/rss.xml" \
|| { echo "curl failed"; exit 0; }
python3 - <<'PY'
from pathlib import Path
import re
import xml.etree.ElementTree as ET
text = Path('rss.xml').read_text()
try:
root = ET.fromstring(text)
print("root:", root.tag)
except Exception as e:
print("tree parse failed:", type(e).__name__, e)
items = re.findall(r'<item\b[^>]*>([\s\S]*?)</item>', text, re.I)
for i, item in enumerate(items[:20]):
m = re.search(r'<description\b[^>]*>([\s\S]*?)</description>', item, re.I)
if not m:
continue
desc = m.group(1)
print(f"item {i+1}:")
print(" raw_begins=[CDATA?:", repr(desc[:12]))
print(" contains_cdata_marker:", "<![CDATA[" in desc)
wrapped = re.match(r'^<!\[CDATA\[(.*?)\]\]>$', desc.strip(), re.S)
if wrapped:
print(" contains_cdata_wrapped:", repr(wrapped.group(1)[:200]))
else:
print("no items/descriptions parsed")
PYRepository: UnifyCN/web-app
Length of output: 728
🏁 Script executed:
#!/bin/bash
set -euo pipefail
tmpdir="$(mktemp -d)"
curl -fsSL -A 'Mozilla/5.0' 'https://nvdpl.libnet.info/rss' -o "$tmpdir/rss.xml" || { echo "curl failed"; exit 0; }
echo "== first 3 <description> values from live feed =="
python3 - <<'PY'
import re
from pathlib import Path
text = Path('rss.xml').read_text()
for i, item in enumerate(re.findall(r'<item\b[^>]*>([\s\S]*?)</item>', text, re.I), start=1):
m = re.search(r'<description\b[^>]*>([\s\S]*?)</description>', item, re.I)
if not m:
continue
desc = m.group(1)
cdata_wrap = re.match(r'^<!\[CDATA\[(.*?)\]\]>$', desc.strip(), re.S)
print(f"item {i}: cdata_wrapped={cdata_wrap is not None}")
if cdata_wrap:
print(f" raw_prefix={desc[:40]!r}")
print(f" after_cdata_prefix={cdata_wrap.group(1)[:200]!r}")
else:
print(f" raw_prefix={desc[:200]!r}")
if i >= 3:
break
PYRepository: UnifyCN/web-app
Length of output: 721
Handle CDATA before parsing Communico descriptions
DESCRIPTION_RE captures <description> contents literally, but Communico can emit <![CDATA[ ... ]]> for HTML. decodeEntities() / htmlToParagraphs() only strip HTML-like blocks and decode entities; they do not remove a surrounding CDATA wrapper, so descriptionHtml and the stored description can contain literal <![CDATA[ / ]]> text.
Strip the CDATA wrapper before extracting the date/body and passing the value through htmlToParagraphs(), and update the “two entity layers” comment to include this one.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@supabase/functions/events-crawler/adapters/communico.ts` at line 37, Update
the Communico description parsing around DESCRIPTION_RE to remove a surrounding
CDATA wrapper before extracting the date/body and passing the content to
htmlToParagraphs(), ensuring descriptionHtml and stored descriptions contain no
literal CDATA markers. Revise the existing “two entity layers” comment to
document the CDATA layer as well.
|
Probed the live feed before acting on this one, and the CDATA finding does not hold for this source. Evidence —
That is exactly the shape the adapter's "two entity layers" comment documents, and exactly what Worth noting the analysis chain behind the finding read a local On the hypothetical, for the record: if CDATA ever did appear, Leaving the code as-is rather than adding a strip for a layer this feed doesn't produce. |
1 participant
What & why
Third of four Phase-2 PRs. Adds two adapters on the registry from #84. Both sources ship
enabled: falseand both are relevance-filtered.Decisions baked in
SFU — LiveWhale JSON, not the ICS the plan specified
events.sfu.capublishes both. The JSON feed carries structured fields plus explicitis_canceled/is_online/is_all_dayflags; the ICS would need RFC-5545 line unfolding,\,unescaping and CRLF handling to arrive at strictly less information. Two measured properties drove the design:Recurring events repeat the same
urlacross every occurrence. One gallery exhibition alone accounted for 852 of the 1,000 feed items. Sinceurlbecomesexternal_link— which is UNIQUE — those collapse to a single row no matter what; the only question is which occurrence survives. Sorting before deduping makes it the soonest, so the stored row is the next date a user could actually attend, and one busy series can't consume the wholeMAX_PER_ORGcap. Live effect: 28 in-window relevant occurrences → 2 real events.927 of 1,000 events are all-day, where
date_utcis local midnight. The time is meaningless for those, only the date matters — which is all either render path displays.The feed is hard-capped at 1,000 items over ~4 weeks.
?max=,/max|N,/starts_after|…,/range|…and/days|Nwere all tested; none widen it. Short of the 4-month window but harmless — the cron is weekly, so events are ingested as the horizon rolls forward.NVDPL — Communico RSS, NVDPL-wide rather than Lynn Valley
RSS is the only readable surface: the calendar is a JS SPA and event pages are client-rendered behind Cloudflare. Two limitations are load-bearing:
events.locationis NOT NULL, so the source suppliesdefaultLocation. Per-branch scoping is impossible regardless: the?l=<branch>filter the site's own UI uses is ignored by the RSS endpoint, which returns the identical system-wide set.Date/Time: Thu, 30 Jul 2026, 10:00am - 11:00am).pubDateis the listing's publish date — often months earlier — and is deliberately unused. All 100 items parse with one pattern; an item that fails to parse is skipped, not guessed at.Those times are local wall-clock with no offset, so they go through a new DST-aware
zonedWallClockToUtcrather than being read as UTC (7–8 hours out) or shifted by a hardcoded-07:00(an hour out for half the year).Changes
adapters/livewhale.ts,adapters/communico.ts— new.lib/dates.ts—zonedWallClockToUtc+ anIntl-derivedzoneOffsetMs.lib/types.ts—SourceKindgainslivewhale/communico.lib/sources.ts— registersfuandnvdpl.next.config.ts— allowlistevents.sfu.caand the Communico image bucket, pinned to the exact hostevents-calendar-public-us-east-2.s3.us-east-2.amazonaws.com(never an*.amazonaws.comwildcard).Both adapters carry the two fixes from #84's review: relevance filtering runs on the full title before truncation, and row mapping uses
Promise.allSettled.Nothing changes in production. Both sources are
enabled: false; no migration, cron job, Vault secret or DB object touched; the function was not deployed; no writes to shared prod.Verification
deno check(incl.dryrun.ts)npx tsc --noEmit/eslintdryrun --source sfudryrun --source nvdplReviewer notes
location: "Virtual and In-Person"butis_online: null, so it typesin-person. Reading the location prose to infer hybrid would be guessing; the feed's own flag is the honest signal.in-person: the feed carries no online/virtual signal, anddefaultLocationis a physical library system.🤖 Generated with Claude Code
Summary by CodeRabbit