Skip to content

feat(industry): add Claude Resources as a page source - #80

Merged
PerfectPan merged 2 commits into
PerfectPan:mainfrom
nyx-cell:codex/claude-resources-page-source
Nov 18, 2026
Merged

PerfectPan merged 2 commits into
PerfectPan:mainfrom
nyx-cell:codex/claude-resources-page-source

Conversation

@nyx-cell

Copy link
Copy Markdown
Contributor

Summary

  • Add Claude Resources (https://claude.com/resources/articles) to industry-feeds.json as a page source with pathPrefixes: ["/resources/articles/"]. claude.com ships no feed: /rss and /feed answer 403, /rss.xml, /atom.xml, /blog/rss.xml, /resources/articles/rss.xml answer 404, and the article pages carry no rel="alternate" feed link. robots.txt is Allow: / and points at sitemap.xml, but that sitemap has no lastmod, so it cannot date an entry either.
  • Fall back to a bare-text date when a card declares no <time>. Claude's cards render Article and Nov 10, 2026 as plain <span>s inside the card anchor and use no <time> element anywhere on the page, so publishedAt stayed undefined and every card was dropped. The fallback only runs when no <time> produced a date, and it only accepts a descendant whose entire text is one calendar date (Nov 10, 2026 or 2026-11-10), so a headline or sentence that merely mentions a date cannot become the publication date; a card that declares no date is still skipped rather than given an invented one.
  • Two unit tests: a plain-text-date card is read with its title and description, and a card whose only date appears inside a headline and a sentence is still rejected.
  • Record the source and the fallback in docs/plans/rfc-official-web-page-sources.md.

Validation

  • pnpm check clean (386 files formatted, no lint or type errors); pnpm build ok; pnpm package:check ok.
  • Tests: 833 passed, one failure that also fails on unmodified main in this environment — src/app/products/subscriptions/flow.test.ts > keeps RSS candidates when GitHub Home is unavailable (5s timeout). tests/golden pass (35 tests).
  • Live CLI dry-run against https://claude.com/resources/articles: source reports status: "ok", itemCount: 11, and industry --day 2026-11-13 --dry-run surfaces the two entries dated that day, including Building always-on agents with its own description from the card.
  • No-regression check on the full 23-source registry for --day 2026-11-13: candidate set before/after differs by exactly the two added Claude entries, with zero title or summary changes on the other 24 candidates. Existing sources use <time>, so the fallback does not execute for them.
  • The production client UA (rss-summary/0.1) receives the same rendered listing as a browser UA, so no browser run or proxy feed is needed.

Risks

  • Text-date matching is deliberately narrow: a card that prints a date in a shape outside Nov 10, 2026 / 2026-11-10 still yields no date, and its entries stay excluded instead of being misdated.
  • claude.com markup may drift; a failed source surfaces as status: "failed" in the source audit rather than an empty brief.
  • Unrelated observation, unchanged by this PR: the three Anthropic page sources report failed (returned no dated links matching its path prefixes) in this environment.

claude.com serves no feed: /rss and /feed answer 403, /rss.xml, /atom.xml,
/blog/rss.xml and /resources/articles/rss.xml answer 404, and the article pages
carry no alternate feed link. The listing page is server-rendered and dated, so
the page adapter can read it once it handles one more markup shape.

Claude's cards print the date as plain text in a <span> and use no <time>
element at all, so `publishedAt` stayed undefined and every card was dropped.
Extraction now falls back to the first descendant whose entire text is one
calendar date, and only when no <time> produced a date. A headline or sentence
that merely mentions a date does not match, and a card that declares no date is
still skipped instead of being given an invented one.

Registered at `https://claude.com/resources/articles`, entries under
`/resources/articles/`.
@nyx-cell
nyx-cell force-pushed the codex/claude-resources-page-source branch from 24f3304 to 4ddeaf1 Compare November 16, 2026 11:11
@PerfectPan
PerfectPan merged commit be10db2 into PerfectPan:main Nov 18, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants