Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 3 additions & 7 deletions .github/workflows/canary.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# One real run of each mode against goodreads.com, daily. tests.yml
# One real run of each mode against goodreads.com, weekly (Mondays). tests.yml
# (deliberately) never touches the site.
#
# No secrets are needed, and that is the point of running it: the README's
# central claim is "no key, no proxy, no account", and an ungated scheduled
# canary is that sentence under test every morning.
# canary is that sentence under test every Monday.
# Every mode ran from a datacentre VPS when this repo was written
# (2026-09-27), AWS WAF's challenge included, which the browser passes by
# itself; a GitHub runner is a datacentre address too. Its first dispatch
Expand All @@ -26,10 +26,6 @@ name: canary
on:
schedule:
# 05:40 UTC. Not on the hour: every scheduled workflow fires at :00.
- cron: "40 5 * * *"
# Mondays: the same jobs, WITH the paid secrets (proxy, Scraping Browser,
# captcha key). On the daily run above those secrets read as empty, so
# it takes the free no-secret route and bills nothing.
- cron: "40 5 * * 1"
workflow_dispatch:

Expand Down Expand Up @@ -64,7 +60,7 @@ jobs:
pages: 3
floor: 85
env:
GOODREADS_PROXY: ${{ (github.event_name != 'schedule' || github.event.schedule == '40 5 * * 1') && secrets.GOODREADS_PROXY || '' }}
GOODREADS_PROXY: ${{ secrets.GOODREADS_PROXY }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
Expand Down
4 changes: 1 addition & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,7 @@ said so at the top of its release notes.

### Changed

- CI only: the canary's optional paid secret `GOODREADS_PROXY` now reaches it only on a new weekly
run (Mondays 05:40 UTC) and on a manual dispatch. The daily run keeps its
schedule and takes the free route without them, so it bills nothing.
- CI only: the canary now runs weekly (Mondays 05:40 UTC) and on a manual dispatch, with its optional secret `GOODREADS_PROXY` passed on every run; the daily schedule is removed.

## [0.2.0] — 2026-10-15

Expand Down
7 changes: 3 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,12 @@ history needs a decision, not a red check on every push.
Then the rest of the presentation, in the order that matters:

1. `python3 smoke_test.py` green, and the canary dispatched at least once.
It runs daily with no secrets at all and is expected to be green,
It runs weekly (Mondays) with no secrets at all and is expected to be green,
because no mode needs a credential and a green badge there is exactly
the claim the README makes. If a run is refused for the runner's
address, the canary's `GOODREADS_PROXY` secret (an exit elsewhere) is
used on the weekly (Monday) run and on a manual dispatch, with no
workflow edit; the daily run stays on the runner's own address and
bills nothing.
used on every scheduled run and on a manual dispatch, with no
workflow edit.
2. The repo description, homepage and topics set: the description names
the site and what it really publishes, the topics are what people search for.
3. Only then the row in the org profile README — and check it with an
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ Surveyed 2026-09-27:

| | reviews | book detail | listings | notes |
|---|---|---|---|---|
| **this repo** | every review, by cursor, with rating / language / text filters and three orderings | 44 columns, rating distribution, per-language review counts | lists, authors, search, shelves, series | three engines, exit codes that tell blocked from empty from partial, run sidecar, daily canary |
| **this repo** | every review, by cursor, with rating / language / text filters and three orderings | 44 columns, rating distribution, per-language review counts | lists, authors, search, shelves, series | three engines, exit codes that tell blocked from empty from partial, run sidecar, weekly canary |
| [maria-antoniak/goodreads-scraper](https://github.com/maria-antoniak/goodreads-scraper) | capped at 300 | yes | shelves | its README says it no longer works since the redesign |
| [havanagrawal/GoodreadsScraper](https://github.com/havanagrawal/GoodreadsScraper) | no | yes | lists, authors | Scrapy; last push 2024 |
| [Apify epctex/goodreads-scraper](https://apify.com/epctex/goodreads-scraper) | optional | yes | search, list, genre, author, shelf | $15/month rental plus compute |
Expand Down Expand Up @@ -268,7 +268,7 @@ key refused and re-read, a refused parameter, a language the book has no
reviews in, a throttle, and a book run from a listing with one book refused.

The [canary](.github/workflows/canary.yml) runs a real scrape of each mode
daily **with no secrets**, which is what keeps "no account needed" honest.
weekly (Mondays) **with no secrets**, which is what keeps "no account needed" honest.
Its first dispatch (2026-09-27, a GitHub-hosted runner, no proxy, region not
checked) came back complete in all three modes: 300 of 79,610 books on the
Best Books Ever list, 5 books of the Lord of the Rings series with their
Expand Down
Loading