Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/workflows/canary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
#
# scraping-browser the path measured to work, and the one that reads real
# rows: a Scraping Browser profile with an Indian exit,
# on the same schedule. It FAILS, with an error naming
# weekly (Mondays) and on dispatch, since every run
# bills a session. It FAILS, with an error naming
# the secret, when MAKEMYTRIP_CDP_ENDPOINT is not set,
# and it fails on a refusal, on "never obtained" (a 401
# is a refused endpoint: set a fresh secret) and on too
Expand All @@ -39,6 +40,9 @@ on:
schedule:
# 05:50 UTC. Not on the hour: every scheduled workflow fires at :00.
- cron: "50 5 * * *"
# Mondays only: the paid job (scraping-browser) runs on this
# schedule (each run bills); the free jobs keep the daily one above.
- cron: "50 5 * * 1"
workflow_dispatch:

permissions:
Expand Down Expand Up @@ -97,6 +101,8 @@ jobs:
if-no-files-found: ignore

scraping-browser:
# Paid path: weekly schedule or a manual dispatch only.
if: github.event_name != 'schedule' || github.event.schedule == '50 5 * * 1'
runs-on: ubuntu-latest
timeout-minutes: 20
env:
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ said so at the top of its release notes.

## [Unreleased]

### Changed

- CI only: the canary's paid `scraping-browser` job now runs weekly (Mondays
05:50 UTC) and on a manual dispatch; the free `datacentre` job still runs daily.

## [0.2.0] — 2026-10-15

> **Behaviour changes.**
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,8 +263,8 @@ no city.
The [canary](.github/workflows/canary.yml) runs twice over. Daily, with no
secrets, from a GitHub runner: a datacentre, so it asserts the refusal is
REPORTED as one (exit 3, never a crash or an "empty listing"); that job
tests the refusal path, not extraction. On the same schedule, through a
Scraping Browser profile: a real 3-page Goa scrape, checked for prices,
tests the refusal path, not extraction. Weekly (Mondays), because each run
bills a session, through a Scraping Browser profile: a real 3-page Goa scrape, checked for prices,
currency, duplicates and zero ratings. That job fails when its secret is
missing or refused, so the badge is green only when real rows were read.

Expand Down
6 changes: 4 additions & 2 deletions smoke_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -368,8 +368,10 @@ def check_build_query():
_, why = _build(url="https://www.makemytrip.com/hotels/hotel-listing/?checkin=10152026"
"&checkout=10162026&city=CTGOI", city="Goa")
check("--url and --city together are refused", why and "--city" in why, why or "")
q, why = _build(url="https://www.makemytrip.com/hotels/hotel-listing/?checkin=10152026"
"&checkout=10162026&city=CTGOI", sort="price-asc")
# A stay far in the future: a fixed date that has passed is refused
# before --sort is looked at.
q, why = _build(url="https://www.makemytrip.com/hotels/hotel-listing/?checkin=12012030"
"&checkout=12022030&city=CTGOI", sort="price-asc")
check("--sort applies to a --url (the address carries no ordering)",
q is not None and q.sort == "price-asc", why or "")
_, why = _build()
Expand Down
Loading