Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 2 additions & 8 deletions .github/workflows/canary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#
# This canary is unusual for this family, and the reason is worth stating at
# the top: **the listing path needs no credentials at all**, so it really can
# run nightly from a bare GitHub runner and be green.
# run weekly from a bare GitHub runner and be green.
#
# BBB renders its listings out of its own JSON endpoint, and that endpoint is
# not behind Cloudflare. Measured 2026-09-16 from a datacenter VPS that gets
Expand All @@ -12,8 +12,7 @@
# exactly the case this proves.
#
# The PROFILE path is different — it fetches a Cloudflare-gated page — so that
# half needs a Scraping Browser endpoint, and runs only on the weekly (Monday)
# schedule and on a manual dispatch, since each run bills. Without the secret it FAILS with an
# half needs a Scraping Browser endpoint. Without the secret it FAILS with an
# ::error:: naming the secret rather than going green: a skipped job did not
# test the site, and a green badge over an untested path is a claim nobody
# measured. Exit 3 (blocked), exit 5 (never reached the page) and zero rows
Expand All @@ -30,9 +29,6 @@ on:
schedule:
# 05:40 UTC. Not on the hour: every scheduled workflow on GitHub fires at
# :00 and the queue there is long.
- cron: "40 5 * * *"
# Mondays only: the paid job (profile) runs on this
# schedule (each run bills); the free jobs keep the daily one above.
- cron: "40 5 * * 1"
workflow_dispatch:

Expand Down Expand Up @@ -173,8 +169,6 @@ jobs:

profile:
name: profile (needs a Scraping Browser secret)
# Paid path: weekly schedule or a manual dispatch only.
if: github.event_name != 'schedule' || github.event.schedule == '40 5 * * 1'
runs-on: ubuntu-latest
timeout-minutes: 20
env:
Expand Down
3 changes: 1 addition & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@ When it does, the release notes lead with it.

### Changed

- CI only: the canary's paid `profile` job now runs weekly (Mondays 05:40 UTC)
and on a manual dispatch; the free `listing` job still runs daily.
- CI only: the whole canary, free and paid jobs alike, now runs weekly (Mondays 05:40 UTC) and on a manual dispatch; the daily schedule is removed.

## [1.1.0] — 2026-10-15

Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ Then the rest of the presentation, in the order that matters:
1. `python3 smoke_test.py` green, and the canary dispatched at least once —
including its SKIP branch, which is what runs when the
`BBB_CDP_ENDPOINT` secret is absent. Note that the canary's LISTING job
runs daily with no secrets at all and is expected to be green — that half
runs weekly (Mondays) with no secrets at all and is expected to be green — that half
needs no credentials, and a green badge there is exactly the claim the
README makes. Only the PROFILE job skips without a secret, because it
fetches a Cloudflare-gated page and a GitHub runner is a datacenter
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -272,10 +272,10 @@ installs each engine in its own virtualenv and fails if that engine's group
reports one, because "skipped, engine absent" reads identically to a real
import error.

The [canary](.github/workflows/canary.yml) runs a real 3-page listing daily —
The [canary](.github/workflows/canary.yml) runs a real 3-page listing weekly (Mondays) —
**with no secrets**, because the listing path needs none, which is also what
keeps that claim honest. The profile half bills a Scraping Browser session, so it runs weekly
(Mondays) and on a manual dispatch, and fails, naming the secret, when no
keeps that claim honest. The profile half runs on the same schedule through a Scraping Browser
session, and fails, naming the secret, when no
`BBB_CDP_ENDPOINT` secret is set.

---
Expand Down
2 changes: 1 addition & 1 deletion landing.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Full numbers are in the [repository README](https://github.com/2scraper/bbb-scra
- Three modes: a keyword **search**, a **category** listing, or a single business **profile**
- Reads BBB's own embedded state payload (`window.__PRELOADED_STATE__`) — the same object its endpoint returns — so there is no fragile DOM scraping anywhere in the primary path
- JSON and CSV export, a documented 53-column `Business` schema, and a `.meta.json` sidecar on every run recording status, pages completed, and **BBB's own result arithmetic**
- 340 offline checks, and a daily canary that runs a real 3-page scrape **with no secrets**
- 340 offline checks, and a weekly canary that runs a real 3-page scrape **with no secrets**
- Optional 2Captcha integration, wired in but never required to get started

## Three things about BBB worth knowing before you start
Expand Down
Loading