Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/claude-code-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,18 @@ jobs:
# github.event.pull_request.user.login == 'new-developer' ||
# github.event.pull_request.author_association == 'FIRST_TIME_CONTRIBUTOR'

# Skipped, not failed, when the token is absent. This repo inherited both
# Claude workflows from its sibling, where CLAUDE_CODE_OAUTH_TOKEN is set
# as a repo secret; here it is not, and the action's own environment
# validation fails the job ("Environment variable validation failed"),
# which put a permanently red check on every pull request. A check that is
# always red teaches everyone to ignore checks. Set the secret to turn
# this on:
# gh secret set CLAUDE_CODE_OAUTH_TOKEN --repo 2scraper/amazon-scraper
# `secrets` is not available in a job-level `if`, so the presence test goes
# through env and is read by the step.
env:
HAS_CLAUDE_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN != '' }}
runs-on: ubuntu-latest
permissions:
contents: read
Expand All @@ -33,6 +45,7 @@ jobs:

- name: Run Claude Code Review
id: claude-review
if: env.HAS_CLAUDE_TOKEN == 'true'
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
Expand Down
13 changes: 13 additions & 0 deletions .github/workflows/claude.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,18 @@ jobs:
(github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) ||
(github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) ||
(github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude')))
# Skipped, not failed, when the token is absent. This repo inherited both
# Claude workflows from its sibling, where CLAUDE_CODE_OAUTH_TOKEN is set
# as a repo secret; here it is not, and the action's own environment
# validation fails the job ("Environment variable validation failed"),
# which put a permanently red check on every pull request. A check that is
# always red teaches everyone to ignore checks. Set the secret to turn
# this on:
# gh secret set CLAUDE_CODE_OAUTH_TOKEN --repo 2scraper/amazon-scraper
# `secrets` is not available in a job-level `if`, so the presence test goes
# through env and is read by the step.
env:
HAS_CLAUDE_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN != '' }}
runs-on: ubuntu-latest
permissions:
contents: read
Expand All @@ -32,6 +44,7 @@ jobs:

- name: Run Claude Code
id: claude
if: env.HAS_CLAUDE_TOKEN == 'true'
uses: anthropics/claude-code-action@v1
with:
claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
Expand Down
27 changes: 26 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,30 @@ CLI toolkit can. Read a **patch** as "fixes" rather than as a promise that
every flag and default is frozen: a default that changes behaviour can ship in
one, and when it does the release notes say so first.

## [0.1.1] — 2026-09-09

Everything here landed after `v0.1.0` was tagged, so the 0.1.0 artifact does
not contain it. Nothing changes for a caller: no flag, column, exit code or
output shape moved.

### Fixed

- **The fixtures carried session material and one real person's data.** Real
page captures bring the session that fetched them (three anonymous, expired
`sessionId` values and their CSRF tokens) and, on a review, a real
customer's display name, profile permalink, review id, photo ids and words.
All of it is now replaced with obvious placeholders — the checks read the
structure of a review, not the person — and a new guard fails the suite if
any of it arrives with a future capture. Found while reviewing what going
public would actually expose.
- **Both Claude workflows failed instead of skipping when their token is
absent.** They were inherited from the sibling repo, where
`CLAUDE_CODE_OAUTH_TOKEN` is a repo secret; unset here, the action's own
environment validation failed the job and put a permanently red check on
every pull request — and a check that is always red teaches everyone to
ignore checks. Setting the secret turns the review back on with no further
change.

## [0.1.0] — 2026-09-08

First release. The repository previously held a different, unrelated Amazon
Expand Down Expand Up @@ -51,7 +75,7 @@ scrapers. **Nothing from the previous code is source- or output-compatible.**
- **A `page` column** beside `position`, because Amazon's `data-index`
restarts on every page and position alone is ambiguous across a paginated
run.
- 239 offline checks in `smoke_test.py`, with fixtures cut from real captures;
- 244 offline checks in `smoke_test.py`, with fixtures cut from real captures;
a daily canary against a real 3-page listing; `tests.yml` on Python 3.9 and
3.12 with an `engine-smoke` job that fails on any unexpected skip.

Expand Down Expand Up @@ -91,4 +115,5 @@ scrapers. **Nothing from the previous code is source- or output-compatible.**
during development was served one; it is exercised only by an offline
fixture written from Amazon's documented markup.

[0.1.1]: https://github.com/2scraper/amazon-scraper/releases/tag/v0.1.1
[0.1.0]: https://github.com/2scraper/amazon-scraper/releases/tag/v0.1.0
23 changes: 15 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -430,7 +430,7 @@ says something about our own two snapshots, not about Amazon. Same for
## Testing

```bash
python3 smoke_test.py # 239 offline checks, no network, no browser
python3 smoke_test.py # 244 offline checks, no network, no browser
pytest # the same suite, as one test
python3 env_config.py # what configuration was picked up (prints no secrets)
```
Expand All @@ -440,13 +440,20 @@ The offline suite passes with no engine library installed at all, and
and fails if anything reports skipped, because "skipped, engine absent" reads
identically to a passing run.

Its fixtures are real captures with `<script>`/`<style>` stripped and nothing
else changed — the parser produces identical rows from the trimmed and
untrimmed forms, which is checked before they are committed. Exactly one
fixture is not a capture: Amazon's image captcha page, written from its
documented markup because no run during development was ever served one. That
is stated in the file, and it means the captcha **solve** path is unproven
against the live site.
Its fixtures are real captures with `<script>`/`<style>` stripped — the parser
produces identical rows from the trimmed and untrimmed forms, which is checked
before they are committed. Two exceptions, both stated in the file itself:

* Amazon's image captcha page is **written from its documented markup**,
because no run during development was ever served one — which is also why
the captcha *solve* path is unproven against the live site.
* The review fixture is **anonymised**. A real capture of a review carries a
real customer's display name, profile permalink, review id and words, and
republishing those inside a public repo is a different act from Amazon
showing them on its own page. Those fields hold obvious placeholders;
everything Amazon generates around them is untouched, because that is what
the checks read. A guard in the suite fails if a session id, a CSRF token or
a real account id ever arrives with a future capture.

Two workflows: `tests.yml` is offline only and never touches amazon.com or a
credential; `canary.yml` runs one real 3-page listing daily and asserts
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "amazon-scraper"
version = "0.1.0"
version = "0.1.1"
description = "Amazon scraper: search results, best-seller grids, product pages and public reviews, across 21 marketplaces."
readme = "README.md"
license = "MIT"
Expand Down
67 changes: 56 additions & 11 deletions smoke_test.py

Large diffs are not rendered by default.

Loading