Repository navigation
Scrub session material and one real person's data out of the fixtures - #2
Merged
Merged
Conversation
Found while checking what making this repo public would actually expose,
rather than by reading the code.
The fixtures are real page captures, and a real page capture carries more than
markup:
* THE SESSION THAT FETCHED IT. Three `sessionId` values, their
`anti-csrftoken-a2z` tokens, `csrfToken` and `offerListingId` values were
baked into the committed HTML. All from anonymous, never-logged-in
browsing and long expired, so nothing here granted access to anything —
but session material does not belong in a repo about to go public, and the
repo's own "no credentials committed" CI check has exactly this spirit
while matching none of these shapes.
* ON A REVIEW, A REAL CUSTOMER. Their display name, profile permalink
(amzn1.account...), review id, the ids of photos they uploaded, their
review title and the words they wrote. All of it is public on amazon.com,
but republishing someone's name and writing inside a public repo is a
different act from Amazon showing them on its own page — and the checks
here read the STRUCTURE of a review, not the person.
Both are now obvious placeholders. Everything Amazon generates around them —
the accessibility filler that is part of the body node's text, the date
wording, the badges, the markup — is untouched, because that is what the
checks actually exercise. The review fixture is now the second of two
fixtures that are not verbatim captures, and it says so where a reader of the
fixtures will look, next to the image-captcha one.
A one-off cleanup would rot on the next capture, so the suite grew a guard:
patterns (not literals) that fail if a real session id, CSRF token or account
id ever arrives with a future fixture, plus an assertion that the review's
author stays a placeholder. Confirmed it fires — a copy of the tree with one
real sessionId put back exits 1 and names it.
244 offline checks. README and CHANGELOG updated to describe both non-verbatim
fixtures rather than claiming everything is a straight capture.
NOTE ON WHAT THIS DOES NOT REACH: the pre-scrub values remain in this
repository's history — in the v0.1.0 tag and in PR #1's own refs, which stay
attached to the PR and cannot be deleted from it. This commit keeps HEAD, all
future clones and every future release artifact clean; removing the history
entirely would need a fresh repository.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Cm7Ebf8wfGq2BvsxFa7rxY
Both were inherited from the sibling repo, where CLAUDE_CODE_OAUTH_TOKEN is
set as a repo secret. It is not set here, so the action's own environment
validation failed the job and put a permanently red check on every pull
request — and a check that is always red teaches everyone to ignore checks.
The presence test goes through job-level env, because the secrets context is
not available in a job-level `if`. Setting the secret turns the review back
on with no further change:
gh secret set CLAUDE_CODE_OAUTH_TOKEN --repo 2scraper/amazon-scraper
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The fixture scrub and the workflow guard landed after v0.1.0 was tagged, so describing them inside 0.1.0 claimed something its artifact does not contain. They get their own release; nothing in it changes a flag, a column, an exit code or the output shape. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while checking what making this repo public would actually expose.
The fixtures are real page captures, and a real capture carries more than markup.
What was in them
"sessionId":"140-5118551-5108920"and two moreanti-csrftoken-a2zvalues,csrfToken,offerListingIda-profile-name,amzn1.account.…,data-reviewid,data-physicalid, the review title and bodyThe session material was anonymous, never logged in, and is long expired — nothing here granted access to anything. But it does not belong in a repo about to be public, and the repo's own "no credentials committed" CI check has exactly this spirit while matching none of these shapes.
The customer's data is public on amazon.com. Republishing someone's name and writing inside a public repo is still a different act from Amazon showing them on its own page — and these checks read the structure of a review, not the person.
What changed
All of it is now an obvious placeholder. Everything Amazon generates around it — the accessibility filler that is part of the body node's text, the date wording, the badges, the markup — is untouched, because that is what the checks actually exercise. Every assertion that named the real values now names the placeholders, and the review fixture is documented as the second of two fixtures that are not verbatim captures, beside the image-captcha one.
A one-off cleanup would rot on the next capture, so the suite grew a guard: patterns, not literals, so a new capture's values are caught too. Confirmed it fires rather than only staying quiet — a copy of the tree with one real
sessionIdput back exits 1 and names it:244 offline checks (was 239).
What this does NOT reach
The pre-scrub values remain in this repository's history — in the
v0.1.0tag and in PR #1's own refs, which stay attached to that PR and cannot be deleted from it. This keepsHEAD, every future clone and every future release artifact clean; removing the history entirely would need a fresh repository. Worth a decision before the repo goes public rather than after.🤖 Generated with Claude Code