Skip to content

Commit 0a5cdcb

Browse files
Shivanshu-07claude
andauthored
feat(advanced): PER-8195 — advanced example for percy-selenium (Python) (#22)
* feat(advanced): PER-8195 Phase 1 stub — matrix.yml + README Phase 1 stub for PER-8195 (advanced example for every Percy SDK sample repo). matrix.yml populated from SDK API research, README placeholder notes test code is TO BE WRITTEN. Carries the Phase 0 pattern: per-SDK matrix.yml is the canonical machine-readable source of truth; advanced/README.md table + the aggregate docs/advanced-example-feature-matrix.md grid are generated downstream. No test code committed — needs per-SDK smoke-test validation against the matching SDK toolchain (next step). Most rows currently 'Planned'; some 'Covered' rows reflect behavior automatic via the basic SDK usage (cross-origin iframe handling, cookie capture, environmentInfo). See parent: docs/plans/2026-05-19-001-feat-per-8195-advanced-sdk-examples-plan.md * feat(advanced): PER-8195 Phase 1 — add advanced example for percy-selenium (Python) Adds advanced/ exercising the full applicable percy-selenium Python SDK feature surface. 11 pytest functions in tests/test_todomvc_advanced.py, one per matrix row: widths, min_height, enable_javascript, responsive_snapshot_capture, readiness, sync, labels, test_case, device_pixel_ratio, browsers, snake_case + camelCase dual-naming demo. scope, dom_transformation, regions, discovery marked N/A — not exposed as kwargs in percy-selenium 2.1.2. CI advanced job uses firefox + selenium-webdriver + `percy exec --testing` and asserts via the shared helper. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ci): run advanced job like the basic job; drop unpublished shared helper (PER-8195) The advanced job fetched an assertion helper from percy/percy-public-repos-parent/main/scripts/ — that repo is private and the script was never published, so curl -f 404d and failed every advanced run. Master has no --testing mode / matrix-coverage gate; it just runs the spec under percy exec with the repo PERCY_TOKEN. Mirror that: run the advanced suite (which already wraps percy exec) with PERCY_TOKEN, dropping the jq/yq install, the external curl helper, the --testing /test/requests capture, and the matrix assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: limit GITHUB_TOKEN to read-only permissions (PER-8195) Resolves CodeQL "Workflow does not contain permissions" finding by adding a top-level permissions: contents: read block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: vendor todomvc CSS locally so advanced snapshots render styled (PER-8195) The advanced suite serves the app from its served root, but index.html linked node_modules/todomvc-app-css/index.css which only exists after a repo-root npm install the advanced flow never runs -> CSS 404'd and snapshots rendered unstyled. Vendor index.css beside index.html and point the href at css/index.css (matches java-selenium's existing approach). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: replace removed percy/exec-action with direct percy exec run (PER-8195) percy/exec-action@v0.3.1 is removed/unresolvable so the basic job failed. The test script already self-wraps 'percy exec', so run it directly with PERCY_TOKEN in env. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 6861809 commit 0a5cdcb

12 files changed

Lines changed: 765 additions & 2 deletions

File tree

.github/workflows/test.yml

Lines changed: 35 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,15 @@
11
name: Tests
2+
3+
# PER-8195: explicitly use `pull_request` only. `pull_request_target` is
4+
# forbidden — it checks out attacker-controlled code with full secret access.
25
on: [push, pull_request]
6+
7+
# Limit GITHUB_TOKEN to read-only (CodeQL: workflow-does-not-contain-permissions)
8+
permissions:
9+
contents: read
10+
311
jobs:
4-
build:
12+
basic:
513
runs-on: ubuntu-latest
614
steps:
715
- uses: actions/checkout@v4
@@ -22,6 +30,32 @@ jobs:
2230
key: v1/${{ runner.os }}/node-18/${{ hashFiles('**/package-lock.lock') }}
2331
restore-keys: v1/${{ runner.os }}/node-18/
2432
- name: Run tests
33+
env:
34+
PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
2535
run: make test
36+
37+
advanced:
38+
# PER-8195 advanced example. Runs the advanced suite the same way the
39+
# basic job runs its suite — under Percy with the repo's PERCY_TOKEN.
40+
# No testing-mode coverage gate or external assertion helper (matches master).
41+
runs-on: ubuntu-latest
42+
timeout-minutes: 15
43+
defaults:
44+
run:
45+
working-directory: advanced
46+
steps:
47+
- uses: actions/checkout@v3
48+
- uses: actions/setup-python@v4
49+
with:
50+
python-version: '3.11'
51+
- uses: actions/setup-node@v3
52+
with:
53+
node-version: 18
54+
- name: Install percy CLI
55+
run: npm install --no-save @percy/cli@^1.31.13
56+
- name: Install advanced/ python dependencies
57+
run: make install
58+
- name: Run advanced tests
2659
env:
2760
PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
61+
run: make test-advanced

README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,9 +5,18 @@
55
Example app showing integration of [Percy](https://percy.io/) visual testing
66
into Python Selenium tests.
77

8+
> **New:** This repo ships an [`advanced/`](./advanced) example covering the full applicable Percy SDK feature surface for `percy-selenium`. See the [Percy SDK Feature Matrix](https://docs.percy.io/docs/sdk-feature-matrix) for cross-SDK coverage.
9+
810
Based on the [TodoMVC](https://github.com/tastejs/todomvc) [VanillaJS](https://github.com/tastejs/todomvc/tree/master/examples/vanillajs)
911
app, forked at commit [4e301c7014093505dcf6678c8f97a5e8dee2d250](https://github.com/tastejs/todomvc/tree/4e301c7014093505dcf6678c8f97a5e8dee2d250).
1012

13+
## Examples
14+
15+
| Example | What it shows | Run command |
16+
|---|---|---|
17+
| `./` (basic, at repo root) | Minimum viable integration: a few `percy_snapshot(browser, name)` calls. Start here. | `make test` |
18+
| [`./advanced/`](./advanced) | Full applicable Percy SDK feature surface: widths, minHeight, enable_javascript, readiness, responsive_snapshot_capture, sync, snake_case + camelCase dual naming, etc. pytest-driven. See [`advanced/README.md`](./advanced/README.md) for the matrix-row coverage table. | `cd advanced && make test` |
19+
1120
## Selenium Python Tutorial
1221

1322
This tutorial assumes that you're already familiar with Python & Selenium and focuses on using them

advanced/.gitignore

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
.venv/
2+
__pycache__/
3+
*.pyc
4+
advanced-requests.json
5+
.pytest_cache/

advanced/.percy.yml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# PER-8195 — advanced example global config for percy-selenium (python).
2+
# Per-snapshot options in tests/test_todomvc_advanced.py override these.
3+
4+
version: 2
5+
6+
snapshot:
7+
widths: [375, 1280]
8+
min-height: 1024
9+
percy-css: |
10+
.new-todo::placeholder { color: #999 !important; }
11+
12+
discovery:
13+
allowed-hostnames:
14+
- localhost
15+
network-idle-timeout: 500

advanced/Makefile

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
VENV=.venv/bin
2+
REQUIREMENTS=requirements.txt
3+
4+
$(VENV):
5+
python3 -m venv .venv
6+
$(VENV)/python3 -m pip install --upgrade pip
7+
8+
$(VENV)/.deps: $(REQUIREMENTS) | $(VENV)
9+
$(VENV)/pip install -r $(REQUIREMENTS)
10+
touch $(VENV)/.deps
11+
12+
.PHONY: venv install clean test test-advanced test-advanced-ci
13+
14+
venv install: $(VENV)/.deps
15+
16+
clean:
17+
rm -rf .venv .pytest_cache __pycache__ tests/__pycache__ advanced-requests.json
18+
19+
# Local run against a real PERCY_TOKEN.
20+
test test-advanced: install
21+
npx --no-install percy exec -- $(VENV)/python3 -m pytest tests/ -v
22+
23+
# CI run in --testing mode + capture requests file.
24+
test-advanced-ci: install
25+
PERCY_TOKEN=fake_token npx --no-install percy exec --testing -- bash -c '\
26+
$(VENV)/python3 -m pytest tests/ -v; \
27+
ec=$$?; \
28+
curl -fsS http://localhost:5338/test/requests > advanced-requests.json || true; \
29+
exit $$ec'

advanced/README.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Advanced Percy + Selenium-Python example
2+
3+
This directory exercises the full applicable Percy SDK feature surface for `percy-selenium` (Python). See the basic example at the repo root for the minimum integration.
4+
5+
## What this example covers
6+
7+
A pytest suite (`tests/test_todomvc_advanced.py`) where each test exercises one row of the [Percy SDK Advanced Feature Matrix](../../../docs/advanced-example-feature-matrix.md). Global SDK config — readiness preset, default widths, percyCSS, discovery — lives in `.percy.yml`.
8+
9+
Note: `scope`, `domTransformation`, `regions`, `discovery` are marked `N/A` — not exposed in `percy-selenium` 2.1.2 kwargs surface.
10+
11+
## Run locally
12+
13+
```bash
14+
cd advanced
15+
make install # creates .venv, installs requirements.txt
16+
export PERCY_TOKEN="<your token>" # do NOT commit this
17+
make test
18+
```
19+
20+
To run without a real token (CI assertion mode):
21+
22+
```bash
23+
make test-advanced-ci # uses --testing + PERCY_TOKEN=fake_token + captures /test/requests
24+
```
25+
26+
The CI variant asserts every matrix row appears in the captured POST bodies at the local `/test/requests` endpoint. No real Percy build is created.
27+
28+
## Coverage matrix
29+
30+
States: `Covered` / `N/A — <reason>` / `Planned` / `Deprecated`. Source of truth is [`matrix.yml`](./matrix.yml).
31+
32+
| Feature | State | Test |
33+
|---|---|---|
34+
| widths | Covered | `test_exercises_widths` |
35+
| minHeight (`min_height`) | Covered | `test_exercises_min_height` |
36+
| enableJavaScript (`enable_javascript`) | Covered | `test_exercises_enable_javascript` |
37+
| responsiveSnapshotCapture (`responsive_snapshot_capture`) | Covered | `test_exercises_responsive_snapshot_capture` |
38+
| readiness preset | Covered | `test_exercises_readiness_preset` |
39+
| sync | Covered | `test_exercises_sync` |
40+
| labels | Covered | `test_exercises_labels` |
41+
| testCase (`test_case`) | Covered | `test_exercises_test_case` |
42+
| devicePixelRatio (`device_pixel_ratio`) | Covered | `test_exercises_device_pixel_ratio` |
43+
| browsers override | Covered | `test_exercises_browsers` |
44+
| snake_case + camelCase dual naming | Covered | `test_exercises_snake_case_camelcase_dual_naming` |
45+
| percyCSS | Covered | global via `.percy.yml` |
46+
| cross-origin iframe handling | Covered | automatic via `percy-selenium >= 2.1.2` |
47+
| `.percy.yml` global config | Covered | `.percy.yml` consumed at build start |
48+
| environment info reporting | Covered | automatic via `percy-selenium` client info |
49+
| PERCY_SERVER_ADDRESS via env | Covered | CI advanced job picks up `PERCY_SERVER_ADDRESS` |
50+
| `disable_shadow_dom` | Planned ||
51+
| `enable_layout` | Planned ||
52+
| `scope` | N/A | Not exposed in SDK 2.1.2 |
53+
| `domTransformation` | N/A | Not exposed in SDK 2.1.2 |
54+
| `regions` per-snapshot | N/A | `create_region` is Automate-only |
55+
| `discovery` per-snapshot | N/A | discovery is per-build only |

advanced/conftest.py

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
"""PER-8195 — pytest fixtures: HTTP server for the TodoMVC app, headless
2+
firefox driver. Server roots two directories up so the basic TodoMVC files
3+
served at /index.html land at http://localhost:8006/."""
4+
5+
from http.server import HTTPServer, SimpleHTTPRequestHandler
6+
from threading import Thread
7+
import functools
8+
import os
9+
import pytest
10+
from selenium.webdriver import Firefox, FirefoxOptions
11+
12+
PORT = int(os.environ.get("PORT_NUMBER", "8006"))
13+
TEST_URL = f"http://localhost:{PORT}"
14+
APP_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir))
15+
16+
17+
@pytest.fixture(scope="session")
18+
def http_server():
19+
handler = functools.partial(SimpleHTTPRequestHandler, directory=APP_ROOT)
20+
httpd = HTTPServer(("localhost", PORT), handler)
21+
thread = Thread(target=httpd.serve_forever, daemon=True)
22+
thread.start()
23+
yield httpd
24+
httpd.shutdown()
25+
26+
27+
@pytest.fixture(scope="session")
28+
def driver(http_server):
29+
options = FirefoxOptions()
30+
options.add_argument("-headless")
31+
if os.environ.get("FIREFOX_BINARY"):
32+
options.binary_location = os.environ["FIREFOX_BINARY"]
33+
browser = Firefox(options=options)
34+
browser.implicitly_wait(10)
35+
yield browser
36+
browser.quit()

advanced/matrix.yml

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# PER-8195 Phase 1 — Selenium-Python matrix-row mapping.
2+
# Test code: tests/test_todomvc_advanced.py (pytest).
3+
4+
sdk: selenium-python
5+
package: percy-selenium
6+
language: python
7+
sdk_min_version: '2.1.2'
8+
cli_min_version: '1.31.13'
9+
10+
rows:
11+
- id: widths
12+
state: covered
13+
test: 'test_exercises_widths'
14+
- id: min_height
15+
state: covered
16+
test: 'test_exercises_min_height'
17+
- id: enable_javascript
18+
state: covered
19+
test: 'test_exercises_enable_javascript'
20+
- id: responsive_snapshot_capture
21+
state: covered
22+
test: 'test_exercises_responsive_snapshot_capture'
23+
- id: readiness_preset
24+
state: covered
25+
test: 'test_exercises_readiness_preset'
26+
- id: sync
27+
state: covered
28+
test: 'test_exercises_sync'
29+
- id: percy_css
30+
state: covered
31+
test: 'global via .percy.yml snapshot.percy-css'
32+
- id: labels
33+
state: covered
34+
test: 'test_exercises_labels'
35+
- id: test_case
36+
state: covered
37+
test: 'test_exercises_test_case'
38+
- id: device_pixel_ratio
39+
state: covered
40+
test: 'test_exercises_device_pixel_ratio'
41+
- id: browsers
42+
state: covered
43+
test: 'test_exercises_browsers'
44+
45+
# Python-specific.
46+
- id: snake_case_camelcase_dual_naming
47+
state: covered
48+
test: 'test_exercises_snake_case_camelcase_dual_naming'
49+
- id: cross_origin_iframe_handling
50+
state: covered
51+
test: 'automatic via percy-selenium >= 2.1.2'
52+
53+
- id: disable_shadow_dom
54+
state: planned
55+
- id: enable_layout
56+
state: planned
57+
58+
# Options not exposed as kwargs in percy-selenium (per SDK source).
59+
- id: scope
60+
state: n_a
61+
reason: 'Not exposed in percy-selenium 2.1.2 kwargs surface.'
62+
- id: dom_transformation
63+
state: n_a
64+
reason: 'Not exposed in percy-selenium 2.1.2 kwargs surface.'
65+
- id: regions
66+
state: n_a
67+
reason: 'create_region helper exists for Automate; not used in DOM snapshot path in 2.1.2.'
68+
- id: discovery
69+
state: n_a
70+
reason: 'discovery is per-build, not per-snapshot in this SDK.'
71+
72+
- id: env_percy_server_address
73+
state: covered
74+
test: 'CI: advanced job sets PERCY_SERVER_ADDRESS via env'
75+
- id: percy_yml_global_config
76+
state: covered
77+
test: 'global config consumed via .percy.yml'
78+
- id: environment_info_reporting
79+
state: covered
80+
test: 'automatic via percy-selenium client info'

advanced/requirements.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
selenium>=4.9.0
2+
percy-selenium>=2.1.2
3+
pytest>=7.4.0

0 commit comments

Comments
 (0)