Tests

feat(import_common): matchowanie autorów PL↔EN (Ewa Marańda ↔ Eva Maranda) #906

Workflow file for this run

	name: Tests

	on:
	push:
	branches:
	- dev
	- master
	pull_request:
	branches:
	- dev

	concurrency:
	group: ${{ github.workflow }}-${{ github.event.pull_request.number \|\| github.ref }}
	cancel-in-progress: true

	# Default minimum permissions; per-job overrides below where extra
	# scopes are needed (notably `packages:` for the GHCR push/pull flow).
	permissions:
	contents: read

	jobs:
	lint:
	# Lint only Python files changed in this push/PR. The repo's
	# convention is that ruff runs on changed files only (see
	# pre-commit config); historic drift in unchanged files is
	# deliberately left alone, so CI must match that scope.
	name: Lint changed files
	runs-on: ubuntu-latest
	timeout-minutes: 5

	steps:
	- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
	with:
	fetch-depth: 0

	- name: Install system build deps
	# uv run triggers a full project sync, and several wheels (pycairo
	# via svglib/xhtml2pdf, psycopg2, python-ldap, cryptography) need
	# native headers to build. Reuse docker/bpp_base/build.txt so the
	# package list stays in sync with the runtime/test images.
	run: \|
	sudo apt-get update
	xargs -a docker/bpp_base/build.txt sudo apt-get install -y

	- name: Install uv
	uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
	with:
	enable-cache: true

	- name: Determine diff base
	id: base
	env:
	EVENT_NAME: ${{ github.event_name }}
	BASE_REF: ${{ github.base_ref }}
	BEFORE_SHA: ${{ github.event.before }}
	run: \|
	if [ "$EVENT_NAME" = "pull_request" ]; then
	git fetch --no-tags --depth=100 origin "$BASE_REF"
	echo "ref=origin/$BASE_REF" >> "$GITHUB_OUTPUT"
	elif [ -n "$BEFORE_SHA" ] && \
	[ "$BEFORE_SHA" != "0000000000000000000000000000000000000000" ]; then
	echo "ref=$BEFORE_SHA" >> "$GITHUB_OUTPUT"
	else
	# First push to a new branch: diff against dev
	git fetch --no-tags --depth=100 origin dev
	echo "ref=origin/dev" >> "$GITHUB_OUTPUT"
	fi

	- name: Collect changed Python files
	id: files
	env:
	DIFF_BASE: ${{ steps.base.outputs.ref }}
	run: \|
	files=$(git diff --name-only --diff-filter=AM \
	"$DIFF_BASE"...HEAD -- '*.py' \| tr '\n' ' ')
	echo "list=$files" >> "$GITHUB_OUTPUT"
	if [ -n "$files" ]; then
	echo "Changed Python files:"
	printf ' %s\n' $files
	else
	echo "No Python files changed."
	fi

	- name: Ruff check
	# `ruff` siedzi w [dependency-groups].dev (PEP 735); `uv run`
	# aktywuje grupe `dev` defaultowo, wiec nie ma `--group dev`
	# / `--extra dev`. $FILES jest env-bounded ponizej (a nie
	# ${{ ... }} inline), wiec nie ma command injection.
	if: steps.files.outputs.list != ''
	env:
	FILES: ${{ steps.files.outputs.list }}
	run: uv run ruff check --force-exclude $FILES

	- name: Ruff format --check
	if: steps.files.outputs.list != ''
	env:
	FILES: ${{ steps.files.outputs.list }}
	run: uv run ruff format --check --force-exclude $FILES

	prepare-image:
	# Build the test-runner image once and push it to GHCR with two
	# tags: `:sha-<commit>` (consumed by downstream test jobs) and
	# `:cache` (registry-mode buildx cache, auto-merged across builds).
	# All test jobs `needs:` this step so the image is built once per
	# workflow run instead of N times per matrix runner.
	name: Build test-runner image
	if: \|
	!(github.ref == 'refs/heads/dev' && contains(github.event.head_commit.message, 'Merge tag'))

	runs-on: ubuntu-latest
	timeout-minutes: 30

	permissions:
	contents: read
	# GHCR push needs `packages: write`. GITHUB_TOKEN inherits this
	# scope only when the job declares it explicitly.
	packages: write

	outputs:
	image: ${{ steps.tag.outputs.image }}

	env:
	PYTHON_VERSION: "3.12"
	DEBIAN_VERSION: trixie
	NODEJS_VERSION: "20"
	# Lowercase repo owner, since GHCR rejects mixed-case namespaces.
	# GitHub Actions ${{ github.repository_owner }} preserves case.
	IMAGE_REPO: ghcr.io/iplweb/bpp-test-runner

	steps:
	- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

	# baseline_check is repo-state-only — it doesn't need the
	# test-runner image, the database, or any baked artefacts. Running
	# it here (instead of inside a test job) means a baseline drift
	# fails fast and skips the expensive image build below.
	- name: Install system build deps (for uv sync)
	run: \|
	sudo apt-get update
	xargs -a docker/bpp_base/build.txt sudo apt-get install -y

	- name: Install uv (for baseline_check)
	# No setup-uv cache here — this job pushes images to GHCR, and
	# zizmor flags cache+publish combos as a poisoning vector. The
	# cache hit would shave a few seconds off a one-shot uv sync,
	# not worth the audit smell.
	uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
	with:
	enable-cache: false

	- name: Install project deps (for baseline_check)
	run: uv sync --frozen --no-install-project

	- name: Check baseline freshness
	env:
	DJANGO_BPP_SKIP_DOTENV: "1"
	run: uv run python src/manage.py baseline_check --max-delta 50

	- name: Set up Docker Buildx
	uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
	with:
	driver: docker-container

	- name: Log in to GHCR
	uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
	with:
	registry: ghcr.io
	username: ${{ github.actor }}
	password: ${{ secrets.GITHUB_TOKEN }}

	- name: Compute image tag
	id: tag
	env:
	SHA: ${{ github.sha }}
	run: \|
	echo "image=${IMAGE_REPO}:sha-${SHA}" >> "$GITHUB_OUTPUT"

	- name: Build & push test-runner image
	uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
	with:
	context: .
	file: docker/bpp_base/Dockerfile
	target: test-runner
	push: true
	# Two tags: immutable sha-pinned (consumed by tests jobs) and
	# the registry-cache helper used to seed `cache-from` next time.
	tags: \|
	${{ steps.tag.outputs.image }}
	build-args: \|
	PYTHON_VERSION=${{ env.PYTHON_VERSION }}
	DEBIAN_VERSION=${{ env.DEBIAN_VERSION }}
	NODEJS_VERSION=${{ env.NODEJS_VERSION }}
	# Registry-mode cache. mode=max stores all intermediate layers,
	# so subsequent builds where uv.lock/Dockerfile/build.txt are
	# unchanged finish in seconds. The first run after rolling out
	# this workflow has no cache and pays the full ~3 min build —
	# warm cache afterwards.
	cache-from: type=registry,ref=${{ env.IMAGE_REPO }}:cache
	cache-to: type=registry,ref=${{ env.IMAGE_REPO }}:cache,mode=max

	tests:
	# Single sharded job covering the full suite (playwright +
	# non-playwright + serial) under `pytest -n auto`. Earlier the
	# workflow split this into three specialised jobs (parallel /
	# serial / playwright); shard-then-mix is faster end-to-end because
	# the per-shard fixed cost (image pull, PG init + baseline load,
	# Redis startup) is paid once instead of three times.
	#
	# @pytest.mark.serial tests are deliberately mixed in here. They
	# don't actually depend on running outside xdist — the marker is
	# used by callers that want to opt out of parallelism, but the
	# tests themselves pass under -n auto. If that ever changes, gate
	# serial tests with @pytest.mark.xdist_group(name="serial") so
	# `--dist=loadgroup` keeps them on a single worker.
	#
	# DB setup: docker-compose.test.yml mounts baseline.sql into
	# /docker-entrypoint-initdb.d/ and sets DJANGO_BPP_TEST_TEMPLATE=bpp,
	# so xdist worker DBs are created via `CREATE DATABASE … WITH
	# TEMPLATE bpp` (instant in-server clone) instead of replaying
	# `psql -f baseline.sql` per worker.
	name: Tests (sharded)
	needs: prepare-image
	if: \|
	!(github.ref == 'refs/heads/dev' && contains(github.event.head_commit.message, 'Merge tag'))

	runs-on: ubuntu-latest
	timeout-minutes: 35

	permissions:
	contents: read
	packages: read

	strategy:
	fail-fast: false
	max-parallel: 8
	matrix:
	python-version: ["3.12"]
	# 0-indexed; bump alongside NUM_SHARDS below if you change it.
	shard: [0, 1, 2, 3, 4, 5, 6, 7]

	env:
	TEST_RUNNER_IMAGE: ${{ needs.prepare-image.outputs.image }}
	COMPOSE_FILES: -f docker-compose.test.yml -f docker-compose.test.ci.yml
	NUM_SHARDS: "8"

	steps:
	- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2

	- name: Log in to GHCR
	uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
	with:
	registry: ghcr.io
	username: ${{ github.actor }}
	password: ${{ secrets.GITHUB_TOKEN }}

	- name: Prepare ci-output dir
	run: mkdir -p ci-output

	- name: Pull test-runner image
	run: docker compose $COMPOSE_FILES pull test-runner

	- name: Start services
	run: docker compose $COMPOSE_FILES up -d db redis

	- name: Wait for services
	timeout-minutes: 2
	run: \|
	until docker compose $COMPOSE_FILES exec db \
	pg_isready -U bpp; do
	echo "Waiting for PostgreSQL..."
	sleep 2
	done
	until docker compose $COMPOSE_FILES exec redis \
	redis-cli ping; do
	echo "Waiting for Redis..."
	sleep 2
	done

	- name: Tests (shard ${{ matrix.shard }}/${{ env.NUM_SHARDS }})
	timeout-minutes: 25
	env:
	SHARD_ID: ${{ matrix.shard }}
	run: \|
	docker compose $COMPOSE_FILES run --rm \
	-e SHARD_ID \
	test-runner uv run pytest \
	-n auto \
	--shard-id "$SHARD_ID" --num-shards "$NUM_SHARDS" \
	--timeout 300 \
	--cov=src/ --cov-branch \
	--cov-report=xml:/ci-output/cov.xml

	- name: Upload coverage to Coveralls (parallel)
	if: always()
	uses: coverallsapp/github-action@5cbfd81b66ca5d10c19b062c04de0199c215fb6e # v2.3.7
	with:
	github-token: ${{ secrets.GITHUB_TOKEN }}
	flag-name: tests-${{ matrix.shard }}
	parallel: true
	file: ci-output/cov.xml

	- name: Cleanup
	if: always()
	run: docker compose $COMPOSE_FILES down -v

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(import_common): matchowanie autorów PL↔EN (Ewa Marańda ↔ Eva Maranda) #906

Workflow file

feat(import_common): matchowanie autorów PL↔EN (Ewa Marańda ↔ Eva Maranda) #906

Uh oh!

Workflow file for this run