[oss] Add GitHub Actions CI for pytest on Python 3.10–3.12

[fstandhartinger]

What changed

• .github/workflows/tests.yml (new): runs on push and pull_request on ubuntu-latest, with a matrix of Python 3.10, 3.11 and 3.12 (fail-fast: false). It caches pip through actions/setup-python (cache key based on pyproject.toml), has a 10-minute job timeout and permissions: contents: read. The steps are python -m pip install -e . pytest and then python -m pytest -q.
• pyproject.toml: adds [tool.pytest.ini_options] with testpaths = ["tests"] and pythonpath = ["."].
• tests/test_protocol.py: test_public_suite_pairs read datasets/public/original.jsonl relative to the current directory. It now builds that path from Path(__file__).resolve().parents[1]. Only the test's path changed; the library and the dataset are untouched.
• README.md: a tests badge line directly under the title.

Diff: 4 files, +34 / −1.

Tests that depended on the local environment (fixed in tests/config, not library code)

1. test_public_suite_pairs only passed when pytest ran from the repo root. Fixed by building the path from the test file's location.
2. tests/test_combinations.py runs from scripts.analyze_combinations import .... This only works with python -m pytest from the repo root, because that puts the current directory on sys.path. Launched any other way, collection fails with ModuleNotFoundError: No module named 'scripts'. pythonpath = ["."] fixes this without moving any scripts.

I found no flaky tests: 3 repeated runs in a row all passed with identical results.

asyncio setting

I did not add asyncio_default_fixture_loop_scope. The PytestDeprecationWarning doesn't appear because pytest-asyncio isn't a dependency or installed. Also, -W error::DeprecationWarning runs pass cleanly on 3.10–3.12. Setting an option for a plugin that isn't installed would itself trigger a "Unknown config option" warning.

How I verified it

Before (main @ b6b8fff, Python 3.13, python -m pytest -q):

108 passed, 5 subtests passed in 0.54s

Before, run from a different directory (python -m pytest -q /home/user/jevbench/tests):

E   ModuleNotFoundError: No module named 'scripts'
1 error in 0.20s

After, in a clean uv venv per version (pip install -e . pytest, then python -m pytest -q):

== 3.10  108 passed, 5 subtests passed in 0.34s
== 3.11  108 passed, 5 subtests passed in 0.37s
== 3.12  108 passed, 5 subtests passed in 0.42s
== 3.13 (system)  108 passed, 5 subtests passed in 0.29s
== run from another directory   108 passed, 5 subtests passed in 0.39s
== run from inside tests/     108 passed, 5 subtests passed in 0.33s

The workflow YAML loads with yaml.safe_load.

Open doubts

• I could not run the workflow on GitHub itself from the sandbox. The first Actions run on this PR is the real check.
• Running the bare pytest command in the sandbox picked up a different interpreter that doesn't have the package installed. That is a local setup issue, so CI uses python -m pytest as specified.
• The badge points at actions/workflows/tests.yml on the default branch, so it shows "no status" until this PR is merged.
• Scope: no datasets/, results/, scoring or published-number changes. No network calls to model APIs.

🤖 Generated with Claude Code

https://claude.ai/code/session_017Fy3BbvTht2E5AAszd968q

---

Generated by Claude Code