Skip to content

Migrate dependency management to uv - #72

Merged
Jan-Williams merged 3 commits into
mainfrom
migrate-to-uv
Jul 27, 2026
Merged

Jan-Williams merged 3 commits into
mainfrom
migrate-to-uv

Conversation

@Jan-Williams

Copy link
Copy Markdown
Owner

Adopt uv for environment/dependency management and clean up packaging cruft that had accumulated in pyproject.toml.

Packaging:

  • Switch build backend from setuptools+setuptools_scm to hatchling+hatch-vcs. hatch-vcs wraps setuptools_scm, so git-tag-derived version strings are unchanged (verified: 0.3.4.dev2+gc0e77dd2b before and after).
  • Drop dead build requirements (wheel, numpy, cython) -- there are no C or Cython sources in the tree.
  • Remove setuptools_scm from runtime dependencies; it is a build-time tool and nothing imports it, but every installed user was pulling it in.
  • Pin the sdist contents with only-include. docs/examples is a symlink to ../examples, and hatchling's tree walk follows symlinks while de-duplicating on (st_dev, st_ino), so whichever directory was visited first won and the other was silently dropped -- a non-deterministic sdist. only-include bypasses the walk entirely. Sdist also shrinks 2.8M -> 140K.

Dependencies:

  • Move dev/docs dependencies to PEP 735 [dependency-groups]; keep gpu and notebooks as extras. Note this breaks pip install -e ".[dev]" in favour of uv sync (or pip install --group dev on pip >= 25.1).
  • Add notebook/ipykernel to the docs group: mkdocs-jupyter runs with execute: true, so the docs build executes every example notebook and needs a kernel. It previously worked only by transitive luck.
  • Mark the gpu extra sys_platform == "linux". JAX publishes CUDA wheels for Linux only, so uv sync --all-extras failed outright on macOS. No capability is lost: Windows GPU users go through WSL2, which reports as Linux.
  • Commit uv.lock for reproducible environments.

CI:

  • Replace pip with astral-sh/setup-uv (cached) across tests/docs/publish.
  • Split lint into its own job that runs once with uv sync --locked, so a lockfile out of sync with pyproject.toml fails loudly. The test matrix deliberately resolves fresh (--upgrade) so upstream jax/equinox/diffrax breakage still surfaces.
  • Add fetch-depth: 0 to tests.yml. It was checking out shallow, so the version derived during install was wrong (0.1.dev1+... rather than the real version).
  • Assert in publish.yml that the built artifact version matches the release tag, since hatch-vcs derives it from git rather than from the tag itself.
  • Narrow publish.yml's top-level permissions to {}; only the publish job needs id-token: write.
  • Add a concurrency group to docs.yml -- gh-deploy force-pushes gh-pages, so overlapping runs raced.

Also adds a coverage config canonicalizing editable vs non-editable paths, and drops a per-file-ignores entry for a setup.py that does not exist.

Co-authored by Claude Opus 5

Adopt uv for environment/dependency management and clean up packaging cruft
that had accumulated in pyproject.toml.

Packaging:
- Switch build backend from setuptools+setuptools_scm to hatchling+hatch-vcs.
  hatch-vcs wraps setuptools_scm, so git-tag-derived version strings are
  unchanged (verified: 0.3.4.dev2+gc0e77dd2b before and after).
- Drop dead build requirements (wheel, numpy, cython) -- there are no C or
  Cython sources in the tree.
- Remove setuptools_scm from runtime dependencies; it is a build-time tool and
  nothing imports it, but every installed user was pulling it in.
- Pin the sdist contents with only-include. docs/examples is a symlink to
  ../examples, and hatchling's tree walk follows symlinks while de-duplicating
  on (st_dev, st_ino), so whichever directory was visited first won and the
  other was silently dropped -- a non-deterministic sdist. only-include
  bypasses the walk entirely. Sdist also shrinks 2.8M -> 140K.

Dependencies:
- Move dev/docs dependencies to PEP 735 [dependency-groups]; keep gpu and
  notebooks as extras. Note this breaks `pip install -e ".[dev]"` in favour of
  `uv sync` (or `pip install --group dev` on pip >= 25.1).
- Add notebook/ipykernel to the docs group: mkdocs-jupyter runs with
  execute: true, so the docs build executes every example notebook and needs a
  kernel. It previously worked only by transitive luck.
- Mark the gpu extra `sys_platform == "linux"`. JAX publishes CUDA wheels for
  Linux only, so `uv sync --all-extras` failed outright on macOS. No capability
  is lost: Windows GPU users go through WSL2, which reports as Linux.
- Commit uv.lock for reproducible environments.

CI:
- Replace pip with astral-sh/setup-uv (cached) across tests/docs/publish.
- Split lint into its own job that runs once with `uv sync --locked`, so a
  lockfile out of sync with pyproject.toml fails loudly. The test matrix
  deliberately resolves fresh (--upgrade) so upstream jax/equinox/diffrax
  breakage still surfaces.
- Add fetch-depth: 0 to tests.yml. It was checking out shallow, so the version
  derived during install was wrong (0.1.dev1+... rather than the real version).
- Assert in publish.yml that the built artifact version matches the release tag,
  since hatch-vcs derives it from git rather than from the tag itself.
- Narrow publish.yml's top-level permissions to {}; only the publish job needs
  id-token: write.
- Add a concurrency group to docs.yml -- gh-deploy force-pushes gh-pages, so
  overlapping runs raced.

Also adds a coverage config canonicalizing editable vs non-editable paths, and
drops a per-file-ignores entry for a setup.py that does not exist.
The run-tests 3.12 job segfaulted (exit 139) while importing orc under
coverage, whereas 3.10, 3.11 and 3.13 all passed with identical dependency
versions -- `uv lock --upgrade` resolves to exactly the same versions as the
committed lock, so this was not an upstream regression.

The traceback showed the interpreter was /usr/lib/python3.12: ubuntu-latest
ships 3.12 as its system Python, so uv's default python-preference of "managed"
(prefer managed, fall back to system) used the system interpreter for that one
leg while downloading python-build-standalone builds for the others. The
previous pip-based workflow never hit this because actions/setup-python installs
a tool-cache interpreter rather than the system one.

Set UV_PYTHON_PREFERENCE=only-managed so every job uses a uv-downloaded
interpreter. This makes the matrix internally consistent, matches local
development, and stops CI depending on what the runner image happens to ship.
Applied to docs.yml and publish.yml as well, so released artifacts and the docs
build do not depend on the runner image either.
Add two pre-merge checks that were missing. docs.yml only ran on pushes to
main, so a broken example notebook was caught by the deploy job after merge --
leaving main red and the published site stale. publish.yml only runs on release,
so a packaging regression would first surface during an actual PyPI upload.

- docs.yml: add a pull_request trigger with a build-docs job that runs
  `mkdocs build --strict` and nothing else. Because mkdocs-jupyter runs with
  execute: true and allow_errors: false, this executes every example notebook.
  Split into two event-gated jobs so gh-deploy keeps contents: write while pull
  requests get read-only access, and so main does not execute the notebooks
  twice (gh-deploy already builds).
- tests.yml: add a build-package job running `uv build` and `twine check`, plus
  assertions that the wheel ships orc/ without leaking src//tests//examples/,
  and that the sdist ships the example notebooks but never the docs/examples
  symlink. Each assertion was negative-tested to confirm it fails when it
  should.

Also correct mistakes in the migration docs:

- installation.md presented `pip install "OpenReservoirComputing[gpu]"` inside
  the install-from-source section, which starts with git clone -- a PyPI command
  under a source heading. Extras now sit with the PyPI instructions and the
  source section uses the local `pip install ".[gpu]"` form.
- Both installation.md and contributing.md offered `pip install --group dev` as
  the pip equivalent of `uv sync`. It is not: verified against pip 26.1 that
  --group installs only the group's tools and not the project, so the correct
  form is `pip install -e . --group dev`.
- Two admonition titles contained backslash-escaped quotes, which
  python-markdown does not parse. Retitled without quotes and confirmed all four
  admonitions render cleanly in the built site.
@Jan-Williams
Jan-Williams merged commit 4d16622 into main Jul 27, 2026
8 checks passed
@Jan-Williams
Jan-Williams deleted the migrate-to-uv branch July 27, 2026 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant