ci(pytorch): derive numpy test pin from vendored torch checkout#6423
ci(pytorch): derive numpy test pin from vendored torch checkout#6423lucbruni-amd wants to merge 5 commits into
Conversation
Instead of hand-copying upstream's per-Python numpy pins into requirements-test.txt (which drifts and is per-torch-branch), derive them at install time from the checked-out torch's own .ci/docker/requirements-ci.txt via derive_test_requirements.py, and wire it into the non-full PyTorch test workflow. Prototype / follow-up to the review discussion on #6400. Relates to pytorch/pytorch#189267.
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
Pre-commit check failed⛔ pre-commit failed Please run locally:
This repo uses |
|
🎉 All checks passed! This PR is ready for review. |
There was a problem hiding this comment.
Pull request overview
This PR updates the lean (non-full) PyTorch wheel test workflow to avoid manual NumPy pin drift by deriving the NumPy requirement directly from the checked-out PyTorch source tree’s .ci/docker/requirements-ci.txt, matching what upstream validates for the exact ref under test.
Changes:
- Remove the bare
numpyentry fromexternal-builds/pytorch/requirements-test.txtand document that NumPy is derived at test time. - Add
external-builds/pytorch/derive_test_requirements.pyto extract and print requirement lines (default:numpy) from the vendored PyTorch checkout’s CI requirements file. - Update
.github/workflows/test_pytorch_wheels.ymlto install the derived NumPy pins after installing the lean test requirements.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| external-builds/pytorch/requirements-test.txt | Drops bare numpy and documents that NumPy is now derived per torch ref. |
| external-builds/pytorch/derive_test_requirements.py | Adds a helper to extract pinned requirement lines (e.g., NumPy) from the checked-out PyTorch repo’s CI requirements file. |
| .github/workflows/test_pytorch_wheels.yml | Installs derived NumPy pins in the non-full wheel test workflow to avoid upstream drift. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
Possibly superseded by #6384. |
Motivation
Follow-up / alternative to the review discussion on #6400. The unpinned
numpyin
external-builds/pytorch/requirements-test.txtbroke the non-full PyTorchtest workflow once a newer NumPy tightened
uint8casting (an upstream OpInfobug, pytorch/pytorch#189267). #6400 fixes it by hand-copying upstream's
per-Python numpy pins, but there is silent drift potential.
This prototype removes the manual copy: it derives numpy from the checked-out
torch's own
.ci/docker/requirements-ci.txtat install time, so we always usewhat upstream validates for the exact ref under test.
Note: the full test workflow already installs the entire
requirements-ci.txt(since #3705), so it is already covered; this only closes the gap in the lean
non-full
test_pytorch_wheels.ymlwithout pulling upstream's ~75-package set.Technical Details
external-builds/pytorch/derive_test_requirements.py: prints the requirementlines for the requested packages (default
numpy) from the vendored checkout's.ci/docker/requirements-ci.txt, preserving environment markers; errors if thefile is missing.
requirements-test.txt: drop barenumpy(now derived).test_pytorch_wheels.yml: after installing the lean reqs, pipe the derivedpins into pip.
It auto-tracks the torch ref (nightly → nightly's numpy,
release/2.9→ 2.9'snumpy), so there is nothing to re-sync and nothing to drop when
pytorch/pytorch#189267 is fixed upstream.
Test Plan
Local verification (below); in CI, the non-full PyTorch test jobs install the
derived numpy and the
thresholduint8 tests pass.Test Result
Ran locally against the vendored checkout:
numpy==1.26.2--requirements-cipointed atROCm/pytorchrelease/2.9→numpy==2.0.2/2.1.2(per-branch tracking)Submission Checklist