ci(pytorch): pin numpy per Python version to mirror upstream test CI

[lucbruni-amd]

Motivation

Fixes #6027
Fixes #6174

numpy was unpinned in external-builds/pytorch/requirements-test.txt (bare
numpy since the file was created in #903), so CI installed the newest release.
A numpy release after 2.4.6 tightened negative-int → uint8 casting to raise
OverflowError, breaking upstream PyTorch's nn.functional.threshold OpInfo
reference (np.where(...).astype(uint8) with value=-9). This surfaced as 4
failing test_reference_numerics_*_nn_functional_threshold_cuda_uint8 tests
across gfx90a / gfx94X / gfx950 / gfx1151 / gfx110X / gfx120X, py3.12–3.14.

Torch version binding: the failure was observed on the branches in the
failing matrix (2.9–2.11), but it is not specific to those versions and is
not fixed in newer torch. The same OpInfo (value: -9 cast to uint8) is
still present unchanged on both the upstream nightly and
release/2.12 (pinned to current HEAD):

• upstream pytorch/pytorch@566bb97: common_methods_invocations.py#L18461-L18475
• ROCm ROCm/pytorch@7a68a17 (release/2.12): common_methods_invocations.py#L17628-L17642

Any torch version hits it given a new-enough numpy.

Technical Details

Upstream PyTorch never hits this because its test CI pins numpy per Python
version. Pin numpy the same way, copied verbatim from pytorch/pytorch
.ci/docker/requirements-ci.txt (still pinned on trunk), so we test against the
exact numpy versions upstream validates against. Test-environment only — does not
touch shipped wheel metadata (the wheel's pyproject leaves numpy unpinned, so
end users still get the latest).

Test Plan

The 4 test_reference_numerics_*_nn_functional_threshold_cuda_uint8 tests should
pass in the full PyTorch test workflow across the affected arches / Python
versions.

Test Result

Pending CI.

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[therock-pr-bot]

✅ All Checks Passed — Ready for Review

Check	Status	Details
🌿 Branch Name	✅ Pass	—
📝 PR Title/Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	PR does not contain code files — Unit Test auto-passed
🔎 pre-commit	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—
🤖 therock-pr-bot	✅ Pass	—

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

[therock-pr-bot]

🎉 All checks passed! This PR is ready for review.

[HereThereBeDragons]

can we have a bit more discussion here @ethanwee1 @jithunnair-amd @pragupta if this is the correct way to do it?

@lucbruni-amd can you please confirm that this is only for torch versions <= 2.11 and in newer versions it is fixed?

for me this would look more like that upstream should fix the tests to be able to handle the new behavior of numpy correctly.
an end user also would just install the latest numpy available for their given python version.

[Copilot]

Pull request overview

Pins NumPy versions in the PyTorch test-only requirements to mirror upstream PyTorch CI, preventing recent NumPy behavior changes from breaking PyTorch reference-numerics tests (notably uint8 casting in nn.functional.threshold references).

Changes:

• Replace the unpinned numpy dependency with Python-version-specific NumPy pins for the PyTorch test environment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[lucbruni-amd]

@HereThereBeDragons

can you please confirm that this is only for torch versions <= 2.11 and in newer versions it is fixed?

You are right, it's not bound to <= 2.11:

Upstream nightly branch: https://github.com/pytorch/pytorch/blob/566bb9719b0a67fe47af72cb69345dd809b5caec/torch/testing/_internal/common_methods_invocations.py#L18461-L18475

ROCm release/2.12 branch: https://github.com/ROCm/pytorch/blob/7a68a172a909169dd46a732f6e97206da1a12ce6/torch/testing/_internal/common_methods_invocations.py#L17628-L17642

(The links were taking a while to open on my browser due to the size of these files)

Updated the PR description to reflect that.

an end user also would just install the latest numpy available for their given python version

That is true - this PR only touches the test-env side of things, which mirrors what upstream is doing. I'm happy to discuss further and open a PR upstream instead if that is preferred.

[lucbruni-amd]

Filed pytorch/pytorch#189267 to get the ball rolling on your suggestion, @HereThereBeDragons.

[ethanwee1]

I agree with @HereThereBeDragons These pins are copied into a TheRock-owned file (external-builds/pytorch/requirements-test.txt) rather than upstream's requirements-ci.txt, so there's no mechanism keeping them in sync so this will silently drift as upstream bumps versions. Could you add a short comment above the block noting the upstream source path (pytorch/pytorch .ci/docker/requirements-ci.txt) and the commit/branch it was copied from? That gives a future maintainer a clear pointer for when to re-sync or drop the pin (e.g. once the OpInfo reference is fixed upstream via pytorch/pytorch#189267).

[lucbruni-amd]

@ethanwee1

Could you add a short comment above the block noting the upstream source path (pytorch/pytorch .ci/docker/requirements-ci.txt) and the commit/branch it was copied from?

Done. This is now a matter of whether we want to pin (and if so, how to handle the drift - I wonder if we can derive such pins from the external-builds/pytorch/pytorch_torch_repo.py source checkout), or just fix these types of issues as they come up.

[HereThereBeDragons]

i think we could pin it for now, but the idea should be not to pin it in the future. just needs too much maintenance - and also a bit unrealistic to just test a single numpy version that does not have your breaking changes in it.

@ethanwee1 is getting this upstream something your team can take care of it? i would feel more confident that it is not forgotten if we have someone assign to it.

[ethanwee1]

Thanks @HereThereBeDragons - yes, my team can take the upstream fix. I'm taking it up.

I've filed a draft PR upstream that fixes the OpInfo reference itself (pre-casting the fill value into the input dtype so np.where no longer strict-casts a negative Python int against unsigned dtypes under NumPy >2.4.6): pytorch/pytorch#189273 (tracking the earlier issue pytorch/pytorch#189267).

Plan: land the upstream fix, then drop the interim numpy pin here once it propagates. Tracking this on my board so it doesn't get lost.

I think this PR is fine to merge as the interim unblock in the meantime.

[lucbruni-amd]

Opened a draft #6423 which implements the alternative solution of deriving the pins automatically, preventing drift.

Would we prefer something like that instead of merging this PR?

EDIT: This PR and the draft are possibly superseded by the mentioned PR below.

[zjliu-amd]

#6384 is trying to solve the same issue (and some more) by installing pytorch's requirements-ci.txt file. This step is also done in the test_pytorch_wheels_full workflow, but it's missing in test_pytorch_wheels workflow and causing this problem. I feel like installing pytorch's requirement file is an easier way to go, unless it was intentionally omitted in test_pytorch_wheels workflow.

[ScottTodd]

I'd rather we go with #6384 instead of this