making a release#4
Merged
Merged
Conversation
Audit the existing MR tests and serialize those that make large allocations (specifically, a pool with 90% of the available device memory). This also allows us to remove serialization from some of the tests which don't make large allocations. - Closes rapidsai#1671 Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Bradley Dice (https://github.com/bdice) - Mark Harris (https://github.com/harrism) URL: rapidsai#1672
…i#1670) Closes rapidsai#173 Authors: - Matthew Murray (https://github.com/Matt711) - Lawrence Mitchell (https://github.com/wence-) Approvers: - Lawrence Mitchell (https://github.com/wence-) - Mark Harris (https://github.com/harrism) URL: rapidsai#1670
…ce_resource_ref() (rapidsai#1661) Closes rapidsai#1660. This adds a constructor to each MR adaptor to take a resource_ref rather than an `Upstream*`. It also updates RMM to use `get_current_device_resource_ref()` everywhere: in containers, in tests, in adaptors, Thrust allocator, polymorphic allocator, execution_policy, etc. Importantly, this PR also modifies `set_current_device_resource()` to basically call `set_current_device_resource_ref()`. This is necessary, because while RMM C++ uses `get_current_device_resource_ref()` everywhere, the Python API still uses the raw pointer API `set_current_device_resource()`. So we need the latter to update the state for the former. This is a temporary bootstrap to help with the refactoring. Authors: - Mark Harris (https://github.com/harrism) Approvers: - Michael Schellenberger Costa (https://github.com/miscco) - Lawrence Mitchell (https://github.com/wence-) - Rong Ou (https://github.com/rongou) - Bradley Dice (https://github.com/bdice) URL: rapidsai#1661
This has been found breaking CCCL CI when building cuDF Authors: - Michael Schellenberger Costa (https://github.com/miscco) Approvers: - Lawrence Mitchell (https://github.com/wence-) - Mark Harris (https://github.com/harrism) URL: rapidsai#1677
Recommending `miniforge` for conda install in installation docs. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Lawrence Mitchell (https://github.com/wence-) - Mark Harris (https://github.com/harrism) URL: rapidsai#1681
* Update fmt (to 11.0.2) and spdlog (to 1.14.1). * simplify get_spdlog * copyright * Apply suggestions from code review Co-authored-by: Bradley Dice <bdice@bradleydice.com> * Apply suggestions from code review * test with changes from rapidsai/rapids-cmake@d7671a3 * Update cmake/thirdparty/get_spdlog.cmake * move rapids-cmake overrides [skip ci] * try reverting get_spdlog export changes [skip ci] * more fiddling with export sets [skip ci] * more exporting [skip ci] * more export set fiddling [skip ci] * more [skip ci] * exports [skip ci] * run a build * restore tests * branch references * remove testing-only changes [skip ci] --------- Co-authored-by: Bradley Dice <bdice@bradleydice.com>
Forward-merge branch-24.10 into branch-24.12
Forward-merge branch-24.10 into branch-24.12
This project currently lists `gcovr` (https://pypi.org/project/gcovr/) as a pip dependency for development. I strongly suspect that that was unintentional... it doesn't look like it has any reliance on getting that package via `pip` (just conda, in the C++ test jobs and for local C++ development). This proposes removing `gcovr` from the list of pip dependencies, so it won't get installed in the DLFW images or other places where `rapids-make-pip-env` from https://github.com/rapidsai/devcontainers is called. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#1688
Rather than hand-coding a fraction of the device memory use the utility routine. - Closes rapidsai#1674 Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Rong Ou (https://github.com/rongou) URL: rapidsai#1689
…hon classes public (rapidsai#1676) Closes rapidsai#1280 Authors: - Matthew Murray (https://github.com/Matt711) - Lawrence Mitchell (https://github.com/wence-) Approvers: - Lawrence Mitchell (https://github.com/wence-) - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#1676
This PR fixes a bug in rapidsai#1676. It makes sure that rmm imports work correctly using both `from rmm._lib...` and `import rmm._lib...` syntax. I'm adding DO NOT MERGE until I do some more testing. Authors: - Matthew Murray (https://github.com/Matt711) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#1693
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: rapidsai#1695
Forward-merge branch-24.10 into branch-24.12
NVIDIA/cuda-python#137 reorganised the low-level binding structure which broke our cross-linking, update to the new name to fix. - Closes rapidsai#1698 Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Matthew Murray (https://github.com/Matt711) - Bradley Dice (https://github.com/bdice) URL: rapidsai#1699
This means that downstream libraries that get their `fmt` dependency from RMM will use `-DFMT_SHARED` in the compile command: this matches what the rapids combined devcontainers do, so we get sccache hits. - Closes rapidsai#1701 Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) URL: rapidsai#1702
Contributes to rapidsai/build-planning#106 Proposes specifying the RAPIDS version in `conda install` calls in CI that install CI artifacts, to reduce the risk of CI jobs picking up artifacts from other releases. Authors: - James Lamb (https://github.com/jameslamb) - Lawrence Mitchell (https://github.com/wence-) Approvers: - Ray Douglass (https://github.com/raydouglass) URL: rapidsai#1696
Small fix to some typos that cropped up in the .gitignore with rapidsai#1676 Authors: - Charles Blackmon-Luca (https://github.com/charlesbluca) - Lawrence Mitchell (https://github.com/wence-) Approvers: - Lawrence Mitchell (https://github.com/wence-) - Mark Harris (https://github.com/harrism) URL: rapidsai#1697
Follow-up to rapidsai#1696 Changes relative to that PR: * switches to pinning CI conda installs to the output of `rapids-version` (`{major}.{minor}.{patch}`) instead of `rapids-version-major-minor` (`{major}.{minor}`), to get a bit more protection in the presence of hotfix releases * restores some exporting of variables needed for docs builds In rapidsai#1696, I'd missed that this project's Doxygen setup is expecting to find `RAPIDS_VERSION_MAJOR_MINOR` defined in the environment. https://github.com/rapidsai/rmm/blob/90a5631e1093ce44c4feceb88fcf557c3dfc043b/ci/build_docs.sh#L36 https://github.com/rapidsai/rmm/blob/90a5631e1093ce44c4feceb88fcf557c3dfc043b/doxygen/Doxyfile#L41 Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Mike Sarahan (https://github.com/msarahan) URL: rapidsai#1703
Contributes to rapidsai/build-planning#104 This PR removes support for accessing rmm's underlying spdlog logger directly. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Bradley Dice (https://github.com/bdice) - Lawrence Mitchell (https://github.com/wence-) - Mark Harris (https://github.com/harrism) URL: rapidsai#1690
Closes rapidsai#1705. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - GALI PREM SAGAR (https://github.com/galipremsagar) URL: rapidsai#1706
Fixes issue brought up in rapidsai/rapids-cmake#634 (comment) where rmm wasn't using rapids_cmake_support_conda_env Authors: - Robert Maynard (https://github.com/robertmaynard) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) - Mark Harris (https://github.com/harrism) URL: rapidsai#1707
This PR is replacing the `VAULT_HOST` variable with `AWS_ROLE_ARN`. This is required to use the new token service to get AWS credentials. Authors: - Jordan Jacobelli (https://github.com/jjacobelli) Approvers: - Paul Taylor (https://github.com/trxcllnt) - Bradley Dice (https://github.com/bdice) URL: rapidsai#1708
Contributes to rapidsai/build-planning#111 Proposes some small packaging/CI changes, matching similar changes being made across RAPIDS. * printing `sccache` stats to CI logs * updating to the latest `rapids-dependency-file-generator` (v1.16.0) * reducing verbosity of `pip wheel` in wheel builds Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Jake Awe (https://github.com/AyodeAwe) URL: rapidsai#1712
Follows up rapidsai#1676 to add deprecation warnings to the `rmm._lib` sub package. Authors: - Matthew Murray (https://github.com/Matt711) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#1713
The `is_host_memory` function has been updated to understand that `cudaMemoryTypeUnregistered` is returned when provided pointers allocated by `malloc` and other host side allocation functions. The `is_device_memory` function has been restricted to report only when device pointer that is usable by the calling cuda context. For that reason the tests now also set the active cuda device for all calling threads. Authors: - Robert Maynard (https://github.com/robertmaynard) Approvers: - Rong Ou (https://github.com/rongou) - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#1714
`std::equal` was basically being used to create a loop, which `std::for_each` is better suited for. See discussion at rapidsai#1963 (comment) Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Bradley Dice (https://github.com/bdice) - Yunsong Wang (https://github.com/PointKernel) URL: rapidsai#1999
…idsai#1983) Continues from rapidsai#1980. Contributes to rapidsai#1779. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) - Rong Ou (https://github.com/rongou) URL: rapidsai#1983
## Summary This pull request has been generated by [StepSecurity](https://app.stepsecurity.io/github/rapidsai/actions/dashboard) as part of your enterprise subscription to ensure compliance with recommended security best practices. Please review and merge the pull request to apply these security enhancements. ## Security Fixes ### Harden Runner Harden-Runner is an open-source security agent for the GitHub-hosted runner to prevent software supply chain attacks. It prevents exfiltration of credentials, detects tampering of source code during build, and enables running jobs without sudo access. - [GitHub Security Guide](https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions) - [The Open Source Security Foundation (OpenSSF) Security Guide](https://github.com/ossf/scorecard/blob/main/docs/checks.md#dangerous-workflow) ### Least Privileged GitHub Actions Token Permissions The GITHUB_TOKEN is an automatically generated secret to make authenticated calls to the GitHub API. GitHub recommends setting minimum token permissions for the GITHUB_TOKEN. - [GitHub Security Guide](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#using-the-github_token-in-a-workflow) - [The Open Source Security Foundation (OpenSSF) Security Guide](https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions) ## Feedback For bug reports, feature requests, and general feedback; please create an issue in [step-security/secure-repo](https://github.com/step-security/secure-repo) or contact us via [our website](https://www.stepsecurity.io/). Authors: - https://github.com/apps/stepsecurity-app Approvers: - AJ Schmidt (https://github.com/ajschmidt8) URL: rapidsai#2000
…pidsai#2003) Fixes rapidsai#2002. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - David Wendt (https://github.com/davidwendt) - Lawrence Mitchell (https://github.com/wence-) URL: rapidsai#2003
…#2004) CUDA 13 removes the `cudaMemAdvise` and `cudaMemPrefetchAsync` function signatures used by RMM. To continue to support CUDA 12 and 13 we move over to the v2 APIs when in CUDA 12, which are now the unversioned APIs in 13. Authors: - Robert Maynard (https://github.com/robertmaynard) Approvers: - Bradley Dice (https://github.com/bdice) URL: rapidsai#2004
Bumps the version of `actions/labeler` to `@v5` and updates the syntax in the `labeler.yml` file to account for breaking changes in that version bump. xref: rapidsai/ops#2968 Authors: - Gil Forsyth (https://github.com/gforsyth) Approvers: - Bradley Dice (https://github.com/bdice) - James Lamb (https://github.com/jameslamb) URL: rapidsai#2006
Issue: rapidsai/build-planning#207 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - Bradley Dice (https://github.com/bdice) URL: rapidsai#2007
Forward-merge branch-25.08 into branch-25.10
This works around the failures observed in rapidsai#1935 by disabling system memory resource tests on systems with HMM and CUDA drivers older than 12.8 (R575). I did some local testing with driver R550, which is old enough to reproduce the bug when used with a Linux kernel new enough to support HMM. I believe the system memory resource works as intended on HMM systems with earlier CUDA drivers, but it appears that [is_device_accessible_memory](https://github.com/rapidsai/rmm/blob/e1d777a5e3dd1306970432a35d0f868ccf67f523/cpp/tests/mr/device/test_utils.hpp#L30-L35) returns false for the system-allocated pointers. This may be a driver bug, as the pointers seem to be device-accessible if, e.g., accessed from a CUDA kernel. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Mark Harris (https://github.com/harrism) - Lawrence Mitchell (https://github.com/wence-) - Robert Maynard (https://github.com/robertmaynard) URL: rapidsai#1944
This adds a dependency on `numba-cuda`. Previously, RMM depended on only `numba`, and the CUDA target in `numba` is [deprecated in favor of `numba-cuda`](https://numba.readthedocs.io/en/stable/cuda/overview.html#cuda-deprecation-status). The fixes needed to support CUDA driver 580+ are in `numba-cuda`, so we adopt it here. The version pinning chosen here is `>=0.14.0,<0.15.0a0` to align with the current version pinning used elsewhere in RAPIDS (cudf, ucxx, etc.). See also: numba/numba#10166 Authors: - Bradley Dice (https://github.com/bdice) Approvers: - James Lamb (https://github.com/jameslamb) URL: rapidsai#2013
rapids_config will use a user defined branch over `RAPIDS_BRANCH` contents Authors: - Robert Maynard (https://github.com/robertmaynard) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) - Jake Awe (https://github.com/AyodeAwe) - Bradley Dice (https://github.com/bdice) URL: rapidsai#2010
This PR updates the numba-cuda version to `>=0.18.0,<0.19.0a0`. xref: rapidsai/cudf#19604 Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#2019
Changes the pylibrmm `Stream.view()` method from `except *` to `noexcept`. This prevents many warnings where the `view()` method. Here is an example: https://github.com/rapidsai/cudf/actions/runs/17040978623/job/48305629155?pr=19446#step:12:735 ``` performance hint: /__w/cudf/cudf/python/pylibcudf/pylibcudf/reduce.pyx:57:20: Exception check after calling 'view' will always require the GIL to be acquired. Declare 'view' as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions. ``` Also changed the `is_default()` to `noexcept` since the underlying librmm function is also declared as `noexcept`. Authors: - David Wendt (https://github.com/davidwendt) Approvers: - Matthew Murray (https://github.com/Matt711) - Bradley Dice (https://github.com/bdice) URL: rapidsai#2020
This PR adds CUDA 13.0.0 devcontainers. Contributes to rapidsai/build-planning#208. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - James Lamb (https://github.com/jameslamb) URL: rapidsai#2012
Some minor improvements to the README. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Tom Augspurger (https://github.com/TomAugspurger) URL: rapidsai#2018
Contributes to rapidsai/build-planning#208 This PR uses CUDA 13.0.0 to build and test. Authors: - James Lamb (https://github.com/jameslamb) - Bradley Dice (https://github.com/bdice) Approvers: - Bradley Dice (https://github.com/bdice) URL: rapidsai#2021
RAPIDS has deployed an autoscaling cloud build cluster that can be used to accelerate building large RAPIDS projects. This contributes to rapidsai/build-planning#209. Authors: - Paul Taylor (https://github.com/trxcllnt) Approvers: - Bradley Dice (https://github.com/bdice) URL: rapidsai#2016
…sai#2022) Aligns with changes being made in other repos for CUDA 13 bringup. Contributes to rapidsai/build-planning#208. Also contributes to rapidsai/build-planning#68. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Kyle Edwards (https://github.com/KyleFromNVIDIA) URL: rapidsai#2022
…ai#2014) Refactors device capability checks, add concurrent managed access checks to skip prefetching tests on WSL (not supported). One motivation for this refactoring is that the previous names really didn't make much sense. We created the `runtime_async_alloc::is_supported()` to check for CUDA 11.2+ support for runtime async allocation. However, we tacked on other features like `is_export_handle_type_supported(cudaMemAllocationHandleType)` and `is_hwdecompress_supported()` into the same struct. Those should have been their own features, not tied to `runtime_async_alloc`. Most of the changes in this PR are structurally similar to that. Closes rapidsai#2023. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Tom Augspurger (https://github.com/TomAugspurger) - Matthew Murray (https://github.com/Matt711) - Mark Harris (https://github.com/harrism) - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#2014
…dsai#2024) We need some fixes from NVIDIA/numba-cuda#424 to fully support `numba==0.60` and CUDA 13. This bumps the `numba-cuda` dependency to `>=0.19.1,<0.20.0a0`. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - James Lamb (https://github.com/jameslamb) URL: rapidsai#2024
This PR updates the rapids-dependency-file-generator hook to get rapidsai/dependency-file-generator#163. Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: rapidsai#2027
Contributes to rapidsai/build-planning#208 Now that rapidsai/shared-workflows#413 is merged, this converts all GitHub Actions references from `@cuda13.0` back to `branch-25.10`. ## Notes for Reviewers This is safe to admin-merge because the change is a no-op... configs on those 2 branches are identical.
…from the CUDA runtime (rapidsai#2028) This PR avoids deadlocks resulting from two Python threads simultaneously contending for both the GIL and the CUDA lock where one is running rmm code and the other is running CUDA code from another Python package calling CUDA functions. Resolves rapidsai#2026 Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Bradley Dice (https://github.com/bdice) URL: rapidsai#2028
Contributes to rapidsai/build-planning#214, updating pins to match the rest of RAPIDS: * `cuda-python` (>=12.9.2 for CUDA 12, >=13.0.1 for CUDA 13) Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Bradley Dice (https://github.com/bdice) URL: rapidsai#2031
…pidsai#2025) Closes rapidsai#1941. Authors: - Bradley Dice (https://github.com/bdice) Approvers: - Yunsong Wang (https://github.com/PointKernel) - Lawrence Mitchell (https://github.com/wence-) URL: rapidsai#2025
This PR allows building with CCCL 3.1.0. Contributes to rapidsai#2011. Authors: - Bradley Dice (https://github.com/bdice) - Basit Ayantunde (https://github.com/lamarrr) Approvers: - Nghia Truong (https://github.com/ttnghia) - Vyas Ramasubramani (https://github.com/vyasr) URL: rapidsai#2017
This PR configures this repo (adds a `.github/release.yml` file) for automatic release notes generation. See https://docs.github.com/en/repositories/releasing-projects-on-github/automatically-generated-release-notes. This will fix such issues as hotfix releases including wrong content (e.g [cugraph v25.04.01](https://github.com/rapidsai/cugraph/releases/tag/v25.04.01) contains 50+ PRs from the full v25.04.00 release when it should only contain the single hotfix PR (#5017).) Example auto-generated release notes: https://github.com/rapidsai/literate-octo-potato/releases/tag/v25.10.00a
Add optional flag argument to both `cuda_stream` and `cuda_stream_pool` creation to allow caller to specify whether the resulting stream should be implicitly synchronising with the default stream. - Closes rapidsai#2029 Authors: - Lawrence Mitchell (https://github.com/wence-) Approvers: - Mark Harris (https://github.com/harrism) - Bradley Dice (https://github.com/bdice) URL: rapidsai#2036
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Testing