[feat] Add HCCL tensor transport for RDT by KaisennHu · Pull Request #21 · Ascend/ray-ascend

KaisennHu · 2026-02-14T07:47:02Z

Description

Implement HCCLTensorTransport to integrate HCCL with RDT tensor transport flow and enable explicit registration
Add HCCL collective backend tests by using register_collective_backend API
Add tests covering HCCL tensor transport usage in RDT
Provide register_hccl_collective_backend, register_hccl_tensor_transport, and register_yr_tensor_transport API
Update the usage doc

Self-Check Result

HCCL collective backend tests by using register_collective_backend API:

[root@devserver-bms-54 ray-ascend]# pytest -v /home/hhc/ray_wp/ray-ascend/tests/collective/test_hccl_via_registry.py
====================================================================== test session starts =======================================================================
platform linux -- Python 3.10.16, pytest-8.3.4, pluggy-1.5.0 -- /usr/bin/python
cachedir: .pytest_cache
Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket=<bucket_type>
rootdir: /home/hhc/ray_wp/ray-ascend
configfile: pyproject.toml
plugins: cov-6.2.1, hydra-core-1.3.2, jaxtyping-0.3.2, anyio-4.8.0, random-order-1.2.0, mock-3.14.1, typeguard-4.3.0
collected 6 items                                                                                                                                                

tests/collective/test_hccl_via_registry.py::test_allreduce PASSED                                                                                          [ 16%]
tests/collective/test_hccl_via_registry.py::test_broadcast PASSED                                                                                          [ 33%]
tests/collective/test_hccl_via_registry.py::test_allgather PASSED                                                                                          [ 50%]
tests/collective/test_hccl_via_registry.py::test_reduce PASSED                                                                                             [ 66%]
tests/collective/test_hccl_via_registry.py::test_reducescatter PASSED                                                                                      [ 83%]
tests/collective/test_hccl_via_registry.py::test_send_recv PASSED                                                                                          [100%]

======================================================================== warnings summary ========================================================================
tests/collective/test_hccl_via_registry.py::test_allreduce
  /usr/local/python3.10.16/lib/python3.10/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= tests coverage =========================================================================
________________________________________________________ coverage: platform linux, python 3.10.16-final-0 ________________________________________________________

Name                                                      Stmts   Miss  Cover   Missing
---------------------------------------------------------------------------------------
ray_ascend/__init__.py                                        0      0   100%
ray_ascend/_version.py                                       13     13     0%   4-34
ray_ascend/collective/__init__.py                             2      0   100%
ray_ascend/collective/hccl_collective_group.py              334    259    22%   30, 34-36, 65-83, 94-102, 118-119, 122-123, 126-130, 136-146, 150-162, 167, 173-181, 197-222, 241-274, 291-315, 326-333, 347-374, 393-426, 441-461, 476-496, 499-512, 524-554, 558-560, 572-585, 589-615, 625-636, 640-642, 647-653, 668-677
ray_ascend/direct_transport/__init__.py                       3      3     0%   1-8
ray_ascend/direct_transport/hccl_tensor_transport.py          9      9     0%   1-22
ray_ascend/direct_transport/yr_tensor_transport.py          116    116     0%   1-253
ray_ascend/direct_transport/yr_tensor_transport_util.py      44     44     0%   1-91
---------------------------------------------------------------------------------------
TOTAL                                                       521    444    15%
Coverage HTML written to dir htmlcov
Coverage XML written to file coverage.xml
============================================================ 6 passed, 1 warning in 60.06s (0:01:00) =============================================================

HCCL tensor transport tests:

[root@devserver-bms-54 ray-ascend]# pytest -v /home/hhc/ray_wp/ray-ascend/tests/direct_transport/test_hccl_tensor_transport.py
====================================================================== test session starts =======================================================================
platform linux -- Python 3.10.16, pytest-8.3.4, pluggy-1.5.0 -- /usr/bin/python
cachedir: .pytest_cache
Test order randomisation NOT enabled. Enable with --random-order or --random-order-bucket=<bucket_type>
rootdir: /home/hhc/ray_wp/ray-ascend
configfile: pyproject.toml
plugins: cov-6.2.1, hydra-core-1.3.2, jaxtyping-0.3.2, anyio-4.8.0, random-order-1.2.0, mock-3.14.1, typeguard-4.3.0
collected 1 item                                                                                                                                                 

tests/direct_transport/test_hccl_tensor_transport.py::test_hccl_tensor_transport PASSED                                                                    [100%]

======================================================================== warnings summary ========================================================================
tests/direct_transport/test_hccl_tensor_transport.py::test_hccl_tensor_transport
  /usr/local/python3.10.16/lib/python3.10/site-packages/ray/_private/worker.py:2052: FutureWarning: Tip: In future versions of Ray, Ray will no longer override accelerator visible devices env var if num_gpus=0 or num_gpus=None (default). To enable this behavior and turn off this error message, set RAY_ACCEL_ENV_VAR_OVERRIDE_ON_ZERO=0
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================================================================= tests coverage =========================================================================
________________________________________________________ coverage: platform linux, python 3.10.16-final-0 ________________________________________________________

Name                                                      Stmts   Miss  Cover   Missing
---------------------------------------------------------------------------------------
ray_ascend/__init__.py                                        0      0   100%
ray_ascend/_version.py                                       13     13     0%   4-34
ray_ascend/collective/__init__.py                             2      0   100%
ray_ascend/collective/hccl_collective_group.py              334    259    22%   30, 34-36, 65-83, 94-102, 118-119, 122-123, 126-130, 136-146, 150-162, 167, 173-181, 197-222, 241-274, 291-315, 326-333, 347-374, 393-426, 441-461, 476-496, 499-512, 524-554, 558-560, 572-585, 589-615, 625-636, 640-642, 647-653, 668-677
ray_ascend/direct_transport/__init__.py                       3      0   100%
ray_ascend/direct_transport/hccl_tensor_transport.py          9      2    78%   21-22
ray_ascend/direct_transport/yr_tensor_transport.py          116     82    29%   47-49, 52, 56, 60, 66-104, 107-125, 140-152, 160-173, 185, 194-220, 238-248
ray_ascend/direct_transport/yr_tensor_transport_util.py      44     22    50%   9-10, 22-23, 46, 49, 53-55, 58-59, 62-63, 66, 71, 79, 82-83, 86-87, 90-91
---------------------------------------------------------------------------------------
TOTAL                                                       521    378    27%
Coverage HTML written to dir htmlcov
Coverage XML written to file coverage.xml
============================================================ 1 passed, 1 warning in 68.25s (0:01:08) =============================================================

Related issues

Closes #9 #3

ascend-robot · 2026-02-14T07:47:21Z

CLA Signature Guide

@KaisennHu , thanks for your pull request.

The following commit(s) are not associated with a signed Contributor License Agreement (CLA).

Commit	Reason
[`7d52991` [feat]: Support hccl tensor tra...](`7d52991`)	the email used in the commit is not linked to a signed CLA! please verify that it matches the email you used when signing the CLA.

To sign CLA, click here.

To check if your email is configured correctly, refer to the FAQs.

Once you've signed the CLA or updating your email, please comment /check-cla to revalidate CLA status.

KaisennHu · 2026-02-14T07:56:39Z

/check-cla

ascend-robot · 2026-02-14T07:56:48Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-14T08:05:07Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-27T01:12:21Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-27T02:25:06Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-27T02:43:04Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-27T03:06:45Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-28T02:19:24Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-28T03:02:22Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-28T03:17:48Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-28T03:40:41Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-28T04:56:21Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-28T07:03:17Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-02-28T07:11:44Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-04-21T07:37:39Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot · 2026-04-21T07:42:23Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

tianyi-ge · 2026-05-29T02:38:33Z

I would suggest a "version compatible design". for older ray versions (without register_collective_backend), hccl and hccl rdt will not be activated but yr and hixl can still be used. if ray>=2.56, then all apis are available @KaisennHu

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

ascend-robot · 2026-05-30T02:43:03Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

ascend-robot · 2026-05-30T07:05:17Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

tianyi-ge · 2026-05-30T07:38:43Z

 import ray
 from ray.util import collective
-from ray_ascend.collective import HCCLGroup
+from ray_ascend.collective.hccl_collective_group import HCCLGroup


import HCCLGroup is not necessary. just create_collective_group

tianyi-ge · 2026-05-30T07:41:05Z

    def backend(cls) -> Backend:
        """Return the backend type for this group."""
-        return Backend.HCCL
+        return "HCCL"


here automatically setattr. so here Backend.HCCL should work

tianyi-ge · 2026-05-30T07:42:14Z

+
+| Ray Version | YR Transport | HCCL Collective | HCCL Tensor Transport (RDT) |
+|-------------|-------------|-----------------|-----------------------------|
+| 2.55        | ✅          | ❌              | ❌                           |


change 2.55.0
to
>=2.55.0, <2.56.0
to avoid ambuguity

tianyi-ge · 2026-05-30T07:43:48Z

@@ -0,0 +1,234 @@
+import pytest


since this test works, how about deleting the pure hccl tests?

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

ascend-robot · 2026-05-30T08:09:07Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

ascend-robot · 2026-05-30T08:14:19Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

ascend-robot · 2026-05-30T08:38:02Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

dpj135 · 2026-05-30T08:59:51Z

+    try:
+        from ray.util.collective.backend_registry import register_collective_backend
+    except ImportError as e:
+        import ray
+
+        raise RuntimeError(
+            f"register_hccl_collective_backend requires Ray >= 2.56, "
+            f"but Ray {ray.__version__} is installed. "
+            f"Please upgrade: pip install 'ray>=2.56'"
+        ) from e
+
+    from .collective.hccl_collective_group import HCCLGroup
+
+    register_collective_backend("HCCL", HCCLGroup)


should we check if hccl is installed? like,

raise ImportError( "YR tensor transport requires the [yr] extra dependency. " "Please install it with: pip install ray-ascend[yr]" ) from e

Being able to import torch_npu means that hccl is definitely installed.

dpj135 · 2026-05-30T09:05:56Z

+)
+
+
+class HCCLTensorTransport(CollectiveTensorTransport):


should we implement get_communicator_metadata()?

not needed. It is aligned with NcclTensorTransport.

dpj135 · 2026-05-30T09:09:22Z

We need to update ray-ascend version

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

ascend-robot · 2026-05-30T09:48:11Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

dpj135 · 2026-06-01T02:22:27Z

+
+    if importlib.util.find_spec("torch_npu") is None:
+        raise ImportError("torch_npu not found")
+    ctypes.CDLL("libhccl.so")


where is libhccl.so? In ascend-toolkit/?

should we remind users to set PATH？

Yes, libhccl.so is indeed part of the ascend-toolkit (CANN). Users are already reminded via check_backend_availability.

dpj135 · 2026-06-01T02:24:12Z

we should update the type annotation of tensor

dpj135

LGTM. There are still some comments.

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

ascend-robot · 2026-06-01T03:01:07Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

ascend-robot · 2026-06-01T08:11:39Z

CLA Signature Pass

KaisennHu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

ascend-robot added the ascend-cla/no label Feb 14, 2026

ascend-robot added ascend-cla/yes and removed ascend-cla/no labels Feb 14, 2026

KaisennHu force-pushed the feat/rdt-hccl branch from 7d52991 to 8732aa7 Compare February 14, 2026 08:04

KaisennHu force-pushed the feat/rdt-hccl branch from 8732aa7 to 0c0828e Compare February 27, 2026 01:12

KaisennHu force-pushed the feat/rdt-hccl branch from 0c0828e to 54a2e6e Compare February 27, 2026 02:24

KaisennHu force-pushed the feat/rdt-hccl branch from 54a2e6e to 5fe39a1 Compare February 27, 2026 02:42

KaisennHu force-pushed the feat/rdt-hccl branch from 5fe39a1 to 161ef28 Compare February 27, 2026 03:06

KaisennHu force-pushed the feat/rdt-hccl branch from 7ebce6d to dc6769b Compare February 28, 2026 03:02

KaisennHu force-pushed the feat/rdt-hccl branch from dc6769b to 1837cad Compare February 28, 2026 03:17

KaisennHu force-pushed the feat/rdt-hccl branch from 1837cad to b7c0480 Compare February 28, 2026 03:40

KaisennHu force-pushed the feat/rdt-hccl branch from b7c0480 to 2f9d321 Compare February 28, 2026 04:56

KaisennHu force-pushed the feat/rdt-hccl branch from 2f9d321 to eda04eb Compare February 28, 2026 07:03

KaisennHu force-pushed the feat/rdt-hccl branch from eda04eb to 3315690 Compare February 28, 2026 07:11

KaisennHu force-pushed the feat/rdt-hccl branch from 3315690 to 548d921 Compare February 28, 2026 07:14

KaisennHu force-pushed the feat/rdt-hccl branch from 16380b6 to a30d0ec Compare April 21, 2026 07:42

zrss mentioned this pull request May 26, 2026

[feat] Support HIXL point-to-point tensor transport for Ray RDT #57

Open

[feat]: Support hccl tensor transport

1ed0bf1

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

KaisennHu force-pushed the feat/rdt-hccl branch from a30d0ec to 1ed0bf1 Compare May 30, 2026 02:42

[feat]: Support hccl tensor transport

1cb149c

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

tianyi-ge reviewed May 30, 2026

View reviewed changes

[feat]: Support hccl tensor transport

4e2acbf

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

[feat]: Support hccl tensor transport

aad41e2

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

[feat]: Support hccl tensor transport

0fb0560

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

tianyi-ge approved these changes May 30, 2026

View reviewed changes

dpj135 reviewed May 30, 2026

View reviewed changes

[feat]: Support hccl tensor transport

58ee47b

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

dpj135 reviewed Jun 1, 2026

View reviewed changes

dpj135 approved these changes Jun 1, 2026

View reviewed changes

[feat]: Support hccl tensor transport

35bfcd1

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

[feat]: Support hccl tensor transport

0b925de

Signed-off-by: Haichuan Hu <kaisennhu@gmail.com>

		)


		class HCCLTensorTransport(CollectiveTensorTransport):

Conversation

KaisennHu commented Feb 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Self-Check Result

Related issues

Uh oh!

ascend-robot commented Feb 14, 2026

CLA Signature Guide

Uh oh!

KaisennHu commented Feb 14, 2026

Uh oh!

ascend-robot commented Feb 14, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 14, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 27, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 27, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 27, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 27, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 28, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 28, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 28, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 28, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 28, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 28, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Feb 28, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Apr 21, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented Apr 21, 2026

CLA Signature Pass

Uh oh!

tianyi-ge commented May 29, 2026

Uh oh!

ascend-robot commented May 30, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented May 30, 2026

CLA Signature Pass

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ascend-robot commented May 30, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented May 30, 2026

CLA Signature Pass

Uh oh!

ascend-robot commented May 30, 2026

KaisennHu commented Feb 14, 2026 •

edited

Loading

KaisennHu Jun 1, 2026 •

edited

Loading