[XPU][NIXL] Add GPUDirect RDMA support for XPU

[zhenwei-intel]

Purpose

Add GPUDirect RDMA support for XPU in NIXL connector.

Requirements：

• UCX must include the fix from UCT/IB/ZE: Enable GPUDirect RDMA for Intel Xe devices openucx/ucx#11187.

Limitations:

• Must be set UCX_NET_DEVICES manually for better performance until UCT/ZE: Add device topology registration openucx/ucx#11180 is merged.
• Current ze-ipc implementation has known performance issue.

Test Plan

Performance data of Llama3.3-70B int4 model with fp8 kvcache on 8xB60, ISL=1500, OSL=150
2P1D vs Non-PD under SLO TTFT<5s, ITL<100ms

• Serve more requests: under SLO, 2P1D achieved a request throughput of 1.06, compared to 0.64 for the Non-PD — 1.65x improvement.

PD commands

prefill

export UCX_TLS=ib,rc,ze_copy

export ZE_AFFINITY_MASK=2,3
export model_name=ibnzterrell/Meta-Llama-3.3-70B-Instruct-AWQ-INT4
export tp_size=2


VLLM_USE_V1=1 VLLM_NIXL_SIDE_CHANNEL_HOST=localhost VLLM_NIXL_SIDE_CHANNEL_PORT=5577 VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_ENABLE_V1_MULTIPROCESSING=1 vllm serve $model_name -tp $tp_size --host localhost --port 7101 --seed 42 --enforce-eager --dtype float16 --gpu-memory-utilization 0.9 --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_both","kv_buffer_device":"xpu"}' --max-model-len 8192 --block-size 64 --no-enable-prefix-caching --kv-cache-dtype fp8

prefill2

export UCX_TLS=ib,rc,ze_copy

export ZE_AFFINITY_MASK=4,5
export model_name=ibnzterrell/Meta-Llama-3.3-70B-Instruct-AWQ-INT4
export tp_size=2


VLLM_USE_V1=1 VLLM_NIXL_SIDE_CHANNEL_HOST=localhost VLLM_NIXL_SIDE_CHANNEL_PORT=5377 VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_ENABLE_V1_MULTIPROCESSING=1 vllm serve $model_name -tp $tp_size --host localhost --port 7102 --seed 42 --enforce-eager --dtype float16 --gpu-memory-utilization 0.9 --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_both","kv_buffer_device":"xpu"}' --max-model-len 8192 --block-size 64 --no-enable-prefix-caching--kv-cache-dtype fp8

decode

export UCX_TLS=ib,rc,ze_copy

export ZE_AFFINITY_MASK=0,1
export model_name=ibnzterrell/Meta-Llama-3.3-70B-Instruct-AWQ-INT4
export tp_size=2


VLLM_USE_V1=1 VLLM_NIXL_SIDE_CHANNEL_HOST=localhost VLLM_NIXL_SIDE_CHANNEL_PORT=5177 VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_ENABLE_V1_MULTIPROCESSING=1 vllm serve $model_name -tp $tp_size --host localhost --port 7201 --seed 42 --enforce-eager --dtype float16 --gpu-memory-utilization 0.9 --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_both","kv_buffer_device":"xpu"}' --max-model-len 8192 --block-size 64 --no-enable-prefix-caching --kv-cache-dtype fp8

proxy

python3 tests/v1/kv_connector/nixl_integration/toy_proxy_server.py --prefiller-hosts localhost  localhost --prefiller-ports 7101 7102 --decoder-host localhost --decoder-port 7201 --host localhost --port 7300

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request introduces support for GPUDirect RDMA on XPU devices by integrating NIXL and UCX. Key changes include updating the UCX build process to enable Intel's oneAPI Level Zero (ZE) API, extending NIXL connector support for XPU KV buffer types, and implementing a critical environment variable setting to prevent potential UCX memory misdetection issues on XPU. These changes are essential for enabling efficient KV cache transfers on XPU platforms.

[gemini-code-assist]

Setting UCX_MEMTYPE_CACHE=n is a critical change to prevent potential memory misdetection issues with UCX on XPU. This directly addresses a potential cause of invalid memory access, which could lead to crashes or data corruption. It's good that this is explicitly documented with a reference to the UCX FAQ.

[Copilot]

Pull request overview

This pull request adds GPUDirect RDMA support for Intel XPU devices in the NIXL KV transfer connector. The changes enable direct memory access between XPU devices for more efficient KV cache transfers during distributed inference.

Changes:

• Enable XPU device memory for NIXL KV transfers (previously only CPU memory was supported)
• Configure UCX to disable memory type caching to avoid GPU memory misdetection
• Update UCX build configuration to enable Intel Level Zero (ZE) support for XPU devices

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
vllm/platforms/xpu.py	Adds UCX_MEMTYPE_CACHE environment variable configuration to prevent memory type misdetection
vllm/distributed/kv_transfer/kv_connector/v1/nixl_connector.py	Adds "xpu" to supported KV buffer devices for XPU platform and simplifies memory type selection logic
tools/install_nixl_from_source_ubuntu.py	Updates UCX build to use specific commit and enable ZE support for XPU compatibility

Comments suppressed due to low confidence (1)

vllm/distributed/kv_transfer/kv_connector/v1/nixl_connector.py:956

• There is a logic error in this code. After line 951, nixl_memory_type will always have a value (either from current_platform.get_nixl_memory_type() or from the ternary expression), so the condition on line 952 checking if nixl_memory_type is None will never be true. This makes the error handling at lines 952-956 unreachable code.

The check on line 952 appears to be a leftover from the previous code structure and should be removed.

        if nixl_memory_type is None:
            raise RuntimeError(
                f"{self.device_type} with {self.kv_buffer_device} kv_buffer "
                "is not supported."
            )

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The UCX_MEMTYPE_CACHE environment variable is being set unconditionally for all XPU configurations, but this setting is specifically for GPUDirect RDMA support which is only relevant when KV transfer is enabled. This could have unintended side effects on configurations that don't use KV transfer.

Consider making this setting conditional on kv_transfer_config being enabled, similar to how other KV transfer related configurations are handled in this method (see line 184).

[Copilot]

The change from checking out a branch (v1.19.x) to a specific commit hash (e5d9887) lacks documentation explaining why this specific commit is needed. This makes it difficult for maintainers to understand:

1. What version or features this commit corresponds to
2. Why this specific commit was chosen over the v1.19.x branch
3. Whether this commit includes important fixes or features for GPUDirect RDMA support

Consider adding a comment explaining what this commit represents and why it was chosen, or document this in the PR description.

Suggested change

	ucx_source_path = os.path.abspath(UCX_DIR)
	ucx_source_path = os.path.abspath(UCX_DIR)
	# NOTE: We pin UCX to a specific, known-good commit instead of tracking
	# a moving branch (e.g., v1.19.x). Commit e5d9887 corresponds to a UCX
	# revision that has been validated with nixl for our GPUDirect RDMA
	# use case. If you update this hash, please ensure the new commit has
	# been tested with nixl and GPUDirect RDMA and update this comment.

[Copilot]

The PR description is empty except for the template. This makes it difficult to understand:

1. What problem this PR is solving
2. How GPUDirect RDMA support was tested
3. What the expected performance or functional improvements are
4. What the changes to UCX configuration (ZE support, specific commit) are intended to accomplish

Please complete the PR description with the purpose, test plan, and test results as indicated by the template checklist.

[zhenwei-intel]

@copilot help review again

[Copilot]

@zhenwei-intel I've opened a new pull request, #3, to work on those changes. Once the pull request is ready, I'll request review from you.