TL/UCP: use memcpy instead of sendrecv in agv

[Sergei-Lebedev]

What

Use ucc_mc_memcpy instead of sendrecv in allgatherv ring

Why ?

UCP doesn't support efficient copy of CUDA buffers in self transport. It does it through host staging buffer

[greptile-apps]

Greptile Summary

Replaced inefficient UCP self-transport sendrecv with ucc_mc_memcpy for local buffer copy in allgatherv ring algorithm. UCP doesn't efficiently copy CUDA buffers through self-transport and uses host staging buffers, while ucc_mc_memcpy provides direct GPU-to-GPU copies.

• Replaced local sendrecv pair with single ucc_mc_memcpy call in non-inplace case
• Adjusted ring algorithm loop from tsize to tsize - 1 iterations to account for local copy being done upfront
• Updated send_idx calculation to remove + 1 offset (now trank - send_posted)
• Updated recv_idx calculation to add - 1 offset (now trank - recv_posted - 1)
• Removed inplace branch that manually set counters, as loop now handles all cases uniformly
• Added profiling events for start and completion
• Simplified error handling by removing separate error label

The index adjustments align with the pattern used in allgather_ring.c and correctly implement the ring algorithm with upfront local copy.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The changes follow an established pattern from allgather_ring.c, with correct mathematical adjustments to loop bounds and index calculations. The replacement of sendrecv with memcpy is a straightforward performance optimization that addresses UCP's inefficient handling of CUDA self-transport. Error handling has been appropriately simplified.
• No files require special attention

Important Files Changed

Filename	Overview
src/components/tl/ucp/allgatherv/allgatherv_ring.c	Replaced inefficient UCP self-transport sendrecv with direct memcpy for CUDA buffers, adjusted ring algorithm loop bounds and indices accordingly

[wfaderhold21]

Looks good

[wfaderhold21]

LGTM! question though: would ucc_mc_memcpy potentially cause a deadlock if using cuda buffers for a scenario of NCCL + UCC? should this be a local copy instead?

[janjust]

/build

[janjust]

/build

[janjust]

/build

[janjust]

/build