Skip to content

Fix SGLang disagg on ionic/RoCE fabrics (AINIC transport env) #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mkuznet1 merged 2 commits into from
Jul 7, 2026
Merged

Fix SGLang disagg on ionic/RoCE fabrics (AINIC transport env) #2

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7decf3e
Fix SGLang disagg on ionic/RoCE fabrics: preserve AINIC transport env
Jul 4, 2026
581a61a
sglang_disagg_server: source host_ip from IPADDRS[NODE_RANK]
Jul 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions scripts/sglang_disagg/sglang_disagg_server.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,31 @@ esac
pip install py-spy
pip install --ignore-installed --force-reinstall flask

# AINIC/ionic RoCE fix: MOONCAKE_COOKBOOK's set_env_vars.sh assumes a Mellanox/mlx5
# fabric. On ionic-based RoCE clusters (e.g. mia1) it (1) hardcodes mlx5 IB device
# names, (2) forces NCCL_IB_DISABLE=1, and (3) mangles NCCL_SOCKET_IFNAME via an
# `ip route` awk parse. Save the transport iface set by the launcher, then re-assert
# the correct ionic values after sourcing so RDMA over ionic actually engages.
_SAVED_NCCL_SOCKET_IFNAME="${NCCL_SOCKET_IFNAME:-}"

source $MOONCAKE_COOKBOOK_PATH/set_env_vars.sh
#trap 'echo "Error occurred. Cleaning up..."; exit 0' ERR

host_ip=$(hostname -I | awk '{print $1}')
# Re-assert RDMA/socket env clobbered by set_env_vars.sh (see note above).
export NCCL_IB_DISABLE=0
[[ -n "${IB_DEVICES:-}" ]] && export IBDEVICES="${IB_DEVICES}"
export NCCL_SOCKET_IFNAME="${_SAVED_NCCL_SOCKET_IFNAME:-eno0}"
export GLOO_SOCKET_IFNAME="${_SAVED_NCCL_SOCKET_IFNAME%%,*}"

# This node's IP is already resolved by run.sh and handed to us (rank-ordered,
# post-rendezvous) via IPADDRS. Reuse IPADDRS[NODE_RANK] so the address we bind
# and advertise (--host, socket_barrier --local-ip) matches exactly what peers
# registered for us in the router/barrier; re-deriving it here can pick a
# different NIC on multi-homed nodes and desync the two.
IFS=',' read -ra IP_ARRAY <<< "$IPADDRS"
host_ip="${IP_ARRAY[NODE_RANK]:-$(hostname -I | awk '{print $1}')}"
host_name=$(hostname)
unset _SAVED_NCCL_SOCKET_IFNAME

# =============================================================================
# Model-Specific Configuration Maps
Expand Down Expand Up @@ -112,7 +132,7 @@ python $MOONCAKE_COOKBOOK_PATH/socket_barrier.py \
# Cluster Topology Configuration
# =============================================================================

IFS=',' read -ra IP_ARRAY <<< "$IPADDRS"
# IP_ARRAY was already parsed from IPADDRS above (where host_ip is derived).

PREFILL_ARGS=""
DECODE_ARGS=""
Expand Down