Repository navigation
start.sh: detect the fabric port on each Spark (fixes #44) - #45
Merged
Merged
Conversation
The default was hardcoded to rocep1s0f0, so a pair cabled on the second port (enp1s0f1np1) failed with no hint. start.sh now lists each Spark's RoCE ports whose link is up and that have an IPv4 address, picks the pair on the same subnet (the cable between them; with several cables this skips the others), falls back to the first active port with a warning, and prints its choice and how to override it. FABRIC_HCA in .env still overrides both ranks. Fixes #44. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #44.
start.shhardcoded the fabric port torocep1s0f0, so a pair cabled on the ConnectX-7's second port (enp1s0f1np1) failed with no hint..env.examplealready claimed the port was detected.Now, on each Spark:
start.shlists the RoCE ports whose link is up and that have an IPv4 address.== fabric: rocep1s0f1 (enp1s0f1np1, 192.168.100.1) here, rocep1s0f1 (enp1s0f1np1, 192.168.100.2) on <worker>; set FABRIC_HCA in .env to choose another portFABRIC_HCA.FABRIC_HCAin.envstill overrides both ranks. The engine still adds the sibling device behind the other PCIe domain of whichever port is chosen.Tested:
install/is unchanged, so the image tag stays4046c81baa07. The manual-install doc now mentions--fabric-hca rocep1s0f1for the second port.🤖 Generated with Claude Code