update features/15_vllm_support notebook - #37
Open
yiksiu-chan wants to merge 2 commits into
Open
yiksiu-chan wants to merge 2 commits into
yiksiu-chan wants to merge 2 commits into
Conversation
khaiwang
reviewed
Jun 2, 2026
khaiwang
left a comment
khaiwang
left a comment
Contributor
There was a problem hiding this comment.
The notebook looks good to me, scripts run well. a few obsolete links need fix, and the vllm-lens dependency needs confirm
| " `.chunk(2, dim=-1)`; the separate projections don't exist.\n", | ||
| "\n", | ||
| " The full inventory with the fix for each lives in the repo's\n", | ||
| " [`intervention-gaps/REPORT.md`](https://github.com/ndif-team/nnsight/blob/main/src/nnsight/modeling/vllm/intervention-gaps/REPORT.md).\n", |
Contributor
There was a problem hiding this comment.
this doc will retire soon, you might want to point to docs/models/vllm.md
| "async backend for transport compression.\n", | ||
| "\n", | ||
| "<pre><code>pip install nnsight \"vllm==0.18.0\" \"triton>=3.1\" zstandard\n", | ||
| "<pre><code>pip install \"nnsight[vllm]\" \"vllm-lens>=1.1.0\" zstandard\n", |
Contributor
There was a problem hiding this comment.
is vllm-lens used anywhere in this notebook or nnsight (just curious)?
| "\n", | ||
| "A full multi-node Docker example, including the Dockerfile and cluster\n", | ||
| "test harness, lives at\n", | ||
| "[`nnsight/src/nnsight/modeling/vllm/examples/multi_node_with_ray/`](https://github.com/ndif-team/nnsight/tree/main/src/nnsight/modeling/vllm/examples/multi_node_with_ray).\n", |
Contributor
There was a problem hiding this comment.
this doc is also relocated to examples/ray/ now
| " distributed executor, and the async engine. Read this if you're modifying\n", | ||
| " the integration or trying to understand why a specific intervention\n", | ||
| " behaves the way it does inside vLLM.\n", | ||
| "- [`src/nnsight/modeling/vllm/examples/multi_node_with_ray/`](https://github.com/ndif-team/nnsight/tree/main/src/nnsight/modeling/vllm/examples/multi_node_with_ray) —\n", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.