Skip to content

Point the nnterp pin at our fork for the text-tower fix - #47

Closed
can-goodfire wants to merge 1 commit into
nnterp-rebasefrom
can/nnterp-fork-pin
Closed

can-goodfire wants to merge 1 commit into
nnterp-rebasefrom
can/nnterp-fork-pin

Conversation

@can-goodfire

Copy link
Copy Markdown
Collaborator

Unblocks us from waiting on ndif-team/nnterp#52 to be reviewed.

Why

nnsight's LanguageModel refuses any config registered with AutoModelForImageTextToText. qwen3_5_moe — the text tower of Qwen3.6-35B-A3B, and the target of the hookpoint-vocabulary work — is registered there and under AutoModelForCausalLM, so the tower loads cleanly and the refusal is over-broad. Without the fix, nnterp cannot load the model at all.

#52 fixes it upstream and is open for review. This repoints the existing git source at our fork in the meantime.

What changes

One line plus its comment:

-nnterp = { git = "https://github.com/ndif-team/nnterp", rev = "b4a3127…" }
+nnterp = { git = "https://github.com/can-goodfire/nnterp", rev = "45f386b…" }

45f386b is the previous upstream pin b4a3127 plus the two commits in #52, so this is a strict superset of what we had — the packaging fix (#49) that motivated the git source in the first place is still in there. The comment now records both reasons and how each retires:

uv lock --upgrade-package nnterp moved only nnterp (1.3.1.dev14+gb4a3127461.3.1.dev16+g45f386b7c); the other 239 packages are unchanged.

Verification

Installed from the fork's public URL rather than a local path, to prove the fork actually unblocks a clean checkout:

uv pip install "nnterp @ git+https://github.com/can-goodfire/nnterp@can/text-tower-loading"

tiny-random/qwen3.5-moe then loads as Qwen3_5MoeForCausalLM with no vision tower attached, and traces to logits.

One caveat worth knowing

This branch still resolves transformers 4.57.1, which carries no qwen3_5_moe. So the fix ships here but cannot be exercised on this lineage until a transformers >= 5.16 bump reaches it. That bump landed downstream on can/protocol-refactor (#46, merged) and whether it should also come to the path to main is a separate call — flagging rather than bundling it.

Also worth noting: can/protocol-refactor has no nnterp dependency at all, so the round-1 hookpoint work is not blocked by #52 — its pytorch_hooks backend loads the tower through AutoModelForCausalLM directly.

🤖 Generated with Claude Code

nnsight's LanguageModel refuses any config registered with
AutoModelForImageTextToText, so it cannot load qwen3_5_moe — the text tower of
Qwen3.6-35B-A3B and the target of the hookpoint-vocabulary work — even though
AutoModelForCausalLM resolves to that tower cleanly. ndif-team/nnterp#52 fixes
it and is open for review; this unpins us from that review landing.

45f386b is the upstream pin b4a3127 plus the two commits in #52, so this is a
strict superset of what we had — the packaging fix (#49) that motivated the git
source in the first place is still in there, and the comment now records both
reasons and how each retires.

Verified installed from the fork's public URL rather than a local path:
nnterp 1.3.1.dev16+g45f386b7c loads tiny-random/qwen3.5-moe as
Qwen3_5MoeForCausalLM with no vision tower attached, and traces to logits.

Note this branch still resolves transformers 4.57.1, which carries no
qwen3_5_moe, so the fix ships here but cannot be exercised until a transformers
>= 5.16 bump reaches this lineage. That bump landed downstream on
can/protocol-refactor (#46) and is a separate decision for the path to main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@can-goodfire

Copy link
Copy Markdown
Collaborator Author

NNterp implemented fixes upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants