Skip to content

TransformersModel.load_adapter: attach a PEFT adapter after construction - #714

Open
JadenFiotto-Kaufman wants to merge 1 commit into
0.8from
feat/load-adapter
Open

JadenFiotto-Kaufman wants to merge 1 commit into
0.8from
feat/load-adapter

Conversation

@JadenFiotto-Kaufman

Copy link
Copy Markdown
Member

The post-hoc form of the peft= constructor kwarg, for #555.

model = TransformersModel("gpt2", task="text-generation")   # meta, no weights
model.load_adapter(adapter_id)   # grafts the adapter's config onto the meta tree
with model.trace(prompt):        # dispatch loads base + adapter weights together
    ...

Before dispatch, only the adapter's config is read: the meta module gains the adapter's architecture, so module paths (and remote requests) match the adapted model, and the real weights arrive with the base's at dispatch. This is what made the fallthrough PreTrainedModel.load_adapter fail there — safetensors cannot load onto meta.

After dispatch, it goes through the same swap the remote env path uses: call again with another id to swap adapters, or None to remove the current one, with the envoy tree rebuilt around the new structure each time.

Mechanically: the swap logic moves out of _remoteable_set_env into _swap_adapter, its rebind closure becomes _rebind, and the env path delegates to them unchanged. Our load_adapter deliberately shadows transformers' method of the same name, which mutates the module structure without the envoy tree noticing. Adapters attached this way ship with remote requests automatically, since _remoteable_get_env reads self.peft.

Four new tests in TestPeft against the local LoRA fixture; full suite green.

Closes #555

🤖 Generated with Claude Code

The post-hoc form of the `peft=` constructor kwarg. Before dispatch it grafts
only the adapter's config onto the meta module, so the tree gains the adapter
modules and remote paths match, and the real weights arrive with the base's at
dispatch; safetensors cannot load onto meta, which is what made the fallthrough
`PreTrainedModel.load_adapter` fail there (#555). After dispatch it goes
through the same swap the remote env path uses, so calling again with another
id swaps adapters and `None` removes the current one, with the envoy tree
rebuilt around the new structure each time.

Mechanically, the swap logic moves out of `_remoteable_set_env` into
`_swap_adapter`, with the `rebind` closure promoted to `_rebind`; the env path
delegates to them unchanged. Our `load_adapter` shadows transformers' method
of the same name deliberately: reached by fallthrough, that one mutates the
module structure without the envoy tree noticing.

An adapter attached this way ships with remote requests automatically, since
`_remoteable_get_env` reads `self.peft`.

Requested by @Butanium in #555.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant