Support multilingual (mdeberta-v3-base) checkpoints#16
Open
julienmarie wants to merge 1 commit into
Open
Conversation
Multilingual GLiNER2 checkpoints (e.g. fastino/gliner2-multi-v1, an
mdeberta-v3-base backbone) decoded to garbage spans, while the English
deberta-v3-base base model worked. Diagnosed by confirming input_ids,
encoder hidden states, subword→word pooling and span_rep were all
byte-identical to the Python reference, which isolated the divergence to
the count-aware scoring head. Four fixes:
1. Read vocab_size from encoder_config/config.json (the top-level config
omits it). It was defaulting to 128011, building the word-embedding
too small for the 250112-token multilingual vocab.
2. downloadModelDirectory now fetches encoder_config/config.json.
3. Build the count-aware layer per config.countingLayer: count_lstm
(CountLSTM projector) vs count_lstm_v2 (CountLSTMv2 transformer). It
was hardcoded to CountLSTMv2, so count_lstm checkpoints loaded those
weights as random — silently garbling every span score (count_embed
feeds the span-scoring einsum). A CountEmbedding protocol abstracts
the two variants; the projector keys get a remap entry.
4. safeSpans invalid-index zeros are Int32, not Float — otherwise
MLX.where promotes the span indices to Float and the gather fails
("cannot gather with indices type") on the CPU backend.
Verified: fastino/gliner2-multi-v1 now matches the Python reference for
EN and FR NER on both CPU and GPU.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes multilingual GLiNER2 checkpoints work — e.g.
fastino/gliner2-multi-v1(mDeBERTa-v3-base backbone, fr/en/es/de/it/pt). Before this, such models loaded without error but decoded to garbage spans (fragment text, ~0.5 confidences); only the Englishdeberta-v3-basebase model produced correct output. This is the "Additional GLiNER models … not yet available" item from the README.Diagnosis
The bug was isolated by comparing intermediates against the Python
gliner2reference on identical input. Forfastino/gliner2-multi-v1, all of these were byte-identical to Python:input_ids(schema + text),span_repoutput.That narrowed the divergence to the count-aware scoring head, where
count_embedprojects the label embeddings before the span-scoring einsum.Changes (4 fixes)
vocab_sizeis read fromencoder_config/config.json. The top-levelconfig.jsonomits it, so it fell back to the default128011and the word-embedding was built too small for the multilingual 250112-token vocab. (GLiNER2.fromPretrained,ExtractorConfig.vocabSizemade mutable.)downloadModelDirectorynow fetchesencoder_config/config.json(added to the snapshot file patterns) so the value above is available.config.countingLayer.count_lstm→CountLSTM(GRU + MLP projector);count_lstm_v2→CountLSTMv2(GRU + DownscaledTransformer). It was hardcoded toCountLSTMv2, socount_lstmcheckpoints loaded those (absent) transformer weights as random, silently garbling every span score. A smallCountEmbeddingprotocol abstracts the two variants; acount_embed.projectorkey-remap entry is added.safeSpansinvalid-index zeros areInt32, notFloat.MLX.wherewas promoting the span indices toFloat, which made the downstream gather fail with "cannot gather with indices type" on the CPU backend.Verification
fastino/gliner2-multi-v1now matches the Python reference on both CPU and GPU:The English
fastino/gliner2-base-v1continues to produce identical results to before (the count-layer switch defaults toCountLSTMv2).Notes
deberta-v3-baseis supported" line can be relaxed.