Skip to content

Support multilingual (mdeberta-v3-base) checkpoints#16

Open
julienmarie wants to merge 1 commit into
MacPaw:mainfrom
julienmarie:fix/multilingual-mdeberta-support
Open

Support multilingual (mdeberta-v3-base) checkpoints#16
julienmarie wants to merge 1 commit into
MacPaw:mainfrom
julienmarie:fix/multilingual-mdeberta-support

Conversation

@julienmarie

Copy link
Copy Markdown

Summary

Makes multilingual GLiNER2 checkpoints work — e.g. fastino/gliner2-multi-v1 (mDeBERTa-v3-base backbone, fr/en/es/de/it/pt). Before this, such models loaded without error but decoded to garbage spans (fragment text, ~0.5 confidences); only the English deberta-v3-base base model produced correct output. This is the "Additional GLiNER models … not yet available" item from the README.

Diagnosis

The bug was isolated by comparing intermediates against the Python gliner2 reference on identical input. For fastino/gliner2-multi-v1, all of these were byte-identical to Python:

  • the built input_ids (schema + text),
  • the encoder hidden states (mean/std and per-token vectors),
  • the subword→word pooled embeddings,
  • the span_rep output.

That narrowed the divergence to the count-aware scoring head, where count_embed projects the label embeddings before the span-scoring einsum.

Changes (4 fixes)

  1. vocab_size is read from encoder_config/config.json. The top-level config.json omits it, so it fell back to the default 128011 and the word-embedding was built too small for the multilingual 250112-token vocab. (GLiNER2.fromPretrained, ExtractorConfig.vocabSize made mutable.)
  2. downloadModelDirectory now fetches encoder_config/config.json (added to the snapshot file patterns) so the value above is available.
  3. The count layer is built per config.countingLayer. count_lstmCountLSTM (GRU + MLP projector); count_lstm_v2CountLSTMv2 (GRU + DownscaledTransformer). It was hardcoded to CountLSTMv2, so count_lstm checkpoints loaded those (absent) transformer weights as random, silently garbling every span score. A small CountEmbedding protocol abstracts the two variants; a count_embed.projector key-remap entry is added.
  4. safeSpans invalid-index zeros are Int32, not Float. MLX.where was promoting the span indices to Float, which made the downstream gather fail with "cannot gather with indices type" on the CPU backend.

Verification

fastino/gliner2-multi-v1 now matches the Python reference on both CPU and GPU:

EN  character: Cato 0.9996, Clove 0.9995, Thresh 0.9995, Peeta 0.992
EN  place:     District 12 0.97, cave 0.74, stream 0.67
FR  personnage: Charles 0.9999, Emma 0.999    lieu: Rouen 0.995

The English fastino/gliner2-base-v1 continues to produce identical results to before (the count-layer switch defaults to CountLSTMv2).

Notes

  • Relation extraction for these checkpoints is still the separate WIP item.
  • The README's "currently only deberta-v3-base is supported" line can be relaxed.

Multilingual GLiNER2 checkpoints (e.g. fastino/gliner2-multi-v1, an
mdeberta-v3-base backbone) decoded to garbage spans, while the English
deberta-v3-base base model worked. Diagnosed by confirming input_ids,
encoder hidden states, subword→word pooling and span_rep were all
byte-identical to the Python reference, which isolated the divergence to
the count-aware scoring head. Four fixes:

1. Read vocab_size from encoder_config/config.json (the top-level config
   omits it). It was defaulting to 128011, building the word-embedding
   too small for the 250112-token multilingual vocab.
2. downloadModelDirectory now fetches encoder_config/config.json.
3. Build the count-aware layer per config.countingLayer: count_lstm
   (CountLSTM projector) vs count_lstm_v2 (CountLSTMv2 transformer). It
   was hardcoded to CountLSTMv2, so count_lstm checkpoints loaded those
   weights as random — silently garbling every span score (count_embed
   feeds the span-scoring einsum). A CountEmbedding protocol abstracts
   the two variants; the projector keys get a remap entry.
4. safeSpans invalid-index zeros are Int32, not Float — otherwise
   MLX.where promotes the span indices to Float and the gather fails
   ("cannot gather with indices type") on the CPU backend.

Verified: fastino/gliner2-multi-v1 now matches the Python reference for
EN and FR NER on both CPU and GPU.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants