chore(deps): update backend dependencies (major)

[renovate]

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Change	Age	Confidence
Django (changelog)	==5.2.17 → ==6.1
django-filter (changelog)	==25.2 → ==26.1
gevent (changelog)	==25.9.1 → ==26.8.0
gunicorn (changelog)	==25.3.0 → ==26.1.0
insightface	==0.7.3 → ==1.0.1
sentence_transformers	==5.7.0 → ==6.0.0
setuptools (changelog)	==82.0.1 → ==84.0.0

---

Release Notes

django/django (Django)

v6.1

Compare Source

v6.0.8

Compare Source

v6.0.7

Compare Source

v6.0.6

Compare Source

v6.0.5

Compare Source

v6.0.4

Compare Source

v6.0.3

Compare Source

v6.0.2

Compare Source

v6.0.1

Compare Source

v6.0

Compare Source

carltongibson/django-filter (django-filter)

v26.1

Compare Source

• Added testing for Django 6.1.

  DRF integration will require the upcoming DRF v3.18.
  See encode/django-rest-framework#9978

• Added testing against Python 3.14.

benoitc/gunicorn (gunicorn)

v26.1.0: gunicorn 26.1.0

Compare Source

New Features

• Glob patterns in reload_extra_files: entries containing *, ? or [
  are treated as patterns, so ui/*/config.json watches every view's config
  without listing them one by one. Patterns are re-expanded on every reload
  check rather than once at startup, so a file created later starts being
  watched without restarting gunicorn, and ** recurses. A pattern matching
  nothing warns instead of failing, since with live expansion it may match later
  (#​1643,
  #​3662).

Security

• Dependency floors raised past known advisories: every declared floor was
  checked against the advisory database. tornado, h2, setuptools and
  pymdown-extensions permitted vulnerable versions and now require the first
  clean release; pytest and httpx were unpinned and now carry floors. The
  tornado example pinned tornado<6, which was both the source of several
  advisories and older than the >=6.5.0 the tornado worker needs, so the
  example could not run as pinned.

Bug Fixes

• SIGHUP did not reload the logger configuration: Arbiter.reload()
  re-read the configuration file but kept using the logger built at startup,
  calling only reopen_files() on its existing handlers. Changes to
  logconfig, logconfig_dict, logconfig_json and loglevel were ignored
  until a full restart, which in containers meant replacing the pod. The
  existing logger now re-runs its setup on reload, so new handlers, formats
  and levels take effect while the process identity and its listeners are
  preserved, and re-running the setup no longer stacks duplicate syslog
  handlers. An invalid log configuration on reload is not fatal either: the
  error is reported on stderr, the previous working configuration is restored
  and the master keeps running with it
  (#​3353).

• Truncated chunked bodies accepted: RFC 9112 section 7.1.2 ends a chunked
  body with 0 CRLF CRLF, the second CRLF being the mandatory empty trailer
  section. ChunkedReader.parse_chunk_size() swallowed the NoMoreData raised
  while scanning for it, so a body cut short right after the last chunk line was
  treated as complete instead of rejected. It now raises
  ChunkMissingTerminator
  (#​3382,
  #​3685).

• --spew crashed on dynamically generated code: the trace hook indexed the
  2-tuple returned by inspect.getsourcelines() by line number rather than
  indexing the list of lines, so a frame with no __file__ raised
  AttributeError: 'int' object has no attribute 'rstrip' on line 1 and
  IndexError beyond it. The tuple is now unpacked and offset by the source's
  starting line (#​3344,
  #​3495).

• Duplicate Host and Content-Type headers accepted: RFC 9110 section 5.3
  allows only one of each, and a repeat cannot be merged into a list, so the
  message means different things to gunicorn and to anything downstream. Both
  are now rejected with InvalidHeader. The check lives in the policy hook
  shared by both parsers, so the pure-Python and fast parsers agree. Duplicate
  Content-Length was already rejected and is unchanged
  (#​3366,
  #​3548).

• Non-worker children reported as failed workers: reap_workers() reaps
  every child through waitpid(-1), including processes the kernel reparented
  onto gunicorn when it runs as PID 1 in a container, but it logged the exit
  status before checking whether the pid was ever a worker. An unrelated process
  produced Worker (pid:N) exited with code M and triggered alerts. More
  seriously, such a process exiting with code 3 or 4 raised HaltServer and shut
  the server down. Ownership is now established first: the dirty arbiter is
  reported as itself, unknown children are reaped silently at debug level, and
  only real workers can halt the server
  (#​3220,
  #​3566).

• Dirty arbiter exits were invisible on SIGCHLD: handle_chld() called
  reap_workers() first, whose waitpid(-1) claimed the dirty arbiter before
  reap_dirty_arbiter() could identify it, so the latter always hit ECHILD and
  its reporting never ran. The dirty arbiter is now reaped first, and
  reap_workers() recognises it if it exits mid-loop.

• Dirty arbiter returned stale responses after a worker timeout: when a
  request reached dirty_timeout the arbiter answered the client with a timeout
  error but kept the worker connection open. The worker's late response was then
  the first message waiting on that socket, so the next request routed to the
  same worker received the previous request's result, and every request after it
  stayed one response behind. The connection is now closed on timeout, so the
  late answer is discarded with it
  (#​3626).

• ASGI connection count leaked on server-initiated close: nr_conns was
  only decremented in connection_lost(), behind a guard keyed on the same
  flag _close_transport() sets first. Every close the server started (a
  Connection: close response, a keepalive timeout, an error abort) leaked one
  count, so ASGIWorker._shutdown() ran the full graceful_timeout and warned
  about connections that were already gone. The guard now uses its own flag, so
  the decrement and the rest of the cleanup run exactly once whichever side
  closes first (#​3661).

• Inotify reloader on cwd-relative extra files: reload_extra_files entries
  with no directory part (for example .env) produced an empty dirname, and
  watching it raised InotifyError with ENOENT. The current directory is now
  watched as . (#​3377,
  #​3667).

• StatsD zero-valued metrics: gauges, counters, histograms and timers
  reporting 0 were silently dropped because the value was tested for
  truthiness. Only None is skipped now
  (#​3676).

• Spurious no-body warning from sendfile(): a HEAD, 204 or 304 response
  served through sendfile() warned about dropped body bytes even when the
  file was empty and nothing was dropped. It now warns only when there are
  bytes to drop, matching write()
  (#​3684).

• Bare except in the gevent websocket example: narrowed to
  except Exception (#​3683).

• ASGI receive() cancellation: Let asyncio.CancelledError propagate
  from BodyReceiver instead of swallowing it and returning
  http.disconnect. Frameworks that cancel their disconnect listener after
  the response completes (Django) no longer see the cancel masked, so
  request_finished fires and close_old_connections() runs. Fixes idle
  database connections leaking since 25.1.0
  (#​3627,
  #​3654).

• Control socket leak on SIGHUP reload: The control thread is now marked
  ready once its loop and server are live, and the stop paths wait on that
  readiness before scheduling shutdown. Reloads no longer leak one thread and
  its selector fd plus unix socket per worker, which eventually raised
  "too many open files"
  (#​3648).

• WSGI body framing on HEAD/1xx/204/304: Mirror the ASGI strip-and-warn
  behavior on the WSGI path. Content-Length is stripped on 1xx/204 per
  RFC 9110 section 6.4.2, body bytes are dropped for no-body responses in
  both write() and sendfile(), and a single warning is logged per request
  (#​3413).

Refactoring

• Pass log arguments to the logger instead of pre-formatting the worker
  termination message in Arbiter.reap_workers()
  (#​3678).

Changes

• packaging is no longer a runtime dependency: it was only ever imported by
  the gevent worker, to compare gevent's version. It moved to the gevent and
  testing extras, so a plain pip install gunicorn pulls in nothing
  (#​3643).

• Fast HTTP Parser: Require gunicorn_h1c >= 0.6.6, which rejects duplicate
  Host and Content-Type headers in the C parser itself. Gunicorn already
  refuses them on both the WSGI and ASGI paths, so this changes nothing that is
  reachable; it moves the rejection to where the bytes are read and lets the
  ASGI corpus exercise those cases against the fast parser directly.

Full changelog: https://gunicorn.org/2026-news/

v26.0.0

Compare Source

Breaking Changes

• Eventlet worker removed: The eventlet worker class has been dropped. Migrate to gevent, gthread, or tornado.

New Features

• ASGI Framework Compatibility Suite: New end-to-end compatibility test harness covering Starlette, FastAPI, Litestar, Quart, Sanic, and BlackSheep. Current grid passes 438/444 tests (98%).
• ASGI Test Suite Expansion: 134 additional ASGI unit tests covering protocol semantics, lifespan, websockets, and chunked framing.

Security

• HTTP/1.1 Request-Target Validation (RFC 9112 sections 3.2.3, 3.2.4):
  • Reject authority-form request-target outside CONNECT
  • Reject asterisk-form request-target outside OPTIONS
  • Reject relative-reference request-targets
• Header Field Hardening (RFC 9110):
  • Reject control characters in header field-value (section 5.5)
  • Reject forbidden trailer field-names (section 6.5.1)
  • Reject Content-Length list form (RFC 9112 section 6.3)
• Request Smuggling Hardening:
  • Tighten keepalive gate and scope finish_body byte cap
  • Keep _body_receiver alive across the keepalive smuggling gate so pipelined requests cannot re-enter a closed body
  • Address parser/protocol findings from a six-point WSGI/ASGI audit
• PROXY Protocol (ASGI): Enforce proxy_allow_ips and tighten v1/v2 parsing in the ASGI callback parser.
• Connection Draining: Drain the connection on close per RFC 9112 section 9.6 to prevent reset-on-close truncation.

Bug Fixes

• Body Framing on HEAD/204/304:
  • Keep Content-Length on HEAD and 304 responses (#​3621)
  • Drop body framing on HEAD/204/304 even when the framework set it
  • Warn once when an ASGI app emits a body for a no-body response
• HTTP/2 ASGI:
  • Fix _handle_stream_ended to set _body_complete in the async HTTP/2 handler so request bodies finalize correctly on stream end
  • Add InvalidChunkExtension mapping and fast-parser support in ASGI tests (#​3565)
• HTTP/1.1 100-Continue: Stop adding Transfer-Encoding: chunked to 100-Continue interim responses.
• WebSocket Close Handshake (RFC 6455):
  • Comply with the close handshake state machine
  • Close the transport after the close handshake completes
  • Fix binary send when the text key is None
• Early Hints: Validate headers in the early_hints callback to match process_headers; pass only the header name to InvalidHeader (#​3588).
• ASGI Framework Fixes:
  • Fix ASGI disconnect handling for Django-style apps
  • Fix Litestar request handling (use raw ASGI receive for body/headers)
  • Fix Litestar HTTP endpoints for compatibility tests
  • Fix Quart headers endpoint to normalize keys to lowercase
  • Fix Quart WebSocket close test app (missing accept())
  • Fix duplicate Transfer-Encoding header for BlackSheep streaming

Refactoring

• Split BodyReceiver._closed into separate transport and body-wait flags for clearer keepalive/EOF semantics.

Changes

• Fast HTTP Parser: Require gunicorn_h1c >= 0.6.5. Drop the last python_only test markers; the C extension is now used wherever available (CPython only; PyPy continues to use the Python parser).
• Test Dependencies: Add h2 and uvloop to the testing extra; remove eventlet.
• Docker Build: Bump GitHub Actions docker/setup-qemu-action, docker/setup-buildx-action, docker/login-action, docker/build-push-action, and docker/metadata-action to current major versions.

Full changelog: benoitc/gunicorn@25.3.0...26.0.0

huggingface/sentence-transformers (sentence_transformers)

v6.0.0: - MultiVectorEncoder for ColBERT & late interaction models, transformers v5, float32 scoring, faster training & encoding

Compare Source

This major release introduces Multi-Vector Embedding models, also known as late interaction or ColBERT-style models, as a fourth model type alongside SentenceTransformer, CrossEncoder, and SparseEncoder. Going forward, you'll be able to use Sentence Transformers for training, inferencing, and interpreting Multi-Vector Embedding models.

It also modernizes the dependency floors to transformers v5, fixes a class of silent scoring bugs caused by half precision, and speeds up both training and encoding.

Install this version with

# Training + Inference
pip install sentence-transformers[train]==6.0.0

# Inference only, use one of:
pip install sentence-transformers==6.0.0
pip install sentence-transformers[onnx-gpu]==6.0.0
pip install sentence-transformers[onnx]==6.0.0
pip install sentence-transformers[openvino]==6.0.0

# Multimodal dependencies (optional):
pip install sentence-transformers[image]==6.0.0
pip install sentence-transformers[audio]==6.0.0
pip install sentence-transformers[video]==6.0.0

# Or combine as needed:
pip install sentence-transformers[train,onnx,image]==6.0.0

[!TIP]
Our Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers blogpost is an excellent place to learn about multi-vector models: loading the various checkpoint formats, encoding and scoring, plugging them into a search stack, running them on page images, and keeping the index affordable.

[!WARNING]
This is a major release with breaking changes. Upgrading from v5.x to v6.0 may require code updates. The changes marked 🚨 below are the ones most likely to affect you, and the Migration Guide has the full list. If you run into issues when upgrading, feel free to open an issue.

MultiVectorEncoder: ColBERT-style late interaction models (#​3794)

Sentence Transformers v6.0 introduces MultiVectorEncoder, for ColBERT-style late interaction retrieval. Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away, which usually means stronger retrieval at the cost of a bigger index. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, with no OCR step in between.

Any PyLate checkpoint and any Stanford-NLP ColBERT checkpoint loads straight into it, and colpali-engine models for visual document retrieval work too, through the same familiar API you already use for dense, sparse, and reranker models.

from sentence_transformers import MultiVectorEncoder

# Download from the 🤗 Hub
model = MultiVectorEncoder("lightonai/LateOn")

query_embeddings = model.encode_query(["Which planet is known as the Red Planet?"])
document_embeddings = model.encode_document([
    "Venus is often called Earth's twin because of its similar size and proximity.",
    "Mars, known for its reddish appearance, is often referred to as the Red Planet.",
    "Jupiter, the largest planet in our solar system, has a prominent red spot.",
    "Saturn, famous for its rings, is sometimes mistaken for the Red Planet.",
])

print(query_embeddings[0].shape)

# (12, 128)

scores = model.similarity(query_embeddings, document_embeddings)
print(scores)

# tensor([[10.7942, 11.1104, 10.9743, 11.0811]])

Mars wins, as it should, though notice how close the four scores are. That is normal for MaxSim: the scores often look similar, but the ranking is still exact. The blogpost explores this in more detail.

Note what you get back: a list of 2D tensors on the model device, one per input, each of shape (num_tokens, embedding_dim). Unlike dense embeddings, you cannot stack these into one rectangular tensor, because every input has its own token count. Pass convert_to_numpy=True for a list of numpy arrays instead, which is what you want once a corpus outgrows device memory.

Multi-vector models are also asymmetric: queries and documents go through different prefixes, different length caps, and different scoring masks. Unlike many dense models, where the two are interchangeable, encode_query and encode_document are required to get correct embeddings.

The MaxSim operator

Scoring uses MaxSim: for each query token, take its highest similarity against any document token, then sum those maxima across the query.

$$\text{MaxSim}(Q, D) = \sum_{Q_i \in Q} \max_{D_j \in D} Q_i \cdot D_j$$

You can read the operator as a soft alignment: every query token points at the one document token that best explains it, and the score is how well the document explains the query overall. The alignment does not have to be lexical, since the token embeddings are contextualized. But when an exact match does matter to you (a product code, a surname, a function name), MaxSim has a token sitting right there to match it, where a single-vector model had to fold it into an average.

Because MaxSim sums over query tokens, its magnitude scales with the query token count, so scores are not comparable across models with different query recipes. If you want scores on a bounded scale, use similarity_fn_name="meanmaxsim", which divides by the query token count and gives you an average cosine similarity in [-1, 1].

Scoring builds a 4-dimensional intermediate of every query token against every document token, which is the largest tensor in the operation. Every scoring function takes a chunk_elements budget that bounds it, defaulting to 100 million elements (roughly 400 MB in float32), so lower it if you run out of memory. Scores and gradients are bit-identical whatever you set it to. maxsim and maxsim_pairwise also take a device, which scores one chunk at a time on that device and moves each result straight back, letting you score a corpus larger than your VRAM on the GPU. Both are reachable through similarity, which forwards any extra keyword arguments to the scoring function:

scores = model.similarity(query_embeddings, document_embeddings, chunk_elements=1_000_000, device="cuda")

When training, pass the budget to the loss instead, with similarity_fct=partial(colbert_scores, chunk_elements=1_000_000). It chunks the document axis, so it composes with the loss-level score_mini_batch_size, which chunks the query axis.

Are they any good?

lightonai/LateOn and lightonai/DenseOn were trained by LightOn on the same data with the same ModernBERT backbone and the same 149M parameters, differing only in whether they keep one vector per token or pool down to one per document. Running both over all 13 NanoBEIR datasets isolates what that choice buys:

NanoBEIR dataset	LateOn (multi-vector, 128d)	DenseOn (dense, 768d)
MSMARCO	0.7194	0.6517
NQ	0.7810	0.7511
HotpotQA	0.9295	0.8802
FEVER	0.9702	0.9612
ClimateFEVER	0.4887	0.4846
DBPedia	0.6836	0.6748
QuoraRetrieval	0.9795	0.9687
Touche2020	0.5938	0.5673
ArguAna	0.5562	0.5660
NFCorpus	0.3949	0.3851
SciFact	0.7978	0.8057
SCIDOCS	0.4469	0.4484
FiQA2018	0.5871	0.6491
Mean	0.6868	0.6764

Late interaction wins on 9 of the 13 datasets and on the mean, by roughly one NDCG point. The four it loses (ArguAna, FiQA2018, SCIDOCS, and SciFact) are the shape of the tradeoff you should expect: a real gain in retrieval quality at the same model size, paid for in index footprint, rather than a universal win on every dataset. The same pair scores 57.22 against 56.20 on the full 15-dataset BEIR, a comparable gap, so the margin is not an artifact of the small benchmark.

That footprint is the real cost. One vector per token instead of one vector per document is a lot more vectors, only partly offset by the smaller dimension. Encoding 4,874 Natural Questions passages with lightonai/LateOn produced 608,414 token vectors, an average of 124.8 per passage:

Representation	Vectors	Dimensions	float32 index
Dense, all-MiniLM-L6-v2	4,874	384	7.5 MB
Dense, gte-modernbert-base	4,874	768	15.0 MB
Multi-vector, LateOn	608,414	128	311.5 MB

That is about 42x the storage of the MiniLM index. Token Pooling cuts the vector count before any of that, real late interaction indexes compress heavily (the same vectors take 88 MB as a fast-plaid PLAID index), and using a multi-vector model as a reranker over a dense first stage avoids building an index at all.

Every checkpoint format loads

Multi-vector checkpoints have been published in several formats over the years. MultiVectorEncoder reads all of them, so loading looks the same whatever the model started life as:

from sentence_transformers import MultiVectorEncoder

# Native Sentence Transformers checkpoints. PyLate builds on the same schema,

# so any PyLate checkpoint loads identically
model = MultiVectorEncoder("lightonai/LateOn")
model = MultiVectorEncoder("mixedbread-ai/mxbai-edge-colbert-v0-17m")
model = MultiVectorEncoder("LiquidAI/LFM2-ColBERT-350M")

# Any Stanford-NLP ColBERT checkpoint, detected via the `HF_ColBERT` architecture

# marker. The inline projection weight and the recipe come from `artifact.metadata`
model = MultiVectorEncoder("colbert-ir/colbertv2.0")
model = MultiVectorEncoder("answerdotai/answerai-colbert-small-v1")

# transformers-native *ForRetrieval ports (ColPali, ColQwen2, ...)
model = MultiVectorEncoder("vidore/colqwen2-v1.0-hf")

# A bare transformer: a fresh random projection is appended, so training is required
model = MultiVectorEncoder("answerdotai/ModernBERT-base")

The recipe knobs that differ per checkpoint (marker prefixes for queries and documents, length caps, whether queries are padded out with [MASK] tokens, and which tokens are skipped when scoring documents) all live in the module configs, so print(model) shows you exactly what you loaded:

model = MultiVectorEncoder("colbert-ir/colbertv2.0")
print(model)
"""
MultiVectorEncoder(
  (0): Transformer({..., 'document_length': 180,
                    'query_expansion': {'strategy': 'fixed', 'attend': False, 'token': None, 'length': 32}})
  (1): Dense({'in_features': 768, 'out_features': 128, 'bias': False, ...})
  (2): MultiVectorMask({'skiplist_words': ['!', '"', '#', ...], 'skiplist_tasks': ['document'], ...})
  (3): Normalize({...})
)
"""

Following the design principle of the rest of the library, this behavior lives in swappable modules rather than in the model class: a Transformer producing contextualized token embeddings, a token-level Dense projecting each of them down, a MultiVectorMask deciding which tokens count during scoring, and a token-level Normalize.

Supported models

These are the checkpoints we test against directly, ranked by retrieval quality. The sentence-transformers tag on the Hub is the list that stays current, and for text retrieval in particular, any PyLate or Stanford-NLP ColBERT checkpoint loads whether or not it carries the tag yet. Where a revision is listed, pass it until the pull request on that repository is merged.

Text retrieval (29 models). NanoBEIR is the mean NDCG@​10 over the 13 NanoBEIR datasets, a fast proxy for English text retrieval quality. A - means the model was not evaluated on it, which is the case for the non-English models.

Model	Parameters	NanoBEIR	Notes
lightonai/LateOn-regularized	149M	0.6897	-
lightonai/LateOn-hpool-regularized	149M	0.6876	-
lightonai/LateOn	149M	0.6868	-
LiquidAI/LFM2.5-ColBERT-350M	353M	0.6864	needs trust_remote_code=True
lightonai/mLateOn	307M	0.6851	-
lightonai/GTE-ModernColBERT-v1	149M	0.6720	-
topk-io/Iso-ModernColBERT	149M	0.6687	-
perplexity-ai/pplx-embed-v1-late-0.6b	596M	0.6662	needs trust_remote_code=True
lightonai/ColBERT-Zero	149M	0.6569	-
answerdotai/answerai-colbert-small-v1	33M	0.6550	-
mixedbread-ai/mxbai-edge-colbert-v0-32m	32M	0.6524	-
LiquidAI/LFM2-ColBERT-350M	353M	0.6441	-
mixedbread-ai/mxbai-edge-colbert-v0-17m	17M	0.6407	-
lightonai/colbertv2.0	110M	0.6201	-
lightonai/LateOn-Code	149M	0.6169	-
lightonai/Agent-ModernColBERT	149M	0.6164	-
lightonai/Reason-ModernColBERT	149M	0.6078	-
colbert-ir/colbertv2.0	110M	0.6053	-
VAGOsolutions/SauerkrautLM-EuroColBERT	212M	0.5982	-
antoinelouis/colbert-xm	853M	0.5915	-
VAGOsolutions/SauerkrautLM-Multi-ModernColBERT	149M	0.5886	-
mixedbread-ai/mxbai-colbert-large-v1	335M	0.5733	revision="refs/pr/4"
lightonai/LateOn-Code-edge	17M	0.5274	-
VAGOsolutions/SauerkrautLM-Multi-Reason-ModernColBERT	149M	0.5267	-
VAGOsolutions/SauerkrautLM-Reason-EuroColBERT	212M	0.4479	-
NeuML/biomedbert-base-colbert	110M	0.4320	-
yjoonjang/colbert-ko-v1	149M	-	-
ytu-ce-cosmos/turkish-colbert	111M	-	-
samheym/GerColBERT	110M	-	-

Visual document retrieval (22 models). These embed page images as documents and text as queries. NanoViDoRe is the equivalent proxy over the ViDoRe benchmark subsamples.

Model	Parameters	NanoViDoRe	Notes
webAI-Official/webAI-ColVec1.1-8b	8.4B	0.6580	needs trust_remote_code=True
webAI-Official/webAI-ColVec1.1-4b	4.5B	0.6520	needs trust_remote_code=True
tencent/EVIE-Preview-4.5B	4.54B	0.6405	-
TomoroAI/tomoro-colqwen3-embed-8b	8.8B	0.6206	needs trust_remote_code=True
TomoroAI/tomoro-colqwen3-embed-4b	4.4B	0.6019	needs trust_remote_code=True
vidore/colqwen2.5-v0.2	3.8B	0.5402	-
vidore/colqwen2.5-v0.1	3.8B	0.5395	-
vidore/colqwen-omni-v0.1	4.4B	0.5309	-
vidore/colpali-v1.3	2.9B	0.4802	-
vidore/colpali-v1.3-hf	2.9B	0.4793	-
vidore/colpali-v1.2	2.9B	0.4691	-
vidore/colqwen2-v1.0	2.2B	0.4685	-
vidore/colqwen2-v0.1	2.2B	0.4526	-
vidore/colpali	2.9B	0.4516	-
vidore/colpali-v1.1	2.9B	0.4314	-
vidore/colsmolvlm-v0.1	2.1B	0.4054	-
vidore/colpali-hard-v1.1	2.9B	0.3949	-
vidore/colSmol-500M	507M	0.3459	-
vidore/colSmol-256M	256M	0.2673	-
ModernVBERT/colmodernvbert	252M	0.2632	-
vidore/colpali-v1.2-hf	2.9B	-	-
vidore/colqwen2-v1.0-hf	2.2B	-	-

Note that NanoBEIR and NanoViDoRe are small benchmarks, so their scores are not a substitute for evaluating on your own data, which is always the right way to pick a model.

Visual, audio, and video document retrieval

Late interaction is the state of the art for visual document retrieval: matching a text query against page images, with charts, tables, and layout intact, and no OCR step. This is what the ColPali family of models does, and those checkpoints run through the same API. Image documents are passed as URLs, local paths, or PIL images:

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("vidore/colqwen2.5-v0.2")

queries = [
    "What is the variable represented on the y-axis of the graph?",
    "Total outlay is maximum in which year?",
]
images = [
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc1.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc2.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc3.jpg",
    "https://huggingface.co/datasets/sentence-transformers/example-documents/resolve/main/doc4.jpg",
]

# A page yields far more vectors than a query: one per image patch
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(images)
print(query_embeddings[0].shape, document_embeddings[0].shape)

# (25, 128) (755, 128)

scores = model.similarity(query_embeddings, document_embeddings)
print(scores)

# tensor([[13.8672, 12.3115, 12.1670, 11.0293],
#         [ 7.2012, 14.7207,  6.9414,  6.9746]])

The code is unchanged from the text case. Underneath, the processor handles the visual prompt and the image patches, and MaxSim scores query text tokens against document image patches. Page images are not the only non-text modality either: text, images, audio, and video are all accepted, and a checkpoint supports whichever of those its processor does, which model.modalities reports.

Because MaxSim is a sum of per-query-token maxima, a ranking decomposes exactly: every point of a document's score belongs to one query token and one document token. The new sentence_transformers.multi_vector_encoder.interpretability module overlays that decomposition onto the page as the standard ColPali heatmap, either aggregated over the query or one map per query token.

Token pooling

If the index footprint worries you, the most effective knob is to store fewer token vectors. HierarchicalTokenPooling implements the token pooling technique from Clavié, Chaffin, and Adams: it clusters each document's token vectors with Ward linkage on cosine similarity and replaces each cluster with its mean, keeping roughly 1 / pool_factor of the tokens.

from sentence_transformers import MultiVectorEncoder
from sentence_transformers.multi_vector_encoder.modules import HierarchicalTokenPooling

model = MultiVectorEncoder("lightonai/LateOn")
pooling = HierarchicalTokenPooling(pool_factor=2)

# 1. Per encode call
document_embeddings = model.encode_document(documents, token_pooling=pooling)

# 2. Standalone, on embeddings you already have saved
pooled = pooling.pool(document_embeddings)

# 3. Baked into the model, so every consumer of the checkpoint gets pooled documents
model.append(HierarchicalTokenPooling(pool_factor=2))
model.save_pretrained("my-pooled-colbert")

By default, pooling applies to documents only, since queries are short and are the side you cannot afford to distort. On the Natural Questions corpus above, the reduction tracks pool_factor closely:

pool_factor	Token vectors	Reduction	float32 index
1 (off)	608,414	1.00x	311.5 MB
2	305,438	1.99x	156.4 MB
3	204,407	2.98x	104.7 MB
4	153,936	3.95x	78.8 MB

The original experiments measured the retrieval cost of this on BEIR and found very little of it: 100.6% of the unpooled performance on average at pool_factor=2, and 99.0% at pool_factor=3. How much it costs on your data is corpus-specific, so measure it with an evaluator before you settle on a factor.

Update Stats

Introducing MultiVectorEncoder has been one of the largest updates to Sentence Transformers, introducing all of the following:

• Code:
  • New Trainer, Training Arguments, Data Collator, Model Card generation + template
  • 4 new modules: MultiVectorMask, BaseTokenPooling, HierarchicalTokenPooling, and LambdaTokenPooling
  • 4 new losses, including cached and knowledge distillation variants
  • 5 new evaluators
  • 4 new similarity functions (maxsim, maxsim_pairwise, mean_maxsim, mean_maxsim_pairwise) plus 6 named ColBERT scorers and 5 XTR scoring entry points
  • 1 new interpretability module for MaxSim heatmaps and similarity maps
  • ONNX and OpenVINO backend support, alongside half precision, Flash Attention, and torch.compile
• Tests:
  • 332 tests for MultiVectorEncoder loading, inference, training, etc., including parity checks against reference scores for 27 pretrained checkpoints
• Docs:
  • All new Training Overview, Loss Overview, and API Reference docs
  • 5 new usage documentation pages
  • 5 new training example documentation pages backed by 14 runnable example scripts
  • A pretrained models page covering 29 text retrieval and 22 visual document retrieval checkpoints
  • Migrating from PyLate and from colpali-engine guides

Resources

• Blogpost:
  • Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers
• Documentation:
  • Multi-Vector Encoder > Usage
  • Multi-Vector Encoder > Pretrained Models
  • Multi-Vector Encoder > Creating Custom Models
  • Multi-Vector Encoder > Speeding up Inference
  • Multi-Vector Encoder > Training Overview
  • Multi-Vector Encoder > API Reference
• Models:
  • Multi-vector models on the Hub

🚨 transformers v5, torch 2.2, and new dependency floors (#​3794)

Sentence Transformers v6.0 requires transformers v5. The v4.x compatibility branches have been removed, which is what allows the new modality handling, chat template support, and unpadding paths to be relied upon rather than feature-detected. The floors that moved:

Dependency	v5.7.0	v6.0.0
transformers	>=4.41.0,<6.0.0	>=5.0.0,<6.0.0
huggingface-hub	>=0.23.0	>=1.3.0,<2.0.0
torch	>=1.11.0	>=2.2
numpy	>=1.20.0	>=1.24.0
scikit-learn	>=0.22.0	>=1.1.0
typing_extensions	>=4.5.0	>=4.10.0
datasets (train)	>=2.0.0	>=2.16.0
accelerate (train)	>=0.20.3	>=1.3.0
optimum-intel[openvino]	unpinned	>=2.0.0

requires-python is unchanged at >=3.10. Note that multi-GPU training with streaming (IterableDataset) datasets needs accelerate>=1.13.0 in practice.

🚨 Higher-precision scoring (#​3892, #​3893, #​3924, #​3926)

Half precision ties too many scores together to rank with. Three separate places where that mattered are now computed in float32.

Reranker scores are the big one. CrossEncoder.predict (and rank) now upcast the logits to float32 before applying the activation function. A sigmoid in bfloat16 saturates and collapses the top candidates onto a handful of tied values, which randomizes their order. Measured on cross-encoder/ettin-reranker-32m-v1 in bfloat16 over three NanoBEIR datasets with 100 candidates per query:

Metric	v6.0.0	v5.7.0
NanoBEIR mean NDCG@​10	0.6795	0.1849
NanoBEIR mean MRR@​10	0.6797	0.3986
Unique scores over 15,040 pairs	710	270

NanoMSMARCO NDCG@​10 alone goes from 0.0965 to 0.7093. If you run a half precision reranker with the default sigmoid activation, its ranking was essentially randomized before this release. Models using activation_fn=nn.Identity() (raw logits) were unaffected, as bf16 logits keep enough relative spacing.

Similarity scores from model.similarity / similarity_pairwise and the cos_sim family are now computed in float32 for float16 and bfloat16 embeddings. With 10,000 realistic cosine scores (mean 0.7, standard deviation 0.05), float32 keeps 9,983 distinct values where float16 keeps 593 and bfloat16 keeps just 93. bfloat16 can represent only 129 distinct values in the whole of [0.5, 1.0).

MaxSim sums over query tokens, reaching magnitudes where the bfloat16 grid is 0.125 wide, so maxsim and maxsim_pairwise accumulate the per-token maxima in float32 and always return float32 scores. The 4-dimensional scoring intermediate stays in the input dtype, so this does not change peak memory.

Note that encode() output dtypes are unchanged. Only the scoring step is upcast. For CrossEncoder.predict, the returned dtype changes only with convert_to_tensor=True or convert_to_numpy=False, as the default numpy output was already float32.

Separately, the multi-vector bf16 benchmarks were re-measured under this float32 accumulation (#​3924). Most of the previously reported bf16 quality drop came from the scoring accumulation rather than from the embeddings: plain bf16 now sits at 99.0% of fp32 retrieval quality (was 95.0%), and bf16 with FlashAttention-2 is indistinguishable from fp32 at 99.96% (was 97.9%).

🚨 Other breaking changes (#​3794, #​3927, #​3935)

• similarity and similarity_pairwise are methods, not properties. Calls like model.similarity(embeddings1, embeddings2) work unchanged, but assigning a custom function to model.similarity is no longer supported: it now silently shadows the method where it previously raised an AttributeError. Set model.similarity_fn_name = "dot" instead, which updates both. Note also that model.similarity.__name__ is now "similarity" rather than the resolved function name, which affected loss get_config_dict() output and generated model cards. The new sentence_transformers.util.similarity_fct_name() resolves it properly and the losses use it.
• A bare list of chat message dictionaries is now one conversation. model.encode([{"role": "user", ...}, {"role": "assistant", ...}]) produces one embedding, where v5.x read it as a batch of two inputs. Wrap each conversation in its own list to encode a batch: model.encode([[msg1], [msg2]]). This applies to SentenceTransformer, SparseEncoder, and MultiVectorEncoder. CrossEncoder is unaffected.
• Custom module classes require trust_remote_code=True (#​3935). Loading a model whose modules.json references a class outside sentence_transformers executes third-party code, and a local directory no longer implies trust. This closes the bypass reported in #​3801 and completes the deprecation cycle announced in v5.6 and v5.7. Unmet, it raises a ValueError naming the class and pointing at the repository or local path to inspect. Trainer checkpoint reloading (load_best_model_at_end, resume_from_checkpoint) keeps working for programmatically built models without the flag.
• quantize_embeddings returns a list of per-input matrices when given a list of 2D arrays, where it previously stacked them into one 3D array. Update callers that indexed the stacked array. An empty list now returns [] instead of raising, and a (0, dim) matrix returns a correctly shaped empty result.
• Multi-process encode(pool=..., precision="int8") now quantizes once after merging the worker results, so the calibration ranges match single-process encoding. Quantized indexes built with v5.x multi-process encoding are not bit-compatible and should be regenerated. Peak memory is higher, because the full float32 matrix is materialized before quantization.
• CrossEncoder.rank returns Python floats ([#​3927](https://redirect.github.com/

✂ Note

PR body was truncated to here.

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.