Summary
TextEmbedding can return vectors that are entirely NaN without raising, warning, or
logging anything. Downstream this is worse than a hard failure: cosine similarity becomes
NaN, ranking silently degenerates to input order, and a vector search appears to work
while returning meaningless results.
I ran into this through jinaai/jina-embeddings-v2-base-de, which is the case already fixed
on main by #623, but the silent-NaN behavior itself looks independent of that model.
Reproduction
from fastembed import TextEmbedding
import numpy as np
m = TextEmbedding(model_name="jinaai/jina-embeddings-v2-base-de")
v = np.array(list(m.embed(["Ein Testsatz."]))[0])
print(v.shape, np.isnan(v).sum()) # (768,) 768 -- no warning, no error
Environment: fastembed 0.8.0 (current PyPI), onnxruntime 1.29.0, providers
['AzureExecutionProvider', 'CPUExecutionProvider'], Python 3.12.12, Windows 11 x86_64.
Every one of the 768 components is NaN. Nothing on stderr, no exception, exit code 0.
Two separate points
1. The fp16 jina-de mapping is fixed on main but not released
#623 ("use original jina de model instead of fp16 due to onnxruntime up...") was merged
2026-04-15 and main now carries model_file="onnx/model.onnx". The newest PyPI release is
0.8.0 from 2026-03-23, so every pip install fastembed still gets the fp16 mapping and, on
onnxruntime >= 1.23, silently produces NaN. Is a release planned? This currently reaches
anyone selecting that model through a downstream package.
As a workaround on 0.8.0, onnx/model_quantized.onnx (int8, 154 MB) works correctly on CPU
and is both smaller and faster than the fp32 file main switched to — in my measurements ~5x
faster than bge-small-en-v1.5 on the same corpus. Possibly worth considering as the
mapping, though I have not compared int8 vs fp32 output quality.
2. The silent part is the more general problem
Independently of which file is mapped, a NaN result is not detectable by callers without
explicitly checking, and nothing in the docs suggests that is necessary. A cheap guard would
turn a silent data-corruption bug into an actionable error, for example checking the first
batch's output once per model load and raising something like
RuntimeError: model '<name>' produced non-finite embeddings with provider '<provider>'.
That would have surfaced #623's root cause immediately at the call site rather than as
unexplained ranking behavior downstream.
I am happy to open a PR for the guard if you think it is worth having and can point me at
where you would want the check to live.
Summary
TextEmbeddingcan return vectors that are entirelyNaNwithout raising, warning, orlogging anything. Downstream this is worse than a hard failure: cosine similarity becomes
NaN, ranking silently degenerates to input order, and a vector search appears to workwhile returning meaningless results.
I ran into this through
jinaai/jina-embeddings-v2-base-de, which is the case already fixedon main by #623, but the silent-NaN behavior itself looks independent of that model.
Reproduction
Environment: fastembed 0.8.0 (current PyPI), onnxruntime 1.29.0, providers
['AzureExecutionProvider', 'CPUExecutionProvider'], Python 3.12.12, Windows 11 x86_64.Every one of the 768 components is
NaN. Nothing on stderr, no exception, exit code 0.Two separate points
1. The fp16 jina-de mapping is fixed on main but not released
#623 ("use original jina de model instead of fp16 due to onnxruntime up...") was merged
2026-04-15 and main now carries
model_file="onnx/model.onnx". The newest PyPI release is0.8.0 from 2026-03-23, so every
pip install fastembedstill gets the fp16 mapping and, ononnxruntime >= 1.23, silently produces NaN. Is a release planned? This currently reaches
anyone selecting that model through a downstream package.
As a workaround on 0.8.0,
onnx/model_quantized.onnx(int8, 154 MB) works correctly on CPUand is both smaller and faster than the fp32 file main switched to — in my measurements ~5x
faster than
bge-small-en-v1.5on the same corpus. Possibly worth considering as themapping, though I have not compared int8 vs fp32 output quality.
2. The silent part is the more general problem
Independently of which file is mapped, a NaN result is not detectable by callers without
explicitly checking, and nothing in the docs suggests that is necessary. A cheap guard would
turn a silent data-corruption bug into an actionable error, for example checking the first
batch's output once per model load and raising something like
RuntimeError: model '<name>' produced non-finite embeddings with provider '<provider>'.That would have surfaced #623's root cause immediately at the call site rather than as
unexplained ranking behavior downstream.
I am happy to open a PR for the guard if you think it is worth having and can point me at
where you would want the check to live.