Skip to content

add embeddinggemma2 results - #793

Merged
KennethEnevoldsen merged 2 commits into
mainfrom
embeddinggemma-2-results
Oct 19, 2026
Merged

add embeddinggemma2 results#793
KennethEnevoldsen merged 2 commits into
mainfrom
embeddinggemma-2-results

Conversation

@KennethEnevoldsen

@KennethEnevoldsen KennethEnevoldsen commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

closes embeddings-benchmark/mteb#5700

Checklist

  • My model has a model sheet, report, or similar
  • My model has a reference implementation in mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here
    • No, but there is an existing PR ___
  • The results submitted are obtained using the reference implementation
  • My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
  • I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

@github-actions

github-actions Bot commented Oct 19, 2026

Copy link
Copy Markdown

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: google/embeddinggemma-2

Results for google/embeddinggemma-2

task_name google/embeddinggemma-2 google/gemini-embedding-001 intfloat/multilingual-e5-large Max result Model with max result In Training Data
AILAStatutes .253 .488 .208 .955 litillabs/octen-law-8b-v1 False
AROCocoOrder .301 nan nan .750 LCO-Embedding/LCO-Embedding-Omni-7B False
AROFlickrOrder .424 nan nan .718 BidirLM/BidirLM-Omni-2.5B-Embedding False
AROVisualAttribution .780 nan nan .769 Salesforce/blip-itm-base-coco False
AROVisualRelation .569 nan nan .592 royokong/e5-v False
AfriSentiClassification .457 .536 .459 .569 tencent/KaLM-Embedding-Gemma3-12B-2511 False
AlloProfClusteringS2S.v2 .508 .564 .333 .611 microsoft/harrier-oss-v1-27b False
AlloprofReranking .769 .818 .694 .854 Octen/Octen-Embedding-8B False
AmazonCounterfactualClassification .858 .882 .697 .970 GeoGPT-Research-Project/GeoEmbedding False
AppsRetrieval .893 .938 .326 .986 google/gemini-embedding-2-preview False
ArXivHierarchicalClusteringP2P .635 .649 .557 .687 NovaSearch/jasper_en_vision_language_v1 False
ArXivHierarchicalClusteringS2S .605 .638 .537 .666 ai-sage/Giga-Embeddings-instruct-3B-0826 False
ArguAna .598 .864 .544 .898 mongodb/voyage-3-m-exp False
ArmenianParaphrasePC .935 .969 .949 .970 tencent/KaLM-Embedding-Gemma3-12B-2511 False
AskUbuntuDupQuestions .606 .642 .592 .753 IEITYuan/Yuan-embedding-2.0-en False
BIOSSES .824 .890 .846 .969 Gameselo/STS-multilingual-mpnet-base-v2 False
BLINKIT2IMultiChoice .709 nan nan .794 jinaai/jina-embeddings-v5-omni-small False
BUCC.v2 .985 .990 .988 1.000 google/gemini-embedding-2-preview False
Banking77Classification .923 .943 .749 .943 google/gemini-embedding-001 False
BeijingOpera .809 nan nan .974 Qwen/Qwen2-Audio-7B False
BelebeleRetrieval .719 .907 .779 .985 sionic-ai/comsat-embed-ko-8b-preview False
BibleNLPBitextMining .123 .207 .166 .990 deepvk/USER-bge-m3 False
BigPatentClustering.v2 .338 .381 .315 .454 microsoft/harrier-oss-v1-27b False
BiorxivClusteringP2P.v2 .471 .539 .372 .842 codefuse-ai/F2LLM-4B False
BirdCLEF .152 nan nan .452 MIT/ast-finetuned-audioset-10-10-0.4593 False
BornholmBitextMining .495 .517 .442 .917 google/gemini-embedding-2-preview False
BrazilianToxicTweetsClassification .309 .280 .177 .381 microsoft/harrier-oss-v1-27b False
BulgarianStoreReviewSentimentClassfication .683 .781 .638 .816 microsoft/harrier-oss-v1-27b False
CEDRClassification .503 .574 .448 .738 ai-sage/Giga-Embeddings-instruct-10B-A1.8B-0826 False
CIFAR100ZeroShot .755 nan nan .911 QuanSun/EVA02-CLIP-bigE-14-plus False
CIRRIT2IRetrieval .220 nan nan .350 mongodb/voyage-multimodal-3 False
CLSClusteringP2P.v2 .380 .427 .404 .757 Qwen/Qwen3-Embedding-8B False
COIRCodeSearchNetRetrieval .869 .811 nan .910 codefuse-ai/F2LLM-v2-14B False
CQADupstackGamingRetrieval .568 .707 .587 .816 IEITYuan/Yuan-embedding-2.0-en False
CQADupstackUnixRetrieval .409 .537 .399 .720 mongodb/voyage-3-m-exp False
CREMADPairClassification .525 nan nan .689 Qwen/Qwen2-Audio-7B False
CREMA_D .197 nan nan .740 Qwen/Qwen2-Audio-7B False
CREMA_DClustering .005 nan nan .324 Qwen/Qwen2-Audio-7B False
CSFDSKMovieReviewSentimentClassification .350 .494 .348 .679 microsoft/harrier-oss-v1-27b False
CTKFactsNLI .788 .876 .798 .899 omarelshehy/arabic-english-sts-matryoshka False
CUB200I2IRetrieval .787 nan nan .862 facebook/dinov2-giant False
CVBenchCount .369 nan nan .698 LCO-Embedding/LCO-Embedding-Omni-7B False
CVBenchDepth .453 nan nan .902 dnotitia/DNA-VL-STEER-2B False
CVBenchDistance .537 nan nan .710 dnotitia/DNA-VL-STEER-2B False
CVBenchRelation .538 nan nan .862 LCO-Embedding/LCO-Embedding-Omni-7B False
CataloniaTweetClassification .492 .545 .504 .798 microsoft/harrier-oss-v1-27b False
ClimateFEVERHardNegatives .230 .311 .260 .591 IEITYuan/Yuan-embedding-2.0-en False
ClothoT2ARetrieval .107 nan nan .587 microsoft/msclap-2022 False
CodeEditSearchRetrieval .805 .816 .504 .921 Bytedance/Seed1.6-embedding-1215 False
CodeFeedbackMT .882 .563 .428 .962 ai-sage/Giga-Embeddings-instruct-10B-A1.8B-0826 False
CodeFeedbackST .821 .853 .743 .908 codefuse-ai/F2LLM-v2-14B False
CodeSearchNetCCRetrieval .966 .847 .778 .979 codefuse-ai/C2LLM-7B False
CodeSearchNetRetrieval .908 .913 .841 .940 mongodb/voyage-code-3 False
CodeTransOceanContest .837 .895 .740 .951 codefuse-ai/F2LLM-v2-14B False
CodeTransOceanDL .366 .315 .313 .442 jinaai/jina-embeddings-v4 False
CommonLanguageAgeDetection .175 nan nan .205 laion/larger_clap_general False
CommonVoiceMini21T2ARetrieval .933 nan nan .831 jinaai/jina-embeddings-v5-omni-small False
Core17InstructionRetrieval .019 .077 -.016 .146 nvidia/llama-embed-nemotron-8b False
CosQA .513 .502 .348 .522 google/text-embedding-005 False
Country211 .208 nan nan .325 google/siglip-so400m-patch14-384 False
Country211ZeroShot .194 nan nan .428 encord-team/ebind-full False
CovidRetrieval .770 .791 .756 .961 TencentBAC/Conan-embedding-v2 False
CyrillicTurkicLangClassification .685 .953 .409 .994 microsoft/harrier-oss-v1-27b False
CzechProductReviewSentimentClassification .584 .682 .571 .767 Bytedance/Seed1.6-embedding-1215 False
DBpediaClassification .918 .948 .883 .993 Qwen/Qwen3-Embedding-4B False
DTD .756 nan nan .811 QuanSun/EVA02-CLIP-bigE-14-plus False
DalajClassification .502 .505 .500 .659 microsoft/harrier-oss-v1-27b False
DiaBlaBitextMining .818 .872 .848 .996 google/gemini-embedding-2-preview False
EstonianValenceClassification .387 .535 .429 .676 microsoft/harrier-oss-v1-27b False
EuroSAT .836 nan nan .939 QuanSun/EVA02-CLIP-bigE-14-plus False
FER2013ZeroShot .620 nan nan .627 BidirLM/BidirLM-Omni-2.5B-Embedding False
FEVERHardNegatives .817 .890 .838 .945 ByteDance-Seed/Seed1.5-Embedding True
FGVCAircraftZeroShot .397 nan nan .692 LCO-Embedding/LCO-Embedding-Omni-7B False
FSD2019Kaggle .320 nan nan .780 EximiusLabs/fusion-embedding-2-2b-preview False
FaroeseSTS .716 .861 .724 .974 Gameselo/STS-multilingual-mpnet-base-v2 False
Fashion200kI2TRetrieval .152 nan nan .241 google/siglip-large-patch16-384 False
FiQA2018 .323 .618 .438 .821 ai-sage/Giga-Embeddings-instruct False
FilipinoShopeeReviewsClassification .372 .485 .353 .528 microsoft/harrier-oss-v1-27b False
FinParaSTS .178 .286 .249 .350 jinaai/jina-embeddings-v5-text-nano False
FinancialPhrasebankClassification .822 .886 .839 1.000 ai-sage/Giga-Embeddings-instruct-3B-0826 False
FleursT2ARetrieval .979 nan nan .735 BidirLM/BidirLM-Omni-2.5B-Embedding False
FloresBitextMining .549 .837 .811 .982 google/gemini-embedding-2-preview False
Food101ZeroShot .794 nan nan .959 encord-team/ebind-full False
GTSRB .798 nan nan .890 QuanSun/EVA02-CLIP-bigE-14-plus False
GTZANAudioReranking .665 nan nan .867 Haon-Chen/e5-omni-7B False
GTZANGenre .576 nan nan .931 Qwen/Qwen2-Audio-7B False
GermanSTSBenchmark .832 .881 .841 .954 Gameselo/STS-multilingual-mpnet-base-v2 False
GigaSpeechT2ARetrieval .820 nan nan .833 LCO-Embedding/LCO-Embedding-Omni-7B False
GreekLegalCodeClassification .263 .438 .371 .805 Bytedance/Seed1.6-embedding-1215 False
GujaratiNewsClassification .874 .920 .767 .934 Bytedance/Seed1.6-embedding-1215 False
HALClusteringS2S.v2 .281 .320 .226 .330 BidirLM/BidirLM-Omni-2.5B-Embedding False
HagridRetrieval .988 .993 .989 .993 google/gemini-embedding-001 False
HatefulMemesI2TRetrieval .786 nan nan .842 google/siglip-so400m-patch14-384 False
HotpotQAHardNegatives .781 .870 .706 .870 google/gemini-embedding-001 True
IEMOCAPGender .664 nan nan .938 EximiusLabs/fusion-embedding-2-2b-preview False
IN22GenBitextMining .736 .937 .767 .995 google/gemini-embedding-2-preview False
ImageCoDe .122 nan nan .152 laion/CLIP-ViT-g-14-laion2B-s34B-b88K False
ImageNetDog15Clustering .886 nan nan .926 facebook/dinov2-giant False
ImdbClassification .910 .950 .887 .974 Qwen/Qwen3-Embedding-8B False
IndicCrosslingualSTS .368 .629 .439 .848 Gameselo/STS-multilingual-mpnet-base-v2 False
IndicGenBenchFloresBitextMining .867 .968 .887 .988 Sailesh97/Hinvec False
IndicLangClassification .855 .877 .203 .993 Bytedance/Seed1.6-embedding-1215 False
IndonesianIdClickbaitClassification .645 .670 .612 .756 nvidia/llama-embed-nemotron-8b False
InfoSeekIT2TRetrieval .135 nan nan .341 Haon-Chen/e5-omni-7B False
IsiZuluNewsClassification .242 .405 .338 .426 microsoft/harrier-oss-v1-27b False
ItaCaseholdClassification .689 .733 .668 .944 bigscience/sgpt-bloom-7b1-msmarco False
JSICK .842 .850 .798 .896 Octen/Octen-Embedding-8B False
JamAltArtistA2ARetrieval .554 nan nan .969 laion/larger_clap_music_and_speech False
JamAltLyricA2TRetrieval .615 nan nan .760 LCO-Embedding/LCO-Embedding-Omni-3B False
KorHateSpeechMLClassification .120 .177 .105 .762 Bytedance/Seed1.6-embedding-1215 False
KorSarcasmClassification .578 .605 .568 .819 ICT-TIME-and-Querit/BOOM_4B_v1 False
KurdishSentimentClassification .642 .864 .771 .940 Bytedance/Seed1.6-embedding-1215 False
LEMBPasskeyRetrieval .720 .385 .383 1.000 DataScience-UIBK/uibk-embedding-v1 False
LegalBenchCorporateLobbying .934 .960 .897 .970 mongodb/voyage-3-large False
MACST2ARetrieval .066 nan nan .417 microsoft/msclap-2022 False
MIRACLRetrievalHardNegatives .661 .704 .592 .731 geevec-ai/geevec-embeddings-1.0 True
MInDS14 .877 nan nan .911 BidirLM/BidirLM-Omni-2.5B-Embedding False
MLQARetrieval .753 .842 .757 .848 geevec-ai/geevec-embeddings-1.0 False
MTOPDomainClassification .954 .980 .902 1.000 mongodb/voyage-3-m-exp False
MacedonianTweetSentimentClassification .580 .718 .619 .755 Qwen/Qwen3-Embedding-4B False
MalteseNewsClassification .287 .374 .239 .694 Bytedance/Seed1.6-embedding-1215 False
MasakhaNEWSClassification .754 .835 .770 .901 Bytedance/Seed1.6-embedding-1215 False
MasakhaNEWSClusteringS2S .460 .574 .399 .737 Bytedance/Seed1.6-embedding-1215 False
MassiveIntentClassification .615 .819 .602 .919 mongodb/voyage-3-m-exp False
MassiveScenarioClassification .716 .873 .651 .993 mongodb/voyage-3-m-exp False
MedrxivClusteringP2P.v2 .426 .472 .343 .720 codefuse-ai/F2LLM-4B False
MedrxivClusteringS2S.v2 .396 .450 .315 .702 codefuse-ai/F2LLM-4B False
MindSmallReranking .314 .329 .302 .344 Kingsoft-LLM/QZhou-Embedding False
MridinghamTonic .314 nan nan .612 Qwen/Qwen2-Audio-7B False
MultiEURLEXMultilabelClassification .042 .053 .052 .097 Bytedance/Seed1.6-embedding-1215 False
MultiHateClassification .573 .725 .636 .862 microsoft/harrier-oss-v1-27b False
NIGHTSI2IRetrieval .262 nan nan .266 dnotitia/DNA-VL-STEER-2B False
NMSQAPairClassification .981 nan nan .976 LCO-Embedding/LCO-Embedding-Omni-7B False
NTREXBitextMining .740 .936 .914 .988 google/gemini-embedding-2-preview False
NepaliNewsClassification .955 .981 .885 .995 jinaai/jina-embeddings-v5-omni-small False
News21InstructionRetrieval -.009 .103 -.001 .114 google/embeddinggemma-300m False
NollySentiBitextMining .345 .687 .724 .838 microsoft/harrier-oss-v1-27b False
NordicLangClassification .659 .860 .802 .958 microsoft/harrier-oss-v1-27b False
NorwegianCourtsBitextMining .930 .934 .940 1.000 google/gemini-embedding-2-preview False
NusaParagraphEmotionClassification .451 .564 .417 .837 Bytedance/Seed1.6-embedding-1215 False
NusaTranslationBitextMining .713 .775 .672 .932 google/gemini-embedding-2-preview False
NusaX-senti .689 .803 .705 .848 Bytedance/Seed1.6-embedding-1215 False
NusaXBitextMining .672 .825 .727 .906 Bytedance/Seed1.6-embedding-1215 False
OVENIT2TRetrieval .160 nan nan .207 LCO-Embedding/LCO-Embedding-Omni-7B False
OdiaNewsClassification .729 .918 .800 .978 microsoft/harrier-oss-v1-27b False
OpusparcusPC .924 .966 .945 .970 microsoft/harrier-oss-v1-27b False
OxfordPets .928 nan nan .951 google/siglip-large-patch16-384 False
OxfordPetsZeroShot .877 nan nan .968 google/siglip-large-patch16-384 False
PAC .671 .717 .703 .881 Bytedance/Seed1.6-embedding-1215 False
PatchCamelyon .693 nan nan .795 encord-team/ebind-full False
PawsXPairClassification .546 .600 .551 .756 Bytedance/Seed1.6-embedding-1215 False
PlscClusteringP2P.v2 .713 .743 .716 .758 ai-sage/Giga-Embeddings-instruct-3B-0826 False
PoemSentimentClassification .565 .597 .507 .892 ai-sage/Giga-Embeddings-instruct-10B-A1.8B-0826 False
PolEmo2.0-OUT .582 .775 .365 .806 microsoft/harrier-oss-v1-27b False
PpcPC .885 .955 .912 .958 microsoft/harrier-oss-v1-27b False
PunjabiNewsClassification .810 .826 .807 .888 Bytedance/Seed1.6-embedding-1215 False
RESISC45 .883 nan nan .928 QuanSun/EVA02-CLIP-bigE-14-plus False
RP2kI2IRetrieval .737 nan nan .728 nyu-visionx/moco-v3-vit-b False
RTE3 .867 .895 .875 .917 Bytedance/Seed1.6-embedding-1215 False
RavdessZeroshot .140 nan nan .406 Haon-Chen/e5-omni-7B False
Robust04InstructionRetrieval -.029 -.024 -.075 .124 Qwen/Qwen3-Embedding-4B False
RomaniBibleClustering .392 .432 .409 .466 microsoft/harrier-oss-v1-27b False
RuBQReranking .707 .738 .756 .809 ai-sage/Giga-Embeddings-instruct-10B-A1.8B-0826 False
SCIDOCS .182 .252 .174 .599 IEITYuan/Yuan-embedding-2.0-en False
SIB200ClusteringS2S .245 .417 .212 .873 LingoIITGN/FlowVec-v1 False
SIBFLEURS .547 nan nan .475 BidirLM/BidirLM-Omni-2.5B-Embedding False
SICK-R .822 .827 .802 .947 Gameselo/STS-multilingual-mpnet-base-v2 False
STS12 .778 .815 .800 .955 Gameselo/STS-multilingual-mpnet-base-v2 False
STS13 .865 .899 .816 .978 Gameselo/STS-multilingual-mpnet-base-v2 False
STS13VisualSTS .773 nan nan .843 LCO-Embedding/LCO-Embedding-Omni-3B False
STS14 .827 .854 .777 .975 Gameselo/STS-multilingual-mpnet-base-v2 False
STS15 .885 .904 .893 .981 Gameselo/STS-multilingual-mpnet-base-v2 False
STS15VisualSTS .857 nan nan .882 LCO-Embedding/LCO-Embedding-Omni-7B False
STS17 .858 .886 .821 .957 jcorners/ingot-8b-r3 False
STS17MultilingualVisualSTS .779 nan nan .826 LCO-Embedding/LCO-Embedding-Omni-7B False
STS22.v2 .697 .717 .643 .799 albertobarnabo/bge-m3-italian False
STSB .814 .855 .824 .920 Gameselo/STS-multilingual-mpnet-base-v2 False
STSBenchmark .875 .891 .873 .950 Kingsoft-LLM/QZhou-Embedding False
STSBenchmarkMultilingualVisualSTS .775 nan nan .826 LCO-Embedding/LCO-Embedding-Omni-7B False
STSES .812 .817 .802 .823 google/embeddinggemma-300m False
SUN397 .730 nan nan .805 QuanSun/EVA02-CLIP-bigE-14-plus False
ScalaClassification .501 .518 .511 .911 microsoft/harrier-oss-v1-27b False
SemRel24STS .654 .731 .645 .811 VPLabs/SearchMap_Preview False
SentimentAnalysisHindi .682 .761 .642 .807 microsoft/harrier-oss-v1-27b False
SinhalaNewsClassification .696 .823 nan .859 microsoft/harrier-oss-v1-27b False
SiswatiNewsClassification .477 .624 .554 .784 Lajavaness/bilingual-embedding-small False
SlovakMovieReviewSentimentClassification .784 .903 .744 .962 microsoft/harrier-oss-v1-27b False
SpartQA .118 .103 .056 .909 geevec-ai/geevec-embeddings-1.0 False
SpeechCommandsZeroshotv0.02 .969 nan nan .974 jinaai/jina-embeddings-v5-omni-small False
SpokenSQuADT2ARetrieval .693 nan nan .767 Haon-Chen/e5-omni-7B False
SprintDuplicateQuestions .965 .969 .931 1.000 KiteFishAI/Nano-Em1-0.6B-v2.1 False
StackExchangeClustering.v2 .641 .921 .464 .921 google/gemini-embedding-001 False
StackExchangeClusteringP2P.v2 .478 .509 .385 .551 Kingsoft-LLM/QZhou-Embedding False
StackOverflowQA .873 .967 .889 .975 codefuse-ai/F2LLM-v2-14B False
StanfordCarsZeroShot .883 nan nan .946 google/siglip-so400m-patch14-384 False
StatcanDialogueDatasetRetrieval .420 .511 .106 .747 geevec-ai/geevec-embeddings-1.0 False
SummEvalSummarization.v2 .351 .383 .314 .397 LCO-Embedding/LCO-Embedding-Omni-7B False
SwahiliNewsClassification .609 .660 .599 .707 codefuse-ai/F2LLM-v2-4B False
SwednClusteringP2P .427 .458 .369 .621 Qwen/Qwen3-Embedding-4B False
SwissJudgementClassification .551 .579 .536 .796 microsoft/harrier-oss-v1-27b False
SyntheticText2SQL .611 .700 .531 .787 Qwen/Qwen3-Embedding-8B False
T2Reranking .662 .680 .663 .732 tencent/Youtu-Embedding False
TERRa .571 .639 .584 .796 ai-sage/Giga-Embeddings-instruct False
TRECCOVID .790 .863 .712 .983 IEITYuan/Yuan-embedding-2.0-en False
Tatoeba .542 .820 .779 .995 google/gemini-embedding-2-preview False
TempReasonL1 .013 .030 .011 .418 microsoft/harrier-oss-v1-27b False
TinyImageNetClustering .688 nan nan .836 QuanSun/EVA02-CLIP-bigE-14 False
Touche2020Retrieval.v3 .434 .524 .496 .762 jcorners/ingot-8b-r3 False
ToxicConversationsClassification .910 .887 .660 .976 mongodb/voyage-3-m-exp False
TswanaNewsClassification .314 .534 .454 .642 Bytedance/Seed1.6-embedding-1215 False
TweetSentimentExtractionClassification .664 .699 .628 .882 mongodb/voyage-3-m-exp False
TweetTopicSingleClassification .730 .711 .653 .879 ai-sage/Giga-Embeddings-instruct-3B-0826 False
TwentyNewsgroupsClustering.v2 .485 .574 .392 .876 GeoGPT-Research-Project/GeoEmbedding False
TwitterHjerneRetrieval .675 .980 .352 .980 google/gemini-embedding-001 False
TwitterSemEval2015 .758 .792 .753 .901 jcorners/ingot-8b-r3 False
TwitterURLCorpus .868 .870 .858 .957 TencentBAC/Conan-embedding-v2 False
UrbanSound8KT2ARetrieval .005 nan nan .010 EximiusLabs/fusion-embedding-2-2b-preview False
VQA2IT2TRetrieval .165 nan nan .231 Haon-Chen/e5-omni-3B False
VehicleSoundClustering .075 nan nan .134 MIT/ast-finetuned-audioset-10-10-0.4593 False
VidoreDocVQARetrieval .396 nan nan .687 webAI-Official/webAI-ColVec1-9b False
VidoreInfoVQARetrieval .773 nan nan .952 webAI-Official/webAI-ColVec1-9b False
VidoreShiftProjectRetrieval .646 nan nan .947 DataScience-UIBK/Argus-Colqwen3.5-4b-v0 False
VidoreSyntheticDocQAAIRetrieval .878 nan nan 1.000 nvidia/nemotron-colembed-vl-8b-v2 False
VidoreTabfquadRetrieval .915 nan nan .981 nvidia/nemotron-colembed-vl-4b-v2 False
VidoreTatdqaRetrieval .420 nan nan .857 DataScience-UIBK/Argus-Colqwen3.5-9b-v0-bf16 False
VisualNewsI2TRetrieval .169 nan nan .464 QuanSun/EVA02-CLIP-bigE-14-plus False
VisualSTS-b-Multilingual .756 nan nan .815 LCO-Embedding/LCO-Embedding-Omni-7B False
VisualSTS17Multilingual .779 nan nan .833 LCO-Embedding/LCO-Embedding-Omni-7B False
VoxCelebSA .450 nan nan .495 BidirLM/BidirLM-Omni-2.5B-Embedding False
VoxPopuliAccentPairClassification .517 nan nan .554 openai/whisper-medium False
VoxPopuliGenderClustering .000 nan nan .607 encord-team/ebind-full False
VoxPopuliLanguageID .996 nan nan .994 speechbrain/m-ctc-t-large False
VoyageMMarcoReranking .606 .667 .682 .837 codefuse-ai/F2LLM-v2-14B False
WITT2IRetrieval .642 nan nan .636 Haon-Chen/e5-omni-7B False
WebLINXCandidatesReranking .121 .110 .078 .225 codefuse-ai/F2LLM-v2-8B False
WebQAT2ITRetrieval .640 nan nan .656 mongodb/voyage-multimodal-3 False
WikiCitiesClustering .819 .916 .755 .950 microsoft/harrier-oss-v1-27b False
WikiClusteringP2P.v2 .271 .282 .256 .332 microsoft/harrier-oss-v1-27b False
WikipediaRerankingMultilingual .874 .922 .891 .931 jinaai/jina-reranker-v3 False
WikipediaRetrievalMultilingual .895 .942 .904 .942 google/gemini-embedding-001 False
WinoGrande .547 .605 .550 .931 microsoft/harrier-oss-v1-27b False
Winoground .120 nan nan .150 LCO-Embedding/LCO-Embedding-Omni-7B False
XM3600T2IRetrieval .650 nan nan .709 dnotitia/DNA-VL-STEER-2B False
XNLI .760 .853 .751 .929 Bytedance/Seed1.6-embedding-1215 False
indonli .548 .607 .517 .672 Bytedance/Seed1.6-embedding-1215 False
Average .594 .686 .581 .772 nan -

Model have high performance on these tasks: VoxPopuliLanguageID,NMSQAPairClassification,CommonVoiceMini21T2ARetrieval,AROVisualAttribution,FleursT2ARetrieval,RP2kI2IRetrieval,WITT2IRetrieval,SIBFLEURS

@KennethEnevoldsen
KennethEnevoldsen enabled auto-merge (squash) October 19, 2026 08:49
@KennethEnevoldsen
KennethEnevoldsen merged commit 54f0815 into main Oct 19, 2026
4 checks passed
@KennethEnevoldsen
KennethEnevoldsen deleted the embeddinggemma-2-results branch October 19, 2026 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Samoed Samoed Samoed approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Evaluate model: google/embeddinggemma-2

2 participants

@KennethEnevoldsen @Samoed