Skip to content

add results for ColUltraVBERT - #792

Merged
KennethEnevoldsen merged 1 commit into
mainfrom
colultravbert-private-vidore
Oct 19, 2026
Merged

KennethEnevoldsen merged 1 commit into
mainfrom
colultravbert-private-vidore

Conversation

@KennethEnevoldsen

Copy link
Copy Markdown
Contributor

closes embeddings-benchmark/mteb#5694

Checklist

  • My model has a model sheet, report, or similar
  • My model has a reference implementation in mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here
    • No, but there is an existing PR ___
  • The results submitted are obtained using the reference implementation
  • My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
  • I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

@KennethEnevoldsen
KennethEnevoldsen enabled auto-merge (squash) October 19, 2026 08:40
@github-actions

Copy link
Copy Markdown

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: athrael-soju/ColUltraVBERT

Results for athrael-soju/ColUltraVBERT

task_name athrael-soju/ColUltraVBERT Max result Model with max result In Training Data
Vidore3NuclearRetrieval .420 .566 tencent/EVIE-8B False
Vidore3NuclearRetrieval.v2 .440 .551 perplexity-ai/pplx-embed-v2-late-9b False
Vidore3TelecomRetrieval .620 .743 tencent/EVIE-8B False
Vidore3TelecomRetrieval.v2 .644 .733 webAI-Official/webAI-ColVec1.1-8b False
Average .531 .648 nan -

@KennethEnevoldsen
KennethEnevoldsen merged commit 13a4eb0 into main Oct 19, 2026
4 checks passed
@KennethEnevoldsen
KennethEnevoldsen deleted the colultravbert-private-vidore branch October 19, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluate model: athrael-soju/ColUltraVBERT

1 participant