Skip to content

MTEB Evaluation Results: JMullings/spectral-embed-v2-140m - #789

Open
jmullings wants to merge 2 commits into
embeddings-benchmark:mainfrom
jmullings:mteb-results-1792351083
Open

jmullings wants to merge 2 commits into
embeddings-benchmark:mainfrom
jmullings:mteb-results-1792351083

Conversation

@jmullings

@jmullings jmullings commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Models Submitted

JMullings/spectral-embed-v2-140m (revision: 6943d55ebe5e317f00701b574c12d4f39ca6f03b)

Checklist

  • My model has a model sheet, report, or similar
  • My model has a reference implementation in mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here
  • No, but there is an existing PR: model: add JMullings/spectral-embed-v2-140m mteb#5716
  • The results submitted are obtained using the reference implementation
  • My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
  • I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

Models: JMullings/spectral-embed-v2-140m
Total results: 19
Submitted by MTEB ResultCache
@KennethEnevoldsen KennethEnevoldsen added the waiting for review of implementation This PR is waiting for an implementation review before merging the results. label Oct 18, 2026
@KennethEnevoldsen

Copy link
Copy Markdown
Contributor

@jmullings please fill out the checklist, notably on the reference implementation

@jmullings

Copy link
Copy Markdown
Contributor Author

@KennethEnevoldsen The PR description and checklist are now fully updated and linked to PR #5716, and the 39 V2 task results have been pushed to this branch. As expected, the CI is currently failing with KeyError: Model 'JMullings/spectral-embed-v2-140m' not found in MTEB registry because it is waiting for #5716 to be reviewed and merged. Ready whenever you are!

@KennethEnevoldsen

Copy link
Copy Markdown
Contributor

@jmullings checklist is not filled out

@jmullings

Copy link
Copy Markdown
Contributor Author

@KennethEnevoldsen Apologies for the delay—the checklist is now filled out and linked to the reference implementation PR (embeddings-benchmark/mteb#5716).

@jmullings

Copy link
Copy Markdown
Contributor Author

@KennethEnevoldsen embeddings-benchmark/mteb#5716 ready for review.

@jmullings

Copy link
Copy Markdown
Contributor Author

@KennethEnevoldsen
All test are passing on embeddings-benchmark/mteb#5716, following the the removal of scikit-learn 1.6.1 package use. Version 2 should now be ready to merge. Many thanks..

@KennethEnevoldsen

Copy link
Copy Markdown
Contributor

@jmullings you commented 3 times on this PR which is still waiting for embeddings-benchmark/mteb#5716 to be merged and is marked as such.

We will look into this once the model PR is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting for review of implementation This PR is waiting for an implementation review before merging the results.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants