Add LUSR leaderboard submission - #130
Merged
Merged
Conversation
rrivera1849
force-pushed
the
submit-lusr
branch
from
August 6, 2026 17:58
759e31e to
43c346a
Compare
Record the LUSR submission and publish its scores. The model's raw metrics are merged into the canonical results tarball (distributed via Google Drive), so this PR carries only the permanent, in-repo artifacts: - SUBMISSIONS.yaml: register the LUSR submission entry. - scripts/models_all.txt: add rrivera1849/LUSR to the default models file. - scripts/benchmark_clustering/config.py: categorize LUSR with the style models. - docs/leaderboard.md: LUSR now appears in the Operational and Definitional tables. submitted_results/ is treated as a temporary scratchpad: a contributor's metrics live there only long enough to be validated and merged into the tarball, then the subtree is cleared.
rrivera1849
force-pushed
the
submit-lusr
branch
from
August 6, 2026 18:11
e4d7ba8 to
a356932
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Submits the LUSR style embedding model (
rrivera1849/LUSR) to the STEB leaderboard.The goal is to get LUSR onto the public leaderboard under both the Operational and Definitional STEB scores. This follows the standard submission flow described in the README "Submitting your model" section.
What changed
SUBMISSIONS.yaml— new entry registering LUSR with itshf_id, run command, and contributor.scripts/models_all.txt— addedrrivera1849/LUSRso the default--models-filepicks it up during score regeneration.scripts/benchmark_clustering/config.py— categorized LUSR alongside the other style/encoder models so it is scored in the correct group.submitted_results/— the full LUSR results subtree (metrics.jsonacross every evaluated dataset and task).Validation
python scripts/validate_submission.pypasses: 1 entry declared, 1 short_name on disk, all checks green.