Skip to content

[BENCH-807] Rewrite roster-dependent tests to state their own models - #597

Merged
coval-cale merged 2 commits into
mainfrom
cale/bench-807-rewrite-roster-tests
Sep 3, 2026
Merged

[BENCH-807] Rewrite roster-dependent tests to state their own models#597
coval-cale merged 2 commits into
mainfrom
cale/bench-807-rewrite-roster-tests

Conversation

@coval-cale

Copy link
Copy Markdown
Contributor

The roster is test data now, so a test that computes its expectation by running a production function over the whole roster checks that function against itself, and a test that searches the roster for a model with some property guards a case that cannot happen. Each test now declares the rows it is about and asserts a literal result.

The seed shrinks to three synthetic rows: one live STT model and two live TTS models with voice pools for the arena to pair. Catalogue tests add their own rows under a made-up provider. Prod-data invariant tests (pools are female/male pairs, live TTS has a pool, scheduled providers have code) go; the admin API enforces them at write time.

@coval-cale
coval-cale requested a review from a team as a code owner September 3, 2026 23:11

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your trial has ended. Reactivate Greptile to resume code reviews.

@coderabbitai

coderabbitai Bot commented Sep 3, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 11cc5418-36b6-4dc4-9481-2752b02219f0

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The roster is test data now, so a test that computes its expectation by
running a production function over the whole roster checks that function
against itself, and a test that searches the roster for a model with some
property guards a case that cannot happen. Each test now declares the rows
it is about and asserts a literal result.

The seed shrinks to three synthetic rows: one live STT model and two live
TTS models with voice pools for the arena to pair. Catalogue tests add
their own rows under a made-up provider. Prod-data invariant tests
(pools are female/male pairs, live TTS has a pool, scheduled providers
have code) go; the admin API enforces them at write time.
@coval-cale
coval-cale force-pushed the cale/bench-807-rewrite-roster-tests branch from 44ba922 to 78d7fc8 Compare September 3, 2026 23:13
@coval-cale
coval-cale changed the base branch from cale/bench-806-delete-the-in-code-model-registry to main September 3, 2026 23:13
@coval-cale
coval-cale enabled auto-merge September 3, 2026 23:18
@coval-cale
coval-cale disabled auto-merge September 3, 2026 23:33
@coval-cale
coval-cale enabled auto-merge September 3, 2026 23:36
@coval-cale
coval-cale added this pull request to the merge queue Sep 3, 2026
Merged via the queue into main with commit c072672 Sep 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants