Skip to content

Repository files navigation

Conductor Eval Logo

Conductor Eval measures how well AI models generate MIDI loops against musical constraints. It runs evaluations through the public conductor-core engine, applies deterministic MIDI checks, saves results, and provides an optional Plotly Dash dashboard for analysis.

Documentation

The detailed documentation is built with MkDocs:

To preview the documentation locally, install the docs dependency group and start MkDocs:

uv sync --locked --group docs
uv run --locked --group docs mkdocs serve

Open http://127.0.0.1:8000/ in your browser.

Alternatively, Read the documentation source as plain markdown.

Setup

For a short local setup, install the locked development environment and run the checks:

uv sync --locked --all-extras
uv run --locked --all-extras pytest -q

See getting started for guidance on running an evaluation.

See analysis to run a dashboard displaying evaluation results.

About

A benchmarking framework for evaluating LLM-based MIDI generation across models, prompts, and generation strategies, helping identify the most effective approaches through automated testing and comparison.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages