Improve Benchmarking UI and dataset alignment flow - #118
Conversation
suung
left a comment
There was a problem hiding this comment.
@nicooxxx FWIW
I would generally think like this
Benchmark Dataset is the exact same as Ground truth. I always found this confusing
So fo rme it is more like "Data you want to test" and either "Benchmark dataset" or "Ground thruth dataset"
Or both.
Hey @nicooxxx I came across this again during #127 .. I think we should fix it, do you want to do it? |
Overview
This is a first iteration of the Benchmarking UI improvements.
The current changes focus mainly on the Dataset and Alignment flow for Ranking and Classification.
V1 – implemented in this draft
Simplified Ranking and Classification dataset setup
Clearer distinction between Ground Truth and Benchmark data
Improved alignment wizard labels and step descriptions
Added help text/tooltips for less obvious fields
Simplified technical wording throughout the dataset flow
Improved consistency of the Benchmarking UI
V2 – next iteration
The next step would focus on the Evaluate flow, especially:
Reusing dataset and column selections from the alignment step
Reducing duplicate configuration
Simplifying Ranking configuration (Top K, Metrics at K)
Simplifying Classification evaluation
Improving the transition from evaluation to results
Feedback
This is still a work in progress. Feedback on the general flow and terminology would be helpful before I continue with the Evaluate section.
Screenshots
Dataset Managment
Ranking Upload GT
Ranking Align GT
Ranking Upload BM
Classification Upload
Classification Align
Evaluate Ranking
Classification Ranking