In the MMLU-Pro data, the parsed letter answer prediction is appended to the response column as PRED: <letter answer>.
It would be cleaner for me if instead there were a separate column parsed_prediction that listed the letter answer instead, so that the response is just the model response and can be processed in other post-hoc analysis without having to strip PRED: from the end of each response.
In the MMLU-Pro data, the parsed letter answer prediction is appended to the
responsecolumn asPRED: <letter answer>.It would be cleaner for me if instead there were a separate column
parsed_predictionthat listed the letter answer instead, so that theresponseis just the model response and can be processed in other post-hoc analysis without having to stripPRED:from the end of each response.