Issues · UKGovernmentBEIS/inspect_ai · GitHub

Labels Milestones New issue

Missing proper livecodebench support

#4050

· patrik-bartak opened

on May 26, 2026

CLI type annotations for --display and --effort are narrower than their click choices

#4031

· RecreationalMath opened

on May 25, 2026

multi_scorer() returns unregistered closure — runtime crash 'Object score does not have registry info' in 0.3.46

#4027

· explosivebit opened

on May 24, 2026

`model_graded_qa` / `model_graded_fact` silently default to `INCORRECT` when the judge's verdict doesn't match the grade regex

#4026

· vladmesh opened

on May 24, 2026

model_graded_fact: answer field empty when judge GRADE parsing fails despite correct completion

#4025

· bodraski opened

on May 24, 2026

chat_history(include_history=True) duplicates the final answer in model-graded grader prompts

#4017

· RecreationalMath opened

on May 22, 2026

Logging from a worker thread inside eval_async crashes under trio (RuntimeError: must be called from async context)

#4003

· Butanium opened

on May 21, 2026

Feature: Pairwise Comparison & Elo Ratings for Model Arena Evaluation

#3994

· showpiecep opened

on May 20, 2026

Add a plugin extension point for modifying tasks at load time (e.g., injecting cross-cutting scorers)

#3973

· surgepower opened

on May 19, 2026

[discussion] Pre-registration primitive for eval claims — PRML interop?

#3897

· sk8ordie84 opened

on May 11, 2026

Missing Score Answer and Explanation in Custom Metrics with Epochs Task Option

#3840

· mohsinconnex opened

on May 5, 2026

Thread `exclude_fields` through `read_eval_log` / `read_eval_log_async` / `samples_df`

#3815

· RoshDSIT opened

on May 2, 2026