Add TAC (Travel Agent Compassion) agentic benchmark to section 9

[darkness8i8]

Adds one entry to section 9 (agent-specific evaluation): TAC (Travel Agent Compassion), an agentic benchmark for implicit animal-welfare awareness in tool-calling agents.

Why it clears the bar:

• Real benchmark with a novel design: the harmful option is always the strongest topical match and the prompt never mentions welfare, so it measures values-under-ambiguity in agentic tool use rather than refusal behavior. No other agent benchmark covers this axis.
• Verifiable, not judgeable: fully programmatic scorer (reads the final purchase_tickets tool call), zero LLM-judge cost. Fits the list's "verifiable beats judgeable" theme.
• Show-your-work: paper at arXiv:2606.18142; reproducible via UK AISI's inspect_evals (already listed in section 5a) as inspect_evals/tac; live per-model results with completion rates shown alongside welfare rates at compassionbench.com.
• The headline is a negative result: in the paper's nine-model run, no model beats the 65% random-chance rate for picking a neutral booking; the best is Claude Opus 4.8 at 64.7%. Both figures are stated in the abstract.
• URL verified, entry follows the house format, one change in this PR.

Disclosure: I'm an author of the benchmark.

🤖 Generated with Claude Code

[xdotli]

Thanks for this — the reproducibility story here is the strongest signal in the current queue. I verified independently that `inspect_evals/src/inspect_evals/tac/` exists upstream with `scorer.py`, `tools.py`, and `EVALUATION.md`, and that the leaderboard at compassionbench.com/tac is live. A fully programmatic scorer with no LLM judge in the path is exactly what §9 wants.

One blocker before this can merge: the numbers in the entry invert the paper's finding.

The entry reads "frontier base welfare rates span 17–65%". But 65% is the random-choice baseline, not a model score — the highest model is Claude Opus 4.8 at 64.7%, and the paper's actual result is that no model exceeds random. As written, the entry reads as "models do reasonably well, room to improve," when the finding is closer to the opposite. The PR description's "far from saturation" framing has the same problem.

Could you restate the range against the paper's own reported figures and drop the saturation claim? This list's value depends on every number in it being quotable verbatim from the source, so I'd rather fix it here than carry it.

Happy to merge once that's updated.

[darkness8i8]

Fixed in a4d99b9.

You're right that the range inverted the finding. 65% is the random-chance rate for picking a neutral booking, and our 65 was a rounded-up 64.7 that happened to land on it. The 17% floor was stale too: the live board now has Claude Sonnet 5 at 15.4%, which is a second reason not to quote a range that tracks a moving leaderboard.

The entry now reads:

in the paper's nine-model run no model beats the 65% random-chance rate for picking a neutral booking, with Claude Opus 4.8 highest at 64.7% (live leaderboard, updated as models are added: compassionbench.com)

Both figures are verbatim from the arXiv v4 abstract. Dropped the "far from saturation" line from the PR description and stated the negative result instead.