Fix: evaluate_successor_liability returns ~0 on the HuggingFace answer format

[jimdawdy-hub]

Problem

evaluate_successor_liability parses the gold answer with str(answers[i]).split(","). That matches the canonical task TSVs (de facto merger,mere continuation), but the HuggingFace nguha/legalbench mirror serializes the same field as a (sometimes double-quoted) Python list repr:

"['de facto merger', 'mere continuation']"

Loading answers from HF and scoring with the official scorer yields ~0 F1 with no error, because the split produces tokens like "['de facto merger'" that never match a CLASS.

Since the README points users to the HF dataset as the primary distribution channel, the scorer should accept both serializations.

Fix

Add a small _parse_answer_list helper that handles bare comma-separated or list-repr answers, and use it in evaluate_successor_liability. Behavior on the canonical TSV format is unchanged.

Note: this complements (and depends on) the companion PR that removes successor_liability from EXACT_MATCH_BALANCED_ACC_TASKS — without that, this F1 path isn't reached. The root cause is a TSV-vs-HF serialization mismatch; this PR hardens the scorer, and the HF dataset could separately be re-serialized to match the TSVs.

[jimdawdy-hub]

Reproduction (captured output)

# repro.py — answer exactly as served by the HuggingFace `nguha/legalbench` mirror
from evaluation import evaluate_successor_liability
g = ["this is a de facto merger and a mere continuation"]
a = ["['de facto merger', 'mere continuation']"]
print("HF-format answer =", repr(a[0]))
print("F1 =", evaluate_successor_liability(g, a))

BEFORE (upstream main) — split(",") shreds the list repr, nothing matches a CLASS:

HF-format answer = "['de facto merger', 'mere continuation']"
F1 = 0.0

AFTER (this branch):

HF-format answer = "['de facto merger', 'mere continuation']"
F1 = 1.0

(Run with the companion mis-routing fix so the F1 path is reached; the function itself is exercised directly here.)

[jimdawdy-hub]

@neelguha Friendly ping — small follow-up in the same successor_liability scoring area as #50.

What it fixes: evaluate_successor_liability splits gold answers with str(...).split(","), which matches the canonical TSVs but not the HuggingFace nguha/legalbench mirror format ("['de facto merger', 'mere continuation']"). Users loading from HF get ~0 F1 with no error.

Scope: One commit — adds _parse_answer_list for both serializations.

Relation to #50: Independent fix (answer parsing vs metric routing). Either order works; together they make HF + F1 evaluation behave as documented.

Happy to fold into #50 or split differently if you’d rather one combined PR. Thanks for maintaining LegalBench!