feat(multilabel): noise-aware losses with an optional per-tag prior

[narugo1992]

Implements the losses described in #3, behind a flag that defaults to the current behaviour.

What this adds

animetimm/multilabel/losses.py — three losses selected by --loss:

• bce — an exact reproduction of BCEWithLogitsLoss(reduction='none'); the default, so the existing training path is byte-for-byte unchanged unless asked otherwise
• asl — Asymmetric Loss (Ridnik et al., ICCV'21)
• pasl — Partial-ASL with class-aware selective treatment, adapted from Ben-Baruch et al., CVPR'22

All three return an unreduced (B, C) tensor and accept soft targets, so mixup and the existing per-tag weighting and reduction keep working untouched.

One deliberate departure from the P-ASL paper: there, labels are explicitly annotated / un-annotated and the class-prior gate can drop a whole class's un-annotated entries. Here every non-positive is an assumed negative, so dropping a tag's negatives wholesale would mean never learning when the tag is absent. The per-tag signal is instead a continuous interpolation of the negative focusing parameter between gamma_neg and gamma_unann; hard dropping is reserved for the per-sample top-k gate. The ablation supports that call — see below.

Two entry points for the per-tag prior, both optional:

# external file
--loss pasl --loss-prior-file tag_reliability.parquet

# a column carried by the dataset's own tags parquet
--loss pasl --loss-prior-column label_reliability

Explicit --loss-prior-file wins over a column; neither present means a flat gamma. The column path is the longer-term home for this — it makes the prior travel with the dataset instead of as a side file.

metrics.py: the F-measure parameter is unified on beta. f1score squared its alpha while _curves did not, and a test pinned that disagreement as a known quirk. Since this is the first non-unit use of the parameter in the codebase, both now mean beta; alpha stays as an alias, and the test is repinned with a comment recording that the change is deliberate. Per-tag beta vectors are now accepted, which is what the β_t = 1/r_t work in #3 needs.

tools/ — the ablation harness: loss_ablation.py (one arm per invocation, identical seed and data order across arms), posthoc_eval.py (uniform re-scoring with a 0.05–0.95 threshold sweep, per-pair audited scores kept for paired bootstrap), report_ablation.py, and the launcher scripts. Paths that cannot be checked in — the survey repo, the audited thumbnails, the scratch directory — come from DBSURVEY_ROOT, DBSURVEY_AUDIT_IMAGES and ABLATION_OUT.

docs/superpowers/specs/2026-08-13-per-tag-noisy-label-training-design.md — the design, the six arms and why each exists, the three evaluation harnesses, and the measured results.

Results

Full numbers are in #3 and in the design doc. The short version:

• ASL over BCE: audited AUC 0.7435 → 0.8733, dbv4 micro-F1 0.4444 → 0.5113, macro-F1 0.0215 → 0.1620. Some of that gap is a 5,000-step budget artefact; a 12,000-step confirmation run is in flight.
• Per-tag γ against a same-mean constant-γ control, on the 86 human-confirmed omissions vs 606 confirmed absences: +0.0242 AUC, 95% CI [+0.0079, +0.0430]. It costs 0.0333 of dbv4-facing macro-F1 — the two rulers disagree in sign, which is the whole point of Noise-aware per-tag losses: ASL is a large win, and our current metric would reject the best variant #3.
• Both hard gates are significantly harmful: top-k −0.0319, diligence modulation a further −0.0111. Hence keeping the class-level treatment continuous.

Tests

67 new tests across test/multilabel/test_losses.py and test/multilabel/test_fbeta.py; suite is 130 passing.

Coverage worth calling out: bce is asserted equal to BCEWithLogitsLoss to floating-point tolerance; the ASL clip path is checked against an independently derived closed form; gradients are checked finite at logits of ±30 with clip on, which is where the naive log(1 - p + clip) formulation overflows; soft targets are checked to interpolate; and TestTrainIntegration exercises both prior entry points end to end, including the precedence rule and the flat-gamma fallback.

Compatibility

--loss defaults to bce and every new flag is inert without it, so existing training commands, configs and resumed runs behave exactly as before. The only behavioural change outside the opt-in path is the alpha → beta unification in metrics.py, which is a no-op at the default value of 1.

Closes nothing yet — #3 has open decisions (which objective to train for, whether key_metric moves to a corrected beta, where the reliability column should live) that should be settled before this drives a production run.

[narugo1992]

The 12,000-step confirmation run finished and it changes the case for this PR, so the summary above needs amending. Details and audit data in #3; the short version:

What still stands. Per-tag gamma, against the same-mean constant-gamma control, on the 86 human-confirmed omissions vs 606 confirmed absences: +0.0312 AUC [+0.0138, +0.0501] at 12,000 steps, up from +0.0242 [+0.0079, +0.0430] at 5,000. Replicated twice, effect grew with budget. pasl_gamma also has the highest pooled audited AUC at the longer budget (0.9162), where it trailed at the shorter one.

What does not. My "ASL over BCE is a large, three-way-consistent win" was two claims fused. The dbv4-facing half is real and does not narrow with budget (macro-F1 gap 0.1405 → 0.1465; 3.8× in the rarest frequency decile). The human-ground-truth half is not: on omission recovery asl_matched − bce goes from +0.0231 significant to −0.0011 not significant, and on the pure-disagreement contrast ASL ends up significantly worse than BCE (−0.0703 [−0.1275, −0.0155]). BCE was simply undertrained at 5,000 steps — its long-tail macro-F1 was exactly 0.0000 there and 0.0658–0.1581 at 12,000.

Consequence for this PR. The code is unchanged and the tests are unchanged; what changes is what the docs claim. docs/superpowers/specs/2026-08-13-per-tag-noisy-label-training-design.md now carries both budgets, states which quantities were stable across them, and separates "ASL improves convergence and tail calibration" from "per-tag gamma improves noise robustness" instead of presenting them as one win. That separation matters for adoption: whoever turns --loss asl on should know it is not the answer to the omission problem, and whoever wants the omission problem addressed needs the prior column too.

Both tools/run_ablation.sh (six arms, 5,000 steps) and tools/run_ablation_long.sh (three arms, 12,000 steps) are in the PR, so the two-budget protocol is reproducible rather than described.