Skip to content

README.md

Latest commit

 

History

History
37 lines (25 loc) · 2.7 KB

README.md

File metadata and controls

37 lines (25 loc) · 2.7 KB
Raw

Hyperparameter tuning

Single source of truth for every tracked ablation run: what was tested, what changed run-to-run, and where the results and frozen configs for each run live.

How-to (launch / add a run): ../documentation/training.md · Full results and interpretation: RESULTS.md

Contents

Path Role
ablations.yaml One entry per run: which base config it extends (paligemma | medgemma) + the parameter deltas (training_overrides / eval_overrides)
verify_ablations.py Round-trip check: resolves every run and asserts the merged config matches expectations — run after editing ablations.yaml
RESULTS.md Full ablation analysis: run index, metrics tables, per-run interpretation, best-configuration trade-offs
runs/{run_name}/ Per-run outputs (gitignored checkpoints; tracked frozen configs + results)

Why a single ablations.yaml instead of one YAML per run

Two base configs — config/training.yaml/config/eval.yaml (V1, PaliGemma) and config/training_medgemma.yaml/config/eval_medgemma.yaml (V2, MedGemma) — capture what nearly all runs share. Most of the 21 runs change only a handful of fields relative to their base, so ablations.yaml records just those deltas rather than duplicating ~85 lines of near-identical YAML per run. src/config/ablations.py deep-merges base + override at load time; see documentation/training.md for the full mechanics.

runs/{run_name}/ contents

Path Description
training.yaml Frozen, fully-resolved training config (written at train time)
evaluation.yaml Frozen, fully-resolved eval config (written at infer/eval time)
checkpoints/ LoRA adapter weights (gitignored) + TensorBoard logs
results/metrics.json Evaluation metrics
results/tb_scalars.csv Exported TensorBoard scalars
results/prediction.csv Test-set generations (gitignored; regenerate with infer.py)

Demo replay uses curated copies of runs 12 and 17 under demo/predictions/.

Run index

21 tracked runs — 6 PaliGemma (V1, runs 01–06) + 15 MedGemma (V2, runs 08–22). Full table with per-run key changes: RESULTS.md.

Best overall (V2): 17_weighted_loss_no_study (study accuracy 0.757, F1 0.735). Best precision (V2): 12_medgemma_ontology_study_loss (precision 0.724, fewer false normals). See RESULTS.md §7 for the full trade-off.