Code accompanying:

Advait Gosai, Arun Kavishwar, Stephanie L McNamara, Soujanya Samineni, Renato Umeton, Alexander Chowdhury, William Lotter. Beyond Diagnosis: Evaluating Multimodal LLMs for Pathology Localization in Chest Radiographs. Machine Learning for Health (ML4H) Symposium, 2025.

Contents:

• gpt_src and oss_src contain code for prompting the GPT and MedGemma models, respectively. Note: An OPENAI_API_KEY is needed to run the GPT scripts, and an ollama service is needed to run the MedGemma scripts.
• compute_results.py: Computes the hit rate determination for each prediction based on the included ground truth csvs.
• figure_heatmap.py: Computes heatmaps of the predictions.