Skip to content

Extend RETFound to OCT Drusen Segmentation with Lightweight Decoder#55

Open
MDSALMANSHAMS wants to merge 7 commits intormaphoh:mainfrom
MDSALMANSHAMS:salman
Open

Extend RETFound to OCT Drusen Segmentation with Lightweight Decoder#55
MDSALMANSHAMS wants to merge 7 commits intormaphoh:mainfrom
MDSALMANSHAMS:salman

Conversation

@MDSALMANSHAMS
Copy link

This PR extends the RETFound foundation model to support pixel-level segmentation tasks.

Specifically, we demonstrate RETFound-based drusen segmentation on the MendeleyOCT dataset by attaching a lightweight convolutional decoder to the pretrained ViT encoder.

Key contributions:

  • Added a lightweight segmentation decoder (models_segmentation.py) that converts ViT patch embeddings to full-resolution masks.
  • Implemented a training pipeline for segmentation (main_segmentation.py) using a combined CrossEntropy + Dice loss.
  • Added reusable training utilities (engine_segmentation.py) for segmentation tasks.
  • Provided a standalone inference script (inference_segmentation.py) for evaluation and overlay visualization.
  • Included an example notebook (examples/RETFound_MendeleyOCT_demo.ipynb) demonstrating dataset preparation, training, and inference.

Dataset notes:

  • OCT images were horizontally cropped (top and bottom) to focus on the retinal region.
  • Binary segmentation masks represent drusen regions.

The implementation keeps the original RETFound encoder unchanged and adds only a minimal decoder, maintaining compatibility with pretrained weights.

This demonstrates the adaptability of RETFound representations for dense prediction tasks beyond classification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant