SAM3-DMS: Decoupled Memory Selection for Multi-target Video Segmentation of SAM3

Ruiqi Shen¹ · Chang Liu^2✉️ · Henghui Ding^1✉️

¹Fudan University    ²Shanghai University of Finance and Economics

demo.mp4

TL;DR: Built upon SAM3, we focus on simultaneous multi-target video segmentation and propose a training-free decoupled memory selection strategy that shifts SAM3's group-level averaging to individual self-assessment, mitigating memory pollution and identity drift in complex scenarios.

⚙️ Installation

# create new conda environment
conda create -n sam3_decoupled python=3.12
conda deactivate
conda activate sam3_decoupled

# for pytorch/cuda dependencies
pip install torch==2.7.0 torchvision --index-url https://download.pytorch.org/whl/cu126

# clone the repo & install packages
git clone https://github.com/FudanCVL/SAM3_decoupled.git
cd SAM3_decoupled
pip install -e .

📥 Getting checkpoints

⚠️ Please request access to the checkpoints on the SAM3 Hugging Face repo. Once accepted, you need to be authenticated to download the checkpoints. You can do this by running the following steps (e.g. hf auth login after generating an access token.)

Please organize the downloaded checkpoint as follows:

├── sam3_ckpt/
│   ├── sam3.pt
│   └── ...

🚀 Training and Inference

We follow the same training and inference pipeline as SAM3. For detailed instructions, please see Evaluation, and Training.

🧪 Demo

We provide additional streamlined script for interactive PCS. You can simply specify a video input (mp4 or jpg folder) and enter text prompts via the command line to generate results.

python interactive_demo.py
Enter video path: # input the video (either mp4 or jpg folder)
Enter text prompt: # input the prompt

📄 Citation

If you find our work useful in your research, please consider citing:

@article{shen2024sam3dms,
  title={SAM3-DMS: Decoupled Memory Selection for Multi-target Video Segmentation of SAM3}, 
  author={Ruiqi Shen and Chang Liu and Henghui Ding},
  year={2026},
  journal={arXiv preprint arXiv:2601.09699},
}