Skip to content

SAM3 Video Runtime - #255

Open
kevchengcodes wants to merge 3 commits into
apple:mainfrom
kevchengcodes:sam3-video-swift-runtime
Open

kevchengcodes wants to merge 3 commits into
apple:mainfrom
kevchengcodes:sam3-video-swift-runtime

Conversation

@kevchengcodes

Copy link
Copy Markdown
Contributor

Summary

This commit ports over the SAM3 video pipeline from the Transformers Python library into a new CoreAIVideoSegmenter Swift library. The majority of the code is purely additive and works on top of the existing SAM3-Video export.

Changes

  • new video-segmenter Swift CLI tool for quick inference and debugging
    • includes parity infrastructure for parity checks against BYO references
  • new 'CoreAIVideoSegmenter` Swift target containing the video pipeline
    • also includes utilities to render mask/box/label results over the input video
  • several new utilities in CoreAIShared

How to Use

Please see models/sam3_video/README.md

Swift runtime for the sam3_video export. Detector runs on every frame and results are associated with existing tracked objects. Some new CoreAIShared utilities. Includes postprocessing and rendering.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant