MOT Label - AI-Powered Multi-Object Tracking Annotation Tool

A lightweight, efficient, web-based tool specifically designed for visualization, review, and editing of Multi-Object Tracking (MOT) datasets, now integrated with YOLOv8 to provide AI-assisted annotation capabilities.

✨ Core Features

AI-Assisted Annotation (Optional, enabled with --yolo)

AI models adopt a Lazy Loading strategy, loading only when first used, significantly reducing application startup time.

• 🤖 One-Click Detection & Recognition:

  • Dual Model Engine: Utilizes both YOLOv8-Pose and YOLOv8-Detection models for one-click pose recognition or object detection.
  • Custom Thresholds: Provides 0.0~1.0 confidence sliders to precisely filter out low-quality predictions before recognition.
  • Safe Preview: AI-detected objects are displayed with special dashed boxes as "previews", and all previewed objects are listed in the panel for you to freely select which ones to add.
  • Keypoint Visualization: For pose recognition results, in addition to bounding boxes, all keypoints are simultaneously drawn on the image.

• 🧠 Smart Correction & Tracking:

  • AI Smart Correction: For an inaccurate annotation box, click "AI Correction" and the program will automatically call the object detection model to find the prediction with the highest IoU overlap with the current box and use its position to automatically correct your annotation.
  • AI Cross-Frame Tracking: Select a target, input the desired number of frames to track, and AI will automatically track the target in subsequent frames and generate trajectories.

Data Processing & Loading

• 🚗 Dynamic Data Loading: Set the dataset root directory dynamically on the homepage with real-time refresh, no need to restart the service. Also retains the functionality of specifying paths via the --mot_root parameter at startup.
• 💪 Robust Data Processing:
  • Seamlessly parses gt.txt and det.txt format label files.
  • Intelligently parses seqinfo.ini files to obtain sequence metadata (image dimensions, frame rate, category definitions).
  • When configuration files or category definitions are missing, built-in MOT17 categories are enabled as fallback, ensuring program robustness.

Visualization & Advanced Navigation

• 🌐 Modern Web Interface: Built with Flask and native JavaScript, featuring a responsive two-column layout on the homepage with clear focus and convenient operations.
• 📈 Visual Timeline: A small timeline below the player marks key frames with annotations, facilitating quick navigation and overview of annotation distribution in the sequence.
• 🖼️ Precise Visualization:
  • Displays video images frame by frame.
  • Precisely draws bounding boxes, tracking IDs, and category names for each target on the image.
• 🖱️ Image Zoom & Drag:
  • Mouse Wheel: Zoom centered on cursor position.
  • Mouse Middle/Alt+Left: Hold and drag to pan the image.
  • Double Click: Reset view to center the image.
• 🎮 Powerful Playback Controls:
  • Basic Controls: Provides "Play/Pause", "Previous Frame", "Next Frame" functionality.
  • Quick Jump: Provides "First Frame" and "Last Frame" buttons for one-click navigation to sequence start or end.
  • Variable Speed Playback: Supports 1x, 2x, 4x, 8x multiple playback speeds for quick browsing.
• 👁️ Multi-Dimensional Filtering:
  • Homepage Filtering: Quickly filter video sequences by detector type (e.g., DPM, FRCNN, SDP).
  • Category Filtering:
    • Dynamically generates a list of all categories that appear in the current sequence.
    • Provides master switch to show/hide all categories with one click.
  • Trajectory Display: Toggle display of all targets' historical motion trajectories with one click.
  • Single Trajectory Isolation: In the editing panel, check "Isolate Trajectory" to show only the current selected target's annotations and trajectories, hiding all other targets for focused editing.

Powerful Annotation Editing Features

• 🖱️ Flexible Target Selection:
  • Single Click: Select a single target (only targets visible in category filtering can be selected).
  • Ctrl/Cmd + Click: Multi-select, supporting up to two targets simultaneously.
• 📝 Property Editing Panel: After selecting a target, the panel automatically displays and provides editing options:
  • Single Selection: Modify ID, category, bounding box coordinates/size, confidence.
  • Dual Selection: "Swap ID" function becomes available.
• 🎯 Scope-Based Editing: All modifications to ID and category can choose "Apply to Current Frame Only" or "Apply to All Frames (Entire Trajectory)".
• ✨ Trajectory Interpolation: Between two known keyframes of a trajectory, automatically perform linear interpolation with one click to quickly complete all missing annotations in between.
• ↔️ ID Swap: When two targets are selected, swap their entire historical trajectories with one click.
• 🗑️ Delete Target: Delete current frame annotations or entire tracking trajectories based on selected scope.
• ✍️ Create New Annotations:
  • Enable Creation Mode: Toggle to enter "Creation Mode", cursor changes to crosshair.
  • Drag to Draw: Hold left mouse button and drag on the image to draw new bounding boxes.
  • Assign ID & Category: After drawing, assign ID and category for the new target in the popup modal.

Save & Export

• 💾 One-Click Save: All modifications made in the frontend can be written back to the server with one click via the "Save Changes" button.
• 📄 Format Consistency: When saving, the backend strictly follows MOT gt.txt standard format (column order, data types) for writing and sorts by ID and frame number, ensuring high compatibility with original file formats.
• Elegant Notifications: Uses non-blocking Toast notifications instead of native alert() for better user experience.

⌨️ Keyboard Shortcuts

• Frame Navigation: ← (Previous Frame), → (Next Frame), Home (First Frame), End (Last Frame)
• Playback Control: Space (Play/Pause)
• Edit Operations: Delete/Backspace (Delete Selected), Esc (Cancel Selection), 1-9 (Quick Category Switch)
• File Operations: Ctrl/Cmd + S (Save)

Preview Layout

⚙️ How to Run

1. Place Dataset: Place your MOT format dataset (e.g., MOT17) in the dataset/mot directory, or specify your dataset path via the --mot_root parameter at startup. Directory structure should be as follows:

   /path/to/your/mot/data/
   └── train/
       └── MOT17-02-FRCNN/
           ├── det/
           │   └── det.txt
           ├── gt/
           │   └── gt.txt
           ├── img1/
           │   ├── 000001.jpg
           │   └── ...
           └── seqinfo.ini
   

2. Install Dependencies:

   • Basic Features:

     pip install -r requirements.txt

   • AI Features (Optional):

     pip install -r requirements-extra.txt

3. Run Service:

   # Basic run
   python server.py --port 5001
   
   # Enable AI features
   python server.py --port 5001 --yolo --mot_root your-mot-dataset-path