Skip to content

GPU path drops the last partial batch when the container over-reports frames; CPU path drops frames when it under-reports #16

Description

@joeljose

Severity: Medium. Audio is lost silently, and the CPU and GPU results differ in length.

GPU path (extract_audio_gpu)

A batch is processed only when len(batch_frames) == batch_size or fc == frame_count - 1. If cap.read() fails before the reported frame_count (containers often over-report), the loop breaks and the frames already in batch_frames are never processed.

Reproduced by running extract_audio_gpu on the CPU (PyTorch CPU build with pytorch_wavelets; device patched to CPU):

container reports 120 frames, 100 decodable, batch 16:
  CPU path returns 100 samples, GPU path returns 96 samples

Up to batch_size - 1 frames are lost at the end of every such video (31 frames at --batch-size 32, about 14 ms of audio at 2200 fps). The progress counter also reports frames_read, not frames processed.

CPU path (extract_audio) and GPU path

Both loop for fc in range(frame_count), so if the container under-reports, all frames after the reported count are silently ignored.

Suggested fix

  • Read until cap.read() fails, not up to frame_count, and use frame_count only for progress and ETA.
  • Flush the remaining batch_frames after the loop. Moving the batch processing into a helper function makes that a single call.
  • Warn when the decoded count differs from the metadata.
  • Add a test with a fake capture that over- and under-reports, checking that both paths return the true frame count. The GPU path can run on the CPU (see the CI issue).

Acceptance criteria

  • CPU and GPU paths return the same number of samples as the decodable frames, whatever CAP_PROP_FRAME_COUNT says.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions