fix issue#6#7
Merged
Merged
Conversation
DuinoDu
added a commit
that referenced
this pull request
Jun 7, 2026
Two FFmpeg patch changes that together let downstream tools recover the mp4 writer's first_pts_us_writer_ anchor from any stream's metadata, without consulting any sidecar file. * read patch -- libavformat/mov.c::mov_parse_stsd_data: Replace the mett branch's blanket avio_skip(pb, size) with a proper walk of the child atoms. The original code read at most 20 bytes of mime via avio_get_str, which truncated the in-buffer copy but only advanced the cursor by 20 -- the rest of the mime string + the tbtm child box were treated as garbage. Now the parser consumes the full ContentEncoding and MimeFormat null-terminated strings (buffers grown to 256), then hands the remaining bytes to mov_read_default so any registered child parser (tbtm, the new tbgt, anything we add later) fires for mett tracks too. Net effect: every mett stream now surfaces the `timebase` (= track_base_unix_us) tag, where previously only RGB / depth carried it. * enc patch -- libavformat/movenc.c: Add `mov_write_tbgt_tag`, a sibling of the existing tbtm writer that emits the writer's track_base_godot_ticks_us (== first_pts_us_writer_) as a 16-byte box (fourcc 'tbgt' + big-endian i64). Call it everywhere tbtm is currently written: the raw1 / FFV1 / HEVC video sample entry blocks, and inside mov_write_mett_tag. The read patch registers a matching mov_read_tbgt that surfaces the value as a `track_base_godot_ticks_us` stream tag. Verified on a fresh Quest 3 capture (Stage 6 muxer + Stage 6 reader): #0 mett (left_controller): tb=...242 tbgt=1870352 #1 mett (right_controller): tb=...242 tbgt=1870352 #2 mett (left_hand): tb=...242 tbgt=1870352 #3 mett (left_input): tb=...242 tbgt=1870352 #4 mett (right_input): tb=...242 tbgt=1870352 #5 mett (head): tb=...242 tbgt=1870352 #6 hev1: tb=...242 tbgt=1870352 #7 FFV1: tb=...242 tbgt=1870352 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix #6.
Details
The root cause was that the Python bindings exposed SpatialML::camera_extrinsics::extrinsics directly (see the old implementation around bindings/spatialmp4.cpp:175), and that field is a cv::Matx<double,4,4>. Pybind11 doesn’t know
how to convert cv::Matx (a fixed-size OpenCV matrix) into a Python object out of the box, so whenever CameraExtrinsics.extrinsics or CameraExtrinsics.as_cvmat() ran, pybind11 couldn’t turn the C++ return value into something Python
understands and raised TypeError: Unregistered type : cv::Matx<double, 4, 4>. Switching the binding to return Eigen matrices—which pybind11 does support—resolved the conversion error.