Deploying our current vision pipeline (v4l2 + CameraNode + VisionNodes) drops framerate of streaming dramatically on PIs
Possible bottlenecks:
- v4l2 camera node constantly streaming both raw and compressed streams constantly
- CameraNode duplicating the v4l2 camera streams after doing minimal compute (rotation, when sometimes rotation isn't necessary).
- These will likely cause constant streaming and repeated compress + decompress calls to the JPEG format captured by pi cams, which can cause unnecessary CPU overhead
- Our vision debug streams likely unconditionally publish compressed images, where the constant compression causes cpu overhead
Proposed fix:
- first fork official ros humble v4l2_camera_node repo. We can likely combine our current CameraNode and v4l2 into a custom v4l2 camera node that does both, where you can specify a rotation parameter. This also utilizes v4l2's lazy compression publishing to avoid compute. (hmm, might be overkill actually)
- Have mode cv debug streams utilize ROS image transport w/ lazy compression, similar to what the official v4l2_camera_node does
- Make sure compressed streams are being used only for debugging/viewing purposes, not for in mission hot control loops (raw).
#152
Deploying our current vision pipeline (v4l2 + CameraNode + VisionNodes) drops framerate of streaming dramatically on PIs
Possible bottlenecks:
Proposed fix:
#152