Skip to content

Investigate vision bottlenecks #169

@yuzhiliu8

Description

@yuzhiliu8

Deploying our current vision pipeline (v4l2 + CameraNode + VisionNodes) drops framerate of streaming dramatically on PIs

Possible bottlenecks:

  • v4l2 camera node constantly streaming both raw and compressed streams constantly
  • CameraNode duplicating the v4l2 camera streams after doing minimal compute (rotation, when sometimes rotation isn't necessary).
    • These will likely cause constant streaming and repeated compress + decompress calls to the JPEG format captured by pi cams, which can cause unnecessary CPU overhead
  • Our vision debug streams likely unconditionally publish compressed images, where the constant compression causes cpu overhead

Proposed fix:

  • first fork official ros humble v4l2_camera_node repo. We can likely combine our current CameraNode and v4l2 into a custom v4l2 camera node that does both, where you can specify a rotation parameter. This also utilizes v4l2's lazy compression publishing to avoid compute. (hmm, might be overkill actually)
  • Have mode cv debug streams utilize ROS image transport w/ lazy compression, similar to what the official v4l2_camera_node does
  • Make sure compressed streams are being used only for debugging/viewing purposes, not for in mission hot control loops (raw).

#152

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions