ffmpeg-rockchip在rock5b上工作不太正常

[TuTuABC]

过去一周我在rock5b上使用ffmpeg-rockchip和gstreamer捕捉HDMI输入，都有点不对劲，gstreamer设置捕获4k60帧时实际只会跑到30帧，ffmpeg-rockchip可以录制4k60帧，但是会有周期性的帧率下降，大概每3分钟左右，帧率开始明显下降到30-40帧，并且还有一个影响很大的问题，画面一直有一些白色闪烁的线条，这是使用硬件mpp编码才会出现，我使用软件编码时画面没有问题。我尝试了radxa os不同内核的系统镜像都是如此，我尝试打印一些日志，掉帧的时候似乎提示mpp已经塞满了，我也尝试录制没有压缩的YUV到内存空间，然后再编码，速度达到了2x，看起来也不像是mpp编码速度太慢。
ffmpeg -loglevel debug -stats_period 1 \ -fflags nobuffer -flags low_delay -avioflags direct -rtbufsize 8192k \ -f v4l2 -input_format nv12 -video_size 3840x2160 -framerate 60 -i /dev/video0 \ -c:v hevc_rkmpp -rc_mode AVBR -b:v 20M -minrate 10M -maxrate 40M \ -y test_long.mkv 2>&1 | tee ffmpeg_long.log
[hevc_rkmpp @ 0xaaaafeb45050] Failed to put frame to encoder input queue: -11 [hevc_rkmpp @ 0xaaaafeb45050] Wrote 19814400 bytes to encoder [hevc_rkmpp @ 0xaaaafeb45050] Received a packet [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder -> type:video pkt_pts:5625 pkt_pts_time:93.75 pkt_dts:5625 pkt_dts_time:93.75 duration:0 duration_time:0 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder <- type:video frame_pts:5630 frame_pts_time:93.8333 time_base:1/60 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] muxer <- type:video pkt_pts:1440000 pkt_pts_time:93.75 pkt_dts:1440000 pkt_dts_time:93.75 duration:0 duration_time:0 size:1572 [hevc_rkmpp @ 0xaaaafeb45050] Wrote 19814400 bytes to encoder [hevc_rkmpp @ 0xaaaafeb45050] Received a packet [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder -> type:video pkt_pts:5626 pkt_pts_time:93.7667 pkt_dts:5626 pkt_dts_time:93.7667 duration:0 duration_time:0 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder <- type:video frame_pts:5631 frame_pts_time:93.85 time_base:1/60 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] muxer <- type:video pkt_pts:1440256 pkt_pts_time:93.7667 pkt_dts:1440256 pkt_dts_time:93.7667 duration:0 duration_time:0 size:570 [hevc_rkmpp @ 0xaaaafeb45050] Failed to put frame to encoder input queue: -11 [hevc_rkmpp @ 0xaaaafeb45050] Wrote 19814400 bytes to encoder [hevc_rkmpp @ 0xaaaafeb45050] Received a packet [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder -> type:video pkt_pts:5627 pkt_pts_time:93.7833 pkt_dts:5627 pkt_dts_time:93.7833 duration:0 duration_time:0 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder <- type:video frame_pts:5632 frame_pts_time:93.8667 time_base:1/60 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] muxer <- type:video pkt_pts:1440512 pkt_pts_time:93.7833 pkt_dts:1440512 pkt_dts_time:93.7833 duration:0 duration_time:0 size:3844 [hevc_rkmpp @ 0xaaaafeb45050] Wrote 19814400 bytes to encoder [hevc_rkmpp @ 0xaaaafeb45050] Received a packet [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder -> type:video pkt_pts:5628 pkt_pts_time:93.8 pkt_dts:5628 pkt_dts_time:93.8 duration:0 duration_time:0 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder <- type:video frame_pts:5633 frame_pts_time:93.8833 time_base:1/60 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] muxer <- type:video pkt_pts:1440768 pkt_pts_time:93.8 pkt_dts:1440768 pkt_dts_time:93.8 duration:0 duration_time:0 size:1013 [hevc_rkmpp @ 0xaaaafeb45050] Failed to put frame to encoder input queue: -11 [hevc_rkmpp @ 0xaaaafeb45050] Wrote 19814400 bytes to encoder [hevc_rkmpp @ 0xaaaafeb45050] Received a packet [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder -> type:video pkt_pts:5629 pkt_pts_time:93.8167 pkt_dts:5629 pkt_dts_time:93.8167 duration:0 duration_time:0 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder <- type:video frame_pts:5634 frame_pts_time:93.9 time_base:1/60 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] muxer <- type:video pkt_pts:1441024 pkt_pts_time:93.8167 pkt_dts:1441024 pkt_dts_time:93.8167 duration:0 duration_time:0 size:4669 [hevc_rkmpp @ 0xaaaafeb45050] Wrote 19814400 bytes to encoder [hevc_rkmpp @ 0xaaaafeb45050] Received a packet [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder -> type:video pkt_pts:5630 pkt_pts_time:93.8333 pkt_dts:5630 pkt_dts_time:93.8333 duration:0 duration_time:0 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder <- type:video frame_pts:5635 frame_pts_time:93.9167 time_base:1/60 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] muxer <- type:video pkt_pts:1441280 pkt_pts_time:93.8333 pkt_dts:1441280 pkt_dts_time:93.8333 duration:0 duration_time:0 size:617 [hevc_rkmpp @ 0xaaaafeb45050] Failed to put frame to encoder input queue: -11 [hevc_rkmpp @ 0xaaaafeb45050] Wrote 19814400 bytes to encoder [hevc_rkmpp @ 0xaaaafeb45050] Received a packet [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder -> type:video pkt_pts:5631 pkt_pts_time:93.85 pkt_dts:5631 pkt_dts_time:93.85 duration:0 duration_time:0 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder <- type:video frame_pts:5636 frame_pts_time:93.9333 time_base:1/60 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] muxer <- type:video pkt_pts:1441536 pkt_pts_time:93.85 pkt_dts:1441536 pkt_dts_time:93.85 duration:0 duration_time:0 size:7283 [hevc_rkmpp @ 0xaaaafeb45050] Wrote 19814400 bytes to encoder [hevc_rkmpp @ 0xaaaafeb45050] Received a packet [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder -> type:video pkt_pts:5632 pkt_pts_time:93.8667 pkt_dts:5632 pkt_dts_time:93.8667 duration:0 duration_time:0 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] encoder <- type:video frame_pts:5637 frame_pts_time:93.95 time_base:1/60 [vost#0:0/hevc_rkmpp @ 0xaaaafeb44d10] muxer <- type:video pkt_pts:1441792 pkt_pts_time:93.8667 pkt_dts:1441792 pkt_dts_time:93.8667 duration:0 duration_time:0 size:1343

[nyanmisaka]

• The v4l2 input device in ffmpeg does not support zero-copy. Therefore, copying 4k60 nv12 has a large overhead.

• "Encoder buffer full" is a debugging message, not an error message.

• The stripes are because you are not using my patched MPP branch linked in the Wiki. The upstream branch contains a known issue where dma_heap is always incorrectly allocating cached buffers, even though we have requested it not to do so in the code.

[MIUICO]

• ffmpeg 中的 v4l2 输入设备不支持零复制。因此，复制4k60 nv12会带来很大的开销。
• “编码器缓冲区满”是调试信息，不是错误信息。
• 条纹是因为你没有使用我在维基里链接的补丁MPP分支。上游分支存在一个已知问题，即dma_heap总是错误分配缓存缓冲区，尽管我们在代码中已请求不要这样做。

感谢回复，我按照wiki中的说明重新安装了一切，画面bug得到完美修复。但随之而来的是帧率大幅度下降，我通过一些60fps的格子视频测得实际帧率只有35fps左右，好像就是不支持零复制带来的问题，请问这个有方法处理吗。
主线mpp：帧率60但是画面有些bug
支线mpp：画面完美但是帧率35
我会有机会获得帧率60+画面完美的解决方案吗

[nyanmisaka]

There's an ongoing project upstream in ffmpeg to add zero-copy drm_prime output support for -f v4l2. So I'm NOT going to reinvent the wheel and will wait for it to be implemented.