Open Test Thread

[hbiyik]

@avafinger

A lot has been added specially hevc and vp8 encoders with scaling support:

https://github.com/hbiyik/FFmpeg/wiki

should be stable to test if you are interested.

[nyanmisaka]

Is there an option to force *_rkmpp_decoder outputting drm_prime hw frames?

[hbiyik]

FFMPEG_RKMPP_PIXFMT=DRMPRIME env value

[nyanmisaka]

HEVC/VP9/AV1 10-bit to H264/HEVC 8bit transcoding can cause system crashes. Sometimes I have to cut the power to reset.

FFMPEG_RKMPP_PIXFMT=DRMPRIME ./ffmpeg -c:v hevc_rkmpp_decoder -i hevc_10bit.mp4 -an -sn -c:v h264_rkmpp_encoder -rc_mode VBR -b:v 6M -maxrate 6M -bufsize 12M -profile:v high -level 4.1 -g:v 120 -f null -

[  771.797702] rk_vcodec: mpp_translate_reg_address:1838: reg[  0]: 0xffffffff fd -1 failed
[  771.797713] rk_vcodec: mpp_task_dump_mem_region:2025: --- dump mem region ---
[  771.797724] mpp_rkvenc2 fdbd0000.rkvenc-core: no memory region mapped
[  771.797737] rk_vcodec: mpp_process_task_default:630: alloc_task failed.
[  771.797747] rkvenc2_wait_result:1995: session 00000000736d7b74 pending list is empty!
[  771.797753] rk_vcodec: mpp_msgs_wait:1634: session 3 wait result ret -5
[  772.019465] rkvdec2_ccu_timeout_work:1643: fdc38100.rkvdec-core, task timeout
[  772.019515] rkvdec2_ccu_timeout_work:1643: fdc48100.rkvdec-core, task timeout
[  772.019586] mpp_rkvdec2 fdc48100.rkvdec-core: resetting...
[  772.019782] mpp_rkvdec2 fdc48100.rkvdec-core: reset done
[  772.019794] mpp_rkvdec2 fdc38100.rkvdec-core: resetting...
[  772.019896] mpp_rkvdec2 fdc38100.rkvdec-core: reset done

Also it seems the post RGA -width 1280 -height 720 doesn't accept 10-bit hw frames.

[h264_rkmpp_encoder @ 0xaaaae92c1dc0] Scaling is only supported for NV12,NV16,YUV420P,YUV422P. drm_prime requested

[hbiyik]

Can you try without forcing drmprime? You actaully dont neee to drm prime, softframes are 0 copy.

Scaling is only possible yuv420/422 p/sp planes, but i noticed when the outputframe is drmprime i dont check it correctly. Ill fix that...

[nyanmisaka]

You actaully dont neee to drm prime, softframes are 0 copy.

Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), nv12(tv, progressive), 1920x1080
But the encoder's input still suggests nv12, which usually implies non-zero-copy.

After removing the env, here comes the rga3 alignment issue or maybe the DMA32 buffer issue on my 16GB RAM board.

Stream mapping:
  Stream #0:5 -> #0:0 (hevc (hevc_rkmpp_decoder) -> h264 (h264_rkmpp_encoder))
Press [q] to stop, [?] for help
[hevc_rkmpp_decoder @ 0xaaaaf9f9af50] Decoder noticed an info change
[hevc_rkmpp_decoder @ 0xaaaaf9f9af50] 10bit NV15 plane will be downgraded to 8bit nv12.
rga_api version 1.8.1_[4]
err hs[0,1088,1080]
Error srcRect
[RgaBlit,782]Error srcRect

fd-vir-phy-hnd-format[12, (nil), (nil), 0, 8192]
rect[0, 0, 1920, 1088, 2816, 1080, 8192, 0]
f-blend-size-rotation-col-log-mmu[8192, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[20, (nil), (nil), 0, 2560]
rect[0, 0, 1920, 1088, 1920, 1088, 2560, 0]
f-blend-size-rotation-col-log-mmu[2560, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail
[hevc_rkmpp_decoder @ 0xaaaaf9f9af50] RGA failed falling back to soft conversion
[hevc_rkmpp_decoder @ 0xaaaaf9f9af50] RGA failed to convert NV15 -> NV12. No Soft Conversion Possible
[hevc_rkmpp_decoder @ 0xaaaaf9f9af50] Failed set frame buffer (code = -1)
[hevc_rkmpp_decoder @ 0xaaaaf9f9af50] Decoder Failed to get frame (code = -1)
Error while decoding stream #0:5: Operation not permitted

And for the HEVC 8-bit 1080p input + post RGA downscaling to 720p.
https://test-videos.co.uk/vids/bigbuckbunny/mp4/h265/1080/Big_Buck_Bunny_1080_10s_30MB.mp4

./ffmpeg -stream_loop -1 -c:v hevc_rkmpp_decoder -i Big_Buck_Bunny_1080_10s_30MB.mp4 -an -sn -c:v h264_rkmpp_encoder -rc_mode CBR -b:v 6M -maxrate 6M -bufsize 12M -profile:v high -level 4.1 -g:v 120 -width 1280 -height 720 -f null -

rga_api version 1.8.1_[4]
err hs[0,1088,1080]
Error srcRect
[RgaBlit,782]Error srcRect

fd-vir-phy-hnd-format[12, (nil), (nil), 0, 2560]
rect[0, 0, 1920, 1088, 2304, 1080, 2560, 0]
f-blend-size-rotation-col-log-mmu[2560, 0, 0, 0, 0, 0, 1]
fd-vir-phy-hnd-format[37, (nil), (nil), 0, 2560]
rect[0, 0, 1280, 720, 1280, 720, 2560, 0]
f-blend-size-rotation-col-log-mmu[2560, 0, 0, 0, 0, 0, 1]
This output the user patamaters when rga call blit fail
[h264_rkmpp_encoder @ 0xaaaaf24ec9e0] RGA failed falling back to soft conversion
[h264_rkmpp_encoder @ 0xaaaaf24ec9e0] Error applying Post RGA

[hbiyik]

Ok ill give it a look in detail tonight

[hbiyik]

the 2nd issue is quite weird.

rect[0, 0, 1920, 1088, 2304, 1080, 2560, 0]

the hstride is given 2304 here, and this is received from mpp directly. for this NV12 frame it should be 1080, or with alignment 1088 1920, but wtf is 2304, and why mpp reports so, is interesting, could be an issue with mpp that i need to dig in.

Update:

1. issue with forcing DRMPRIME and having everything crashed, fixed in ed61669
2. Issue with forcing DRMPRIME and scaling gives wrong limitation error, fixed in bf96d57
3. Issue Big_Buck_Bunny_1080_10s_30MB.mp4 conversion crashes the rga: This is most likely an MPP bug, i understood completely why it is happening and i do not want to patch it from ffmpeg because obviously mpp is providing wrong stride, raised a bug issue mpp provides wrong hstride when decoding this hevc file rockchip-linux/mpp#422, i expect a simple bugfix from mpp.

[nyanmisaka]

Thx for the update. With the fixes:

• forcing DRMPRIME + 10-bit nv15 input => chroma shift in output
  http://www.larmoire.info/jellyfish/media/jellyfish-10-mbps-hd-hevc-10bit.mkv
  

• forcing DRMPRIME + 10-bit nv15 input + post RGA => mismatched output resolution and chroma shift

  Stream #0:0: Video: hevc (Main) (hev1 / 0x31766568), drm_prime(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 6000 kb/s, 23.98 fps, 24k tbn (default)
    Metadata:
      BPS-eng         : 5794040
      DURATION-eng    : 01:30:06.401000000
      NUMBER_OF_FRAMES-eng: 129624
      NUMBER_OF_BYTES-eng: 3915613217
      _STATISTICS_WRITING_APP-eng: mkvmerge v42.0.0 ('Overtime') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2020-01-14 15:39:31
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      encoder         : Lavc60.3.100 hevc_rkmpp_encoder
rga_api version 1.8.1_[4]
[hevc_rkmpp_encoder @ 0xaaaaefd216c0] Reconfigured with w=1280, h=720, format=nv12.

Video
ID                             : 1
Format                         : HEVC
Format/Info                    : High Efficiency Video Coding
Format profile                 : Main@L5.1@Main
Codec ID                       : hev1
Codec ID/Info                  : High Efficiency Video Coding
Duration                       : 23 s 524 ms
Bit rate                       : 4 409 kb/s
Maximum bit rate               : 6 000 kb/s
Width                          : 1 920 pixels
Original width                 : 1 280 pixels
Height                         : 1 080 pixels
Original height                : 720 pixels
Display aspect ratio           : 16:9
Frame rate mode                : Constant
Frame rate                     : 23.976 (24000/1001) FPS
Color space                    : YUV
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Scan type                      : Progressive
Bits/(Pixel*Frame)             : 0.089
Stream size                    : 12.4 MiB (100%)
Menus                          : 2
Codec configuration box        : hvcC

• forcing DRMPRIME + 8-bit nv12 input + w/ and w/o post RGA => corrupted output, TODO?
  

[hbiyik]

this should all work, however i switched to librga and it is being a bi**h, i am working on it.
Anything else in the meanwhile??

[hbiyik]

ah one question, which player are you testing the drmprime output? is it kodi or something else?

[nyanmisaka]

I can't figure out why librga exists independently of mpp. This resulted in developers having to maintain compatibility between them.

sw frames seem to work fine in the encoder. Video quality looks better than AMD graphics cards, albeit slower.

I've been using the command line to test the encoder. Kodi might be a good option for testing ffmpeg as a library. Or refer to some tools of the author of rpi-ffmpeg such as https://github.com/jc-kynesim/hello_drmprime.

[hbiyik]

all of the issues you reported today and yesterday including the one i thought mpp related should be fixed in 1d57c70

There is something fishy with mpp to my understanding however this was not the root cause of the issue.

About Sw frames, even when so-called non-drmprime planes are decoded with rkmpp_decoder they are still hardware planes most of the time, simply mmapped to drm device. Especially when the NV12 plane is used, there is no copy at all. So i would call them hybrid and in the transcoding scenarios they are mainly NV12, so hardware frames mapped to AVFrame. It is kinda tricky but this the actual reason why i started this fork in the first place.

In short, when transcoding forcing DRMPRIME does not make any difference in terms of performance, you can also verify this with the throughput and resource usage. This is the reason i did not test those parts with transcoding with DRMPRIME forced, so thanks for that :), lots of issues were found and fixed (hopefully). i had tested drmprime only when getting kmsgrab input with drmprime bgr0 frames.

[nyanmisaka]

Overall works great after the latest changes, with some exceptions.

• In post RGA cases, avctx->{width,height} should be finalized before calling rkmpp_config(), otherwise MPP_ENC_GET_HDR_SYNC cannot generate the correct extradata for the HEVC encoder. Also, with forcing DRMPRIME enabled, the changes to avctx->{width,height} are ignored. You can check the resolution of the encoded file with ffprobe.

• The default level of HEVC encoder is an invalid value: 255. Better to have a new option "auto=0" and let the mpp runtime to handle the default level of the H26x encoders.

It seems that my experience with desktop GPUs doesn't fully apply to Arm/Rockchip. As you said, forcing DRMPRIME does not improve performance. I also couldn't find any encoder preset to trade off between speed and quality. So the claimed encoding speed of 8k30 cannot be equivalent to single 4k120 or 1080p480. Maybe in parallel encoding it will work.

As for the HEVC encoder of rk3588, it doesn't support NV15 input, which means it cannot encode Main 10 profile/10-bit video, maybe they will add it in the next generation of HW. But for now it's best to remove it so as not to confuse users.

[hbiyik]

thanks for the detailed feedback, all is fixed at latest: cf6e176,

about profile 8.5:
profile value 255 was so called profile 8.5 for hevc which is a special profile that enforces no limitation. I was sceptic about the h265 parser in mpp so i used that one, but now changed to default 0.

my understanding of this is that, only way to trade of speed is to reduce the input size. I think there is a constant pixel per cycle process rate. but i am also not sure, i have not yet benchmarked the encoder performance. And the best way to do it would be to first compare with mpi_enc_test things that come default with mpp.

open question: do you know any way to produce NV24, YUV444P, NV16, YUV422P, BGR24, YUYV422, UYVY422, BGRA, BGR0, NV12, YUV420P formatted DRM PRIME frames with ffmpeg so that i can push them to encoder.

currently i can only test NV12, NV16, YUV420P (rkmpp_decoder) & BGR0 (kmsgrab) drm prime frames and rest is not tested due to lack of input.

[veldspar]

alright, I checked out your git, I did modify the configure line a little from your wiki however:

./configure --enable-rkmpp --enable-version3 --enable-lib
drm --enable-nonfree --enable-gpl --enable-version3 --enable-libx264 --enable-librtmp --enable-shared --enable-static --enable-libx265 --enable-libmp3lame --enable-libpulse --enable-openssl --enable-libopus --enable-libvorbis --enable-libaom --enable-libass --enable-libdav1d --enable-libx265 --enable-libvpx

I used that configure line because i also used it for jjm2473's fork of rkmpp enabled ffmpeg, so i have a comparison. the build went on clean, no errors from the first try on, however a warning during linking.

After build it complained about missing libavdevice.so.60 on first try(ffmpeg --encoder) - I found it in the libavdevice subfolder of your git pull. After that it comlained about all the rest of the common libs being missing one by one. simple guess would be that I didnt install ffmpeg. after adjusting my LD_LIBRARY_PATH for testing ffmpeg loads.

so far so good. ffmpeg -encoders | grep rk lists the h264, hevc and vp8 encoders
ffmpet -decoders | grep rk lists h263, h264, hevc, mpeg1/2/4, vp8 and vp9 hardware decoders. I'm starting to like this.

Now lets give it a try - the goal is to trancsode.
./ffmpeg -i in.mkv -c:v hevc -c:a copy /extern/nn.hevc.mp4

and i get a segfault. same goes when trying to encode to h264

EDIT: I tried a clean build with the configure line from the git repos wiki, but i also keep getting segfault when trying to transcode a video, doesnt matter whether i transcode to h264 or hevc. Input in all cases has been h264 full hd

As for the linker warning, this is what i get:
LD ffprobe_g /usr/bin/ld: /lib/aarch64-linux-gnu/libtirpc.so.3: warning: common of rpc_createerr@@GLIBC_2.17' overridden by definition from /lib/aarch64-linux-gnu/libc.so.6
/usr/bin/ld: /lib/aarch64-linux-gnu/libtirpc.so.3: warning: common of rpc_createerr@@GLIBC_2.17' overridden by definition from /lib/aarch64-linux-gnu/libc.so.6 /usr/bin/ld: /lib/aarch64-linux-gnu/libtirpc.so.3: warning: common of rpc_createerr@@GLIBC_2.17' overridden by definition from /lib/aarch64-linux-gnu/libc.so.6
STRIP ffplay
`

Tiny attachment: I had installed a current mpp, however that got installed into /usr/local/lib and the system mpp was used instead. turns out your ffmpeg isnt compatible with the mpp that comes from radxas repo. a simple override using LD_LIBRARY_PATH fixed that issue, and the segfault is gone. Might be worth adding that to the wiki.