Add Nvidia Jetson support

[mguzzina]

Changes
I've added the support to build ffmpeg natively on arm64., then added the jetson-ffmpeg patch and the support for the nvidia image version I'm currently using. I've tested it by re-encoding the (relevant) jellyfin test videos on a Jetson NX.

Issues
I guess this is related to https://features.jellyfin.org/posts/1118/add-jetson-ffmpeg-nvmpi-support

[nyanmisaka]

First of all, thank you for your efforts on Jetson. From my understanding, you added a builder that only runs in a local Jetson Jetpack 5 environment, which prevents it from being built in GH actions. Furthermore, Jetpack 5 will reach its EOL in Q3 2026, and its underlying Ubuntu 20.04 reached its EOL last year, meaning package availability and security will be compromised.

Currently, the older Jetson models available at reasonable prices with hardware encoders — Nano, TX1/TX2 and Xavier/Xavier NX are not supported by Jetpack 6. Jetson Nano, TX1/TX2 even rely on Jetpack 4, an ancient version based on Ubuntu 18.04. The newer entry-level Jetson Orion Nano, which is supported by Jetpack 6, doesn't include a hardware encoder, which is embarrassing.

This raises the question of whether it's worthwhile to invest our spare time in the Jetson platform. It's not simply about patching ffmpeg, it involves needing us to rewrite patches to support hardware contexts (DMA-BUF & NvBuffer) within ffmpeg and hardware filters to support CUDA or hardware ISPs, which is the baseline for other HWA methods we support. Otherwise, Jetson users will encounter performance issues and start complaining about Jellyfin:

Q: why does enabling hardware transcoding still consume a lot of CPU resources and even unable to keep up with real-time speed?

• A1: 10bit (h265) to 8bit (h264) pixel format conversion depends on the CPU.
• A2: Scaling video size depends on CPU.
• A3: De-interlacing filters depend on the CPU.
• A4: HDR to SDR tone-mapping depends on the CPU.
• A5: Subtitle burn-in depends on CPU.

This is also where you leave a TODO in your Jellyfin server PR. You can't skip them and claim you support hardware transcoding, especially on ARM platforms with limited CPU performance.

Therefore, I suggest you fork and maintain it downstream to meet your minimum needs. If NVIDIA can bring back the hardware encoder on entry-level Jetsons in the future and support the video-codec-sdk, we would be happy to reconsider it (they have already done so recently on the high-end Jetson Thor platform).

It would be great if @gnattu could leave some comments as well.

[mguzzina]

First of all, thank you for your efforts on Jetson.

You're very welcome :)

From my understanding, you added a builder that only runs in a local Jetson Jetpack 5 environment, which prevents it from being built in GH actions. Furthermore, Jetpack 5 will reach its EOL in Q3 2026, and its underlying Ubuntu 20.04 reached its EOL last year, meaning package availability and security will be compromised.

If being built in GH actions is something you'd like I can try to set it up, it seemed more work than worth, especially since I'm basically supporting an already dead OS version. It might make sense to support either Jetpack 6 or 7, however I have no way of testing those as I only have a Xavier NX.

Jetson users will encounter performance issues and start complaining about Jellyfin:

I'm not sure at all about this, so correct me if I'm wrong, but isn't everything currently already on CPU? If that is the case then I don't see why adding codecs support would affect users complaining about slow transcoding performance as it is already slow.

This is also where you leave a TODO in your Jellyfin server PR. You can't skip them and claim you support hardware transcoding, especially on ARM platforms with limited CPU performance.

That is more of a skill issue on my part, I don't really know enough about ffmpeg and jellyfin to handle filters. It made more sense to first check on how you feel about Jetson support in general before getting into the actual hard stuff.

Therefore, I suggest you fork and maintain it downstream to meet your minimum needs.

Maybe "needs" is too strong a word as I did this for fun more than necessity, it seemed an interesting challenge and a good opportunity to learn. I'm happy to submit any change you're also happy with, if it doesn't work out then at least I tried.

If NVIDIA can bring back the hardware encoder on entry-level Jetsons in the future and support the video-codec-sdk, we would be happy to reconsider it (they have already done so recently on the high-end Jetson Thor platform).

One can hope, however I wouldn't hold my breath, to my knowledge they still haven't even announced the Nano/NX tier of Thor/Blackwell.

Thank you for your time and your comprehensive reply.

[nyanmisaka]

If being built in GH actions is something you'd like I can try to set it up, it seemed more work than worth, especially since I'm basically supporting an already dead OS version.

The difficulty in integrating with upstream components, and the need to create a separate Docker image for each Jetpack version outside of arm64, is enough to keep it in the downstream for now, unless its transcoding performance justifies it.

I'm not sure at all about this, so correct me if I'm wrong, but isn't everything currently already on CPU? If that is the case then I don't see why adding codecs support would affect users complaining about slow transcoding performance as it is already slow.

People already have not too much expectations regarding the transcoding performance of ARM processors on SBCs, and our documentation already states that running Jellyfin video transcoding on an ARM SBC is not recommended unless it's running on a specific ARM SBC with well-implemented hardware transcoding.

If we prematurely add an ARM platform without full hardware transcoding support, users will have overly high expectations and might even start buying them to set up their home servers. However, we won't be able to provide them with effective technical support. This is not desirable.

Therefore, the immature fork should be left for downstream development until its hardware transcoding capabilities reach the same level as other HWA methods before we promote it.

[mguzzina]

Therefore, the immature fork should be left for downstream development until its hardware transcoding capabilities reach the same level as other HWA methods before we promote it.

If I were to support filters as well would you be more inclined to merge it or do you think it would sill be too impractical to merge? Would there be any more requirement I'd need to fulfill?

I'm thinking about going further but I don't know if I'll be able to get Jetson level to a satisfactory level, I'd just like a list of required features (unless the Q/A section of your previous comment is that already) so I know what to expect.
Are transcoding capabilities enough to merge such a feature or is Jetson just not worth it regardless in your opinion?

[nyanmisaka]

If I were to support filters as well would you be more inclined to merge it or do you think it would sill be too impractical to merge?

Someone needs to add hwcontext_nvmpi support for it, you can find something similar about hwcontext in the ffmpeg source code. It also needs to support filters like {scale,overlay,yadif,tonemap}_cuda, or their nvmpi variants. Here is a list of hardware filters we used in other HWA methods.

If all the basic features used in Jellyfin hardware transcoding are supported in the future, we will review and merge them.

Are transcoding capabilities enough to merge such a feature or is Jetson just not worth it regardless in your opinion?

The hardware is qualified at least on Xavier NX, but software support in ffmpeg is currently very poor, as mentioned above.