Imx8mplus h264 video encoder on mainline linux >= 6.17

We are creating a camera application that shall record high-resolution videos from Sony sensors (imx219 as a well-tested baseline for now).
Based on the latest yocto build, we have a mainline kernel (6.17) and libcamera (0.5.2) controlling the rkisp1 ISP for camera control. Besides, mesa 25 & libteflon are confirmed working for NPU support.

I saw that the hantro-vpu video decoder blocks have been mainlined, but in order to save videos, we need an efficient way for encoding data. What is the prospect of supporting the Hantro VC8000E video encoder on either the current kernel or a mainline version of 6.12?

The limiting factor is lack of support for libcamera on the 6.12 nxp kernel - there is a very old version bound to a custom nxp driver for the rkisp1 that is incompatible with the mainline implementation.

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.


Hantro VC8000E Video Encoder Support on Toradex Platforms

Based on the available Toradex documentation, I can provide some context about video encoding capabilities, though specific information about the Hantro VC8000E encoder on mainline kernels is not explicitly covered in the current sources.

Current Video Encoding Support

The Toradex documentation shows that video encoding is supported through GStreamer pipelines using hardware acceleration elements(1). For encoding, the recommended approach uses elements like vpuenc_h264 for H.264 encoding with VPU acceleration(1).

Here’s an example encoding pipeline from the documentation:

# gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! videoconvert ! vpuenc_h264 ! video/x-h264 ! h264parse ! qtmux ! filesink location=videotestsrc.mp4

(1)

VPU Support Challenges

The community discussions reveal some important limitations regarding VPU support on Toradex platforms. For instance, there are known performance issues with VPU acceleration on certain modules like the iMX8DX with Torizon OS 6, where users experience high CPU usage and poor performance when hardware acceleration isn’t properly utilized(2).

Additionally, there’s an important note from Toradex support indicating that “at the current moment there is no official VPU support for any SOM on Torizon OS 7.X”(2), though they do plan to implement this feature.

Recommendations for Your Use Case

Given your specific requirements for:

  • High-resolution video recording from Sony sensors
  • Mainline kernel (6.17) with libcamera
  • Efficient encoding capabilities

The current Toradex documentation suggests using GStreamer with available hardware acceleration elements. You can also try the encodebin element for automatic pipeline creation:

# gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! encodebin ! filesink location=videotestsrc.mp4

(1)

Next Steps

For specific information about Hantro VC8000E encoder support on mainline kernels 6.12+ and compatibility with your libcamera setup, I’d recommend checking the current Toradex BSP documentation and kernel configuration for your specific hardware platform, as this level of detail about future kernel support and specific encoder hardware isn’t covered in the available sources.

The GStreamer framework provides a flexible foundation for multimedia processing that can adapt to different hardware acceleration capabilities as they become available(1).

Best regards, Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

Hey, @svogl,

The mainline kernel seems to include some configuration regarding the encoder, but I couldn’t find any specific drivers for it. Furthermore, I tested a Gstreamer pipeline in the module and couldn’t use the encoder, so I think it’s really missing from the mainline, though we have it in the downstream kernel.

In any case, an upstream driver for the VC8000E would come from NXP, not us. I suggest you ask about it in an NXP forum.

Best regards,