Hardware video encoding on Verdin iMX8MP

[j.kugele]

Hi, I am currently trying to get the AR0521 camera running on a Verdin iMX8MP on a Dahlia carrier board. I got the camera raw stream running, but as soon as I use the hardware video encoder or decoder in a gstreamer call, the pipeline fails immediately with this error message

vpuenc_h264-0: Could not initialize supporting library

I also saw this post from roughly 2 years ago, which mentions the same issue:

Verdin iMX8MP Gstreamer issue with encoder and decoder

Hi @dandresky ! There was indeed some problem with Gstreamer and Verdin iMX8M Plus recently: Toradex System/Computer on Modules - Linux BSP Release, ELB-4699 I just installed the latest image available: root@verdin-imx8mp-06849036:~# tdx-info Software summary ------------------------------------------------------------ Bootloader: U-Boot Kernel version: 5.15.77-6.1.0-devel+git.bbbc03a7f625 #1 SMP PREEMPT Wed Dec 7 16:40:51 UTC 2022 Kernel command line: root=PARTU…

As the hardware video encoder and decoder is advertised on the Toradex website, I hope somebody can help me to get it running?
Could you unlock me for uploading files, then I can share how I created the Docker image for the camera?

[ToradexAI]

Hi, I’m the Toradex AI Assistant. The Toradex Support team will follow up with you shortly!

Did you like my answer?

---

Hello,

It sounds like you’re encountering a challenge with the hardware video encoder/decoder on your Verdin iMX8MP when using GStreamer. The error message “vpuenc_h264-0: Could not initialize supporting library” typically indicates an issue with the necessary libraries or configurations.

Here are some steps you can take to troubleshoot and potentially resolve this issue:

1. Update Software: Ensure you are using the latest version of TorizonCore and any related containers. Updates often include important fixes.

2. GStreamer Plugins: Verify that all required GStreamer plugins are installed in your Docker container. The vpuenc_h264 element needs specific plugins that might not be included by default.

3. Library Dependencies: Check for any missing library dependencies. Review your Dockerfile to ensure all necessary packages for the encoder/decoder are included.

4. Device Tree Overlays: Make sure the correct device tree overlay is applied. For the AR0521 camera, use the verdin-imx8mp_ar0521_overlay.dts to ensure proper hardware configuration.

5. Known Issues: Look into any known issues or errata related to the Verdin iMX8MP and Dahlia carrier board. There might be specific workarounds or patches available.

6. Simplify the Pipeline: Try running a simpler GStreamer pipeline to isolate the problem. This can help determine if the issue lies with the encoder/decoder or another part of the pipeline.

I hope these suggestions help you get the hardware encoder/decoder running smoothly. If you have further questions, feel free to ask.

Best regards,
Toradex AI Assistant

Please, also check if these links can help you solve your problem:

AR0521 Camera Firmware Issue

Hello dear community, I am currently trying to set up e-con Systems camera “5MP AR0521 Color” via MIPI CSI interface with a Toradex SoM. My issue is regarding camera firmware errors and missing recognition of the camera by the device. Here is my setup: Verdin IMX8M Plus on Dahlia Carrier Board Yocto-built tdx-reference-multimedia image (OS) with added bitbake layer for camera driver and device tree overlays (as described at First Steps with CSI Camera Set 5MP AR0521 Color (Linux) | Toradex D…

Verdin iMX8MP Gstreamer issue with encoder and decoder

I am trying to run a gstreamer pipeline on the Verdin iMX8MP mounted on the Dahlia Carrier Board. However, when running the pipeline, I am coming up against the error “vpuenc_h264-0: Could not initialize supporting library”. I built a custom Yocto image using the tdx-reference-multimedia-image and a few minor customizations. The pipeline I am trying to run: gst-launch-1.0 videotestsrc ! video/x-raw, format=I420, width=640, height=480 ! vpuenc_h264 ! filesink location=videotestsrc.mp4 I also …

Verdin iMX8MP GStreamer Hardware Decoder/Encoder.

[bruno.tx]

Hello @j.kugele,

Which variant of the Verdin iMX8MP are you using?
The output of sudo tdx-info should contain all the necessary information.

Best Regards,
Bruno

[j.kugele]

Hi,
today I built the Docker image of the vpuGeneric sample again from the newest commit of the bookworm-new branch. This time it went through without any modifications.

The camera works, but using the video encoder I still get the same error:

# update-alternatives --set libg2d.so.1 /usr/lib/aarch64-linux-gnu/libg2d-viv.so.2.1.0
# gst-launch-1.0 videotestsrc num-buffers=300 ! video/x-raw, width=1920, height=1080, framerate=60/1 ! videoconvert ! vpuenc_h264 ! video/x-h264 ! h264parse ! qtmux ! filesink location=videotestsrc.mp4
Setting pipeline to PAUSED ...
====== VPUENC: 4.7.2 build on Jan  8 2025 10:48:49. ======
        wrapper: 3.0.0 (VPUWRAPPER_ARM64_LINUX Build on Jan  8 2025 10:47:54)
        vpulib: 1.1.1
        firmware: 1.1.1.0
ERROR: from element /GstPipeline:pipeline0/vpuenc_h264:vpuenc_h264-0: Could not initialize supporting library.
Additional debug info:
../gst-libs/gst/video/gstvideoencoder.c(1804): gst_video_encoder_change_state (): /GstPipeline:pipeline0/vpuenc_h264:vpuenc_h264-0:
Failed to start encoder
ERROR: pipeline doesn't want to preroll.
Failed to set pipeline to PAUSED.
Setting pipeline to NULL ...
Freeing pipeline ...

sudo tdx-info
Password: 

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           5.15.129-6.5.0-devel+git.6f8fd49366db #1-TorizonCore SMP PREEMPT Mon Dec 23 14:11:20 UTC 2024
Kernel command line:      root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/a992185cd12dfcb0b279996dd90e68a45a466dfb41c80243d0c1e8940eff155c/0
Distro name:              NAME="TorizonCore"
Distro version:           VERSION_ID=6.5.0-devel-20241225134819-build.0
Distro variant:           VARIANT="Docker"
Hostname:                 verdin-imx8mp-15503045
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin iMX8M Plus WB on Verdin Development Board
Toradex version:          0058 V1.1B
Serial number:            15503045
Processor arch:           aarch64
------------------------------------------------------------

I am currently using self-built Torizon OS 6.5.0 with the AR0521 Yocto overlay. Which one is the latest version where video encoding was confirmed working?

[j.kugele]

PS: It seems that multiple of the NXP provided libraries have undefined referenced symbols. Is this normal?

ldd -r /usr/lib/libfslvpuwrap.so.3.0.0
        linux-vdso.so.1 (0x0000ffff92c78000)
        libcodec.so.1 => /lib/libcodec.so.1 (0x0000ffff92bd0000)
        libhantro_vc8000e.so.1 => /lib/libhantro_vc8000e.so.1 (0x0000ffff92b60000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff929b0000)
        libhantro.so.1 => /lib/libhantro.so.1 (0x0000ffff92960000)
        libg1.so.1 => /lib/libg1.so.1 (0x0000ffff92810000)
        /lib/ld-linux-aarch64.so.1 (0x0000ffff92c3b000)
undefined symbol: pow   (/lib/libhantro_vc8000e.so.1)
undefined symbol: sqrt  (/usr/lib/libfslvpuwrap.so.3.0.0)

ldd -r /usr/lib/libhantro_vc8000e.so.1
        linux-vdso.so.1 (0x0000ffff869b9000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff86760000)
        /lib/ld-linux-aarch64.so.1 (0x0000ffff8697c000)
undefined symbol: pow   (/usr/lib/libhantro_vc8000e.so.1)

[bruno.tx]

Hello @j.kugele,

Thanks for the additional information.
The 0058 Verdin iMX8M Plus Quad 4GB WB IT module has a working VPU, so we can look into what else may be the issue here.

Which command are you using to run the container?

With the following command, I can get the VPU encoding working using the same pipeline that you used:

docker run --rm -it -e MACHINE=verdin-imx8mp -v /tmp:/tmp -v /var/run/dbus:/var/run/dbus -v /dev:/dev -v /sys:/sys --device-cgroup-rule='c 226:* rmw' --device-cgroup-rule='c 199:* rmw' --device-cgroup-rule='c 81:* rmw' --device-cgroup-rule='c 234:* rmw' --device-cgroup-rule='c 235:* rmw' --device-cgroup-rule='c 253:* rmw' <container name>

This command has a few additional cgroup rules which are needed for the VPU.
I will add them to the developer website page as well.

Best Regards,
Bruno

[j.kugele]

Hi Bruno,
thanks with this command I can use the hardware video encoder/decoder. Is there some way to detect if cgroup rules are missing, or how did you find that out?

For reference for other users, here are some usefull gstreamer pipelines:

Record valid .mp4 file:

gst-launch-1.0 v4l2src device='/dev/video2' num-buffers=300 ! "video/x-raw, format=RGB16, framerate=30/1, width=1920, height=1080" ! videoconvert ! vpuenc_h264 ! video/x-h264 ! h264parse ! qtmux ! filesink location=videotestsrc.mp4

Playback file (gstreamer automatically uses hardware H264 decoding):

gst-launch-1.0 filesrc location=videotestsrc.mp4 ! decodebin ! fpsdisplaysink video-sink=waylandsink text-overlay=false sync=false

Send MPEGTS network stream to host computer:

gst-launch-1.0 v4l2src device='/dev/video2' ! "video/x-raw, format=RGB16, framerate=30/1, width=1920, height=1080" ! videoconvert ! vpuenc_h264 ! video/x-h264 ! h264parse config-interval=-1 ! mpegtsmux ! udpsink host=<host ip> port=5000

To playback with vlc open “udp://@:5000” (host firewall might need to be adjusted)

[bruno.tx]

Hello @j.kugele,

Thanks for the update.

@j.kugele

Is there some way to detect if cgroup rules are missing, or how did you find that out?

To find out which device cgroup rules are needed for each device you can check the output of ls -la /dev. This is described in the following page: Best Practices with Peripheral Access | Toradex Developer Center

Otherwise, to find which cgroups are needed, it will depend on the task and which devices are needed.
Initially I added the relevant mxc devices’ cgroup rules and afterwards searched for the missing one by trying different configurations.
There may be a better way to do this, but this was simple enough.

Best Regards,
Bruno