Hi Jeremias,
Thank you for the feedback.
After several tests, I found out that the issue was actually related to the camera itself. I was initially using a USB camera with an integrated microphone and autofocus (not sure if those features are relevant to the problem). After switching to another camera, I was able to achieve the required frame rate without any problems — decoding MJPEG at 60 FPS, 720p resolution.
Additionally, hardware-accelerated MJPEG decoding worked perfectly using v4l2jpegdec. With a monochrome camera I tested, I even reached 120 FPS at the same resolution.
Regarding the previous camera, I noticed several QoS warnings in the logs. Regardless of the video sink used, the frame rate was limited to around 8 FPS. When using autovideosink with sync=false, I could get up to 16 FPS. However, using applications like Cheese on Linux or the Camera app on Windows, I had no problem achieving the expected 30 FPS.
Do you have any suggestions regarding this? It seems to be a limitation or configuration issue in GStreamer, as I could reproduce the exact same behavior on my notebook — both with GStreamer and FFplay (16 FPS with sync=false, and 8 FPS otherwise).
I’m currently facing some performance issues when converting video formats. Using videoconvert or autovideoconvert leads to significant performance drops. While researching, I came across the i.MX8 GStreamer User Guide and found that the plugin imxvideoconvert_g2d should provide hardware-accelerated color space conversion.
However, when I try to use it, I get the following error:
g2d_open: 2D/VG PIPE not found!
g2d_open: fail with status -13
I tried modifying the Dockerfile I previously shared to manually download and build the following components:
i* mx-parser
# imx-codec
RUN wget -q https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/imx-codec-4.7.2.bin \
&& chmod +x imx-codec-4.7.2.bin \
&& ./imx-codec-4.7.2.bin --auto-accept \
&& cd imx-codec-4.7.2 \
&& ./configure --enable-armv8 --disable-vpu --prefix=/usr \
--libdir=/usr/lib/aarch64-linux-gnu --includedir=/usr/include \
&& make -j${nproc} \
&& make install DESTDIR=/vpusysroot \
&& rm -r /vpusysroot/usr/lib/aarch64-linux-gnu/imx-mm/video-codec \
&& mv /vpusysroot/usr/lib/aarch64-linux-gnu/imx-mm/audio-codec/lib* \
/vpusysroot/usr/lib/aarch64-linux-gnu/ \
&& cp -r /vpusysroot/* / \
&& cd .. && rm -rf imx-codec-4.7.2.bin imx-codec-4.7.2
# imx-parser
RUN wget -q https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/imx-parser-4.7.2.bin \
&& chmod +x imx-parser-4.7.2.bin \
&& ./imx-parser-4.7.2.bin --auto-accept \
&& cd imx-parser-4.7.2 \
&& ./configure --enable-armv8 --prefix=/usr \
--libdir=/usr/lib/aarch64-linux-gnu --includedir=/usr/include \
&& make -j${nproc} \
&& make install DESTDIR=/vpusysroot \
&& cp -r /vpusysroot/* / \
&& cd .. && rm -rf imx-parser-4.7.2.bin imx-parser-4.7.2
RUN wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/imx-vpu-hantro-1.27.0.bin && \
chmod +x imx-vpu-hantro-1.27.0.bin && \
./imx-vpu-hantro-1.27.0.bin --auto-accept && \
cd imx-vpu-hantro-1.27.0 && \
make -j$(nproc) PLATFORM=IMX8QXP all && \
libdir=/usr/lib/aarch64-linux-gnu/ make DEST_DIR=/vpusysroot PLATFORM=IMX8QXP install && \
cd .. && \
cp -r /vpusysroot/* / && \
rm -rf imx-vpu-hantro-1.27.0.bin imx-vpu-hantro-1.27.0 /vpusysroot && \
\
wget https://www.nxp.com/lgfiles/NMG/MAD/YOCTO/imx-vpu-hantro-vc-1.9.0.bin && \
chmod +x imx-vpu-hantro-vc-1.9.0.bin && \
./imx-vpu-hantro-vc-1.9.0.bin --auto-accept && \
mkdir -p /vpusysroot/usr/include/ /vpusysroot/usr/lib/aarch64-linux-gnu/ && \
cp -r imx-vpu-hantro-vc-1.9.0/usr/include/* /vpusysroot/usr/include/ && \
cp -r imx-vpu-hantro-vc-1.9.0/usr/lib/* /vpusysroot/usr/lib/aarch64-linux-gnu/ && \
cp -r /vpusysroot/* / && \
rm -rf imx-vpu-hantro-vc-1.9.0.bin imx-vpu-hantro-vc-1.9.0 /vpusysroot
RUN git clone https://github.com/nxp-imx/imx-vpuwrap.git && \
cd imx-vpuwrap && \
git checkout MM_04.07.02_2210_L5.15.y && \
autoreconf -Wcross --verbose --install --force && \
./configure --prefix=/usr --libdir=/usr/lib/aarch64-linux-gnu --disable-static && \
make -j$(nproc) && \
make install DESTDIR=/vpusysroot && \
cp -r /vpusysroot/* / && \
rm -rf /vpusysroot/usr/share && \
cd .. && \
rm -rf imx-vpuwrap /vpusysroot
# imx-gst1.0-plugin
RUN git clone https://github.com/nxp-imx/imx-gst1.0-plugin.git \
-b MM_04.07.02_2210_L5.15.y \
&& cd imx-gst1.0-plugin \
&& meson -Dplatform=MX8 --prefix=/usr --libdir=/usr/lib/aarch64-linux-gnu \
-Dc_args="-I/usr/include -I/usr/include/imx" build \
&& ninja -v -j$(nproc) -C build \
&& DESTDIR=/vpusysroot ninja -v -j$(nproc) -C build install \
&& cp -r /vpusysroot/* / \
&& cd .. && rm -rf imx-gst1.0-plugin
I attempted different approaches, including forcing PLATFORM=IMX8QXP and testing variations with or without the -vc variant of the VPU drivers. Unfortunately, I couldn’t get the G2D-based plugins (imxvideoconvert_g2d, imxcompositor_g2d) to work.
Interestingly, I had access to another board based on the i.MX8DX with the latest Yocto BSP and the Reference Multimedia Image. On this device, the G2D-based plugins worked flawlessly — both imxvideoconvert and imxcompositor load and function without issues. That leads me to suspect it could be a kernel module, driver, or device node issue.
I’m also able to access the GPU through OpenGL, so it doesn’t seem to be a general GPU problem.
Here’s the relevant part of my docker-compose.yml setup, in case it’s related to permissions or device access:
version: '3.8'
services:
hud:
image: luccasparentex/hud:v1_t6_modified
container_name: hud
privileged: true
network_mode: host
cap_add:
- SYS_TTY_CONFIG
volumes:
- /dev:/dev
- /tmp:/tmp
- /run/udev/:/run/udev/
device_cgroup_rules:
- 'c 4:* rmw'
- 'c 253:* rmw'
- 'c 13:* rmw'
- 'c 226:* rmw'
- 'c 10:223 rmw'
- 'c 199:0 rmw'
Do you have any suggestions on what I could try? If you can share any reference on how the video acceleration components were set up in the original Dockerfile, I could check if there’s something similar for the NXP i.MX8DX. I’m not sure if this is a kernel, driver, or configuration issue. I’ve also tried running with sudo, but perhaps something like a missing cgroup_rule or permission is still blocking it.
Any suggestions or insights would be greatly appreciated.
Best regards
