Imx6s colibri - Display with GPU yocto

Technical Support
1

Cloned yocto project, built the tdx multimedia reference image, I successfully enabled HDMI, and rendered video. Using mainline tree, 6.6

gst-launch-1.0 -v filesrc location=BigBuckBunny.mp4 ! qtdemux name=d d.video_0 ! queue ! h264parse ! v4l2h264dec ! videoconvert ! fbdevsink.

The above command worked, but the video rendering is too slow, Core & RAM usage is 100%.
I assume VPU working (so file decoding is good) but GPU not working. On studying further, it took me to various directions, vivanta, egl, etc… But, not ended with solution.

If similar issue already identified, and is there any ready solution to try?

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.

I’d be glad to help troubleshoot your video rendering performance issue! To provide the most accurate solution, could you let me know which hardware platform and OS version you’re using? You can run the tdx-info command to get detailed information about your setup - see the tdx-info documentation for details.

Performance Issue Analysis

Your pipeline is using software-based video conversion and display, which explains the high CPU/RAM usage. The issue is likely that you’re using fbdevsink instead of hardware-accelerated display sinks(1).

Recommended Solutions

1. Use Hardware-Accelerated Display Sink

Instead of fbdevsink, try using waylandsink for better GPU utilization(1):

gst-launch-1.0 filesrc location=BigBuckBunny.mp4 ! qtdemux ! h264parse ! vpudec ! waylandsink

(1)

2. Alternative Pipeline with Hardware Acceleration

You can also use the decodebin element which automatically creates a decoding pipeline(1):

gst-launch-1.0 filesrc location=BigBuckBunny.mp4 ! decodebin ! waylandsink

(1)

3. Remove Software Video Conversion

The videoconvert element in your current pipeline forces software processing. The hardware-accelerated decoders can output directly to the display sink without this conversion step(1).

Key Points for Hardware Acceleration

  • VPU Decoder: Use vpudec instead of v4l2h264dec for better hardware acceleration(1)
  • Display Sink: Use waylandsink which works with the Weston Compositor for hardware-accelerated display(1)
  • Remove Software Steps: Eliminate videoconvert to avoid forcing software processing(1)

The waylandsink displays raw video frames on a Wayland display server (Weston) and should significantly reduce your CPU usage while providing proper GPU utilization(1).

Best regards, Toradex AI Assistant.

Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

3

Firstly, after boot - the weston desktop grey UI appeared in the screen. But sametime, the kmesg shown oom error, and it was retrying. I clearly understood, weston was eating the ram space, and hence disabled weston service at begining. So, wayland cannot run.

image
image1872×717 101 KB

But, I think, weston is also hanging, because it was not able to utlize the GPU.

4

Understanding the OOM Issue

Your OOM (Out of Memory) error with Weston suggests a fundamental issue with GPU initialization or configuration. This is commonly seen when the GPU drivers aren’t properly loaded or configured, forcing Weston to fall back to software rendering which consumes excessive RAM.

Alternative Display Solutions

Since Weston is causing memory issues, you have several alternatives for video playback on your Yocto BSP system:

1. X-Server with Hardware Acceleration

From BSP 5 onward, Wayland/Weston is the default, but X-Server is still supported(1). For i.MX6-based modules, the vivante DDX driver provides hardware acceleration(1):

# Check your current X-Server configuration
cat /etc/X11/xorg.conf

(1)

The default configuration selects the vivante driver for i.MX6 modules, which should provide better GPU utilization than software rendering(1).

2. Direct Framebuffer with Hardware Decoding

Since your VPU decoding is working, you can continue using the framebuffer approach but optimize the pipeline:

# Try this optimized pipeline
gst-launch-1.0 filesrc location=BigBuckBunny.mp4 ! qtdemux ! h264parse ! vpudec ! fbdevsink

3. Troubleshooting GPU Issues

Before re-enabling Weston, check if your GPU drivers are properly loaded:

# Check for GPU device nodes
ls -la /dev/dri/

# Check loaded modules
lsmod | grep -i gpu
lsmod | grep -i vivante

# Check kernel messages for GPU initialization
dmesg | grep -i gpu
dmesg | grep -i vivante

Memory Optimization

To address the OOM issue when you do use Weston:

  1. Reduce color depth: The default X-Server configuration uses 16-bit color depth to save memory bandwidth(1)
  2. Check available memory: Ensure you have sufficient RAM for your use case
  3. Monitor GPU memory usage: Some GPU memory might be reserved and not visible to the system

Next Steps

  1. First, verify your hardware setup using tdx-info as mentioned earlier
  2. Check if GPU drivers are properly initialized
  3. Try the optimized GStreamer pipeline with vpudec
  4. If Weston is required, investigate the GPU initialization failure before re-enabling it

The key is ensuring hardware acceleration is working at both the decode (VPU) and display (GPU) levels to avoid the software fallback that’s causing your performance issues.

Best regards, Toradex AI Assistant.

Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.