Gstreamer and DMABuf for GPU-VPU operations

Technical Support
2

Hi, I’m the Toradex AI Assistant.

I might make mistakes. Our support team will step in if needed. If I answered your question, please tick the “Solution” checkbox. For follow-up questions to me, please use @ToradexAI in your post.

Did you like my answer? :+1: :-1:

GPU-to-VPU Pipeline Bottleneck on Verdin iMX8M Plus

The issue you’re experiencing is related to the memory transfer bottleneck between the GPU and VPU components. The Verdin iMX8M Plus supports hardware-accelerated graphics processing, but there are specific considerations for efficient pipeline configuration.

Graphics Backend Support

Toradex BSPs support Wayland as the graphics backend on i.MX 8 Series modules starting from BSP 5.0(1). Your current BSP 6.8.0 should have proper Wayland support for OpenGL ES acceleration(1).

Hardware Acceleration Validation

Toradex validates OpenGL ES using glmark2 benchmarks on the Verdin iMX8M Plus, achieving a score of 953 under specific test conditions(1). This confirms that GPU acceleration is functional on your hardware.

Pipeline Optimization Recommendations

  1. Direct Memory Buffer Usage: Your pipeline already uses io-mode=dmabuf for the v4l2src, which is good for avoiding memory copies.

  2. Format Considerations: The gldownload element falling back to SystemMemory suggests a format or memory allocation mismatch. Try using compatible formats that both GPU and VPU can handle efficiently.

  3. Alternative Pipeline Structure: Consider bypassing the GPU processing if it’s causing bottlenecks. For camera capture and encoding, a direct path might be more efficient:

gst-launch-1.0 -v v4l2src device=/dev/video2 ! video/x-raw ! videoconvert ! waylandsink

(2)

BSP Version Considerations

Based on community feedback, there are known issues with VPU encoding in BSP 7.x versions. A community discussion indicates that v4l2h264enc has problems in recent BSP versions, with users being advised to use BSP 6 until fixes are available(3).

Since you’re already on BSP 6.8.0, upgrading to BSP 7 might introduce additional VPU-related issues rather than solving your current problem.

Suggested Troubleshooting Steps

  1. Test VPU independently: Verify that v4l2h264enc works properly without GPU processing
  2. Check memory allocation: Use GST_DEBUG=*:3 to identify memory allocation issues
  3. Try alternative formats: Experiment with different video formats between gldownload and v4l2h264enc
  4. Consider direct encoding: If GPU processing isn’t essential, use direct camera-to-encoder pipeline for better performance

The Verdin iMX8M Plus features a Neural Processing Unit (NPU) and Video Processing Unit (VPU) designed to offload processing tasks(4), but efficient pipeline configuration is crucial for optimal performance.

Best regards, Toradex AI Assistant.