High cpu usage on VPU encoder with Verdin IMX8MP

Technical Support
, ,
1

I am using the VPU on the verdin imx8mp to encode a video stream from the fsm:go framos imx662 camera with 10fps via a gstreamer pipeline. The pipline looks roughly as follows:

appsrc name=source ! queue max-size-buffers=2 ! vpuenc_h264 bitrate=4000 gop-size=30 ....

Here I provide the appsrc with the camera frames in NV12 format.

I also verified the high cpu usage is not coming from anything before and I can see a big increase in cpu usage when vpuenc_h264 is part of the pipline compared to when it is not.

My question now is how can i mitigate this high cpu usage? Htop shows roughly 70% cpu usage on only 10fps 1920x1080. The image frame passing from framos camera to the appsrc is also zero-copy. So I am wondering if there is maybe some colorspace conversion done within my pipline still on cpu side. Do I maybe have to pass another format to the vpu unstead of NV12?

This is on Torizon 6, since the VPU did not seem to work on Torizon 7.

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.

VPU CPU Usage Analysis for Verdin iMX8M Plus

Based on the documentation and your setup with the Framos FSM:GO IMX662 camera on Verdin iMX8M Plus with Torizon 6, here are several factors that could contribute to high CPU usage:

Pipeline Optimization Recommendations

For the Verdin iMX8M Plus, the recommended GStreamer pipeline structure for hardware-accelerated encoding includes specific elements(1):

gst-launch-1.0 videotestsrc ! videoconvert ! vpuenc_h264 ! video/x-h264 ! h264parse ! qtmux ! filesink location=videotestsrc.mp4

(1)

Potential CPU Usage Sources

  1. Format Conversion: While NV12 is typically optimal for VPU encoding, ensure no unnecessary format conversions are happening. The videoconvert element might be performing CPU-based conversions(2).

  2. Buffer Management: Your current queue configuration (max-size-buffers=2) might be causing buffer management overhead. Consider adjusting buffer sizes or using hardware buffer pools.

  3. VPU Configuration: The VPU encoder parameters like bitrate=4000 and gop-size=30 should be optimized for your use case.

Debugging Steps

To identify the CPU usage source, you can use GStreamer debugging tools(2):

export GST_DEBUG=2

(2)

This will show warning and error messages that might indicate where CPU-intensive operations are occurring.

Torizon OS 7 VPU Status

Regarding VPU support in Torizon OS 7, the documentation indicates that there is currently no official VPU support for any SoM on Torizon OS 7.x, though this is planned for future development(3). This confirms your observation about VPU functionality in Torizon 7.

Container Configuration

Ensure your container has proper access to VPU devices and is using the appropriate base image. For hardware-accelerated multimedia on iMX8M Plus, the wayland-base-vivante container image is recommended(3).

Next Steps

  1. Add debugging output to identify bottlenecks
  2. Experiment with different buffer configurations
  3. Verify that hardware acceleration is actually being used by checking /dev/video* devices
  4. Consider pipeline restructuring to eliminate unnecessary conversions

Could you provide the exact version of Torizon OS 6 you’re running? This will help provide more specific guidance for your setup.

Best regards, Toradex AI Assistant.

Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

3

Dear Marcel,

Thanks for reaching out to Toradex.

Could you please share more details?

The following questions will help us better understand your case.

  1. Can you provide your entire pipeline for evaluation?
  2. Considering your approach, is there a reason you are using max-size-buffers=2? Both threads will wake and sleep frequently - verify if tune=zerolatency option is available and vpuenc is using the correct driver.
  3. Is appsrc output matching vpu input?

Have you defined specifically the caps? e.g: appsrc ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! vpuenc_h264

I’m looking forward to your reply.

Best regards,

Henrique Freitas
Toradex