High processor consumption when playing video and streaming from the Internet

Technical Support
, , ,
1

Hello,

I have Apalis imx8 board, with ixora carrier board, and I installed Torizon os 7.1.0+build.4 (scarthgap).

I’m trying to play a video file through QML 6, but the processor consumption is around 90%, and when I try to stream from the internet, the processor consumption is 160%.

I tried to play the video using:
gst-launch-1.0 filesrc location=/home/torizon/test_video.mp4 ! decodebin name=dec ! videoconvert ! waylandsink
But the cpu consumption was around 74%.

I think it seems that the hardware video decoding is not working, or there is some other problem.

These are the packages that I added to my container:
"qt6-multimedia-dev", "qml6-module-qtmultimedia", "libqt6multimedia6", "gstreamer1.0-x", "qml-module-qtgstreamer", "gstreamer1.0-tools", "gstreamer1.0-plugins-base", "gstreamer1.0-plugins-good", "gstreamer1.0-plugins-bad", "gstreamer1.0-plugins-ugly", "libgstreamer1.0-dev", "libgstreamer-plugins-base1.0-dev", "libgstreamer-plugins-bad1.0-dev", "gstreamer1.0-alsa", "gstreamer1.0-gl", "gstreamer1.0-gtk3", "gstreamer1.0-pulseaudio", "gstreamer1.0-libav", "v4l-utils", "pulseaudio", "pulseaudio-utils", "gstreamer1.0-plugins-base-apps", "apulse", "libpulse-dev", "libasound2", "alsa-tools", "alsa-utils", "alsa-oss", "vlc", "qt6-webengine-dev", "libqt6webenginequick6", "qml6-module-qtwebengine", "libqt6webenginecore6", "libqt6webenginecore6-bin", "libqt6webenginewidgets6", "qt6-webengine-dev-tools", "qt6-webview-dev", "libqt6webview6", "qml6-module-qtwebview", "iproute2", "can-utils", "net-tools", "gpiod", "libqt6serialport6", "libqt6serialport6-dev", "qt6-connectivity-dev", "bluez", "dbus", "nano", "libcap2-dev", "libcap2", "libcap2-bin", "nethogs", "bluez-alsa-utils"

Is there any missing package to run the hardware video decoder?

Is there any setting I need to add to Qt to run a hardware video decoder?

Or is there another problem that is not related to hardware video decoding?

How can I solve this problem please? Or look into it?

Best regards,
Tarek

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes. Our support team will step in if needed. If I answered your question, please tick the “Solution” checkbox. For follow-up questions to me, please use @ToradexAI in your post.

Did you like my answer? :+1: :-1:

Hello Tarek,

Thank you for your detailed description. You’re correct in suspecting that hardware video decoding is likely not being used, based on the high CPU usage you observed with both your Qt/QML application and the GStreamer pipeline.

1. Use the VPU Hardware Decoder in GStreamer

Your current GStreamer pipeline:

gst-launch-1.0 filesrc location=/home/torizon/test_video.mp4 ! decodebin name=dec ! videoconvert ! waylandsink

This pipeline uses decodebin, which often falls back to software decoding. For efficient hardware (VPU) decoding on Apalis iMX8, use the following pipeline, which leverages the hardware-accelerated vpudec element:

gst-launch-1.0 filesrc location=/home/torizon/test_video.mp4 ! qtdemux ! h264parse ! vpudec ! waylandsink

References & Further Reading:

2. Check if Hardware Decoder Plugin Is Available

Verify if the vpudec plugin is present:

gst-inspect-1.0 | grep vpudec

or

gst-inspect-1.0 | grep h264

If vpudec does not appear, your container might be missing the specific hardware-accelerated GStreamer plugins for NXP i.MX8. These are typically included in the gstreamer1.0-plugins-imx package or similar (exact package naming might vary depending on your base image). Verify if you have this package and add it if not.

3. Debugging GStreamer

To see more verbose output and pinpoint errors:

export GST_DEBUG=2

Then run your GStreamer pipeline again. More debug info may hint at missing elements or other misconfigurations.

4. Qt/QML Integration

Qt’s QMediaPlayer and related QML multimedia components under the hood use GStreamer by default on Linux. However, they may select the first available decoder (decodebin), which could be a software decoder. Qt does not provide a built-in way to specify an explicit GStreamer pipeline out-of-the-box.

Therefore, to ensure hardware-accelerated decoding in Qt/QML, you must:

  • Make sure the hardware decoder (vpudec) is present and registered in GStreamer.
  • Remove software decoders from your image, if feasible, to force GStreamer to use the hardware decoder.
  • Alternatively, you might write a custom QML/Qt pipeline using GStreamer directly (coding manually with the GStreamer API).
  • Monitor the process list to ensure decoding isn’t falling back to CPU.

5. Next Steps

  • Test with the hardware-accelerated pipeline in the terminal.
    If CPU usage drops significantly (typically under 15-20%), hardware decoding works.

  • Check your container image for the hardware decoder plugins.
    If they’re missing, expand your package list as described above.

  • Review Qt’s debug output (run your QML app with debugging enabled) to see which GStreamer pipeline is being constructed.

6. References

Summary:

  • Use a GStreamer pipeline with vpudec for video playback.
  • Ensure the VPU plugins are installed (check with gst-inspect-1.0).
  • Try reducing software decoder presence, or use the hardware pipeline directly.
  • For QML, check debug output to ensure hardware decoders are used.

Let me know if you have further questions or need clarification!

Best regards,
Toradex AI Assistant.

3

This command:
gst-launch-1.0 filesrc location=/home/torizon/test_video.mp4 ! qtdemux ! h264parse ! vpudec ! waylandsink
Gives:
WARNING: erroneous pipeline: no element "vpudec"

The command: gst-inspect-1.0 | grep vpudec gives: Nothing
The command: gst-inspect-1.0 | grep h264 gives:

codectimestamper: h264timestamper: H.264 timestamper
libav: avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
libav: avenc_h264_omx: libav OpenMAX IL H.264 video encoder encoder
openh264: openh264dec: OpenH264 video decoder
openh264: openh264enc: OpenH264 video encoder
rtp: rtph264depay: RTP H264 depayloader
rtp: rtph264pay: RTP H264 payloader
typefindfunctions: video/x-h264: h264, x264, 264
uvch264: uvch264deviceprovider (GstDeviceProviderFactory)
uvch264: uvch264mjpgdemux: UVC H264 MJPG Demuxer
uvch264: uvch264src: UVC H264 Source
videoparsersbad: h264parse: H.264 parser

Adding: gstreamer1.0-plugins-imx to torizonPackages.json file gives: E: No packages found

4

Greetings @tarekmhnds,

The gstreamer packages inside the container don’t have support yet for utilizing the VPU on the i.MX8 hardware. This would explain the high CPU usage you are seeing.

This is known to us already and we do have plans to re-package the gstreamer packages in our containers for proper VPU, hardware acceleration support. Currently I don’t have an exact timeline to give you regarding when this will be done.

Is there a timeline for when you will need this in your project? I can try to raise this internally and see what is possible.

Best Regards,
Jeremias