High processor consumption when playing video and streaming from the Internet

Technical Support
, , ,
1

Hello,

I have Apalis imx8 board, with ixora carrier board, and I installed Torizon os 7.1.0+build.4 (scarthgap).

I’m trying to play a video file through QML 6, but the processor consumption is around 90%, and when I try to stream from the internet, the processor consumption is 160%.

I tried to play the video using:
gst-launch-1.0 filesrc location=/home/torizon/test_video.mp4 ! decodebin name=dec ! videoconvert ! waylandsink
But the cpu consumption was around 74%.

I think it seems that the hardware video decoding is not working, or there is some other problem.

These are the packages that I added to my container:
"qt6-multimedia-dev", "qml6-module-qtmultimedia", "libqt6multimedia6", "gstreamer1.0-x", "qml-module-qtgstreamer", "gstreamer1.0-tools", "gstreamer1.0-plugins-base", "gstreamer1.0-plugins-good", "gstreamer1.0-plugins-bad", "gstreamer1.0-plugins-ugly", "libgstreamer1.0-dev", "libgstreamer-plugins-base1.0-dev", "libgstreamer-plugins-bad1.0-dev", "gstreamer1.0-alsa", "gstreamer1.0-gl", "gstreamer1.0-gtk3", "gstreamer1.0-pulseaudio", "gstreamer1.0-libav", "v4l-utils", "pulseaudio", "pulseaudio-utils", "gstreamer1.0-plugins-base-apps", "apulse", "libpulse-dev", "libasound2", "alsa-tools", "alsa-utils", "alsa-oss", "vlc", "qt6-webengine-dev", "libqt6webenginequick6", "qml6-module-qtwebengine", "libqt6webenginecore6", "libqt6webenginecore6-bin", "libqt6webenginewidgets6", "qt6-webengine-dev-tools", "qt6-webview-dev", "libqt6webview6", "qml6-module-qtwebview", "iproute2", "can-utils", "net-tools", "gpiod", "libqt6serialport6", "libqt6serialport6-dev", "qt6-connectivity-dev", "bluez", "dbus", "nano", "libcap2-dev", "libcap2", "libcap2-bin", "nethogs", "bluez-alsa-utils"

Is there any missing package to run the hardware video decoder?

Is there any setting I need to add to Qt to run a hardware video decoder?

Or is there another problem that is not related to hardware video decoding?

How can I solve this problem please? Or look into it?

Best regards,
Tarek

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes. Our support team will step in if needed. If I answered your question, please tick the “Solution” checkbox. For follow-up questions to me, please use @ToradexAI in your post.

Did you like my answer? :+1: :-1:

Hello Tarek,

Thank you for your detailed description. You’re correct in suspecting that hardware video decoding is likely not being used, based on the high CPU usage you observed with both your Qt/QML application and the GStreamer pipeline.

1. Use the VPU Hardware Decoder in GStreamer

Your current GStreamer pipeline:

gst-launch-1.0 filesrc location=/home/torizon/test_video.mp4 ! decodebin name=dec ! videoconvert ! waylandsink

This pipeline uses decodebin, which often falls back to software decoding. For efficient hardware (VPU) decoding on Apalis iMX8, use the following pipeline, which leverages the hardware-accelerated vpudec element:

gst-launch-1.0 filesrc location=/home/torizon/test_video.mp4 ! qtdemux ! h264parse ! vpudec ! waylandsink

References & Further Reading:

2. Check if Hardware Decoder Plugin Is Available

Verify if the vpudec plugin is present:

gst-inspect-1.0 | grep vpudec

or

gst-inspect-1.0 | grep h264

If vpudec does not appear, your container might be missing the specific hardware-accelerated GStreamer plugins for NXP i.MX8. These are typically included in the gstreamer1.0-plugins-imx package or similar (exact package naming might vary depending on your base image). Verify if you have this package and add it if not.

3. Debugging GStreamer

To see more verbose output and pinpoint errors:

export GST_DEBUG=2

Then run your GStreamer pipeline again. More debug info may hint at missing elements or other misconfigurations.

4. Qt/QML Integration

Qt’s QMediaPlayer and related QML multimedia components under the hood use GStreamer by default on Linux. However, they may select the first available decoder (decodebin), which could be a software decoder. Qt does not provide a built-in way to specify an explicit GStreamer pipeline out-of-the-box.

Therefore, to ensure hardware-accelerated decoding in Qt/QML, you must:

  • Make sure the hardware decoder (vpudec) is present and registered in GStreamer.
  • Remove software decoders from your image, if feasible, to force GStreamer to use the hardware decoder.
  • Alternatively, you might write a custom QML/Qt pipeline using GStreamer directly (coding manually with the GStreamer API).
  • Monitor the process list to ensure decoding isn’t falling back to CPU.

5. Next Steps

  • Test with the hardware-accelerated pipeline in the terminal.
    If CPU usage drops significantly (typically under 15-20%), hardware decoding works.

  • Check your container image for the hardware decoder plugins.
    If they’re missing, expand your package list as described above.

  • Review Qt’s debug output (run your QML app with debugging enabled) to see which GStreamer pipeline is being constructed.

6. References

Summary:

  • Use a GStreamer pipeline with vpudec for video playback.
  • Ensure the VPU plugins are installed (check with gst-inspect-1.0).
  • Try reducing software decoder presence, or use the hardware pipeline directly.
  • For QML, check debug output to ensure hardware decoders are used.

Let me know if you have further questions or need clarification!

Best regards,
Toradex AI Assistant.

3

This command:
gst-launch-1.0 filesrc location=/home/torizon/test_video.mp4 ! qtdemux ! h264parse ! vpudec ! waylandsink
Gives:
WARNING: erroneous pipeline: no element "vpudec"

The command: gst-inspect-1.0 | grep vpudec gives: Nothing
The command: gst-inspect-1.0 | grep h264 gives:

codectimestamper: h264timestamper: H.264 timestamper
libav: avdec_h264: libav H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 decoder
libav: avenc_h264_omx: libav OpenMAX IL H.264 video encoder encoder
openh264: openh264dec: OpenH264 video decoder
openh264: openh264enc: OpenH264 video encoder
rtp: rtph264depay: RTP H264 depayloader
rtp: rtph264pay: RTP H264 payloader
typefindfunctions: video/x-h264: h264, x264, 264
uvch264: uvch264deviceprovider (GstDeviceProviderFactory)
uvch264: uvch264mjpgdemux: UVC H264 MJPG Demuxer
uvch264: uvch264src: UVC H264 Source
videoparsersbad: h264parse: H.264 parser

Adding: gstreamer1.0-plugins-imx to torizonPackages.json file gives: E: No packages found

4

Greetings @tarekmhnds,

The gstreamer packages inside the container don’t have support yet for utilizing the VPU on the i.MX8 hardware. This would explain the high CPU usage you are seeing.

This is known to us already and we do have plans to re-package the gstreamer packages in our containers for proper VPU, hardware acceleration support. Currently I don’t have an exact timeline to give you regarding when this will be done.

Is there a timeline for when you will need this in your project? I can try to raise this internally and see what is possible.

Best Regards,
Jeremias

5

Greetings @jeremias.tx,

I’m building a control and entertainment system for a Mercedes minivans. In this system, I play MP4 video files of the seat movements, sunroof movement, TV, etc. We also have Live TV channels, and the user can play video files from a USB flash drive. Therefore, hardware video decoding support is an important part of the project.

We will install the system on a test minivan within a month, and leave it under testing for about 2 months.

Do you have a workaround for this problem?

If I convert our current video file format from MP4 to AVI or any other format, will that solve the problem? Which video file format takes less CPU load, please?

Best regards,
Tarek

6

Do you have a workaround for this problem?

We do have a sort of proof-of-concept container example for using the VPU on our SOMs: torizon-samples/vpu/generic/vpuGeneric at bookworm-new · toradex/torizon-samples · GitHub

Though this was only tested to work on Torizon OS 6.X, it probably won’t work correctly at all on Torizon OS 7.X. This would mean you’d have to use the previous OS version if that is acceptable for you. While this example was more of a proof-of-concept rather than officially tested, it did show the capability to utilize the VPU for more efficient video playback.

If you do use this there are some things to note:

  • For the versions of Gstreamer provided by NXP to work with the versions of packages provided by Debian we had to freeze the Debian package feed in this container to a past snapshot. Therefore the Debian packages in this container while still Bookworm will not be the latest available.
  • In the example Dockerfile change the BASE_VERSION variable at the top of the Dockerfile from 3 to 3.3 to avoid a build issue.

Other than this the only other alternative for VPU acceleration would be to wait for us to officially support it in Torizon OS 7.X.

If I convert our current video file format from MP4 to AVI or any other format, will that solve the problem? Which video file format takes less CPU load, please?

Uncompressed, or low-compression video formats like AVI as you suggested, should in theory take less CPU load. Though how much it would improve your situation by it’s hard to say, you’d probably need to test it to see a direct comparison.

Best Regards,
Jeremias

8

Hello @jeremias.tx ,

I have no problem downgrading to Torizon 6, and I did that.

After this, I tested these 2 commands, according to the example at the github, but both didn’t work:

gst-launch-1.0 filesrc location=/home/torizon/test.mp4 ! qtdemux ! h264parse ! queue ! v4l2h264dec ! imxvideoconvert_g2d ! waylandsink
WARNING: erroneous pipeline: no element “v4l2h264dec”

gst-launch-1.0 filesrc location=/home/torizon/text.mp4 ! qtdemux ! h264parse ! queue ! vpudec ! waylandsink
WARNING: erroneous pipeline: no element “vpudec”

Did I forget to add any packages to my container? Packages are the same at the first question.

If I run:
gst-launch-1.0 filesrc location=/home/torizon/test.mp4 ! decodebin name=dec ! videoconvert ! waylandsink
I got the same problem of high cpu consumption around 75%.

How to solve this please?

The operating system information on the Apalis board is::

ID=torizon
NAME=“TorizonCore”
VERSION=“6.8.2+build.30 (kirkstone)”
VERSION_ID=6.8.2-build.30
PRETTY_NAME=“TorizonCore 6.8.2+build.30 (kirkstone)”
DISTRO_CODENAME=“kirkstone”
BUILD_ID=“30”
ANSI_COLOR=“1;34”
VARIANT=“Docker”

And the os information on the container is:

PRETTY_NAME=“Debian GNU/Linux 12 (bookworm)”
NAME=“Debian GNU/Linux”
VERSION_ID=“12”
VERSION=“12 (bookworm)”
VERSION_CODENAME=bookworm
ID=debian
HOME_URL=“https://www.debian.org/”
SUPPORT_URL=“Debian -- User Support”
BUG_REPORT_URL=“https://bugs.debian.org/”

Best regards,
Tarek

9

Did you build the container image properly as per the README? I just built the container image myself for Apalis i.MX8, and I can see the v4l2h264dec element is present in the container:

# Inside the example VPU container
root@apalis-imx8-06738453:/home/torizon# gst-inspect-1.0 | grep v4l2h264dec
video4linux2:  v4l2h264dec: V4L2 H264 Decoder

That said, I did have other issues trying to play an mp4 video file with the VPU. It’s likely this example reference is outdated, or not working in some way. Since it was only intended as an initial proof-of-concept. Though strangely mp4 playback works as expected on Verdin i.MX8M Plus. So perhaps it’s an issue with i.MX8 specifically.

Again though It’s probably preferable to wait for proper VPU support from our team rather than try to get this outdated reference working. I’ll raise your issue with the team and try to see if we can get this VPU work prioritized in Torizon 7.

Best Regards,
Jeremias

10

Hello @jeremias.tx ,

No I didn’t build any special image, I just downloaded the Torizon 6.8.2 version via Toradex Easy Installer.

Ok I will wait for VPU support in Torizon 7.
Thank you very much

Best regards,
Tarek

11

We’ll try to inform you once there is news to share regarding this.

No I didn’t build any special image, I just downloaded the Torizon 6.8.2 version via Toradex Easy Installer.

Also just to clarify this point, I was referring to the container image, not the OS image.

Best Regards,
Jeremias

1 Like