Gst-launch-1.0 video playback fail on verdin-am62

ali · July 11, 2024, 10:31am

Hi,
I’m testing video playback performance on verdin-am62 SOM. with Verdin-AM62_Reference-Multimedia-Image. I couldn’t play successfully test videos.

Here is the tdx-info.

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           6.1.80+git.0d9f1748ac17 #1 SMP PREEMPT Thu Mar 28 13:48:11 UTC 2024
Kernel command line:      root=PARTUUID=2f7e697a-02 ro rootwait console=tty1 console=ttyS2,115200 consoleblank=0 earlycon=ns16550a,mmio32,0x02800000
Distro name:              NAME="TDX Wayland with XWayland"
Distro version:           VERSION_ID=6.6.0-build.12
Distro variant:           -
Hostname:                 verdin-am62-15313415
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin AM62 on Verdin Development Board
Toradex version:          0073 V1.1B
Serial number:            15313415
Processor arch:           aarch64
------------------------------------------------------------

I’m guessing GPU is not initialized here. Even though I have installed the pvr module, it doesn’t help it before weston launches. The reason why I dig into this is I get “libEGL warning: egl: failed to create dri2 screen” whatever I launch based on gui.

insmod /lib/modules/$(uname -r)/extra/pvrsrvkm.ko

Here is the testing responses:

root@verdin-am62-15313415:~/video# gst-launch-1.0 videotestsrc ! video/x-raw ! autovideosink
Setting pipeline to PAUSED ...
libEGL warning: egl: failed to create dri2 screen
Pipeline is PREROLLING ...
Got context from element 'autovideosink0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
0:00:31.7 / 99:99:99.

Could you please confirm that reference image successfully playback video ?
How can I fix the “libEGL warning: egl: failed to create dri2 screen” problem ?
Any help is appreciated.
Thanks

ToradexAI · July 11, 2024, 10:32am

Hi, I’m the Toradex AI Assistant. I’m in early beta, and I would appreciate your feedback.
The Toradex Support team will follow up with you shortly!

Did you like my answer?

Hello,

Based on the information provided, it seems you are encountering issues with video playback on the Verdin AM62 SoM, specifically with the Reference Multimedia Image and encountering the “libEGL warning: egl: failed to create dri2 screen” error. This issue typically indicates a problem with the initialization of the graphics stack, which is crucial for video playback and graphical applications.

First, ensure that the PVR graphics driver (pvrsrvkm.ko) is correctly installed and loaded before the Weston compositor starts, as the driver is essential for hardware-accelerated graphics on platforms utilizing PowerVR GPUs. The error message suggests that there might be an issue with the EGL initialization, which is often related to the graphics driver or the configuration of the Weston compositor.

Here are a few steps to troubleshoot and potentially resolve the issue:

Verify Driver Installation: Confirm that the PVR kernel module is correctly installed and loaded at boot time. You can add the insmod command to your system’s startup scripts to ensure it’s loaded before the graphical environment starts.
Weston Configuration: Check the Weston configuration file (usually located at /etc/xdg/weston/weston.ini) for any settings that might be incompatible with your setup. Ensure that the configuration is set to use the correct graphics backend and that it matches the capabilities of your hardware.
Environment Variables: Setting the GST_GL_API environment variable to gles2, as mentioned, is crucial for GStreamer to utilize OpenGL ES for video rendering. Ensure this variable is set in the environment where you are launching your GStreamer pipeline.
GStreamer Pipeline: The pipeline you’re testing with (gst-launch-1.0 videotestsrc ! video/x-raw ! autovideosink) is a good starting point. However, if the issue persists, consider specifying the GstGLDisplayWayland explicitly in your pipeline or experimenting with different video sinks (e.g., glimagesink) to see if the issue is related to the autovideosink selection.
Debugging: Utilize GStreamer’s debugging capabilities to gain more insight into what might be going wrong. You can increase the verbosity of the GStreamer logs by setting the GST_DEBUG environment variable (e.g., export GST_DEBUG=3 for more detailed logs).
Update Your System: Ensure your system is up to date with the latest software versions provided by Toradex, including

Please, also check if these links can help you solve your problem:

Web Browser / Kiosk Mode with Torizon OS | Toradex Developer Center.

bruno.tx · July 11, 2024, 3:47pm

Hello @ali,

This issue could be caused by the lack of a valid videosink for GStreamer.
Do you have a display connected to the system? If so, using which interface on which carrier board?

I can also confirm that your pipeline (gst-launch-1.0 videotestsrc ! video/x-raw ! autovideosink) works with the default Toradex Reference Multimedia Image, using a DSI to HDMI converter with an HDMI display connected.

Now, talking specifically about video playback, please keep in mind that the Verdin AM62 has no VPU, so hardware-accelerated video decoding (or encoding) is not possible.
This means that if you have encoded video that you want to play, a software decoder will need to be used and the performance will likely be degraded for higher resolutions.
This is true for all versions of the Verdin AM62.

Is video decoding something that you need?
If so, in which formats?

Best Regards,
Bruno

ali · July 11, 2024, 5:06pm

Hello Bruno,
Appreciated your quick and knowledgeable and meaningful response for me.

Do you have a display connected to the system? If so, using which interface on which carrier board?

Yes, I have a custom board has a custom screen (connected native lvds interface) as well which uses “verdin-am62_mezzanine_panel-cap-touch-10inch-lvds_overlay.dtbo” and screen works fine.(weston desktop, xclock is running default) I didn’t test it on Mallow board but if you suggest it to be used, I can go for it.

I can also confirm that your pipeline (gst-launch-1.0 videotestsrc ! video/x-raw ! autovideosink ) works with the default Toradex Reference Multimedia Image, using a DSI to HDMI converter with an HDMI display connected

Thanks for confirming. It works on my side as well but not perfectly still I have “libEGL warning: egl: failed to create dri2 screen” warning which explains we get this warning because Verdin-AM52 doesn’t have any VPU. Thank you I missed it on your comparison table (IMX8MM, IMX8MP and AM62)

We have QT based app which has video idle loop and trying to port our app working on AM62. Our video format is webm. Before dealing with app, I’m trying to be ensure that we have correct video-sink plugin on gstreamer. As you complement that the first work is here successfully playing video with gstreamer. When idle loop video is running on the QT app, video decoding is awful and CPU usage increased too much.

Do you suggest any gstreamer command for webm format should be used to play a video?

Many Thanks
Best Regards,
Ali

bruno.tx · July 11, 2024, 6:46pm

Hello @ali,

I don’t think that this is necessary, as your display is working.

Actually, libEGL would not be used to handle a VPU, therefore this warning indicates something else.
The most likely cause is that the 0073 Verdin AM62 Dual 1GB ET also lacks a GPU, therefore libEGL is not available.
It could also be that autovideosink first tries another video sink which uses libEGL and later on falls back to the correct waylandsink, which can be used with Weston.
Can you check if you see the same problem when running the following pipeline?

gst-launch-1.0 -v videotestsrc ! video/x-raw,width=1280,height=800 ! fpsdisplaysink text-overlay=false video-sink=waylandsink sync=false

This pipeline will run the video out of sync with the display and the current time, but will show the best possible throughput of the pipeline and can be useful when testing performance.

Webm is a container format for VP8, VP9 or AV1 encoded video, therefore you will need to ensure that you have both a webm-compatible demuxer such as matroskademux and a software decoder for the encoding present in your file, VP8, VP9 or AV1.

The following pipeline should be able to play a webm video once the conditions above are met:

GST_DEBUG=3 gst-launch-1.0 filesrc location=video.webm ! decodebin ! videoconvert n-threads=4 ! fpsdisplaysink text-overlay=false video-sink=waylandsink sync=false

Please note that I added some debug configurations so that you see a meaningful error message if the decoder is missing. For example, using the Toradex Reference Multimedia Image and trying to play a VP8 file on the Verdin AM62 gives the following error:

root@verdin-am62-15207029:~/video# GST_DEBUG=3 gst-launch-1.0 filesrc location=testvideo_vp8.webm ! decodebin ! videoconvert n-threads=4 ! fpsdisplaysink text-overlay=false video-sink=waylandsink sync=false
Setting pipeline to PAUSED ...
0:00:00.094838754  1018 0xaaab09133360 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc0> pad not activated yet
Pipeline is PREROLLING ...
Missing element: VP8 decoder
...
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0: Internal data stream error.
Additional debug info:
../gst-plugins-good-1.20.7/gst/matroska/matroska-demux.c(6075): gst_matroska_demux_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMatroskaDemux:matroskademux0:
streaming stopped, reason not-linked (-1)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

That being said, as the AM62 lacks a VPU, the software decoding will always place a high load on the CPU and may only be viable for very low resolutions.
If the video is small enough, it could be viable to store it as an mjpeg, which will also require software decoding of the jpeg images, but could place a lower load on the CPU, at the cost of a bigger file size.

Best Regards,
Bruno

ali · July 11, 2024, 9:07pm

Hi @bruno.tx ,

Can you check if you see the same problem when running the following pipeline?
gst-launch-1.0 -v videotestsrc ! video/x-raw,width=1280,height=800 ! fpsdisplaysink

Yes, it works as you promised. It doesn’t give EGL warning as you mentioned.

The following pipeline should be able to play a webm video once the conditions above are met:
GST_DEBUG=3 gst-launch-1.0 filesrc location=video.webm ! decodebin ! videoconvert n-threads=4 ! fpsdisplaysink text-overlay=false video-sink=waylandsink sync=false

I got same result with yours. Then I have tested our custom video on my custom Yocto image, it plays the video but not good.

GST_DEBUG=3 gst-launch-1.0 filesrc location=screensaver.webm ! decodebin ! videoconvert n-threads=4 ! fpsdisplaysink text-overlay=false video-sink=waylandsink sync=false
Setting pipeline to PAUSED ...
0:00:00.129461800   893 0xaaaaedf5d360 WARN                 basesrc gstbasesrc.c:3688:gst_base_src_start_complete:<filesrc0> pad not activated yet
Pipeline is PREROLLING ...
0:00:00.164220739   893 0xaaaaedf7b9e0 FIXME          matroskademux matroska-demux.c:682:gst_matroska_demux_parse_colour:<matroskademux0> Unsupported subelement 0x55b7 in Colour
0:00:00.164333321   893 0xaaaaedf7b9e0 FIXME          matroskademux matroska-demux.c:682:gst_matroska_demux_parse_colour:<matroskademux0> Unsupported subelement 0x55b8 in Colour
Redistribute latency...
0:00:00.226850440   893 0xaaaaedf7b9e0 WARN      matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<matroskademux0:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.226971998   893 0xaaaaedf7b9e0 WARN      matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<matroskademux0:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.227008565   893 0xaaaaedf7b9e0 WARN      matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<matroskademux0:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.227038606   893 0xaaaaedf7b9e0 WARN      matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<matroskademux0:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.227068452   893 0xaaaaedf7b9e0 WARN      matroskareadcommon matroska-read-common.c:759:gst_matroska_read_common_parse_skip:<matroskademux0:sink> Unknown CueTrackPositions subelement 0xf0 - ignoring
0:00:00.227381046   893 0xaaaaedf7b9e0 FIXME          matroskademux matroska-demux.c:682:gst_matroska_demux_parse_colour:<matroskademux0> Unsupported subelement 0x55b7 in Colour
0:00:00.227425009   893 0xaaaaedf7b9e0 FIXME          matroskademux matroska-demux.c:682:gst_matroska_demux_parse_colour:<matroskademux0> Unsupported subelement 0x55b8 in Colour
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Got EOS from element "pipeline0".
Execution ended after 0:00:10.913299859
Setting pipeline to NULL ...
Freeing pipeline ...

The custom video plays back nicely on imx6 by using “eglvivsink” since it has VPU. Previously, I was testing on imx6 with soft demuxes like mastroska without using eglvivsink(since elgvivsink is depreciated for new versions) , I didn’t get good result as well.

That being said, as the AM62 lacks a VPU, the software decoding will always place a high load on the CPU and may only be viable for very low resolutions.
If the video is small enough, it could be viable to store it as an mjpeg, which will also require software decoding of the jpeg images, but could place a lower load on the CPU, at the cost of a bigger file size

Agreed. Appreciated your valuable comment on this. This is a R&D work for us, exploring and selection between verdin-imx8mp, verdin-imx8mm and verdin-am62. Everything works fine already verdin-imx8mm. I got satisfactory answers for verdin-am62, probably we will use in future for another reason.
Thank you so much!

Best Regards
Ali