colibri-imx6 GStreamer issue

Hi,

I updated our board from Thud to Kirkstone and managed to successfully port each feature it had but there is an issue with (or related to) Gstreamer.

On the old system, video where smooth, but on the new one those are choppy.

I had one kernel Oops after less than 10 minutes, not really reproducible.

kernel[516]: [ 526.479394] Internal error: Oops: 7 [#1] SMP ARM

[ 526.535206] LR is at __schedule+0x220/0x5d4
[ 526.539390] pc : [<80194eec>] lr : [<808f79c4>] psr: 600b0013
[ 526.545657] sp : 8a561df8 ip : 8c07e040 fp : ffffffff
[ 526.550880] r10: 6cf77644 r9 : 8a561e74 r8 : 8a560000
[ 526.556106] r7 : 8a561e30 r6 : 8c145d40 r5 : ffffe000 r4 : 88560000
[ 526.562632] r3 : 40404040 r2 : ffd8c07c r1 : 00000000 r0 : 8fb391c0
[ 526.569163] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 526.576298] Control: 10c5387d Table: 1d6f404a DAC: 00000051
[ 526.582046] Process aqueue:src (pid: 799, stack limit = 0xfdd5ae30)
[ 526.588315] Stack: (0x8a561df8 to 0x8a562000)

Videos are played though QtMultimedia from a QML object “Video”

The system doesn’t use X11 neither Wayland, but EGLFS. I had to adapt a little local.conf because it refused to compile by adding :

PREFERRED_VERSION_gstreamer1.0 = ‘1.20.3.imx’
PREFERRED_VERSION_gstreamer1.0-plugins-base = ‘1.20.3.imx’
PREFERRED_VERSION_gstreamer1.0-plugins-good = ‘1.20.3.imx’
PREFERRED_VERSION_gstreamer1.0-plugins-bad = ‘1.20.3.imx’

Actually, I’m not really sure that those versions are correct for the setup. I had choice with a version without the .imx at the end, but it seemed more logic to use this version (I tried both without improvement on the video).

The board is based on a Toradex Colibri iMX6DL 512MB IT V1.1B

other local.conf changes are :

MACHINE = ‘colibri-imx6’
DISTRO = ‘tdx-x11’
ACCEPT_FSL_EULA = ‘1’
LICENSE_FLAGS_ACCEPTED = ‘commercial’
DISTRO_FEATURES:remove = ‘x11 wayland directfb vulkan 3g’
MACHINES_UPSTREAM_DISTRO_DEFAULT = ‘apalis-imx6 colibri-imx6 colibri-imx6ull colibri-imx6ull-emmc colibri-imx7 colibri-imx7-emmc’
IMAGE_FSTYPES += ‘tar.bz2’
IMAGE_FSTYPES:remove = ‘teziimg’

I might have made a mistake somewhere but can’t figure it.

Toradex doesn’t provide EGLFS image example anymore, so I can’t compare with a working example.

Does someone have any clue of how to make video works on this board ?

Hi @aridet !

It is quite a long jump :sweat_smile:
Good to know that you were able to at least port and build it.

But this is unfortunate.

I would like to ask you to try your video playback on a default Reference Multimedia Image from Toradex (Download Links | Toradex Developer Center).

Could you please elaborate more on why you need X11 and EGLFS specifically?

Best regards,

Hi,

Thanks for your answer, I will try your suggestion and will report the result, but we don’t use X11, we only use EGLFS

I have removed x11 from DISTRO_FEATURE

DISTRO_FEATURES:remove = ‘x11 wayland directfb vulkan 3g’

That right that I also have :

DISTRO = ‘tdx-x11’

But I don’t know what other distro I can use if I only want EGLFS ?
meta-toradex-distro only provide x11 and xwayland

Regards,

Hello @aridet ,
Please have a look at this thread in the community. It might help you

Best regards,
Josep

Hi,

So I tried with the multimedia reference image based on 6.2.0. I downloaded it directly from TEZI on the board, and videos are smooth, but it’s based on xwayland, not EGLFS.

The difference I could spot is that when I launched the video from gst-play, it’s displayed on the screen , while on my EGLFS version, nothing is displayed (but trace show the video being played). I didn’t try from a Qt application, but the QtCinematicExperience launched on start had 60FPS and the moving sun was absolutely smooth too.

About the latest link, I do have a EGLFS platform plugin, I launch my app with following environment variable :

QT_QPA_PLATFORM=eglfs
QT_QPA_EGLFS_INTEGRATION=eglfs_viv

Maybe it’s the QT_QPA_EGLFS_INTEGRATION that does not link to the correct thing, but without that, app won’t start at all. do you know how can find which integration is available ?

I will still try a build from scratch because I may have build a system with x11 before but can’t remember. But I’m pretty sure that when I change local.conf, it rebuild everything without using sstate-cache.

Regards,

Hi,

Some new from my issue. I rebuilt the image from scratch without success. Video still choppy when played.

I also built Qt5_CinematicExperience, and I do have 60 fps (on the top left corner). The sun animation is also really smooth. So the acceleration is enabled at least when using Qt only.

That let me think the issue is with GStreamer only. Maybe it is not built correctly, or I use the wrong version, or wrong plugin or something else ? I don’t know where to start to check if everything is fine.

For example, gst-launch-1.0 --version returns GStreamer 1.20.3, but I’m unable to know if this is the 1.20.3.imx version I asked in preferred version or if it’s the standard version.

Regards,

Hi @aridet

Please use:

DISTRO = "tdx-xwayland-upstream"

Starting with BSP6 only upstream is supported for iMX6 CPUs.

This will include some other drivers for the GPU and VPU and Gstreamer should work fine with them. Probably gstreamer is using software decoding in your current setup.

You can then still remove all the features as you did for EGLFS.

DISTRO_FEATURES:remove = ‘x11 wayland directfb vulkan 3g’

I hope this helps.

Regards,
Stefan

Hi,

I tried to go the upstream way, but I’m facing a new issue, my device tree are not built anymore and not shipped in the final image.

I changed the KERNEL_DEVICETREE variable to KERNEL_DEVICETREE:use-mainline-bsp because otherwise, the variable was overwriten by another recipe (saw with bitbake -e).

I also saw that now there is a Makefile to build devicetrees, so I added mine alongside the one for aster, eval-v3, iris and iris-v2 (which are built by default), but there are not built.

I couldn’t find a way to integrate them.

Do you have any clues ?

Regards,
Aurélien.

Edit : oups my bad, after a lot of tries I put the KERNEL_DEVICETREE:use-mainline-bsp only on my image recipe and not in the linux-toradex-mainline_%.bbappend.

Now it fails to compile because it can’t find some label or path, I will investigate on that.

Hi,

So I changed DISTRO to tdx-xwayland-upstream, and had to update a little our device tree.

I have some new issues I can’t understand.

Fisrt one, is the most critical, I cannot run 2 Qt applications anymore at the same time. The second application fail to start while spamming the following error message every 10ms :

unknown: Could not queue DRM page flip on screen UNKNOWN1 (Permission denied)

If I kill the first application, then it start fine

Both application never draw on the display at the same time, so it was fully compatible with the framebuffer use.

Second issue is again linked to display. we have a framebuffer vnc server, and when I open a client and connect to it, I have a very strange behavior.

From a black screen, if I execute this command :

cat /dev/urandom >/dev/fb0

Then I have on my display full garbage on the screen which is expected. on the vnc client I have only half the screen displayed, the right half remain black? If I take a screenshot, and analyze with gimp, I can confirm what I see.

Finally, If I start my Qt application, it effectively paint on my display, but the vnc client still display half of old garbage and half a black screen.

I think framebuffer if messed in this configuration.

Note that I had to change QT_QPA_EGLFS_INTEGRATION=eglfs_viv to QT_QPA_EGLFS_INTEGRATION=eglfs_kms to be able to start Qt application.

Third issue is, as with previous version, videos are still choppy when launched from the Qt Application.
I have a black screen when I launch them from command line gst-play-1.0 + video name, with following warning output :

WARNING No decoder available for type ‘audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1190, rate=(int)48000, channels=(int)2’.
WARNING debug information: …/gst-plugins-base-1.20.5/gst/playback/gsturidecodebin.c(960): unknown_type_cb (): /GstPlayBin:playbin/GstURIDecodeBin:uridecodebin0

I don’t get how gstreamer can decode (even badly) the video launched from QtMultimedia, and is not able to play the video from basic command line (this exact same command line worked in very old system)

So I would say that overall this upgrade is worse. I still don’t understand why exactly. As official multimedia image played correctly video at least from command line (I don’t remember if I tried from Qt Application, but I remember that it was fully accelerated).

Regards,
Aurélien

Hi @aridet !

Can you please share the modifications you performed on your layers and recipes?

Then I can try to reproduce your issue. Also, which kind of display you are using?

Are you able to reproduce your issue using the Qt Cinematic Experience that is shipped with Toradex’s Reference Multimedia Image?

Best regards,