TorizonCore 6.1 cannot render anymore 3D content on Verdin iMX8M-Mini

vix · October 13, 2023, 6:29am

Hi @jeremias.tx,
I’m not sure if this helps but I opened this issue on GH for meta-freescale, when I had been working with the Mini.
But I think the concept are more or less tjhe same.
As far as I understand, this is the patch needed (for Kirkstone) and it comes from on NXP LF5.15.71_2.2.0.

/---------/
EDIT:
I read fron NXP i.MX Linux Release Notes that on September, 29th 2023 LF6.1.36_2.1.0 has been released.
One of the new features is
GPU driver upgraded to 6.4.11.p2.0 with Vulkan enablement, bug fixes, and performance optimizations.
/---------/

If I’m right Toradex is in a process of migration of the kernel for iMX8M-Mini and Plus from the NXP-based downstream to the mainline upstream.
But this is only experimental.
Does the GPU-acceleration work in the experimental upstream?

jeremias.tx · October 13, 2023, 9:16pm

Thank you for sharing the additional information I’ll pass it on to our team here.

If I’m right Toradex is in a process of migration of the kernel for iMX8M-Mini and Plus from the NXP-based downstream to the mainline upstream.

I wouldn’t say this exactly. While we do provide a mainline build for i.MX8M Plus/Mini it’s entirely experimental and many hardware related features are not expected to work. Though I wouldn’t say we’re “in a process of migration”. We would like to leverage mainline if possible but the mainline support for these SoCs is still quite lacking compared to the older SoCs (i.e. i.MX6/7) which have had time to mature in mainline. So until we can get a similar level of functionality for these SoCs in mainline, we will keep our official maintenance effort on the downstream kernels based on the NXP releases.

Does the GPU-acceleration work in the experimental upstream?

As far as I’m aware no. There has been some progress to support the i.MX8* SoCs with the mainline open-source GPU drivers (etnaviv). But, as far as I’m aware this is currently still a work in progress.

Best Regards,
Jeremias

vix · October 18, 2023, 6:28am

Which is the NXP release that TorizonCore is basen on at the moment?
Is the latest LF6.1.36_2.1.0?
Or an older release?
This is the release list from NXP documentation.

I cannot find this information

jeremias.tx · October 18, 2023, 6:48pm

You can find what NXP release we are based on from this page here: Embedded Linux Release Matrix | Toradex Developer Center

For our 6.X images we are based NXP BSP L5.15_2.1.0. Though I believe for our recently released 6.4.0 images we have slightly updated to NXP BSP L5.15_2.2.0. Though this is just a minor upgrade and not a drastically different kernel version change.

The reason we are on this release is because our Yocto builds are on Yocto Kirkstone. In Kirkston NXP uses the 5.15* releases so that’s what we use. In future Yocto versions newer NXP releases are probably used.

Best Regards,
Jeremias

vix · October 19, 2023, 6:46am

Do you mean NXP BSP L5.15.71_2.2.0 https://github.com/nxp-imx/meta-imx/tree/kirkstone-5.15.71-2.2.0 ?

I think I got the point.
I’ve just opened this topic on NXP Forum to check if some backporting is possible.

jeremias.tx · October 19, 2023, 5:40pm

Do you mean NXP BSP L5.15.71_2.2.0 GitHub - nxp-imx/meta-imx at kirkstone-5.15.71-2.2.0 ?

Well yeah 5.15* we just omit the minor versioning numbers for brevity.

I think I got the point.
I’ve just opened this topic on NXP Forum to check if some backporting is possible.

I’m curious if NXP would consider this.

Best Regards,
Jeremias

vix · October 23, 2023, 7:56am

Hi @jeremias.tx

from the answer it doesn’t seem that backporting is an option, because VeriSilicon provides GPU code in binary format.
But this doesn’t mean that binary for kirkstone-5.15.71 has not been released.

Investigating this issue I’ve just found Etnaviv.
Not sure if this can be an option (in case it works better than the official, I mean).

jeremias.tx · October 23, 2023, 7:25pm

Etnaviv is meant to be used on upstream kernels. You could give this a try but it would mean a completely custom TorizonCore using a kernel version that we don’t guarantee functionality for on Verdin i.MX8M.

Best Regards,
Jeremias

vix · October 30, 2023, 8:50am

Hi @jeremias.tx
do you think that approach suggested by NXP in their answer here can work in Torizon OS?

jeremias.tx · October 30, 2023, 5:41pm

do you think that approach suggested by NXP in their answer here can work in Torizon OS?

It’s honestly hard for me to say. I have some doubts it’s just that simple to copy and paste backport like this. Especially between two major versions of the kernel. That said given the proprietary nature of these drivers we’re not really the experts on these drivers.

Best Regards,
Jeremias

vix · February 5, 2024, 1:07pm

Hi @jeremias.tx
can you share some info from the developer team on the investigation on GPU not working properly?

jeremias.tx · February 6, 2024, 6:51pm

@vix Let me see what our developer team has available to share. I know some work has been done on this topic since last you asked.

Best Regards,
Jeremias

vix · February 14, 2024, 7:03am

Hi @jeremias.tx
have you had some feedback from the developer team?

jeremias.tx · February 15, 2024, 8:55pm

@vix I just heard back from our developer team.

They did some fixes and have an early version of the improved Chromium container. Please try this:

docker run -d --rm --name=chromium     -v /tmp:/tmp -v /var/run/dbus:/var/run/dbus     -v /dev/galcore:/dev/galcore --device-cgroup-rule='c 199:* rmw'     --security-opt seccomp=unconfined --shm-size 256mb     leonardoheldattoradex/chromium:150224 https://webglsamples.org/aquarium/aquarium.html

Currently the container image is stored in the dockerhub repo of one of the developers as it’s not officially released yet. But you can still try it. The developer was able to get much better performance on the aquarium demo then you reported prior. For the weston container just use the currently release container image for that.

Please let us know how this fixed version works for you, we appreciate the feedback.

Best Regards,
Jeremias

vix · February 20, 2024, 8:26am

Hi @jeremias.tx,
we just test the linked container (unfortunately) with no performance improvements.
We move to the Verdin iMX8M Plus Quad 2GB Wi-Fi / Bluetooth IT (PN: 0064) which will be the candidate for our production.

Below some info that can be useful:

Torizon

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           5.15.129-6.4.0+git.67c3153d20ff #1-TorizonCore SMP PREEMPT Wed Sep 27 12:30:36 UTC 2023
Kernel command line:      root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/49c8910be07b7c3eddf687e18488c490227ca44b62f5a68b8c8db01798468b41/0
Distro name:              NAME="TorizonCore"
Distro version:           VERSION_ID=6.4.0-build.5
Hostname:                 verdin-imx8mp-15128331
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin iMX8M Plus WB on Yavia Board
Toradex version:          0064 V1.1A
Serial number:            15128331
Processor arch:           aarch64
------------------------------------------------------------

Containers

leonardoheldattoradex/chromium:150224
torizon/weston-vivante:3.2

Screenshot fish tank

Screenshot gpu-top

leon.tx · February 20, 2024, 3:10pm

Hi @Vix, just to clarify: are you looking for performance improvements or having issues with rendering? Since you made the thread and between the replies from Jeremias, we fixed an issue with the Chromium GPU rendering.

Regarding performance and gputop, we now have gputop in our feeds, not yet released though. I think, from your screenshots, you were running gputop in a different process namespace than the one running chromium, so gputop won’t see the chromium process.

If you want Chromium to show up, run with --privileged, exec into it as root, edit /etc/apt/sources.list.d/toradex.sources to not use a snapshot (use URIs: https://feeds.toradex.com/debian) and run sudo apt update && apt install -y gputop.

Here are some tests, comparing my version to the one currently in our repo (minor tweaks and options we’re using to develop). I’ve used Torizon 6.5.0 (MP) and 6.3.0 (MM).

iMX8MM running leonardoheldattoradex/chromium:150224

iMX8MM running torizon/chromium:3

iMX8MP running leonardoheldattoradex/chromium:150224

iMX8MP running torizon/chromium:3

There’s clear activity in the GPU usage with write/reads and continuous memory being allocated to it. Also, although not completely reliable WebGL and WebGL2 report “Hardware Accelerated”.

Of course, Chromium is not completely GPU-bound, and certain operations will utilize the CPU anyway.

Note that we use the fishtank/blob webgl examples with some very harsh settings. Having 500 or 1000 objects isn’t necessarily representative of real-world applications.