Weston-vivante launch with gray screen

Hi!. We have an application in .NET. When we are flashing the image, this image already integrates our containers as explained in (Pre-provisioning Docker Containers onto a Torizon OS image | Toradex Developer Center). The situation that occurs is that after several restarts of the application, Weston enters the gray screen as if we were sending the --developer flag. We have already reviewed the case of Weston-vivante:2 launches differently. And it was verified that our application depends on the start of Weston; however, the gray screen appears occasionally.

Here are the logs of the application when it launches correctly.

Mar 19 23:02:03 verdin-imx8mm-14684133 systemd[1]: Starting Docker Compose service with docker compose…
Mar 19 23:02:03 verdin-imx8mm-14684133 systemd[1]: Started Docker Compose service with docker compose.
Mar 19 23:02:04 verdin-imx8mm-14684133 docker-compose[904]: Container torizon-weston-1 Creating
Mar 19 23:02:04 verdin-imx8mm-14684133 docker-compose[904]: Container torizon-weston-1 Created
Mar 19 23:02:04 verdin-imx8mm-14684133 docker-compose[904]: Container torizon-libera_8-1 Creating
Mar 19 23:02:05 verdin-imx8mm-14684133 docker-compose[904]: Container torizon-libera_8-1 Created
Mar 19 23:02:05 verdin-imx8mm-14684133 docker-compose[904]: Container torizon-weston-1 Starting
Mar 19 23:02:05 verdin-imx8mm-14684133 docker-compose[904]: Container torizon-weston-1 Started
Mar 19 23:02:05 verdin-imx8mm-14684133 docker-compose[904]: Container torizon-libera_8-1 Starting
Mar 19 23:02:06 verdin-imx8mm-14684133 docker-compose[904]: Container torizon-libera_8-1 Started

Here are the logs of the application when the gray screen appears.

Mar 22 15:14:05 verdin-imx8mm-14684208 systemd[1]: Starting Docker Compose service with docker compose…
Mar 22 15:14:05 verdin-imx8mm-14684208 systemd[1]: Started Docker Compose service with docker compose.
Mar 22 15:14:06 verdin-imx8mm-14684208 docker-compose[913]: Container torizon-weston-1 Created
Mar 22 15:14:06 verdin-imx8mm-14684208 docker-compose[913]: Container torizon-libera_8-1 Created
Mar 22 15:14:06 verdin-imx8mm-14684208 docker-compose[913]: Container torizon-weston-1 Starting
Mar 22 15:14:07 verdin-imx8mm-14684208 docker-compose[913]: Container torizon-weston-1 Started
Mar 22 15:14:07 verdin-imx8mm-14684208 docker-compose[913]: Container torizon-libera_8-1 Starting
Mar 22 15:14:07 verdin-imx8mm-14684208 docker-compose[913]: Container torizon-libera_8-1 Started

Here is the information from the tdx-info script

Software summary

Bootloader: U-Boot
Kernel version: 5.15.129-6.5.0-devel+git.6f8fd49366db #1-TorizonCore SMP PREEMPT Tue Feb 27 22:23:06 UTC 2024
Kernel command line: root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/f2e676822951ef81195d8dd948be456e336cf286e09243987eb8b68bb417a685/0
Distro name: NAME=“TorizonCore”
Distro version: VERSION_ID=6.5.0-devel-20240228202213-build.0
Distro variant: VARIANT=“Docker”
Hostname: verdin-imx8mm-14684133

Hardware info

HW model: Toradex Verdin iMX8M Mini on Verdin Development Board
Toradex version: 0057 V1.1B
Serial number: 14684133
Processor arch: aarch64

docker-compose.yml (2.0 KB)

Greetings @Isaga,

I haven’t heard of such an observation before from others. Your logs don’t reveal too much info, so it’s not clear at the moment what might be happening on your system.

This “grey screen” that appears, how often would you say this occurs?

Also can you determine whether it’s Weston or your .NET application that is outputting the grey screen?

Finally, when this issue occurs could you try and get logs from the individual Weston and .NET containers? Perhaps one of these containers is throwing some kind of error or warning.

Best Regards,
Jeremias

As a data point, we also observed a single instance of a system somehow entering (or crashing?) to developer mode weston (and no application) under normal operation. In our case we are using Cog or Chromium to serve a browser-based UI, on TorizonCore 6.3.0 build 4 on a Verdin 8MP

We have not yet seen this recur beyond this sole occurrence and did not get to inspect it at the time.

Thanks jeremias. We are aware that it’s something strange because we’ve been working with the same application since TC 5.5.0 and we hadn’t identified that problem before

This “grey screen” that appears, how often would you say this occurs?

At the moment, we can say that around 40% of the flashed modules are experiencing this issue. We have been conducting tests, and when it occurs, we remove the containers and perform a ‘systemctl restart docker’. This causes the application to start, but not in all cases.

Finally, when this issue occurs could you try and get logs from the individual Weston and .NET containers? Perhaps one of these containers is throwing some kind of error or warning.

Sure, I’ll attach the logs
weston_bad.txt (6.7 KB)
weston_ok.txt (7.8 KB)
app-ok.txt (828 Bytes)

Also can you determine whether it’s Weston or your .NET application that is outputting the grey screen?

We’re currently running tests to confirm that.

Best Regards,

we’ve been working with the same application since TC 5.5.0 and we hadn’t identified that problem before

I see you’re now on 6.5.0. Did you only start seeing this issue when transitioning to this newer version of the OS?

Looking at your Weston logs I see one major difference between the “ok” and “bad” state. In the “bad” state it looks like XWayland never starts, these logs are missing in the “bad” state but are present in the “ok” state:

...
[17:32:59.751] Loading module '/usr/lib/aarch64-linux-gnu/libweston-9/xwayland.so'
[17:32:59.874] Registered plugin API 'weston_xwayland_v1' of size 32
[17:32:59.875] Registered plugin API 'weston_xwayland_surface_v1' of size 16
[17:32:59.877] xserver listening on display :0
...
[17:33:01.105] Spawned Xwayland server, pid 47
[     1] wl_drm_is_format_supported, format = 0x30335241
[     2] wl_drm_is_format_supported, format = 0x30335258
[     3] wl_drm_is_format_supported, format = 0x30334241
[     4] wl_drm_is_format_supported, format = 0x30334258
Disabling glamor and dri3 support, XWAYLAND_NO_GLAMOR is set
Failed to initialize glamor, falling back to sw
[17:33:01.273] xfixes version: 5.0
[17:33:01.297] created wm, root 71
The XKEYBOARD keymap compiler (xkbcomp) reports:
> Warning:          Unsupported maximum keycode 569, clipping.
>                   X11 cannot support keycodes above 255.
Errors from xkbcomp are not fatal to the X server

If your application relies on X then not having XWayland running would be problematic indeed.

Looking at your docker-compose.yml I noticed based on the way you’re starting the weston-vivante container that this must using the 2/bullseye tag for this container. Is that correct?

If that is the case could you adapt to instead use the 3/bookworm tag. Our 2/bullseye based containers were designed and optimized to run on Torizon OS 5.X. While they can run on 6.X it is much more preferable to use the 3/bookworm based containers instead. As these containers were designed and optimized to be used on 6.X. Perhaps this might be the issue itself.

Please try using the 3/bookworm based weston-vivante container instead and let me know if the issue still appears. Be aware that this newer version of the container requires slightly different arguments to be launched, you can’t just use the same arguments as you’ve been using. You can see in many of our articles like this: Debian Containers for Torizon | Toradex Developer Center

To see how to launch the 3/bookworm based weston-vivante container.

Best Regards,
Jeremias