Mouse and touch screen don't work with cog-imx8 container

I have a Verdin iMX8M-Plus with:

  • Torizon OS 6.8.0-devel-202409+build.26
  • torizon/weston-vivante:3.5.0-20240910
  • torizon/cog:3.4.0-20240910

I conected a HDMI display with resolution 1280 x 1024 60 Hz and touch-screen connected through USB.
I connected an USB mouse too (to double check).
I set docker-compose.yaml to navigate to www.toradex.com

Here is what I see:

  • cog is not open fullscreen but it takes only a portion of the screen

  • touch screen doesn’t work (it works if I use chromium instead of cog)

  • the arrow pointer of the mouse is visible over the dark blue background of weston only if it’s outside of cog (maybe it’s below cog and not over?)

Can you reproduce this behavior?
Is it possible to have touch screen and mouse working with cog container?

Hi, I’m the Toradex AI Assistant. The Toradex Support team will follow up with you shortly!

Did you like my answer?

:+1: :-1:


Based on the context provided, it seems that you are experiencing issues with the Cog container on a Verdin iMX8M-Plus module where the touch screen and mouse are not functioning as expected. Here are some points to consider:

  1. Fullscreen Issue: Cog not opening in fullscreen might be related to the configuration of the Weston compositor or the Cog container itself. Ensure that the Weston configuration (weston.ini) is set up correctly to support fullscreen applications. You might need to adjust the weston.ini file to ensure that the Cog application is launched in fullscreen mode.

  2. Touch Screen and Mouse Functionality: The touch screen and mouse not working with Cog but working with Chromium suggests a possible configuration or compatibility issue with Cog. Since Cog is a minimal launcher for WebKit WPE, it might not handle input devices the same way Chromium does. Ensure that the necessary input devices are correctly passed to the Cog container. You might need to check if the touch and mouse devices are being recognized by the container and if any additional configuration is required.

  3. Mouse Pointer Visibility: The issue with the mouse pointer being visible only outside of Cog could be related to how Cog handles input focus and rendering. This might require further investigation into the Cog container’s configuration or Weston settings.

Unfortunately, without more specific details or logs, it’s challenging to provide a definitive solution. You might want to check the logs of both the Weston and Cog containers for any errors or warnings that could provide more insight into the issue. Additionally, consulting the Toradex community or support channels might provide more targeted assistance for this specific behavior.

Please, also check if these links can help you solve your problem:

GPU performance in Chromium container on Torizon OS iMX8M-Plus.

Greetings @vix,

This seems to be a known issue with Cog:

For the mouse cursor, in my experience the cursor is actually “invisible”. But, it’s still possible albeit difficult to move the invisible cursor and click on things. The mouse scroll wheel even works as well.

Now looking at the Cog repo again I can see there was a fix/improvement for cursor behavior: Fixed uninitialized wl_shm for cursor · Pull Request #512 · Igalia/cog · GitHub

The next issue though is that it seems these commits are only available in Cog 0.17.1 onward: wl: Fix wrong wl_shm for cursor · Igalia/cog@fe83f9b · GitHub

If you see our Dockerfile for the Cog container image: torizon-containers/debian-docker-images/imx/cog/Dockerfile at stable · torizon/torizon-containers · GitHub

We’re simply just installing Cog from the Debian feeds. Which means we’re installing this version 0.16.1 of Cog: Debian -- Details of package cog in bookworm

Meaning we can’t easily test if these commits even “fix” the issue or not. In summary, it seems like these issues are something that need to be addressed on the Cog side of things.

Best Regards,
Jeremias

Hi @jeremias.tx
with this kind of issues, I think that cog could not be a realistic alternative to implement a kiosk (at the moment, at least).
The mouse it’s only for my test, but the touch doesn’t work at all. I don’t talk about multitouch, swiping and so on, I talk about a simple one-finger touch.
It doesn’t work at all.

I had already seen that cog package for Debian bookworm is stuck to 0.16.1 (quite old) and so I opened an issue to the mantainers to ask for having it updated.
Cog mantainers suggested to build 0.18.4.
I tried and I can see that the mouse works as expected.
The one-finger touch doesn’t work reliably, but scroll works. Strange.

/----------/
EDIT:
Do you think that torizon/cog container could be built including the updated cog 0.18.4?
cog 0.16.1 is really old (and buggy)

Moreover, cog can be started maximized setting the environment variable in docker-compose.yaml

environment:
  COG_PLATFORM_WL_VIEW_FULLSCREEN: 1

Hey Vix,

I had already seen that cog package for Debian bookworm is stuck to 0.16.1 (quite old) and so I opened an issue to the mantainers to ask for having it updated.

It’s very unlikely that the Debian maintainers (who are probably different than the Cog maintainers from Igalia) will bump it. Major-bumping within a Debian Stable release is mainly done for security or stability, not for getting new software.

Do you think that torizon/cog container could be built including the updated cog 0.18.4?
cog 0.16.1 is really old (and buggy)

Jeremias and I let our project management know about this, so it’s a matter of prioritization.

One could always of course use Cog from Alpine (cog - Alpine Linux packages) or use a multi-staged build to build cog and transplant the binaries to a torizon/wayland-base as well.

Saluti,

1 Like

Hi @leon.tx

I asked to lgailia because the official mantainer for the Debian package is Alberto Garcia from lgalia.
But I understand their position.

use a multi-staged build to build cog and transplant the binaries to a torizon/wayland-base as well.

This is what I tried to do with this PR I’ve just opened to commontorizon.
I’ve opened to commontorizon (instead of torizon/containers) because I understood that commontorizon is a little bit more “experimental” than torizon/containers.
But I’m not sure if I understood properly.
If you prefer, I can open the same PR to troizon/containers

I asked to lgailia because the official mantainer for the Debian package is Alberto Garcia from lgalia.

Ah, I see! That makes sense.

build and install cog 0.18.4 in the container by escherstair · Pull Request #21 · commontorizon/Containerfiles · GitHub

Yea, something like that should work (actually quite smart to build it from the .dsc instead of source code, makes it for a cleaner Dockerfile overall, better than my previous suggestions).

I wouldn’t merge this change on torizon-containers specifically because we need to make sure our package state is “consistent” with the upstream Debian, basically meaning we’d backport the whole package into our feed, but as you said Common Torizon is more experimental so it should be fine there.

Have you experienced any of the previous bugs you related with this newer version?

Cheers,

1 Like

With this new release, mouse cursor works as expected (not hidden).
Touch screen is a little bit strange: scroll works perfectly, but one-finger click is not consistent. I cannot say it doesn’t work, but sometimes it doesn’t work…
I need to investigate

/* EDIT: -------- */
The suggestion from cog developers is to rebuild the latest stable wpewebkit too:

dget http://deb.debian.org/debian/pool/main/w/wpewebkit/wpewebkit_2.46.1-1.dsc
cd wpewebkit-2.46.1
sed -i 's/dpkg-dev[^,]*,//' debian/control
apt build-dep .
debian/rules binary

I tried but compiling on the Verdin SoM is really really slow.
I need to find away to cross compile wpewebkit to build a .deb package that can be copied and installed to the SoM.

We have a composite Action on GitHub for that: GitHub - torizon/torizon-deb-ci: A composite action consisting of all actions needed to build, test and sign Debian packages for Torizon OS.
Here’s how it’s used to build Weston, for example .gh/w/deb-builder: build also arm64 and amd64 · torizon/weston-deb@e854372 · GitHub.

We made this so people that want to tinker with the packages don’t need to deal with what I lovingly call Debian bureaucracy :slight_smile:

Thanks @leon.tx,
I haven’t understood the purpose of the tool.
The problem is that I use an Azure environment (DevOps + Azure Container Registry) and not github. So I’m not so expert about github actions.
I’ll try to use it on my forked commontorizon/container project to see if I’m able to have it working.

If I’m right, I imagine that it’s not straightforward using the action (that uder the hood uses torizon/build-deb-action that is a shell wrapper around dpkg-buildpackage) with the proposed way from lgalia (dget + ... + debian/rules)

I opened this issue on github because as far as I understand, using dget to download the sources to be built is not a supported scenario (at the moment).

Hi @leon.tx
I tried using torizon/torizon-deb-ci to build cog and wpewebkit, but I haven’t been able to have it working.
See here.
Do you have any suggestions?

Hey @vix

the issue is basically that libwpewebkit-2.0-dev doesn’t exist in Bookworm, so it tries to install that dependency and it fails: Debian -- Package Search Results -- libwpewebkit-2.0-dev.

Now you need to create that package and make it available from a package feed so it can pull that during the build (like we do here for Weston weston-deb/.github/workflows/deb-builder.yml at scarthgap/bookworm/12.0.3 · torizon/weston-deb · GitHub, because it has dependencies only available in our feed, such as libdrm-imx, imx-gpu-viv etc).

Packaging stuff for Debian is quite labour intensive, all dependencies need to be completely available.

This is what I learnt.
As I wrote, there is one thing I don’t understand: if I start from the torizon/cog container (which has cog 0.16.1 installed) and I run the same commands, I don’t see the missing libwpewebkit-2.0-dev dependency. But I don’t understand why.
For this reason I tried to install the official cog 0.16.1 into a plain bookworm container, but I see the libwpewebkit-2.0-dev dependency issue.

Can you explain this difference, please?

Moreover, it’s impossible for me to build wpewebkit, because on github free plan there is the 6 hour time limit for a job in a github action (see here).
A github paid plan is necessary to build wpewebkit

/---- EDIT ----/
I tried with Github Actions self-hosted runner but I can’t use it because torizon/torizon-deb-ci already specifies runs-on

Do you think that a new input parameter to specify the runner can be added?
Something like

on:
  workflow_call:
    inputs:
      runner:
        description: 'Runner to be used for this workflow'
        required: false
        type: string
        default: 'ubuntu-latest'

What do you think @leon.tx ?

Hello @vix,

Sorry for the delay on this topic.

Your suggestion here could make sense.
I think we did not implement this reusable workflow this way as it was not needed for our needs.

Therefore, if you want to be able to use an input for the runner, I would recommend that you fork this repository and do the changes needed.
If you want, you can submit a pull request and the Torizon team will evaluate if this can be merged into the repository or not.

Best Regards,
Bruno

Done here.
I hope that the Torizon team could merge it into the trunk.
Let me know.

1 Like

It has been merged into main branch.
But I can’t proceed because it seems that github doesn’t allow this:
Called workflows cannot be queued onto self-hosted runners across organizations/enterprises. Failed to queue this job. Labels: 'self-hosted'.
As far as I can read on github,

it seems not possible to use a public reusable workflow with a self-hosted runner

This is a show stopper for me

Hello @vix,

It looks like this is true: Reusing workflows - GitHub Docs

Therefore, I think the only way out here is to fork our reusable workflow in your organization/account and reference the reusable workflow repository that you have.
This requires rebasing from time to time, but this seems to be a GitHub limitation.

Best Regards,
Bruno