I’m just getting starting with a IMX8M-mini board and running a container with weston with VNC/RDP enabled (as seen here) and chromium to display a webpage (as seen here). I have my board connected to an external display which shows the expected webpage and responds to inputs from a connected mouse. However, I would also like to get remote access to the GUI.
I can get access to the GUI with both VNC and RDP (even though only one can be enabled at a time). However, it’s not responding to the remote mouse inputs. On top of this, it seems there is a huge lag 30 seconds sometimes. I’m accessing the board in a local network.
Kernel version: 5.15.77-6.2.0+git.aa0ff7e3554e #1-TorizonCore SMP PREEMPT Wed Mar 29 15:33:40 UTC 2023
Distro name: NAME="TorizonCore"
Distro version: VERSION_ID=6.2.0-build.2
Hostname: verdin-imx8mm-07218371
I ran into a similar problems to what you did, but with Yocto rather than Torizon Core.
For the lag you’re experiencing, I believe that’s because the VNC/RDP implementation in Weston 9/10 (which I assume is what you’re using in Torizon Core) is very immature. In Yocto, I have used Weston built from a recent main branch commit, and the VNC lag is gone (as far as I could tell).
Now for the input issue;
At it’s core, the issue is caused because Chromium does not correctly handle Weston seats (as far as I can tell). When you connect via RDP/VNC, Weston will create a new seat for input purposes, but Chromium will ignore that which means it can’t receive input from the new seat.
If I recall correctly, the solution was to patch some files in Chromium:
In ui/ozone/platform/wayland/host/wayland_connection.*;
Add some way to track extra seats, need to be able to release them too
Make sure when Chromium starts, the primary seat is still put into the correct field, rather than however you track extra seats. Some functionality depends on accessing the primary seat.
In ui/ozone/platform/wayland/host/wayland_seat.*
I think I attached some extra information from Wayland (that Chromium discards) to help distinguish seats
Make sure to initialize all the relevant stuff for a seat, should be fairly similar to whatever the primary seat does perhaps?
Honestly I did this a while ago. I don’t know how useful this is to you, but I might try to create a patch based on the meta-browser Yocto layer.
I was able to reproduce this on my side as well. I believe the observation from FLAC is correct and that this is a chromium specific issue. I confirmed this by trying the Cog browser, which does react accordingly to VNC mouse inputs.
I don’t believe this issue has been reported to us before so I can bring it up with our team. However, if the issue is really with Chromium specifically our team may be limited with what we can do in a quick time-frame.
I’m not sure how Toradex internally builds their Chromium containers, but I hope this helps enable VNC/RDP input functionality.
For reference we take the usual Chromium release and re-package it for Debian. The reason we re-package it is because the Chromium that comes form the standard Debian feed is built/optimized for X11 instead of Wayland. So in theory you could build Chromium with this patch and package it for Debian inside a container for use.
@FLAC , thanks for you effor and the patch @jvieira , please let us know the outcome of your tests, and if they are positive, you can mark @FLAC 's answer as solution.
I got a similar problem but its not even working correctly with cog for me.
My setup is a verdin-imx8mp on a custom carrier board with a resistive touchscreen.
I use TorizonCore 6.3.0 as a base image and added some devicetree overlays for our carrier board.
I use the following commands to start the docker containers:
After the containers are started and there is a new connection via VNC, the screen is displayed correctly on the VNC client, but cog does not react to input.
I get the following logs when connecting via VNC:
Oct 10 14:07:29 verdin-imx8mp-07321126 3fcd6e829bde[801]: [14:07:29.236] New VNC client connected Oct 10 14:07:29 verdin-imx8mp-07321126 86dcd6b14608[801]: (cog:1): Cog-Wayland-DEBUG: 14:07:29.244: Using 'wl_seat' interface obtained from the Wayland registry. Oct 10 14:07:29 verdin-imx8mp-07321126 86dcd6b14608[801]: (cog:1): Cog-Wayland-DEBUG: 14:07:29.246: Ignoring 'zwp_input_method_v1' interface obtained from the Wayland registry.
When I only start a weston container without cog, the input is working (from VNC and touchscreen).
The only way to get the input to cog for me is, to first start weston, then connect via VNC and after that start the cog container.
But when I do so, the input from the touchscreen does not work anymore.
@jeremias.tx Did you do anything different to get cog and VNC working?
@erik.weber I just did a re-test and I observed the same behavior you described. I think before I was keeping VNC running with the same Weston container, while switching between Cog and Chromium containers to do a comparison. I guess before I didn’t do a test where I started Weston and Cog from scratch and then connect with VNC, which is then problematic.
I’ll report this internally.
In the meantime, is this an important or blocking issue for you?
That issue you linked from the upstream Cog repo definitely seems like the problem here. I guess Cog only accepts a single input source at a time. This would explain why you only have either VNC or your touchscreen working and not both.
Is not actually available in the master branch or any release. It seems it’s only available in some side development branches.
I suppose one could theoretically take this commit and try to re-base it to the current Cog release and re-build with the fix. But this commit is 3 years old now so I doubt it would apply very cleanly, and that is assuming the code even works the same as it did when this commit was created.
Unfortunately we might be at the mercy here of this getting fixed in Cog upstream. Is using the patch with Chromium as mentioned earlier in this thread an option for you?
Hi @jeremias.tx@FLAC I am getting same issue, Mouse not work when I launch chromium with weston rdp. I am using chromium-ozone-wayland_96.0.4664.110. Is there any fix or patch for this version ?
