Screen Capture Issue on Wayland Application Running with Weston Compositor on iMX7D SoM

[Saikrishna]

Hello Toradex Community,

I’m developing an application on a Toradex iMX7D SoM running TorizonCore with Weston as the compositor in a Wayland based environment. My application uses GTK for the GUI, and I’m trying to implement screen capture functionality. My application is running inside a Docker container, and I’ve tried multiple approaches to capture the screen without success.

Here’s what I’ve attempted so far:

1. Using Grim and Fgrab: I installed and tried using both grim and fgrab tools for screen capturing. While the tools execute without errors, the captures result in blank screens.
2. Framebuffer Approach (/dev/fb): I also tried capturing the framebuffer directly by accessing /dev/fb0, but that didn’t give the expected results either, still leading to blank captures.
3. Ensuring the Right Display is Targeted: In my docker-compose.yml, I’ve ensured the Wayland display socket is correctly mapped:

volumes:
   - /run/user/1000/wayland-0:/run/user/1000/wayland-0
environment:
   - WAYLAND_DISPLAY=wayland-0

Generally, in board WAYLAND_DISPLAY is showing blank. So, I’ve also set WAYLAND_DISPLAY to wayland-0 and tried for screen capture, but not use it is setting back to blank again.

To confirm which compositor is running I checked with ps aux | grep weston

torizon@colibri-imx7-emmc-15022057:~$ ps aux | grep weston
root      1194  0.0  0.0   3324   372 ?        S    12:08   0:00 setsid -w -f openvt -w -f -s -c 7 -e -- bash -c /usr/bin/weston-launch --tty=/dev/tty7 --user=torizon -- --current-mode --use-pixman > >(tee /proc/1/fd/1) 2> >(tee /proc/1/fd/2)
root      1195  0.0  0.1   4532  1884 tty7     Ss+  12:08   0:00 bash -c /usr/bin/weston-launch --tty=/dev/tty7 --user=torizon -- --current-mode --use-pixman > >(tee /proc/1/fd/1) 2> >(tee /proc/1/fd/2)
root      1196  0.0  0.2   8696  2640 tty7     S+   12:08   0:00 /usr/bin/weston-launch --tty=/dev/tty7 --user=torizon -- --current-mode --use-pixman
root      1197  0.0  0.0   4532   896 tty7     S+   12:08   0:00 bash -c /usr/bin/weston-launch --tty=/dev/tty7 --user=torizon -- --current-mode --use-pixman > >(tee /proc/1/fd/1) 2> >(tee /proc/1/fd/2)
root      1198  0.0  0.0   4532   896 tty7     S+   12:08   0:00 bash -c /usr/bin/weston-launch --tty=/dev/tty7 --user=torizon -- --current-mode --use-pixman > >(tee /proc/1/fd/1) 2> >(tee /proc/1/fd/2)
torizon   1208  2.5  3.2  67332 33268 tty7     S+   12:08   2:21 /usr/bin/weston --current-mode --use-pixman
torizon   1228  0.0  0.7  16524  7896 tty7     S+   12:08   0:00 /usr/lib/arm-linux-gnueabihf/weston-keyboard
torizon   1239  0.0  1.4  23004 14340 tty7     S+   12:08   0:00 /usr/lib/arm-linux-gnueabihf/weston-desktop-shell
torizon   1538  0.0  0.1   2156  1060 pts/0    S+   13:40   0:00 grep weston

When running Weston, I observed that launching Weston with the --current-mode option fails with a “Broken pipe” error, and there’s also a warning regarding XDG_RUNTIME_DIR not being configured correctly, log mentioned below for your reference.

root@colibri-imx7-emmc-14952875:/home/torizon# weston --current-mode --use-pixman
Date: 2024-09-28 UTC
[11:05:11.042] weston 9.0.0
               https://wayland.freedesktop.org
               Bug reports to: https://gitlab.freedesktop.org/wayland/weston/issues/
               Build: 9.0.0
[11:05:11.044] Command line: weston --current-mode --use-pixman
[11:05:11.046] OS: Linux, 6.1.71-6.5.0+git.38fb82ecd144, #1-TorizonCore SMP Tue Dec 19 14:52:07 UTC 2023, armv7l
[11:05:11.048] warning: XDG_RUNTIME_DIR "/tmp/1000-runtime-dir" is not configured
correctly.  Unix access mode must be 0700 (current mode is 700),
and must be owned by the user (current owner is UID 1000).
Refer to your distribution on how to get it, or
http://www.freedesktop.org/wiki/Specifications/basedir-spec
on how to implement it.
[11:05:11.056] Using config file '/etc/xdg/weston/weston.ini'
[11:05:11.059] Output repaint window is 7 ms maximum.
[11:05:11.062] Loading module '/usr/lib/arm-linux-gnueabihf/libweston-9/wayland-backend.so'
[11:05:11.132] Registered plugin API 'weston_windowed_output_api_v1' of size 8
[11:05:11.133] Creating 1024x640 wayland output at (0, 0)
[11:05:11.138] wayland-backend: Using xdg_wm_base
[11:05:11.196] Output 'wayland0' enabled with head(s) wayland0
[11:05:11.197] Compositor capabilities:
               arbitrary surface rotation: yes
               screen capture uses y-flip: no
               presentation clock: CLOCK_MONOTONIC_RAW, id 4
               presentation clock resolution: 0.000000001 s
[11:05:11.202] libwayland: unable to lock lockfile /tmp/1000-runtime-dir/wayland-0.lock, maybe another compositor is running
[11:05:11.205] Loading module '/usr/lib/arm-linux-gnueabihf/weston/desktop-shell.so'
[11:05:11.212] launching '/usr/lib/arm-linux-gnueabihf/weston-keyboard'
[11:05:11.219] Loading module '/usr/lib/arm-linux-gnueabihf/weston/screen-share.so'
[11:05:11.241] Screen share failed: No wl_shm found
[11:05:11.243] Loading module '/usr/lib/arm-linux-gnueabihf/libweston-9/xwayland.so'
[11:05:11.254] Registered plugin API 'weston_xwayland_v1' of size 16
[11:05:11.255] Registered plugin API 'weston_xwayland_surface_v1' of size 8
[11:05:11.257] xserver listening on display :1
[11:05:11.258] fatal: unhandled option: --current-mode
root@colibri-imx7-emmc-14952875:/home/torizon# Failed to process Wayland connection: Broken pipe
failed to create display: Broken pipe

Despite these attempts, I’m still getting blank captures. Could anyone provide guidance on further troubleshooting or share alternative methods that might work for capturing the screen in a Wayland + Weston setup?

Thanks in advance for any help!

Software summary

Bootloader: U-Boot
Kernel version: 6.1.71-6.5.0+git.38fb82ecd144 #1-TorizonCore SMP Tue Dec 19 14:52:07 UTC 2023
Kernel command line: root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/b9a2dcf7fbf43a6103fa51110ba4e2541289013d869f8e87182cf862a2f2bedb/0 console=null
Distro name: NAME=“TorizonCore Upstream”
Distro version: VERSION_ID=6.5.0-build.8
Distro variant: VARIANT=“Docker”
Hostname: colibri-imx7-emmc-14952875

Hardware info

HW model: Toradex Colibri iMX7D 1GB (eMMC) on Colibri Evaluation Board V3
Toradex version: 0039 V1.1B
Serial number: 14952875
Processor arch: armv7l

[ToradexAI]

Hi, I’m the Toradex AI Assistant. The Toradex Support team will follow up with you shortly!

Did you like my answer?

---

It seems like you’re encountering issues with screen capturing in a Wayland + Weston setup on a Toradex iMX7D SoM. Here are a few suggestions and considerations based on the context provided:

1. Grim and Fgrab: These tools might not be fully compatible with your current setup or the Wayland compositor configuration. Ensure that the Wayland environment is correctly set up and that the tools are compatible with the version of Weston you are using.

2. Framebuffer Approach: Accessing /dev/fb0 directly might not work as expected in a Wayland environment, as Wayland does not use the framebuffer in the same way X11 does. This approach is generally not recommended for Wayland.

3. Wayland Display Socket: Ensure that the Wayland display socket is correctly mapped and accessible within your Docker container. The environment variable WAYLAND_DISPLAY=wayland-0 should match the socket name used by Weston.

4. XDG_RUNTIME_DIR Warning: The warning about XDG_RUNTIME_DIR suggests a potential misconfiguration. Ensure that the directory has the correct permissions and ownership. The directory should be owned by the user running Weston and have permissions set to 0700.

5. Weston Configuration: Check the weston.ini configuration file for any settings that might affect screen capturing. Ensure that the necessary modules for screen sharing or capturing are loaded.

6. Alternative Methods: Consider using Weston’s built-in screencast functionality. According to the context, you can start and stop screencast recording with the <mod-key> + r combination. Ensure that the modifier key is correctly set in the weston.ini file.

7. Debugging: Use WAYLAND_DEBUG=1 to get more detailed logs about what is happening with the Wayland connection. This might provide more insights into why the screen capture is resulting in blank screens.

If these suggestions do not resolve the issue, it might be beneficial to consult additional resources or reach out to the Toradex community for further assistance.

Please, also check if these links can help you solve your problem:

https://bbs.archlinux.org/viewtopic.php?id=177123

Torizon Display calibration not working

Hi I would like to calibrate my 7’’ capacitive touch display (toradex hw) as described here: Touch Screen Calibration (Torizon) | Toradex Developer Center. But the display is black, nothing happens. I see in the log that some cursors could not be loaded. [2446-ddee.png] colibri-imx8x-06748730:~$ docker run -ti --rm --privileged -v /dev:/dev -v /run/udev/:/run/udev/ -v /etc/udev/rules.d:/etc/udev/rules.d -e ACCEPT_FSL_EULA=1 torizon/arm64v8-debian-weston-touch-calibrator NXP EULA has already …

https://community.nxp.com/t5/i-MX-Processors/Permanent-calibration-problem-on-weston/m-p/1265864.

[Saikrishna]

3. Wayland Display Socket: Ensure that the Wayland display socket is correctly mapped and accessible within your Docker container. The environment variable WAYLAND_DISPLAY=wayland-0 should match the socket name used by Weston.

However, after setting the WAYLAND_DISPLAY=wayland-0, wayland display socket (wayland-0) is not being created in the /run/user/1000 directory which of course has user permissions. So how can I get this socket created?

[jeremias.tx]

Greetings @Saikrishna,

I recall Weston-wayland can be quite picky about what can get information from the screen including screenshots. In our own internal CI testing for screenshots we use weston-screenshooter. Though to use this tool you need to enable debug mode in weston, otherwise it will block the screenshot attempt. Looking online this is what most people seem to recommend for screenshots in Weston-wayland.

For what purpose exactly do you need a screen capture function?

Best Regards,
Jeremias

[Saikrishna]

Hi @jeremias.tx,

For what purpose exactly do you need a screen capture function?

We are developing a medical application that allows users to take screen captures for observation and diagnostic purposes. Our application uses GTK for the GUI and runs inside a Docker container.

I reviewed the following resource, which was referenced in the weston.ini file:
Working with Weston on Torizon OS | Toradex Developer Center.
By adding binding-modifier=super in the weston.ini file, I can capture the screen using the Super (Windows) key. However, this only works when using the Weston terminal, not the application terminal.

Our primary requirement is to capture the screen directly from the application (running inside the Docker container). Please let us know if any further information is needed to help resolve this issue.

Regards,
Saikrishna

[jeremias.tx]

Our primary requirement is to capture the screen directly from the application

This is probably pretty application specific. There is another thread here about mimicing keyboard events in Weston that may be of help to you: Mimic Keyboard for Weston

Best Regards,
Jeremias