Core dump not dumping

Timo · November 9, 2022, 12:42pm

I’m trying to get my iMX8x to generate a core dump if my app meets an untimely demise.

The board has yocto oe based on the multimedia-reference-image.

Of course I followed the instructions here https://developer.toradex.com/linux-bsp/how-to/linux-features/enable-and-analyse-core-dumps-in-linux/

#zcat /proc/config.gz | grep CONFIG_COREDUMP
CONFIG_COREDUMP=y

To test this I have tried to generate signals

insert a std::raise(SIGKILL); into the code of my primary application;
kill PID from the command line;
Compile and run the following code :-

#include <stdio.h>

#include <assert.h>

int main()
{
	printf("Hello World!\n");
	float Inf = 1;
	Inf /= 0;
	printf( "%f", Inf );
	printf("Still here!\n");
	assert( 0 );
	printf("Still here !!!!\n");
	return 1/0;
}

Compiled as DEBUG and run from the command line :-

Hello World!
infStill here!
Crash: …/Crash/main.c:12: main: Assertion `0’ failed.
Aborted

I have tried setting the /proc/sys/kernel/core_pattern to both “/tmp/core” and also “core”.

I have tried setting /proc/sys/fs/suid_dumpable to 1 instead of 2

I have editted /etc/security/limits.conf to contain

* soft core unlimited

(see this )
though since I didn’t log out or reboot, the ulimit -c unlimited should still apply.

Nothing seems to work ! I do not see “(core dumped)” on the command line, and I can’t find a core file in /, /tmp/, /opt/Crash/bin/, /home/root/

What did I miss ?!

Timo · November 9, 2022, 12:53pm

I also just tried editting /etc/sysctl.d/99-sysctl.conf to have :-

# Change name of core file to start with the command name
# so you get things like: emacs.core mozilla-bin.core X.core
kernel.core_pattern = /tmp/core

And I added this to /etc/security/limits.conf :-

* hard core 1024

which didn’t help…

I presume the primary issue is that the core is not being dumped, rather than that I can’t find it.

josep.tx · November 16, 2022, 8:19am

Hello @Timo ,

The board has yocto oe based on the multimedia-reference-image

Are you using one of our reference images or is that a custom build?

Best regards,
Josep

Timo · November 16, 2022, 9:18am

Hi Josep - thanks for looking

A custom build bitbake tdx-reference-multimedia-image with a couple of modifications - but nothing that I can imagine that would affect the kernal…

Surely I must have missed something dumb ? But if it looks as if I am doing the right things, I can try again with the reference image.

josep.tx · November 16, 2022, 9:33am

Hello @Timo ,

I can try again with the reference image

Yes, please follow the tutorial with one of our reference images, and let us know the results.

Best regards,
Josep

Timo · November 16, 2022, 2:31pm

Hmmm. Not looking so good. I used TEZI to install the latest reference multimedia image from the server

root@colibri-imx8x-06858934:~# cat /etc/issue
TDX Wayland with XWayland 5.7.0+build.20 (dunfell) \n \l
Colibri-iMX8X_Reference-Multimedia-Image

Then did the following. The only difference from the tutorial is the name of the core file, to which I didn’t add any template placeholders. Let me know if you see something odd :-

root@colibri-imx8x-06858934:~# hello
Hello world
root@colibri-imx8x-06858934:~# systemctl status wayland-app-launch
● wayland-app-launch.service - Start a wayland application
     Loaded: loaded (/lib/systemd/system/wayland-app-launch.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-11-16 14:14:11 UTC; 2min 14s ago
    Process: 728 ExecStart=/usr/bin/wayland-app-launch.sh (code=exited, status=0/SUCCESS)
   Main PID: 772 (Qt5_CinematicEx)
      Tasks: 5 (limit: 1550)
     Memory: 83.3M
     CGroup: /system.slice/wayland-app-launch.service
             └─772 /usr/share/cinematicexperience-1.0/Qt5_CinematicExperience --fullscreen

Nov 16 14:14:08 colibri-imx8x-06858934 systemd[1]: Starting Start a wayland application...
Nov 16 14:14:11 colibri-imx8x-06858934 wayland-app-launch.sh[728]: /usr/bin/wayland-app-launch.sh: line 14: cd: HOME not set
Nov 16 14:14:11 colibri-imx8x-06858934 systemd[1]: Started Start a wayland application.
Nov 16 14:14:12 colibri-imx8x-06858934 wayland-app-launch.sh[772]: libpng warning: iCCP: known incorrect sRGB profile
Nov 16 14:14:12 colibri-imx8x-06858934 wayland-app-launch.sh[772]: libpng warning: iCCP: known incorrect sRGB profile
Nov 16 14:14:12 colibri-imx8x-06858934 wayland-app-launch.sh[772]: qt.qpa.wayland: Wayland does not support QWindow::requestActivate()
root@colibri-imx8x-06858934:~# ps aux | grep Qt
root         772 60.4  4.6 707528 84984 ?        Rl   14:14   1:57 /usr/share/cinematicexperience-1.0/Qt5_CinematicExperience --fullscreen
root         796  0.0  0.0   2748   644 pts/0    S+   14:17   0:00 grep Qt
root@colibri-imx8x-06858934:~# ulimit -c unlimited
root@colibri-imx8x-06858934:~# cat /proc/sys/kernel/core_uses_pid
1
root@colibri-imx8x-06858934:~# cat /proc/sys/kernel/core_pattern
core
root@colibri-imx8x-06858934:~# echo "/tmp/core" > /proc/sys/kernel/core_pattern
root@colibri-imx8x-06858934:~# cat /proc/sys/kernel/core_pattern
/tmp/core
root@colibri-imx8x-06858934:~# synch
-sh: synch: command not found
root@colibri-imx8x-06858934:~# sync
root@colibri-imx8x-06858934:~# ls -l /tmp/
total 560
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-bluealsa.service-kH6nIf
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-systemd-logind.service-7HKxAg
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-systemd-timesyncd.service-eR2aUg
-rw-r--r-- 1 root root 570877 Nov 16 14:14 viv_gc_img_builtin.lib
root@colibri-imx8x-06858934:~# cat /proc/sys/fs/suid_dumpable
0
root@colibri-imx8x-06858934:~# echo 2 /proc/sys/fs/suid_dumpable
2 /proc/sys/fs/suid_dumpable
root@colibri-imx8x-06858934:~# kill 772
root@colibri-imx8x-06858934:~# ls -l /tmp/
total 560
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-bluealsa.service-kH6nIf
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-systemd-logind.service-7HKxAg
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-systemd-timesyncd.service-eR2aUg
-rw-r--r-- 1 root root 570877 Nov 16 14:14 viv_gc_img_builtin.lib
root@colibri-imx8x-06858934:~# systemctl start wayland-app-launch
root@colibri-imx8x-06858934:~# ps aux | grep Qt
root         813 68.5  3.8 707532 70324 ?        Sl   14:23   0:08 /usr/share/cinematicexperience-1.0/Qt5_CinematicExperience --fullscreen
root         819  0.0  0.0   2748   620 pts/0    S+   14:23   0:00 grep Qt
root@colibri-imx8x-06858934:~# kill -813 [PID]
-sh: kill: 813: invalid signal specification
root@colibri-imx8x-06858934:~# kill -11 813
root@colibri-imx8x-06858934:~# ls -l /tmp/
total 560
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-bluealsa.service-kH6nIf
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-systemd-logind.service-7HKxAg
drwx------ 3 root root     60 Nov 16 14:14 systemd-private-9960267ee2794b999a187fdacd62b56f-systemd-timesyncd.service-eR2aUg
-rw-r--r-- 1 root root 570877 Nov 16 14:14 viv_gc_img_builtin.lib
root@colibri-imx8x-06858934:~#

josep.tx · November 17, 2022, 10:21am

Hello @Timo ,
I followed the instructions in the tutorial and it worked.
I started with a recovery and then installed the multimedia image.

colibri-imx8x-06748677 login: root
Last login: Thu Nov 17 10:08:31 UTC 2022
root@colibri-imx8x-06748677:~# cat /etc/issue
TDX Wayland with XWayland 5.7.0+build.20 (dunfell) \n \l
Colibri-iMX8X_Reference-Multimedia-Image
root@colibri-imx8x-06748677:~# ulimit -c unlimited
root@colibri-imx8x-06748677:~# echo 1 > /proc/sys/kernel/core_uses_pid
root@colibri-imx8x-06748677:~# echo "/tmp/core-%e-%s-%u-%g-%p-%t" > /proc/sys/kernel/core_pattern
root@colibri-imx8x-06748677:~# echo 2 > /proc/sys/fs/suid_dumpable

Then I get the PID of the demo app (Qt5_CinematicExperience) to send a signal 11 to it.

root@colibri-imx8x-06748677:~# ps aux |grep cinematic
root         755 62.0  4.7 707556 86584 ?        Sl   10:08   0:44 /usr/share/cinematicexperience-1.0/Qt5_CinematicExperience --fullscreen
root         769  0.0  0.0   2748   664 ttyLP3   S+   10:09   0:00 grep cinematic
root@colibri-imx8x-06748677:~# kill -11 755
[   93.554207] audit: type=1701 audit(1668679795.788:3): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=755 comm="Qt5_CinematicEx" exe="/usr/share/cinematicexperience-1.0/Qt5_CinematicExperience" sig=11 res=1
root@colibri-imx8x-06748677:~#

No signs of a core dump when listing the contents of the /tmp folder

root@colibri-imx8x-06748677:~# ls -la /tmp
total 568
drwxrwxrwt 10 root root    260 Nov 17 10:09 .
drwxr-xr-x 20 root root   4096 Nov 17 10:08 ..
drwxrwxrwt  2 root root     40 Nov 17 10:08 .ICE-unix
drwxrwxrwt  2 root root     40 Nov 17 10:08 .Test-unix
-r--r--r--  1 root root     11 Nov 17 10:08 .X0-lock
drwxrwxrwt  2 root root     60 Nov 17 10:08 .X11-unix
drwxrwxrwt  2 root root     40 Nov 17 10:08 .XIM-unix
-rw-r--r--  1 root root      0 Nov 17 10:08 .automount-mmcblk0p1
drwxrwxrwt  2 root root     40 Nov 17 10:08 .font-unix
drwx------  3 root root     60 Nov 17 10:08 systemd-private-4441f7a46b8a4e509df353cd17f8e36d-bluealsa.service-ycmhpf
drwx------  3 root root     60 Nov 17 10:08 systemd-private-4441f7a46b8a4e509df353cd17f8e36d-systemd-logind.service-b3Dmfj
drwx------  3 root root     60 Nov 17 10:08 systemd-private-4441f7a46b8a4e509df353cd17f8e36d-systemd-timesyncd.service-ccFpEh
-rw-r--r--  1 root root 570877 Nov 17 10:08 viv_gc_img_builtin.lib

Then I restarted the demo app and sent the signal 11 to it .

root@colibri-imx8x-06748677:~# /usr/share/cinematicexperience-1.0/Qt5_CinematicExperience &
[1] 779
root@colibri-imx8x-06748677:~# libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
kill -11 779
[  130.585458] audit: type=1701 audit(1668679832.821:4): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=779 comm="Qt5_CinematicEx" exe="/usr/share/cinematicexperience-1.0/Qt5_CinematicExperience" sig=11 res=1
root@colibri-imx8x-06748677:~#
[1]+  Segmentation fault      (core dumped) /usr/share/cinematicexperience-1.0/Qt5_CinematicExperience
root@colibri-imx8x-06748677:~#

This time the core was dumped .
Then I listed the contents of the /tmp folder

root@colibri-imx8x-06748677:~#
root@colibri-imx8x-06748677:~# ls -la /tmp/
total 132528
drwxrwxrwt 10 root root       280 Nov 17 10:10 .
drwxr-xr-x 20 root root      4096 Nov 17 10:08 ..
drwxrwxrwt  2 root root        40 Nov 17 10:08 .ICE-unix
drwxrwxrwt  2 root root        40 Nov 17 10:08 .Test-unix
-r--r--r--  1 root root        11 Nov 17 10:08 .X0-lock
drwxrwxrwt  2 root root        60 Nov 17 10:08 .X11-unix
drwxrwxrwt  2 root root        40 Nov 17 10:08 .XIM-unix
-rw-r--r--  1 root root         0 Nov 17 10:08 .automount-mmcblk0p1
drwxrwxrwt  2 root root        40 Nov 17 10:08 .font-unix
-rw-------  1 root root 168927232 Nov 17 10:10 core-Qt5_CinematicEx-11-0-0-779-1668679832
drwx------  3 root root        60 Nov 17 10:08 systemd-private-4441f7a46b8a4e509df353cd17f8e36d-bluealsa.service-ycmhpf
drwx------  3 root root        60 Nov 17 10:08 systemd-private-4441f7a46b8a4e509df353cd17f8e36d-systemd-logind.service-b3Dmfj
drwx------  3 root root        60 Nov 17 10:08 systemd-private-4441f7a46b8a4e509df353cd17f8e36d-systemd-timesyncd.service-ccFpEh
-rw-r--r--  1 root root    570877 Nov 17 10:08 viv_gc_img_builtin.lib
root@colibri-imx8x-06748677:~#

And the core file was there.

Can you try the same for your program and let me know if it works?

Best regards,
Josep

josep.tx · December 2, 2022, 8:21am

Hello @Timo ,
Do you have any news about this issue?

Best regards,
Josep

Timo · December 2, 2022, 8:46am

Hi @josep.tx

¡Ugh! No - sorry - I have been distracted by other important things. The core dump is just in case - naturally it is impossible that my software crashes

Let’s close the issue - I must have done something dumb - it’s unbeleivable that it should not work…

Timo · February 16, 2023, 8:57am

I have finally got around to this again.

Just to confirm - it is working. Just a question of getting all your ducks in a line…

In my test above for example, I notice now, I made a typo :-

echo 2 /proc/sys/fs/suid_dumpable

( I missed the > !)

Other things to bear in mind are that these settings do not persist, and that the directory for the core must exist (thus simply “core” will do nothing).

Thanks for the support !

Tim