Kernel panic when booting linux when M4 was programmed

Technical Support
, , , ,
1

Hi,

I followed the instructions from here to evaluate the M4 processor on my brand-new Colibri MX7 board.
Whenever i load a demo program in u-boot i can see the M4s debug output on my console. However when i attempt to boot the linux system i get the kernel panic below (even with the very simple hello_world example). The pre-installed image version is “Colibri-iMX7_LXDE-Image 2.7.2 20170410”.
I read that i have to take care to not use resources with both CPUs. Could this be the reason? Do i need to compile the kernel by my own and to edit the device tree?

Thaks for help!

Booting from NAND...
ubi0: default fastmap pool size: 200
ubi0: default fastmap WL pool size: 100
ubi0: attaching mtd1
ubi0: attached by fastmap
ubi0: fastmap pool size: 200
ubi0: fastmap WL pool size: 100
ubi0: attached mtd1 (name "mtd=4", size 508 MiB)
ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 4060, bad PEBs: 4, corrupted PEBs: 0
ubi0: user volume: 4, internal volumes: 1, max. volumes count: 128
ubi0: max/mean erase counter: 2/1, WL threshold: 4096, image sequence number: 0
ubi0: available PEBs: 0, total reserved PEBs: 4060, PEBs reserved for bad PEB handling: 76
Read 0 bytes from volume kernel to 81000000
No size specified -> Using max size (5113344)
Read 0 bytes from volume dtb to 82000000
No size specified -> Using max size (43480)
Kernel image @ 0x81000000 [ 0x000000 - 0x4e0600 ]
## Flattened Device Tree blob at 82000000
   Booting using the fdt blob at 0x82000000
   Using Device Tree in place at 82000000, end 8200d9d7
   Updating MTD partitions...

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.1.39-2.7.2+gba2f75f (linuxdev@linuxdev.toradex.int) (gcc version 6.2.1 20161016 (Linaro GCC 6.2-2016.11) ) #3 SMP Mon Apr 10 03:50:20 CE7
[    0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine model: Toradex Colibri iMX7D on Colibri Evaluation Board V3
[    0.000000] Reserved memory: created CMA memory pool at 0x86000000, size 128 MiB
[    0.000000] Reserved memory: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] PERCPU: Embedded 12 pages/cpu @8fb82000 s16844 r8192 d24116 u49152
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 64516
[    0.000000] Kernel command line: ubi.mtd=ubi root=ubi0:rootfs rootfstype=ubifs ubi.fm_autoconvert=1 console=tty1 console=ttymxc0,115200n8 consoleblank=0
[    0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Memory: 116668K/260096K available (6406K kernel code, 285K rwdata, 2152K rodata, 312K init, 380K bss, 12356K reserved, 131072K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xa0800000 - 0xff000000   (1512 MB)
[    0.000000]     lowmem  : 0x80000000 - 0xa0000000   ( 512 MB)
[    0.000000]     modules : 0x7f000000 - 0x80000000   (  16 MB)
[    0.000000]       .text : 0x80008000 - 0x80863cc0   (8560 kB)
[    0.000000]       .init : 0x80864000 - 0x808b2000   ( 312 kB)
[    0.000000]       .data : 0x808b2000 - 0x808f94c0   ( 286 kB)
[    0.000000]        .bss : 0x808fc000 - 0x8095b16c   ( 381 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] Architected cp15 timer(s) running at 8.00MHz (phys).
[    0.000000] clocksource arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1d854df40, max_idle_ns: 440795202120 ns
[    0.000006] sched_clock: 56 bits at 8MHz, resolution 125ns, wraps every 2199023255500ns
[    0.000018] Switching to timer-based delay loop, resolution 125ns
[    0.000452] mxc_clocksource_init 3000000
[    0.000464] Ignoring duplicate/late registration of read_current_timer delay
[    0.000481] clocksource mxc_timer1: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 637086815595 ns
[    0.001642] Console: colour dummy device 80x30
[    0.002270] console [tty1] enabled
[    0.002296] Calibrating delay loop (skipped), value calculated using timer frequency.. 16.00 BogoMIPS (lpj=80000)
[    0.002338] pid_max: default: 32768 minimum: 301
[    0.002455] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.002483] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.003111] Initializing cgroup subsys freezer
[    0.003150] Initializing cgroup subsys debug
[    0.003202] CPU: Testing write buffer coherency: ok
[    0.003503] Setting up static identity map for 0x80008280 - 0x800082d8
[    0.005441] Brought up 2 CPUs
[    0.005473] SMP: Total of 2 processors activated (32.00 BogoMIPS).
[    0.005495] CPU: All CPU(s) started in SVC mode.
[    0.006016] devtmpfs: initialized
[    0.019095] VFP support v0.3: implementor 41 architecture 2 part 30 variant 7 rev 5
[    0.019493] clocksource jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.019555] futex hash table entries: 512 (order: 3, 32768 bytes)
[    0.025579] pinctrl core: initialized pinctrl subsystem
[    0.026817] NET: Registered protocol family 16
[    0.032674] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.060384] cpuidle: using governor ladder
[    0.090402] cpuidle: using governor menu
[    0.103815] DDR type is DDR3!
[    0.111067] imx7d_enet_clk_sel: failed to get enet_out clock, assuming ext. clock source
[    0.111156] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[    0.111191] hw-breakpoint: maximum watchpoint size is 8 bytes.
[    0.111536] imx7d-pinctrl 30270000.lpsr-gpr: initialized IMX pinctrl driver
[    0.111736] imx7d-pinctrl 302c0000.iomuxc-lpsr: initialized IMX pinctrl driver
[    0.112993] imx7d-pinctrl 30330000.iomuxc: initialized IMX pinctrl driver
[    0.114642] MU is ready for cross core communication!
[    0.124376] mxs-dma 33000000.dma-apbh: initialized
[    0.126212] VCC_USB[1-4]: supplied by 5V
[    0.126682] SCSI subsystem initialized
[    0.127008] usbcore: registered new interface driver usbfs
[    0.127102] usbcore: registered new interface driver hub
[    0.127225] usbcore: registered new device driver usb
[    0.127489] 30800000.aips-bus:usbphy_nop1 supply vcc not found, using dummy regulator
[    0.127785] 30800000.aips-bus:usbphy_nop2 supply vcc not found, using dummy regulator
[    0.128807] i2c i2c-0: IMX I2C adapter registered
[    0.128840] i2c i2c-0: can't use DMA
[    0.129330] i2c i2c-3: IMX I2C adapter registered
[    0.129361] i2c i2c-3: can't use DMA
[    0.129490] Linux video capture interface: v2.00
[    0.129561] pps_core: LinuxPPS API ver. 1 registered
[    0.129584] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.129643] PTP clock support registered
[    0.130169] Advanced Linux Sound Architecture Driver Initialized.
[    0.131564] Switched to clocksource arch_sys_counter
[    0.142572] NET: Registered protocol family 2
[    0.143404] TCP established hash table entries: 2048 (order: 1, 8192 bytes)
[    0.143467] TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
[    0.143527] TCP: Hash tables configured (established 2048 bind 2048)
[    0.143602] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    0.143652] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    0.143835] NET: Registered protocol family 1
[    0.144153] RPC: Registered named UNIX socket transport module.
[    0.144182] RPC: Registered udp transport module.
[    0.144203] RPC: Registered tcp transport module.
[    0.144224] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.146229] virtio_rpmsg_bus virtio0: rpmsg host is online
[    0.146354] imx rpmsg driver is registered.
[    0.147327] Bus freq driver module loaded
[    0.155320] VFS: Disk quotas dquot_6.6.0
[    0.155545] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[    0.157466] NFS: Registering the id_resolver key type
[    0.157533] Key type id_resolver registered
[    0.157555] Key type id_legacy registered
[    0.157592] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.158132] ntfs: driver 2.1.32 [Flags: R/W].
[    0.158618] fuse init (API version 7.23)
[    0.162246] io scheduler noop registered
[    0.162284] io scheduler deadline registered
[    0.162488] io scheduler cfq registered (default)
[    0.164468] backlight supply power not found, using dummy regulator
[    0.166465] 30730000.lcdif supply lcd not found, using dummy regulator
[    0.232043] mxsfb 30730000.lcdif: failed to find mxc display driver 
[    0.240421] Console: switching to colour frame buffer device 80x30
[    0.258139] mxsfb 30730000.lcdif: initialized
[    0.260952] imx-sdma 30bd0000.sdma: no event needs to be remapped
[    0.262785] imx-sdma 30bd0000.sdma: loaded firmware 4.2
[    0.267628] imx-sdma 30bd0000.sdma: initialized
[    0.270407] 30860000.serial: ttymxc0 at MMIO 0x30860000 (irq = 280, base_baud = 1500000) is a IMX
[    0.990153] console [ttymxc0] enabled
[    0.996235] Unhandled fault: external abort on non-linefetch (0x1008) at 0xa08c0090
[    1.007387] pgd = 80004000
[    1.011842] [a08c0090] *pgd=8e000811, *pte=30890653, *ppte=30890453
[    1.019987] Internal error: : 1008 [#1] SMP ARM
[    1.026284] Modules linked in:
[    1.031104] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 4.1.39-2.7.2+gba2f75f #3
[    1.041950] Hardware name: Freescale i.MX7 Dual (Device Tree)
[    1.049553] task: 8e058000 ti: 8e05c000 task.ti: 8e05c000
[    1.056830] PC is at serial_imx_probe+0x214/0x3f0
[    1.063423] LR is at clk_core_get_rate+0x3c/0x60
[    1.069941] pc : [<80355870>]    lr : [<8049ffd4>]    psr: 20000113
[    1.069941] sp : 8e05de48  ip : 00000000  fp : 00000000
[    1.085303] r10: 8e170880  r9 : fffffffa  r8 : 00000119
[    1.092431] r7 : 8e172a10  r6 : 8e172a00  r5 : 8e065a40  r4 : 8e0b4410
[    1.100860] r3 : a08c0000  r2 : 00000001  r1 : 00000000  r0 : 016e3600
[    1.109272] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
[    1.120290] Control: 10c5387d  Table: 8000406a  DAC: 00000015
[    1.127970] Process swapper/0 (pid: 1, stack limit = 0x8e05c210)
[    1.135955] Stack: (0x8e05de48 to 0x8e05e000)
[    1.142254] de40:                   8e34af00 8e174410 807bee20 8e172a10 8e172a10 fffffffe
[    1.154289] de60: 808d6f90 fffffdfb 00000000 808a295c 00000007 80361398 8094d8c8 8e172a10
[    1.166376] de80: 808d6f90 00000000 00000000 8035fc24 8e172a10 808d6f90 8e172a44 00000000
[    1.178615] dea0: 00000000 8035fe28 00000000 808d6f90 8035fd9c 8035e2cc 8e05ea5c 8e166f34
[    1.190981] dec0: 808d6f90 8e369e80 808d7f40 8035f360 807bf738 808d6fd4 808d6f90 808d6f90
[    1.203423] dee0: 808b9860 8e36fb00 80884e28 80360454 00000000 808b9860 8e36fb00 80884e4c
[    1.215952] df00: 808b9860 80009678 808bfb64 808a2938 00000000 801414ec 808bfc00 8e120600
[    1.228641] df20: 00000000 808be4c0 80836f68 0000003d 8fb9c469 8004517c 00000000 807d58dc
[    1.241478] df40: 00000006 00000006 8fb9c477 8078655c 808be4a8 000000a6 808fc000 000000a6
[    1.254504] df60: 808fc000 808fc000 808ac13c 808a2954 808a295c 80864db8 00000006 00000006
[    1.267666] df80: 808645a0 806171bc 00000000 806171bc 00000000 00000000 00000000 00000000
[    1.280884] dfa0: 00000000 806171c4 00000000 8000f328 00000000 00000000 00000000 00000000
[    1.294215] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[    1.307627] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 8f6f7dd7 43f98bfb
[    1.321075] [<80355870>] (serial_imx_probe) from [<80361398>] (platform_drv_probe+0x48/0xa4)
[    1.334782] [<80361398>] (platform_drv_probe) from [<8035fc24>] (driver_probe_device+0x174/0x2a8)
[    1.348932] [<8035fc24>] (driver_probe_device) from [<8035fe28>] (__driver_attach+0x8c/0x90)
[    1.362645] [<8035fe28>] (__driver_attach) from [<8035e2cc>] (bus_for_each_dev+0x68/0x9c)
[    1.376100] [<8035e2cc>] (bus_for_each_dev) from [<8035f360>] (bus_add_driver+0xf4/0x1e8)
[    1.389560] [<8035f360>] (bus_add_driver) from [<80360454>] (driver_register+0x78/0xf4)
[    1.402852] [<80360454>] (driver_register) from [<80884e4c>] (imx_serial_init+0x24/0x40)
[    1.416239] [<80884e4c>] (imx_serial_init) from [<80009678>] (do_one_initcall+0x8c/0x1d0)
[    1.429719] [<80009678>] (do_one_initcall) from [<80864db8>] (kernel_init_freeable+0x144/0x1d4)
[    1.443733] [<80864db8>] (kernel_init_freeable) from [<806171c4>] (kernel_init+0x8/0xe8)
[    1.457136] [<806171c4>] (kernel_init) from [<8000f328>] (ret_from_fork+0x14/0x2c)
[    1.470008] Code: e5d43118 e3130002 0a000017 e5943008 (e593a090) 
[    1.478805] ---[ end trace fe16bd731d3da7b7 ]---
[    1.486133] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    1.486133] 
[    1.502930] CPU0: stopping
[    1.508069] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G      D         4.1.39-2.7.2+gba2f75f #3
[    1.521298] Hardware name: Freescale i.MX7 Dual (Device Tree)
[    1.529454] [<8001572c>] (unwind_backtrace) from [<8001279c>] (show_stack+0x10/0x14)
[    1.541902] [<8001279c>] (show_stack) from [<80618cb4>] (dump_stack+0x84/0x98)
[    1.553814] [<80618cb4>] (dump_stack) from [<80014a8c>] (handle_IPI+0x14c/0x16c)
[    1.565887] [<80014a8c>] (handle_IPI) from [<800093e4>] (gic_handle_irq+0x5c/0x60)
[    1.578131] [<800093e4>] (gic_handle_irq) from [<80013240>] (__irq_svc+0x40/0x54)
[    1.590282] Exception stack(0x808b3f48 to 0x808b3f90)
[    1.597666] 3f40:                   00000001 00000000 00000000 8001e520 808b2000 808b55a8
[    1.610412] 3f60: 80620400 00000000 808b3f98 00000000 00000001 00000000 01000000 808b3f90
[    1.623127] 3f80: 8000fd4c 8000fd50 60000013 ffffffff
[    1.630460] [<80013240>] (__irq_svc) from [<8000fd50>] (arch_cpu_idle+0x38/0x3c)
[    1.642403] [<8000fd50>] (arch_cpu_idle) from [<8005ee00>] (cpu_startup_entry+0x250/0x2bc)
[    1.655321] [<8005ee00>] (cpu_startup_entry) from [<80864c68>] (start_kernel+0x3a4/0x3b0)
[    1.668421] [<80864c68>] (start_kernel) from [<8000807c>] (0x8000807c)
[    1.677483] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    1.677483]
2

Great, the kernel is loading, thanks. However, the example rpmsg_pingpong_freertos still does not work like expected. The M4 prints RPMSG String Echo FreeRTOS RTOS API Demo... RPMSG Init as Remote
but i am missing init M4 as REMOTE and i cannot see any outputs after modprobe imx_rpmsg_tty…

3

The modprobe command does not print anything and there is no /dev/ttyRPMSG at all.

4

Did you disable UART_B as documented here and setup defargs? It would seem that UART_B is enabled on Linux side as the stack trace originated from imx_serial_init.

5

Are the u-boot environment variables set as documented here? Are the FreeRTOS binaries generated from latest source or are you using precompiled binaries from here? Just tested using 2.7 B2 and precompiled binaries I did not see any issues.

root@colibri-imx7:~# modprobe imx_rpmsg_tty     
[   16.298464] imx_rpmsg_tty rpmsg0: new channel: 0x400 -> 0x0!
[   16.304548] Install rpmsg tty driver!
root@colibri-imx7:~# uname -a
Linux colibri-imx7 4.1.39-2.7.2+gba2f75f #3 SMP Mon Apr 10 03:50:20 CEST 2017 armv7l GNU/Linux
6

The u-boot variables are set correctly. The precompiled binaries V1.2 are working (or at least the str_echo demo), binaries from v1.0.1 don’t. My self-compiled binaries are not working. I downloaded the sources by git: git clone -b colibri-imx7-m4-freertos-v8 git://git.toradex.com/freertos-toradex.git freertos-colibri-imx7/

7

There are some changes across FreeRTOS, u-boot and kernel because of which the current updated FreeRTOS branch will only work the most recent u-boot and kernel. There are two options. If you want to stick to the 2.7 B2 image, you can checkout this commit and your FreeRTOS build should work with 2.7 B2 image.

git checkout -b myRTOS 6f0eaf1814e57d187acf57b3fed6561e26a8a21e

Else you may use an updated u-boot and kernel for working with the latest FreeRTOS changes.

Also note that we would be releasing 2.7 B3 image soon which would integrate all the required changes for FreeRTOS to work out of the box.

8

Changing the branch did not resolve my issues. In fact, the example programs do not produce any output on UART_B (all other programs, no matter which did) and modprobe imx_rpmsg_tty seems to do nothing as well.

9

Can you try with this preliminary 2.7.3 image? The latest FreeRTOS branch should work correctly with the latest u-boot and kernel. In the meanwhile I will try taking a look at why the older FreeRTOS release binaries work but building from the branch which was used for the very same release does not. We apologize for the inconvenience.

10

I flashed the given image and my self-compiled str_echo_freertos demo is now working. Thanks!