Toradex yocto BSP upgrade from version 5.7.0 to 6.4.0 colibri-imx6ull fails to boot

Hi,

We have been using colibri-imx6ull module with a build based on yocto bsp 5.7.0 and are now in process of upgrading it to version 6.4.0. The old yocto was based on toradex’s kernel git, which contained colibri-imx6ull_defconfig. The new version is upstream and does not contain this file.
The build with the defconfig included in the linux-toradex-mainline recipe does not boot. There is a toradex_v6_v7.config file which has instructions to produce a defconfig based on imx_v6_v7_defconfig.

My questions

  • Do you have a place-in replacement file for colibri-im6ull_defconfig for the new BSP?
  • If not, what is the recommended way to produce one?
  • Can the files included in the recipe linux-toradex-mainline (defconfig, toradex_imx_v6_v7.config) be adapted for this purpose?
  • Can I use the old colibri-im6ull_defconfig with the new kernel, or what changes are needed for it to work?

Hello @Tuomas_Tuhkanen_IM ,
In your post you say:

Have you tried to build the 6.4.0 minimal image and check that boots correctly?

Best regards,
Josep

Yes, I tested the tdx reference minimal image and it does boot. However, there are some differences to our setup.

  • It uses devicetree imx6ull-colibri-wifi-eval-v3.dtb, we use one based on imx6ull-colibri-wifi-iris-v2.dtb
  • In our original attempt the U-Boot was still based on 5.7.0 bsp

I have tried updating U-Boot to bsp 6.4.0, but that brought up another issue: our u-boot environment is located on a ubi partition and the new u-boot fails to load it with many ecc errors. I think the issue is that the ubifs in the 6.4.0 version has crc16.c added.

We are using mender, which sets the u-boot environment in the ubi. This was working fine with previous version. We’ll contacting them as well, to see if they have any solutions.

Is there any reason the u-boot from 5.7.0 would not work with bsp 6.4.0 kernel? It would simplify things since it does work with mender. I noticed that load addresses were changed.

As for kernel boot problems, we’ll try things like removing changes from our devicetree (we have not changed kernel code). Are there anything else we should be trying?

Mender creates a tezi package which writes the u-boot environment with image that is just zeroes. Is there a way to create the environment

This is the image.json that mender creates:

{
    "config_format": "2",
    "autoinstall": true,
    "name": "Image Based on Toradex Embedded Linux Reference Minimal Image (development)",
    "description": "Minimal image without graphical interface that just boots",
    "version": "Mender kirkstone",
    "release_date": "2023-12-01",
    "prepare_script": "prepare.sh",
    "wrapup_script": "wrapup.sh",
    "marketing": "mender-tezi-metadata/marketing_mender_toradex.tar",
    "icon": "mender-tezi-metadata/mender_toradex_linux.png",
    "supported_product_ids": [
        "0036",
        "0040",
        "0044",
        "0045"
    ],
    "mtddevs": [
        {
            "name": "u-boot1",
            "content": {
                "rawfile": {
                    "filename": "u-boot-nand.imx",
                    "size": 1
                }
            }
        },
        {
            "name": "u-boot2",
            "content": {
                "rawfile": {
                    "filename": "u-boot-nand.imx",
                    "size": 1
                }
            }
        },
        {
            "name": "u-boot-env",
            "content": {
                "rawfile": {
                    "filename": "uboot.env",
                    "size": 1
                }
            }
        },
        {
            "name": "ubi",
            "content": {
                "rawfile": {
                    "dd_options": "bs=8M",
                    "filename": "image-colibri-imx6ull.ubimg.gz"
                }
            }
        }

The uboot.env is just zerofilled data. Should we adjust these options?

U-Boot ubi errors:

CPU:   Freescale i.MX6ULL rev1.1 900 MHz (running at 396 MHz)
CPU:   Commercial temperature grade (0C to 95C) at 42C
Reset cause: POR
DRAM:  512 MiB
Spread Spectrum pll initialized
Core:  79 devices, 17 uclasses, devicetree: separate
NAND:  512 MiB
MMC:   FSL_SDHC: 0
Loading Environment from UBI... ubi0 error: ubi_io_read: error -74 (ECC error) while reading 64 bytes from PEB 0:0, read 64 bytes
ubi0 error: ubi_io_read: error -74 (ECC error) while reading 2048 bytes from PEB 0:2048, read 2048 bytes
...
ubi0 error: ubi_io_read: error -74 (ECC error) while reading 64 bytes from PEB 1931:0, read 64 bytes
ubi0 error: ubi_io_read: error -74 (ECC error) while reading 2048 bytes from PEB 1931:2048, read 2048 bytes
ubi0 error: ubi_io_read: error -74 (ECC error) while reading 22528 bytes from PEB 4:4096, read 22528 bytes
ubi0 error: ubi_io_read: error -74 (ECC error) while reading 22528 bytes from PEB 5:4096, read 22528 bytes
ubi0 error: vtbl_check: bad CRC at record 2: 0x7efdbff7, not 0x040000
Volume table record 2 dump:
        reserved_pebs   397
        alignment       1
        data_pad        0
        vol_type        1
        upd_marker      0
        name_len        4
        name            data
        crc             0x040000
ubi0 error: vtbl_check: bad CRC at record 2: 0x7efdbff7, not 0x040000
Volume table record 2 dump:
        reserved_pebs   397
        alignment       1
        data_pad        0
        vol_type        1
        upd_marker      0
        name_len        4
        name            data
        crc             0x040000
ubi0 error: process_lvol: both volume tables are corrupted
ubi0 error: ubi_attach_mtd_dev: failed to attach mtd4, error -22
UBI error: cannot attach mtd4
UBI error: cannot initialize UBI, error -22
UBI init error 22
Please check, if the correct MTD partition is used (size big enough?)

Update:

We were able to fix the kernel boot issue.

diff --git a/arch/arm/boot/dts/imx6ull-colibri-wifi.dtsi b/arch/arm/boot/dts/imx6ull-colibri-wifi.dtsi
index 237757d..f66b401 100644
--- a/arch/arm/boot/dts/imx6ull-colibri-wifi.dtsi
+++ b/arch/arm/boot/dts/imx6ull-colibri-wifi.dtsi
@@ -174,7 +174,27 @@
 &iomuxc {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_gpio1 &pinctrl_gpio2 &pinctrl_gpio3
-		&pinctrl_gpio4 &pinctrl_gpio7>;
+		&pinctrl_gpio4 &pinctrl_gpio7 &pinctrl_leds &pinctrl_buttons>;
+    /* removing this block solved the issue */
+    /* imx6ull-colibri { */
+        pinctrl_leds: gpio_leds {
+            fsl,pins = <
+                MX6UL_PAD_LCD_ENABLE__GPIO3_IO01 0x13030 /* SODIMM 44 - GREEN 1 */
+                MX6UL_PAD_LCD_DATA07__GPIO3_IO12 0x13030 /* SODIMM 46 - GREEN 2 */
+                MX6UL_PAD_LCD_DATA09__GPIO3_IO14 0x13030 /* SODIMM 48 - GREEN 3 */
+                MX6UL_PAD_LCD_DATA11__GPIO3_IO16 0x13030 /* SODIMM 50 - RED 1 */
+                MX6UL_PAD_LCD_DATA12__GPIO3_IO17 0x13030 /* SODIMM 52 - RED 2 */
+                MX6UL_PAD_LCD_DATA13__GPIO3_IO18 0x13030 /* SODIMM 54 - RED 3 */
+            >;
+        };
+
+        pinctrl_buttons: gpio_keys {
+            fsl,pins = <
+                MX6UL_PAD_LCD_DATA19__GPIO3_IO24 0x10000 /* SODIMM 37 - POWER SWITCH */
+                MX6UL_PAD_GPIO1_IO04__GPIO1_IO04 0x1b030 /* SODIMM 19 - RESET BUTTON */
+            >;
+        };
+    /* }; */
 
 };
 

The issue was that the pin setup was wrapped in imx6ull-colibri block. Once we removed it, the kernel was able to boot.

The u-boot issue remains, but I think it is better to make more specific post about it and close this one as solved.