I’m trying to enable HAB in our verdim-imx8mm modules using your meta-toradex-security meta-layer.
I’m using BSP 6.3.0 (also tried 6.1.0 with same results), and building the reference image: tdx-reference-minimal-image.
Build goes fine, no errors, but after flashing the image using TEZI I get a reboot loop with this messages in the serial console:
U-Boot SPL 2022.04-6.3.0-devel+git.c71ae7141f30 (May 15 2023 - 16:20:01
+0000)
DDRINFO: start DRAM init
DDRINFO: DRAM rate 3000MTS
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
“Synchronous Abort” handler, esr 0x96000021
elr: 00000000007e428c lr : 00000000007e4280
x 0: 00000000007f6534 x 1: 0000000042200020
x 2: 00000000ffffffed x 3: 0000000042200020
x 4: 000000000091fef8 x 5: 00000000009319d8
x 6: 000000000000003a x 7: 0000000000000001
x 8: 0000000000000001 x 9: 0000000000000002
x10: 0000000000000001 x11: 0000000000000001
x12: 0000000000000001 x13: 0000000000000000
x14: 0000000000004190 x15: 0000000000004180
x16: 00000000007f447c x17: 0000000000004090
x18: 000000000091fe40 x19: 0000000000000000
x20: 0000000071c900f8 x21: 0000000000802000
x22: 000000000091fdc8 x23: 0000000030350480
x24: 0000000000000000 x25: 0000000000910000
x26: 0000000030390070 x27: 0000000072000000
x28: 0000000000000000 x29: 000000000091fd30
Code: 9400250e 340000e0 2a0003e2 f9400fe0 (f9400401)
Resetting CPU …
resetting …
U-Boot SPL 2022.04-6.3.0-devel+git.c71ae7141f30 (May 15 2023 - 16:20:01
+0000)
DDRINFO: start DRAM init
DDRINFO: DRAM rate 3000MTS
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
“Synchronous Abort” handler, esr 0x96000021
elr: 00000000007e428c lr : 00000000007e4280
x 0: 00000000007f6534 x 1: 0000000042200020
x 2: 00000000ffffffed x 3: 0000000042200020
x 4: 000000000091fef8 x 5: 00000000009319d8
x 6: 000000000000003a x 7: 0000000000000001
x 8: 0000000000000001 x 9: 0000000000000002
x10: 0000000000000001 x11: 0000000000000001
x12: 0000000000000001 x13: 0000000000000000
x14: 0000000000004190 x15: 0000000000004180
x16: 00000000007f447c x17: 0000000000004090
x18: 000000000091fe40 x19: 0000000000000000
x20: 0000000071c900f8 x21: 0000000000802000
x22: 000000000091fdc8 x23: 0000000030350480
x24: 0000000000000000 x25: 0000000000910000
x26: 0000000030390070 x27: 0000000072000000
x28: 0000000000000000 x29: 000000000091fd30
My settings on the local.conf are as simple as:
INHERIT += “tdx-signed”
TDX_IMX_HAB_CST_DIR = “/workdir/hab”
TDX_IMX_HAB_CST_CERTS_DIR = “/workdir/hab/crts”
Tried 3 different modules, with same result. All of them are known to boot fine with any other image.
My build environment is an standard CROPS docker (crops/poky Ubuntu 22.04)
Any idea if the meta-toradex-security is known to be working with verdin-imx8mm?
I just did a test build for Verdin i.MX8MM. I used our default.xml manifst at the time of writing for our reference BSP and the latest commit for meta-toradex-security. With this configuration the resulting image booted just fine for me. As a reference here’s the commits of each layer used in my build:
All that said the meta-toradex-security meta-layer should have support for Verdin i.MX8MM with the latest commits.
As for your issue I don’t believe we’ve seen this specific error before when working with this meta-layer. If you check the README for meta-toradex-security there’s switches to enable/disable HAB and FIT image signing respectively. If you disable HAB in the build does this result in a functioning image, on the other hand does disabling FIT image signing fix things? I’m curious what is exactly causing this issue for you in the build since I can’t seem to reproduce.
Tried the same build setting TDX_IMX_HAB_ENABLE = “0” and the resulting image boots absolutely fine. It only fails when enabling HAB.
In case of fuses are wrong/not set at all, I’m expecting HAB warnings while trying to boot a signed image, but not the error I’m getting, is this assumption correct?
Looking at your build commits, we have the same commit hash for meta-toradex-security but differ on other layers. Perhaps could you try matching the commit hashes used in my build and see if that resolves things?
Tried the same build setting TDX_IMX_HAB_ENABLE = “0” and the resulting image boots absolutely fine. It only fails when enabling HAB.
Okay that’s good to know.
In case of fuses are wrong/not set at all, I’m expecting HAB warnings while trying to boot a signed image, but not the error I’m getting, is this assumption correct?
Correct, without any fuses set the most HAB does is throw some warnings via hab_status in U-Boot. But it doesn’t actually enforce anything until you choose to close the device by setting the closing device fuse. Which is why your issue here is strange to me since it sounds like you didn’t close the device yet.
One more question, could you share what specific hardware variant of the Verdin i.MX8MM you are testing this on? Just share what is on the sticker of the module itself.
Looking at your build commits, we have the same commit hash for meta-toradex-security but differ on other layers. Perhaps could you try matching the commit hashes used in my build and see if that resolves things?
Could you please share the manifest/tag you are using for repo? For my tests I’m simply using repo init using provided branches for BSP 6.x
I tested the following tags of the manifests:
refs/tags/6.3.0
refs/tags/6.4.0-devel-202308
Which is why your issue here is strange to me since it sounds like you didn’t close the device yet.
100% sure I did not. It boots fine with an unsigned bootloader. Also as I said, I’m testing in 3 different modules, and only configured fuses for one of them.
One more question, could you share what specific hardware variant of the Verdin i.MX8MM you are testing this on? Just share what is on the sticker of the module itself.
Using the default.xml keep in mind.Then all I did was add the latest commit for the meta-toradex-security meta-layer. Other than that everything else in the layers is standard and unmodified.
VERDIN IMX8MM Q 2GB WB IT
V1.1B
Okay the hardware I used to test is nearly the same (1.1A instead of 1.1B), but this shouldn’t make a difference.
Please let me know if the image works after using the same commit hashes as I used in my build.
U-Boot SPL 2022.04-6.4.0-devel+git.7bd2074193e1 (Jul 21 2023 - 09:10:57 +0000)
DDRINFO: start DRAM init
DDRINFO: DRAM rate 3000MTS
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
Failed to initialize : -19
Normal Boot
WDT: Started watchdog@30280000 with servicing (60s timeout)
Trying to boot from MMC1
hab fuse not enabled
Authenticate image from DDR location 0x401fcdc0...
NOTICE: BL31: v2.6(release):lf_v2.6-g3c1583ba0a
NOTICE: BL31: Built : 11:00:38, Nov 21 2022
U-Boot 2022.04-6.4.0-devel+git.7bd2074193e1 (Jul 21 2023 - 09:10:57 +0000)
CPU: i.MX8MMQ rev1.0 1600 MHz (running at 1200 MHz)
CPU: Industrial temperature grade (-40C to 105C) at 48C
Reset cause: POR
DRAM: 2 GiB
Core: 116 devices, 23 uclasses, devicetree: separate
WDT: Started watchdog@30280000 with servicing (60s timeout)
MMC: FSL_SDHC: 0, FSL_SDHC: 1, FSL_SDHC: 2
Loading Environment from MMC... OK
In: serial
Out: serial
Err: serial
Model: Toradex 0055 Verdin iMX8M Mini Quad 2GB WB IT V1.1B
Serial#: 06895003
Carrier: Toradex Dahlia V1.1A, Serial# 10816565
SEC0: RNG instantiated
BuildInfo:
- ATF 3c1583b
Setting variant to wifi
flash target is MMC:0
Net: eth0: ethernet@30be0000
Fastboot: Normal
Normal Boot
Hit any key to stop autoboot: 0
MMC: no card present
switch to partitions #0, OK
mmc0(part 0) is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
6010 bytes read in 4 ms (1.4 MiB/s)
## Executing script at 50280000
86 bytes read in 2 ms (42 KiB/s)
12013462 bytes read in 77 ms (148.8 MiB/s)
Bootargs: root=PARTUUID=abbf5818-02 ro rootwait console=tty1 console=ttymxc0,115200 consoleblank=0 earlycon
## Loading kernel from FIT Image at 50300000 ...
Using 'conf-freescale_imx8mm-verdin-wifi-dev.dtb' configuration
Verifying Hash Integrity ... sha256,rsa2048:dev+ OK
Trying 'kernel-1' kernel subimage
Description: Linux kernel
Type: Kernel Image
Compression: gzip compressed
Data Start: 0x5030010c
Data Size: 11579521 Bytes = 11 MiB
Architecture: AArch64
OS: Linux
Load Address: 0x48200000
Entry Point: 0x48200000
Hash algo: sha256
Hash value: 74a0ec6e13db579b1f6c745052a449297245dfab8e98a9e23a296c1613724403
Verifying Hash Integrity ... sha256+ OK
## Loading fdt from FIT Image at 50300000 ...
Using 'conf-freescale_imx8mm-verdin-wifi-dev.dtb' configuration
Verifying Hash Integrity ... sha256,rsa2048:dev+ OK
Trying 'fdt-freescale_imx8mm-verdin-wifi-dev.dtb' fdt subimage
Description: Flattened Device Tree blob
Type: Flat Device Tree
Compression: uncompressed
Data Start: 0x50e4ce3c
Data Size: 67219 Bytes = 65.6 KiB
Architecture: AArch64
Load Address: 0x50200000
Hash algo: sha256
Hash value: dedd0172c7d591a5ebd5a1c6b1aa9e921f8620591436b641f603f9e20c0a59dc
Verifying Hash Integrity ... sha256+ OK
Loading fdt from 0x50e4ce3c to 0x50200000
## Loading fdt from FIT Image at 50300000 ...
Using 'conf-verdin-imx8mm_dsi-to-hdmi_overlay.dtbo' configuration
Verifying Hash Integrity ... sha256,rsa2048:dev+ OK
Trying 'fdt-verdin-imx8mm_dsi-to-hdmi_overlay.dtbo' fdt subimage
Description: Flattened Device Tree blob
Type: Flat Device Tree
Compression: uncompressed
Data Start: 0x50e6e26c
Data Size: 2317 Bytes = 2.3 KiB
Architecture: AArch64
Load Address: 0x50240000
Hash algo: sha256
Hash value: 27e1f978fb696ecd821a27a4983c2d3671013dbecbf14ccbcbbd23d4aaa31457
Verifying Hash Integrity ... sha256+ OK
## Loading fdt from FIT Image at 50300000 ...
Using 'conf-verdin-imx8mm_spidev_overlay.dtbo' configuration
Verifying Hash Integrity ... sha256,rsa2048:dev+ OK
Trying 'fdt-verdin-imx8mm_spidev_overlay.dtbo' fdt subimage
Description: Flattened Device Tree blob
Type: Flat Device Tree
Compression: uncompressed
Data Start: 0x50e713c0
Data Size: 561 Bytes = 561 Bytes
Architecture: AArch64
Load Address: 0x50240000
Hash algo: sha256
Hash value: 34c3c748a085815c94364b3379e6cb73b45679359e3ae7a7820f3630a4ecc5d5
Verifying Hash Integrity ... sha256+ OK
Booting using the fdt blob at 0x50200000
Uncompressing Kernel Image
Loading Device Tree to 00000000bdee2000, end 00000000bdef59b7 ... OK
Starting kernel ...
I’m using exactly the same docker container I was using previously so build env. is guaranteed to be 100% identical.
Since changing the sources fixes the issue: Any idea of the change that could be fixing the issue?
So my idea was to use very recent commits for the other meta-layers as well.
The issue here is that meta-toradex-security isn’t part of our manifest for our Yocto reference build, which can cause layer incompatibilities, which is what looked like happened here. I’ve already brought this up internally and we’ll see if we can improve this.
Unfortunately not. The meta-toradex-security doesn’t work on regular BSP 6.x therefore the issue is still open.
I’m not sure I understand. You said you just did a Yocto build where it worked no? And what are you defining as “regular BSP 6.x”?
As I said before all I did was a repo init using the latest default.xml at the time of writing. Then added meta-toradex-security, other than that everything is our default BSP.
Your earlier attempts failed as it appears you were using older versions of the manifest, and therefore older versions of the other meta-layers, that were in-compatible with the latest version of meta-toradex-security.
This is true, but I was expecting some clarification of why it failed so we can backport the fix to the previous version of the BSP we are using and have tested / verified.
As you can probably imagine, it is a little bit scary for us to update the BSP to a newer version as it would require to retest and verify all our system.
Also , I would also suggest to add a note on the README.md file of meta-toradex-security to clarify from which exact version of the BSP 6.x is expected to work, otherwise other users could assume “kirkstone” means any BSP 6.x version as I did.
The commit hash got bumped for the U-Boot recipe. Now I’m not sure which specific change in U-Boot fixes this but it’s pretty clear now what change affected this.
For reference here’s the hashes I used in my build:
Everything is standard 6.3.0 hash except meta-toradex-nxp as I stated previously.I also had to bump meta-toradex-bsp-common because there was a commit in meta-toradex-nxp that depended on a change in meta-toradex-bsp-common.
Between 6.3.0 and 6.4.0 there’s only a 3 commit difference in meta-toradex-nxp. Here are the 3 commits:
--- git.orig/board/toradex/verdin-imx8mm/spl.c
+++ git/board/toradex/verdin-imx8mm/spl.c
@@ -59,7 +59,7 @@ void spl_board_init(void)
ret = uclass_get_device_by_driver(UCLASS_MISC, DM_DRIVER_GET(caam_jr), &dev);
if (ret)
- printf("Failed to initialize %s: %d\n", dev->name, ret);
+ printf("Failed to initialize caam_jr(FSL_CAAM) : %d\n", ret);
}
}
fixes the problem with ““Synchronous Abort” handler, esr 0x96000021”
I think this patch should be applied in the u-boot source code. Is there any way to propose this change for u-boot repository ?
Turns out is not fully fixed, visible now in 6.3.0 and 6.4.0 and what @dobrev proposes not only fixes the crash but proves that caam_jr module is for some reason not available and looks like this is the root cause of the issue.
I’m now back to this topic, and tested it. It works: The printf seems to try to print an uninitialized pointer (dev->name) leading to a crash.
After patching this, the crash is gone, and the confirmation of caam_jr not being loaded in SPL is clear.
Do you have any idea of why caam_jr is not being loaded? I guess some option is missing in meta-toradex-security.
BTW I’ve tested a board with hab fuses set and no errors from HAB are printed in the console even when I intentionally use wrong signing keys, I’m assuming this is yet another indicator of caam_jr not being enabled in SPL.
