Cannot update ahab close device with Toradex EasyInstaller

Technical Support
, , ,
1

Dear all,

This is a question that has been asked several times, regarding how to update a firmware for a HAB closed device, with Toradex EasyInstaller.

For example, the following answer was given for a question about the Colibri i.MX7 board.

https://community.toradex.com/t/cannot-use-toradex-easy-installer-with-closed-device/10340
https://community.toradex.com/t/toradex-easy-installer-downloading-failed-on-closed-device-hab-colibri-i-mx7/10927

Since we are using apalis i.MX8QM and the u-boot version is quite updated, the same solution could not be used.
One idea was to partially modify the u-boot of Torizon OS and create an imx-boot built with the tdx-signed configuration and incorporate it into Easy Installer.
In this way, the Easy Installer u-boot will boot even on an ahab close target.
However, the Easy Installer kernel does not boot and stops in the middle of the kernel boot process.

Does anyone have any advice or information?

Below is the u-boot boot log of the modified EasyInstaller.

U-Boot 2022.04-6.6.1-devel+git.d262075124dc (Jan 01 1970 - 00:00:00 +0000)

CPU:   NXP i.MX8QM RevB A53 at 1200 MHz at 40C

DRAM:  4 GiB
Core:  195 devices, 21 uclasses, devicetree: separate
MMC:   FSL_SDHC: 0, FSL_SDHC: 1, FSL_SDHC: 2
Loading Environment from MMC... OK
In:    serial
Out:   serial
Err:   serial
Model: Toradex 0037 Apalis iMX8QM 4GB WB IT V1.1E
Serial#: 14715176

 BuildInfo:
  - SCFW 6638c032, SECO-FW c9de51c0, IMX-MKIMAGE 3bfcfccb, ATF 3c1583b
  - U-Boot 2022.04-6.6.1-devel+git.d262075124dc

Detect USB boot. Will enter fastboot mode!
flash target is MMC:0
Net:   eth0: ethernet@5b040000
Fastboot: Normal
Boot from USB for uuu
Hit any key to stop autoboot:  0
We are in is_imx8
Getting phy ctrl

Best regards,
Yoshinobu KATO

2

Hi, I’m the Toradex AI Assistant. I’m in early beta, and I would appreciate your feedback.
The Toradex Support team will follow up with you shortly!

Did you like my answer?

:+1: :-1:

Hello,

Updating the firmware on a High Assurance Boot (HAB) closed device, such as the Apalis i.MX8QM, using the Toradex Easy Installer can indeed present unique challenges. The issue you’re encountering, where the modified Easy Installer’s U-Boot boots but the kernel does not, suggests a potential mismatch or incompatibility in the boot environment or the kernel image itself.

Given the complexity of booting a HAB closed device, the approach of modifying U-Boot for Torizon OS and creating an imx-boot built with the tdx-signed configuration is sound. This method should theoretically allow the Easy Installer’s U-Boot to boot even on an AHAB closed target. However, the kernel stopping mid-boot indicates that the kernel or its boot parameters may not be correctly configured or compatible with the modified U-Boot environment.

Here are a few steps and considerations to troubleshoot and potentially resolve the issue:

  1. Verify Compatibility: Ensure that the kernel version and the modified U-Boot are compatible. Incompatibilities could arise from changes in device tree configurations, kernel image types, or boot arguments.

  2. Review Boot Arguments: The kernel boot arguments (passed by U-Boot to the kernel at boot time) are crucial for successful booting. Ensure that these arguments are correctly set for your specific hardware configuration and that they match the requirements of the kernel version you are using.

  3. Check Device Tree: The device tree blob (DTB) used during boot must correspond to your hardware configuration. An incorrect or outdated DTB could cause the kernel to halt during the boot process.

  4. Debug Output: Utilize the debug output from U-Boot and the early kernel boot stages to identify where the boot process halts. This information can provide valuable clues about what might be going wrong.

  5. Kernel Configuration: Review the kernel configuration to ensure that it supports your hardware, especially the boot medium and any peripherals critical to the boot process. Missing drivers or incorrect configurations can prevent the kernel from booting.

  6. Secure Boot Configuration: Since you’re working with a HAB/AHAB closed device, ensure that all necessary cryptographic operations (such as signing of the kernel and device tree) are correctly performed and that the secure boot keys are properly configured in U-Boot.

  7. U-Boot Environment: Ensure that the U-Boot environment variables, especially those related to booting (like bootcmd), are correctly set for your modified setup. An

Please, also check if these links can help you solve your problem:

Using DIP Switches to change the boot process | Toradex Developer Center.

3

Greetings @Yo_KATO,

I would like to ask you some questions so that I may better understand your use-case and requirements, if that is okay with you.

First of all, what is your overall goal here? You say you want to “update firmware” on your closed device. Easy Installer could technically do this since it re-flashes the entire the device, but are you specifically looking to re-flash the entire device? Or will any mechanism to update the firmware be sufficient?

Also could you clarify what you mean by “firmware”? I mean are you looking to update the OS, bootloader, application code, everything?

Lastly, just to understand is this something you are seeking to do in your process often? Typically when you close a device with AHAB the intent is that it is now locked down and secure. Which is why I’m curious why you are trying to now access the device like this.

Best Regards,
Jeremias

4

Hello @jeremias.tx,

Thank you very much for your reply.
I’m sorry for the lack of clarity in my inquiry.

First of all, what is your overall goal here?

I’m not thinking of a special process.
I want to re-flash the entire device for the closed device.
I don’t fully understand how to do this.

Also could you clarify what you mean by “firmware”?

“firmware” means the whole, including bootloader, OS, and application.

Lastly, just to understand is this something you are seeking to do in your process often?

We believe that we will probably update the software about once a year to improve functionality and fix bugs.

Best Regards,
Yoshinobu KATO

5

We believe that we will probably update the software about once a year to improve functionality and fix bugs.

When your devices are in production do you always plan to use Easy Installer to “update” your software? Or do you plan to be using some kind of software update tool/service?

The complication is that to re-flash the device with Easy Installer. You’d need to update the bootloader artifacts in Easy Installer with a bootloader that has been signed with your information. Then you would need to package this signed bootloader back into Easy Installer so it can boot. Or at least that is the theory behind it. This isn’t something we have documented as a proper process.

This is all needed because Easy Installer uses it’s own bootloader to boot and this bootloader is obviously not signed. This is why you see the boot fail to complete, since the device is closed it should not boot any unsigned bootloaders.

If you were using a software update service then it would be a bit simpler in my opinion, as then you just need to make sure you’re updating using signed software where appropriate.

But, is there a reason you’re trying to do this with Easy Installer maybe?

Best Regards,
Jeremias

6

Thank you for your detailed explanation and helpful advice.

We plan to use the OTA procedure for usual updates.
However, at the moment, we have not completed our study to see if OTA procedure can be applied to devices that are closed and contain secure file systems.

We are also concerned that if we make significant enhancements to the target system, there may be cases where software updates cannot be performed using the OTA procedure.
We were studying the possibility of updating the software using Easy Installer as a backup method in such a worst-case scenario.

As you mentioned, we think that using a software update service such as OTA is a good idea.
However, we have not completed our research on the possibility of using an update service such as OTA for devices that are closed and contain secure file systems such as dm-verity and compose-fs.

Do you have any information on the experience and possibility of update services in such cases?

Best regards,
Yoshinobu KATO

7

However, at the moment, we have not completed our study to see if OTA procedure can be applied to devices that are closed and contain secure file systems.

To understand, are you studying multiple OTA tools/frameworks? Or are you talking about our Torizon cloud services specifically here? You said in your initial post that you are using Torizon OS so I’ve been assuming you are at least familiar with our update offerings, but please correct me if this is incorrect.

Our Torizon cloud services should be able to update a closed/secure device. Assuming the parts you are updating are appropriately signed with the correct key information. So for example if you have a signed bootloader with AHAB, if you update the bootloader then the bootloader you are updating to should also be similarly signed as well. Otherwise after the update the system won’t boot the unsigned bootloader.

From what I’ve gathered from this conversation it sounds like this is all just future planning for your product yes? As in you are just thinking/speculating how you might update your closed/secure devices in the future, yes? Or is your requirements still different from this?

Best Regards,
Jeremias

9

Sorry for the very late response.

Thank you very much for the information about Torizon Cloud service’s OTA Remote Update service.
I understood that Torizon Cloude service meets our requirements.

As for OTA, it is still under planning and investigation, and I have not been able to figure out its functionality since I have not touched it so far.
I am sure that other methods of OTA besides Torizon Cloud are also being investigated.

For this device, it may be installed in a closed local network, so there may be cases where OTA cannot be used.
Therefore, we are still considering the possibility of using EasyInstaller to re-flash the closed system.

Currently, we are modifying imx-boot of meta-toradex-security to incorporate imx-boot of meta-toradex-tezi of EasyInstaller.
Could you give me some advice on what to do?

Best regards,
Yoshinobu KATO

10

For this device, it may be installed in a closed local network, so there may be cases where OTA cannot be used.

Just to inform you, our Torizon Cloud platform does also have an offline update feature for such a situation.

Currently, we are modifying imx-boot of meta-toradex-security to incorporate imx-boot of meta-toradex-tezi of EasyInstaller.
Could you give me some advice on what to do?

What I’m about to say is mostly theoretical and not a tried and true process. Easy Installer comes with it’s own u-boot binary that it uses to boot the device to the Easy Installer image. Since you’ve closed your device it correctly will not boot unsigned bootloaders. In theory then you would need to replace this u-boot binary with an equivalent binary that has been signed by the keys that correspond to what you used to close this device in the first place.

With basic AHAB the only thing it cares about is whether the bootloader is correctly signed or not. With this it should at least progress past the bootloader. Again though this isn’t something that has been strictly tested, as Easy Installer was not originally designed to be used on a closed device like this.

Best Regards,
Jeremias

11

Based on your advice, we have ported the imx-boot signature process and are now able to re-flash closed devices.
Actually, we merged imx-mkimage included in meta-toradex-security of Torizon OS with imx-mkimage included in meta-toradex-tezi of TEZI.

Merged “meta-toradex-security/recipes-bsp/imx-mkimage/imx-boot_1.0.bbappend” into “meta-toradex-tezi/recipes-bsp/imx-mkimage/imx-boot_1.0.bbappend”

Copied “meta-toradex-security/classes/imx-hab.bbclass” into “meta-toradex-tezi/classes/”
Copied “meta-toradex-security/recipes-bsp/imx-mkimage/files/” into “meta-toradex-tezi/recipes-bsp/imx-mkimage/files/”
Copied “meta-toradex-security/recipes-bsp/imx-mkimage/imx-boot-hab.inc” into “meta-toradex-tezi/recipes-bsp/imx-mkimage/”

In addition to these settings, we have configured local.conf to sign imx-boot.

TDX_IMX_HAB_ENABLE = “1”
TDX_IMX_HAB_CST_CRYPTO =
TDX_IMX_HAB_CST_KEY_SIZE =
TDX_IMX_HAB_CST_DIG_ALGO =

Thank you for all the useful advices and information.
I would like to complete my question about updating the CLOSED device.

After this, I would like to look into OTA services, including Toradex Cloud.

Best regards,
Yoshinobu KATO

1 Like
12

Perfect, I am glad you were able to progress here. Thank you also for sharing your process as it can be helpful for others.

Best Regards,
Jeremias