Bootloader versions

DuncanNapier · May 15, 2024, 10:28am

We’ve seen a further issue with this watchdog reboot our system when in suspend state.
(pre Torizon 6.6.1 watchdog cause a reboot when in suspend state, fixed by Torizon 6.6.1)
We’ve upgraded a few boards using the OTA mechanism from app.torizon.io, we found moving from an Easy Installer flashed 6.5.0 OS to a 6.6.1 we saw the that the issue of reboot kept happening. We solved this by downgrading the bootloader subsystem to 5.7.4, this resolved the issue. is there any issue running the older bootloader with the newer OS. and is there plan to release bootloaders that match the OS release

ToradexAI · May 15, 2024, 10:29am

Hi, I’m the Toradex AI Assistant. I’m in early beta, and I would appreciate your feedback.
The Toradex Support team will follow up with you shortly!

Did you like my answer?

To address the issue of the watchdog causing reboots in suspend state, which was fixed in Torizon 6.6.1, and your subsequent experience with OTA updates from app.torizon.io, it’s important to understand the relationship between the bootloader and the OS version in Toradex devices.

Running an older bootloader version with a newer OS version, as you’ve done by downgrading the bootloader to 5.7.4 to resolve the reboot issue on a system upgraded to Torizon 6.6.1, is generally supported. The bootloader versions provided by Toradex are designed to ensure backward compatibility with newer OS versions to a certain extent. This means that newer versions of the OS should be able to boot with older versions of the bootloader, although this is not the ideal configuration for several reasons:

Feature Support: Newer OS versions may introduce features or optimizations that rely on corresponding updates in the bootloader. While the system may operate normally for basic functionality, you might not benefit from all the improvements or features introduced in the newer OS versions.
Security Updates: Bootloader updates often include security enhancements that are critical for maintaining the integrity of the system. Using an older bootloader might expose the system to vulnerabilities that have been addressed in later versions.
Stability and Performance: Compatibility notwithstanding, the optimal performance and stability of the system are usually achieved when both the OS and bootloader are aligned in terms of versions, as they are tested and released as a pair to ensure they work well together.

Regarding your question about plans to release bootloaders that match the OS release, Toradex typically aligns bootloader updates with major OS releases to ensure compatibility and support for new features. However, specific release schedules and plans for bootloader updates are subject to Toradex’s development roadmap and testing cycles to ensure reliability and stability.

It’s advisable to monitor Toradex’s release announcements and update the bootloader to the version recommended for your OS version when such updates are available. This approach ensures that you benefit from the latest features, performance improvements, and security patches, maintaining the optimal operation of your device.

Best regards, Toradex AI Assistant

Please, also check if these links can help you solve your problem:

Bootloader Updates in Torizon OS | Toradex Developer Center.

rudhi.tx · May 17, 2024, 3:30pm

Hello @DuncanNapier,

Thank you for reporting this problem.
The bootloader versions starting from 5.7.2 should work with Torizon OS 6 versions. This means technically it should not break anything. However, we are aware of this issue and are working on it. I will let you know of any updates.
Also, we have some information on the versions of the bootloader here: Bootloader Updates in Torizon OS | Toradex Developer Center

rudhi.tx · May 22, 2024, 2:56pm

Hello @DuncanNapier,

I hope you are doing well!
I have been discussing this issue internally to gather some more information and I am curious to know if you have ever had the watchdog reboot issue on Torizon OS 6.5.0 installed from Easy Installer. Have you also tested it on 6.5.0?

lblackbeard · June 17, 2024, 2:58pm

Hi @rudhi.tx ,

Lloyd here, I am a colleague of Duncan’s.

Unfortunately, after flashing with Torizon 6.6.1, the problem of watchdog resets while sleeping still persists. I have also tried swapping the u-boot.imx with a copy from Torizon 5.7.2 and the problem persists. The same thing happens when I swapped the u-boot.imx with a Torizon 5.7.2 copy downloaded from artifactory.
Strangely, using the OTA platform to change the BL to 5.7.2 does work, even though I’ve verified that the sha256 in /var/sota/storage/bootloader is the same as the one I flashed.

Do you have any idea what it could be about the OTA BL upgrade that allows the system to enter suspend for extended periods while we end up with a watchdog reset when the same BL binary is present through flashing?

Cheers,
Lloyd

lblackbeard · June 18, 2024, 10:25am

I’ve gone ahead and looked into some of the boot config stuff that isn’t the actual u-boot binary and I can’t seem to find anything.

####################################################
AFTER OTA

root@Nilsa:/boot# cat loader.1/entries/ostree-1-torizon.conf
title TorizonCore Upstream 6.6.1+build.14 (kirkstone) 6.6.1+build.14-tcbuilder.20240613143034 (ostree:0)
version 1
options quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/0
linux /ostree/torizon-1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/vmlinuz-6.1.84-6.6.1+git.347385861c50
initrd /ostree/torizon-1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/initramfs-6.1.84-6.6.1+git.347385861c50.img
fdtdir /ostree/torizon-1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/dtb

root@Nilsa:/boot# cat /proc/cmdline
root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/0

uEnv.txt
bootargs=quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/0

dmesg | grep wdt
[    3.394473] systemd[1]: Using hardware watchdog 'rn5t618-wdt', version 0, device /dev/watchdog0
[    3.396253] systemd[1]: Watchdog running with a timeout of 32s.

BEFORE OTA (TEZI 6.6.0+build.6)

torizon@Nilsa:/boot$ cat loader.1/entries/ostree-1-torizon.conf
title TorizonCore Upstream 6.6.1+build.14 (kirkstone) 6.6.1+build.14-tcbuilder.20240613143034 (ostree:0)
version 1
options quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/0
linux /ostree/torizon-1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/vmlinuz-6.1.84-6.6.1+git.347385861c50
initrd /ostree/torizon-1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/initramfs-6.1.84-6.6.1+git.347385861c50.img
fdtdir /ostree/torizon-1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/dtb

torizon@Nilsa:/boot$ cat /proc/cmdline
root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/0

uEnv.txt
bootargs=quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/1b84dd657e151d1da0c1a218c6a8bffc266137155bcd0fb48c0600ddb3e6bd14/0

dmesg | grep wdt
[    3.546104] systemd[1]: Using hardware watchdog 'rn5t618-wdt', version 0, device /dev/watchdog0
[    3.547843] systemd[1]: Watchdog running with a timeout of 32s.

####################################################

I’d appreciate any input I can get on this please.

lblackbeard · June 18, 2024, 11:10am

After reading this:

I decided to attempt flashing the device with the 6.6.1 bootloader using an up-to-date version of Tezi (6.6, was using 5.5). So far so good, the device is able to suspend for longer than 32s. Here’s how it’s looking:

Tezi 5.5 + BL 6.6.1 + Torizon 6.6.1 + NoOTA => watchdog resets on suspend
Tezi 5.5 + BL 5.7.2 + Torizon 6.6.1 + NoOTA => watchdog resets on suspend
Tezi 5.5 + BL 5.7.2 + Torizon 6.6.1 + OTA-BL 5.7.2 => success
Tezi 5.5 + BL 5.7.2 + Torizon 6.6.1 + OTA-BL 6.1.0 => watchdog resets on suspend
Tezi 5.5 + BL 5.7.2 + Torizon 6.6.1 + OTA-BL 6.2.0 => watchdog resets on suspend

Tezi 6.6 + BL 5.7.2 + Torizon 6.6.1 + NoOTA => watchdog resets on suspend
Tezi 6.6 + BL 6.6.1 + Torizon 6.6.1 + NoOTA => success

rudhi.tx · June 18, 2024, 2:10pm

Hello @lblackbeard,

Thanks for sending us all the details and also your test results. I am trying to reproduce the issue here on our side. May I know how you are trying to put the module into suspend mode? Also, what are your wakeup sources? Sending the exact steps here would be helpful.