Update in progress stuck

Hello

I just did an OS update OTA and it got stuck with “Update in progress”.

I looked at the output of aktualizr, but I don’t know what the problem is:

torizon@verdin-imx8mp-15604835:~$ journalctl -fu aktualizr*
Feb 03 14:57:49 verdin-imx8mp-15604835 aktualizr-torizon[955]: Certificate valid from: Jan 19 13:27:25 2026 GMT until: Jan 19 13:27:25 2126 GMT
Feb 03 14:57:49 verdin-imx8mp-15604835 aktualizr-torizon[955]: Offline Updates are disabled
Feb 03 14:57:49 verdin-imx8mp-15604835 aktualizr-torizon[955]: Registering with D-Bus
Feb 03 14:57:51 verdin-imx8mp-15604835 aktualizr-torizon[955]: Event: SendDeviceDataComplete
Feb 03 14:57:51 verdin-imx8mp-15604835 aktualizr-torizon[955]: Current versions in storage and reported by OSTree do not match
Feb 03 14:57:51 verdin-imx8mp-15604835 aktualizr-torizon[955]: Event: UpdateCheckComplete, Result - No updates available
Feb 03 15:02:51 verdin-imx8mp-15604835 aktualizr-torizon[955]: Current versions in storage and reported by OSTree do not match
Feb 03 15:02:51 verdin-imx8mp-15604835 aktualizr-torizon[955]: Event: UpdateCheckComplete, Result - No updates available
Feb 03 15:07:51 verdin-imx8mp-15604835 aktualizr-torizon[955]: Current versions in storage and reported by OSTree do not match
Feb 03 15:07:51 verdin-imx8mp-15604835 aktualizr-torizon[955]: Event: UpdateCheckComplete, Result - No updates available

I can’t cancel the update or install a new one. What else should I look at to know what the problem might be?

Kind regards

Ian

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.


Based on the logs you’ve shared, the message “Current versions in storage and reported by OSTree do not match” is not an error but rather an informational statement(1). This message appears when Aktualizr’s SQL storage contains different version information than what OSTree reports, which can happen in several scenarios(1).

Understanding the Log Message

The message occurs because Aktualizr stores version information only if it came from an update(1). If you’ve never performed an OS/OSTree update through Aktualizr, the storage would be empty while OSTree reports the current version(1). This mismatch is expected and not inherently problematic(1).

Troubleshooting Your Stuck Update

Since your logs show “No updates available” but you mention the update is stuck at “Update in progress,” there may be a disconnect between the Torizon Cloud interface and the actual device state. Here are steps to investigate:

1. Check OSTree Status

To understand your current OSTree deployment status, you can examine the OSTree state directly. OSTree manages multiple bootable filesystem versions and handles atomic updates(2).

2. Review Complete Aktualizr Logs

Your current logs only show the polling activity. To get a complete picture, check the full Aktualizr logs around the time the update was initiated:

sudo journalctl -u aktualizr* --since "1 hour ago"

3. Verify Update Configuration

Aktualizr is configured via .toml files in /usr/lib/sota/conf.d/ and /etc/sota/conf.d/(3). The default polling frequency is every 5 minutes(3), which matches your log timestamps.

4. Check for Pending Reboots

Torizon OS is configured to automatically reboot after successful OS updates(3). If automatic reboot is disabled, the update process might appear stuck. Check if there’s a pending reboot:

systemctl status ostree-pending-reboot.path

Next Steps

  1. Restart Aktualizr: Try restarting the Aktualizr service to refresh its state:

    sudo systemctl restart aktualizr-torizon
    
  2. Check Device Status: Verify your device’s connection status in the Torizon Cloud interface to ensure it’s properly communicating with the server.

  3. Review Full Logs: Examine the complete Aktualizr logs from when the update was initiated to identify any error messages or status changes that might not appear in the current polling logs.

The “Current versions in storage and reported by OSTree do not match” message itself is not the cause of your stuck update - it’s a normal informational message(1).

Best regards, Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

Here are more logs from aktualizr:

torizon@verdin-imx8mp-15604835:~$ sudo journalctl -u aktualizr* --since "1 hour ago"
Password:
Feb 03 14:47:15 verdin-imx8mp-15604835 systemd[1]: Started Aktualizr SOTA Client.
Feb 03 14:47:15 verdin-imx8mp-15604835 (-torizon)[771]: aktualizr-torizon.service: Referenced but unset environment variable evaluates to an empty string: AKTUALIZR_CMDLINE_PARAMETERS
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Aktualizr version tdx-a8e5ac1a4 starting
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Reading config: "/usr/lib/sota/conf.d/20-sota-device-cred.toml"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Reading config: "/usr/lib/sota/conf.d/30-rollback.toml"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Reading config: "/usr/lib/sota/conf.d/40-hardware-id.toml"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Reading config: "/usr/lib/sota/conf.d/50-secondaries.toml"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Reading config: "/usr/lib/sota/conf.d/60-polling-interval.toml"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Reading config: "/usr/lib/sota/conf.d/70-reboot.toml"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Use existing SQL storage: "/var/sota/sql.db"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Couldn`t import data: empty path received
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Initializing docker-compose Secondaries...
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Use existing SQL storage: "/var/sota/storage/docker-compose/sql.db"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: No valid Director metadata found in storage: Could not load latest root
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: No valid Image metadata found in storage: Could not load latest root
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Adding Secondary with ECU serial: 54c0f96627a1955aa39a2ca874c5f07f0503a11ebafc1bfb489ddcdbc8cc03cc with hardware ID: docker-compose
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Initializing torizon-generic Secondaries...
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Use existing SQL storage: "/var/sota/storage/bootloader/sql.db"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: No valid Director metadata found in storage: Could not load latest root
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: No valid Image metadata found in storage: Could not load latest root
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Adding Secondary with ECU serial: e6ad8d356c9ad3fdd738789735c51b6a28193a67bbb8fcc79e34efe5bbf33c6e with hardware ID: verdin-imx8mp-bootloader
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Initializing torizon-generic Secondaries...
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Use existing SQL storage: "/var/sota/storage/fuse/sql.db"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: No valid Director metadata found in storage: Could not load latest root
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: No valid Image metadata found in storage: Could not load latest root
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Adding Secondary with ECU serial: 70d3f09b764ec6ae71217e34b2a0898b22858c3844494eeaa82b86d77d93d7b5 with hardware ID: verdin-imx8mp-fuses
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Initializing torizon-generic Secondaries...
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: Use existing SQL storage: "/var/sota/storage/m7-firmware/sql.db"
Feb 03 14:47:15 verdin-imx8mp-15604835 aktualizr-torizon[771]: No valid Image metadata found in storage: The timestamp metadata was expired.

Here is the output for the pending reboot:

torizon@verdin-imx8mp-15604835:~$ systemctl status ostree-pending-reboot.path
* ostree-pending-reboot.path - Aktualizr OSTree Update Automatic Reboot
     Loaded: loaded (/usr/lib/systemd/system/ostree-pending-reboot.path; enabled; preset: enabled)
     Active: active (waiting) since Tue 2026-02-03 14:47:06 UTC; 29min ago
   Triggers: * ostree-pending-reboot.service

Notice: journal has been rotated since unit was started, output may be incomplete.

Restarting aktualizr doesn’t work, the update still doesn’t install.

Analysis of Your Update Issue

Based on your additional logs, I can see several important details about your system’s state. The logs show that Aktualizr is initializing multiple secondaries including docker-compose, bootloader, fuses, and M7 firmware components (1). The “No valid Director metadata found in storage” and “No valid Image metadata found in storage” messages indicate that the device hasn’t successfully completed the initial metadata synchronization with the update server (1).

Key Observations

1. Metadata Issues

Your logs show multiple instances of “Could not load latest root” which suggests the device is having trouble establishing proper communication with the Torizon Cloud update infrastructure (1).

2. Automatic Reboot Service Status

The ostree-pending-reboot.path service is active and waiting, which is the correct state. This service monitors for pending OSTree updates and triggers automatic reboots when needed (1).

3. Multiple Secondary Components

Your system is configured with several secondary update components:

  • docker-compose secondary
  • bootloader secondary
  • fuses secondary
  • M7 firmware secondary

This indicates a complex update configuration that may require all components to be properly synchronized (1).

Troubleshooting Steps

1. Check OSTree Status Directly

Run the following command to see the actual OSTree deployment status:

sudo ostree admin status

This will show you the current and pending deployments, helping identify if an update is actually staged.

2. Verify Network Connectivity

Ensure your device can reach the Torizon Cloud servers:

curl -I https://dgw.torizon.io

3. Check for Blocked Updates

Torizon OS provides mechanisms to allow and block updates (2). Verify if updates are currently blocked:

sudo systemctl status aktualizr-torizon

4. Enable Debug Logging

To get more detailed information about what’s happening, you can temporarily increase the logging level. Create a custom configuration file (1):

sudo mkdir -p /etc/sota/conf.d/
cat <<EOF | sudo tee /etc/sota/conf.d/99-custom-loglevel.toml
[logger]
loglevel = 1
EOF

Then restart Aktualizr:

sudo systemctl restart aktualizr-torizon

5. Manual Update Check

You can force Aktualizr to check for updates immediately rather than waiting for the polling interval:

sudo systemctl kill -s SIGUSR1 aktualizr-torizon

Understanding the Update Process

Torizon updates are designed to be power-cut tolerant and atomic operations (2). The system uses OSTree for reliable updates with automatic rollback capabilities if issues occur (2). When an update is “stuck,” it’s often due to metadata synchronization issues rather than the actual update process failing.

Next Steps

  1. Check OSTree status to understand the current deployment state
  2. Enable debug logging to get more detailed information about the update process
  3. Verify network connectivity to ensure communication with Torizon Cloud
  4. Monitor the logs after enabling debug mode to identify specific error conditions

If these steps don’t resolve the issue, the problem may be related to the specific update package or server-side configuration that requires further investigation.

Best regards, Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

Greetings @i.staehli,

Could you provide the device UUID of this specific device, as well as the email of the account this device is provisioned to.

Also, if possible could you leave this specific device in this state in the meanwhile? It will allow our team to investigate what could have happened to this device to put it into this state.

Finally, do you happen to know when the last time this device performed an update?

Best Regards,
Jeremias

Hi @jeremias.tx

I digged a bit more into the issue and found a way to reproduce it. What I did is:

  1. Have the software version A installed on the device

  2. Install the software version B with Torizon Cloud OTA

  3. Manually rollback the software to version A

    1. Check the installed versions with ostree admin status

    2. Rollback with sudo ostree admin deploy to software A

    3. Reboot the device

  4. Install the software version B with Torizon Cloud OTA (again)

    1. The device will be stuck in “Update in progress”

Maybe the issue is that I requested to install two times the same software (version B) one after the other? I saw that it’s not allowed to install twice the same software via Torizon Cloud.

Finally, the “solution” we found to get out of this state is to delete the device from Torizon Cloud and provision it again. At the moment when developing in-house we can do this easily, but later when the devices are in the field with the customers this will no longer be an option.

Manually rollback the software to version A

Why did you do this step? Is this a step you will also do when the device is in the field?

The solution would be to never do this kind of step. All changes/updates to the OS should be managed through the update client proper. If you change the OS outside of an update, then the update client doesn’t accurately know what happened.

Basically what happened is the following:

  • You install version B via OTA update.
  • Update succeeds and the update client stores in it’s persistent database that version B is what is installed.
  • You manually rollback to version A.
  • This action did not go through the update client. As far as the update client knows, the device still has version B installed.
  • You do another OTA update to version B.
  • The device gets the update request from the server, but goes “wait I already have version B installed, why should I do anything?”.
  • The device doesn’t do anything because it thinks it doesn’t need to.
  • The server gets stuck with “Update in progress”, because it’s waiting for a result from the device. But the device doesn’t have a result to give because it didn’t do anything.

In summary, if you plan to manage a software component (i.e the OS), with our OTA updates. You should only let OTA updates change the state of that component. Otherwise you may get weird results like this.

Best Regards,
Jeremias

@jeremias.tx Thanks for your explanation.

I rolled back manually, because I bricked the device with a corrupt update OTA (I will open another issue for this).

In the field we wouldn’t do this, so I guess it won’t be an issue. I was wondering why it got stuck and there is no option to cancel, but that is not a normal use case as you explained.

Kind regards,

Ian

Glad we were able to help clarify.

Best Regards,
Jeremias