Device stuck in "Update in progress"

Hello

We have a device that is currently stuck in “Update in progress”. We installed the update via Torizon Cloud and it seems to have installed it, but it never finished completely.

Before it had the software version 3.0+rc2 and the update installed the version 3.0. Here we checked the ostree status:

sudo ostree admin status
* torizon 884b7447a1ed64e2fcaf02a00b2155d1623b6d7f35c86064744b6039495a1d9f.0
    Version: 3.0
    origin refspec: 884b7447a1ed64e2fcaf02a00b2155d1623b6d7f35c86064744b6039495a1d9f
  torizon 8362c43e1cdc93dcfb071a770a9fe53145ddc8fc5989da33dc3614e7240e36f8.0 (rollback)
    Version: 3.0+rc2
    origin refspec: torizon:release-candidate/scewo-go-image

Is this a bug in the Torizon Cloud UI and is there a way to cancel it? At the moment we can’t install a new software, because we get the error that the update is pending.

Kind regards
Ian

Hi, I’m the Toradex AI Assistant.

I might make mistakes or occasionally get you wrong. Our support team will step in if needed.


Based on your OSTree status output, the update appears to have been successfully deployed to the device (version 3.0 is marked with an asterisk *, indicating it’s the current active deployment)(1). However, the Torizon Cloud UI still shows “Update in progress,” which suggests that Aktualizr hasn’t reported the update completion back to the cloud.

Troubleshooting Steps

1. Check Aktualizr Service Status

First, verify that the Aktualizr service is running properly(2):

sudo systemctl status aktualizr

If it’s not running, restart it(2):

sudo systemctl restart aktualizr

2. Check Aktualizr Logs

Review the Aktualizr logs to identify any issues(2):

journalctl -f -u "aktualizr*"

Look for error messages or indications that Aktualizr is unable to communicate with Torizon Cloud or report the update status.

3. Verify Update Completion

The update process includes checks to determine if a boot was successful(1). After an OS update, Aktualizr should set U-Boot environment variables upgrade_available and bootcount back to 0 to indicate a successful update(1).

You can check if the device considers the boot successful by examining the Greenboot status, as Torizon OS uses Greenboot to validate successful boots(1).

4. Manual Status Report

If Aktualizr is running but hasn’t reported the status, restarting the service should trigger it to check the current system state and report back to Torizon Cloud(2).

Possible Causes

The “Update in progress” state could be caused by:

  • Network connectivity issues preventing Aktualizr from communicating with Torizon Cloud
  • Aktualizr service not running or crashed during the final reporting phase
  • A mismatch between the local Aktualizr storage and the actual OSTree deployment(3)

Next Steps

After restarting Aktualizr and checking the logs, wait a few minutes for the service to poll the cloud and report the current status. The default polling interval is 5 minutes(2). If the issue persists, the logs should provide more specific information about what’s preventing the status update.

Best regards,
Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

Here are some further commands:

torizon@verdin-imx8mp-08739953:~$ sudo systemctl status aktualizr
Password: 
● aktualizr-torizon.service - Aktualizr SOTA Client
     Loaded: loaded (/usr/lib/systemd/system/aktualizr-torizon.service; enabled; preset: enabled)
     Active: active (running) since Mon 2026-07-27 10:34:43 CEST; 38min ago
   Main PID: 849 (aktualizr-toriz)
      Tasks: 6 (limit: 1646)
     Memory: 13.7M (peak: 15.1M)
        CPU: 2.378s
     CGroup: /system.slice/aktualizr-torizon.service
             └─849 /usr/bin/aktualizr-torizon

Jul 27 11:04:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Put manifest request failed: 0  HTTP 400
Jul 27 11:04:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Error sending manifest!
Jul 27 11:04:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Invalid role in root.json
Jul 27 11:04:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Invalid role in root.json
Jul 27 11:04:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Event: UpdateCheckComplete, Result - No updates available
Jul 27 11:09:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Put manifest request failed: 0  HTTP 400
Jul 27 11:09:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Error sending manifest!
Jul 27 11:09:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Invalid role in root.json
Jul 27 11:09:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Invalid role in root.json
Jul 27 11:09:46 verdin-imx8mp-08739953 aktualizr-torizon[849]: Event: UpdateCheckComplete, Result - No updates available

And restarting the aktualizr:

torizon@verdin-imx8mp-08739953:~$ sudo systemctl restart aktualizr
torizon@verdin-imx8mp-08739953:~$ journalctl -f -u "aktualizr*"
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Certificate issuer: CN=ota-devices-CA
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Certificate valid from: Jun 23 11:53:09 2026 GMT until: Jun 23 11:53:09 2126 GMT
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Offline Updates are disabled
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Registering with D-Bus
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Event: SendDeviceDataComplete
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Put manifest request failed: 0  HTTP 400
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Error sending manifest!
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Invalid role in root.json
Jul 27 11:12:56 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Invalid role in root.json
Jul 27 11:12:57 verdin-imx8mp-08739953 aktualizr-torizon[2720]: Event: UpdateCheckComplete, Result - No updates available

The logs reveal the root cause of your issue. Aktualizr is encountering errors when trying to communicate with Torizon Cloud:

Root Cause

The key errors are:

  • Put manifest request failed: 0 HTTP 400
  • Error sending manifest!
  • Invalid role in root.json

These errors indicate that Aktualizr cannot successfully send the device manifest (which includes the current installation status) back to Torizon Cloud. This is why the cloud UI still shows “Update in progress” — it never received confirmation that the update completed successfully(1).

Why This Happens

The “Invalid role in root.json” error suggests there may be a metadata synchronization issue between your device and Torizon Cloud. This can occur when:

  • The device’s Uptane metadata (root.json) is out of sync with the server
  • There was a change in your Torizon Cloud fleet configuration
  • Provisioning credentials have issues

Recommended Solutions

1. Check Device Connectivity

Verify the device can reach Torizon Cloud servers:

curl -I https://app.torizon.io

2. Force Metadata Refresh

Try forcing Aktualizr to refresh its metadata from the server. You can delete the local metadata cache and restart:

sudo systemctl stop aktualizr
sudo rm -rf /var/sota/storage/metadata
sudo systemctl start aktualizr
journalctl -f -u "aktualizr*"

Warning: Only do this if you’re comfortable with the risk. This forces a fresh metadata sync.

3. Re-provision the Device

If the metadata refresh doesn’t work, you may need to re-provision the device to Torizon Cloud. This would require:

  • Obtaining fresh provisioning credentials from Torizon Cloud
  • Re-provisioning the device with those credentials

4. Contact Toradex Support

Since this appears to be a server-side metadata issue (HTTP 400 errors), you may need assistance from Toradex support to resolve the root.json validation problem on the cloud side. The device itself has successfully deployed the update locally, but the cloud-side validation is failing.

Current State

The good news is that your device is running the updated software (version 3.0)(1). The OSTree status confirms this. The issue is purely with the reporting mechanism back to Torizon Cloud, not with the actual update installation.

Best regards,
Toradex AI Assistant.


Did you like my answer? :+1: :-1:

If I answered your question, please tick the “Solution” checkbox.

For follow-up questions to me, please use @ToradexAI in your post.

Greetings @i.staehli,

Just to summarize, the device was correctly updated, but the update itself is not being marked as completed on the server side. Is that correct?

Looking at your logs the main error that is causing this is probably this line here:

Put manifest request failed: 0  HTTP 400

The manifest the device is trying to send here contains the result of the update, and is what tells the server that the update is “completed”. Without the manifest the server has no idea if the update is done or not so it just stays in the “Update in progress” state.

Now we could force cancel the update on the server side. But, if this device can’t send the manifest at all, then the issue will just happen again.

Let’s try to gather more info.

  • First of all, an you reproduce this exact issue on another device?
  • Can you set the log level of Aktualizr to trace to see if the increased logs will reveal more info on why the manifest can’t be set. Please share the logs once you have done so. To change the log-level of Aktualizr please see here: Torizon Update Client - Aktualizr Commands & Settings | Toradex Developer Center
  • Also just to confirm, does this specific device have any known network issues? Or perhaps it’s in a poor network environment?

Best Regards,
Jeremias

Hi @jeremias.tx

Yes, the device was updated correctly, but the update was not marked as completed.

  1. So far we only had the issue on this one device. It’s one of the three devices that we have in Germany with our sales co-workers and we send the updates remotely OTA.
  2. Below you can find the full log of aktualizr after increasing the log level
  3. The device is in a poor network environment (Germany) and has internet connection via a SIM card

aktualizr.log (50.8 KB)

Oh I see what the issue is now from the verbose logs you provided. This is a known issue that is on our list to fix.

The workaround to fix this is to delete the following two files on the affected device:

  • /var/sota/storage/fuse/fuse.yml
  • /var/sota/storage/fuse/target_name

Once these files are deleted, restart Aktualizr. This should fix the state of the device, and hopefully the manifest will properly send to the server and conclude the “Update in progress” state on the server-side.

Can you try this and report back please.

Best Regards,
Jeremias

@jeremias.tx Thanks, that seemed to have solved the issue.

I looked at those two files and they were empty before deleting them:

root@verdin-imx8mp-08739953:/var/sota/storage/fuse# ls -l
total 160
-rw-r--r-- 1 root root      0 Jun 23 13:53 fuse.yml
drwx------ 2 root root   4096 May 29  2025 metadata
-rw-r--r-- 1 root root   1675 May 29  2025 sec.private
-rw-r--r-- 1 root root    451 May 29  2025 sec.public
-rw-r--r-- 1 root root 151552 May 29  2025 sql.db
-rw-r--r-- 1 root root      0 Jul 28 08:12 storage.lock
-rw-r--r-- 1 root root      0 Jun 23 13:53 target_name

Glad that resolved your issue here. Our team will of course work to fix this bug properly when possible.

Best Regards,
Jeremias

Hi @i.staehli,

I just wanted to inform you that this bug was fixed in our OS recently. The next official release that will have this fix is 7.8.0.

Best Regards,
Jeremias