Offline updates usb fail

Technical Support
, , , , , ,
1

Hello,

I am using verdin imx8m-plus SoM on Yavia board and I am trying to test offline updates. The torizon os version is 6.5.0 (release)

I started with the online updates, I followed the documentation and used VSCode QT app template (named it torizontest), created the first version of it and uploaded with tcb-publish-platform runner to a Torizon Cloud test account. Up to this point everything worked flawlessly. I provisioned the device and I was able to run online update, by initiating it from Torizon Cloud UI.
The first version of torizontest was installed correctly and I saw the progress in the aktualizr-torizon service logs.
Then I switched to offline updates, according to the docs (turned off online updates) and restarted the service. Then I made a minor change in the torizontest code and deployed a version 2 to the cloud. Then I defined a lockbox from Torizon UI and then downloaded it using torizoncore-builder. I got update directory on my local filesystem. I copied the directory to a usb media and inserted it into usb port on yavia board.
From the aktualizr logs I can see that the updates check starts but then I get the message - No updates found and that’s it. Restarted the board, reset the whole SoM, tried different packages combination and no luck so far.

My question is - am I doing it the right way? I tried to do offline update with a newer Torizon OS image (6.6.0 devel) and no luck as well. I can see that the check for updates happens when I attach the usb media, but it always ends up with the same message of no updates.

3

Greetings @ollamh,

Just to make sure I understand. You are trying to do an offline update to update the containers on your system correct? You were able to successfully define and download a Lockbox containing your update package. But, when you attach the USB drive containing your Lockbox to the system the update did not occur. Did I understand everything correctly?

Could you provide the Aktualizr logs of the attempted offline update please. Preferably share all the logs from when Aktualizr first started if you can. I want to see what is happening and whether Aktualizr even sees your USB drive or not.

Best Regards,
Jeremias

4

Hi @jeremias.tx , thank you for the quick response!

Yes, correct. Moreover, I was able to do online update installing v1 of my package before trying the offline update with v2.

Here are the logs:

From today - full logs, tried to reinsert usb and got the same message:

root@verdin-imx8mp-15230140:~# journalctl -u aktualizr*
Apr 11 13:07:39 verdin-imx8mp-15230140 systemd[1]: Started Aktualizr SOTA Client.
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Aktualizr version tdx-d71f18a1 starting
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Reading config: "/usr/lib/sota/conf.d/20-sota-device-cred.toml"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Reading config: "/usr/lib/sota/conf.d/30-rollback.toml"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Reading config: "/usr/lib/sota/conf.d/40-hardware-id.toml"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Reading config: "/usr/lib/sota/conf.d/50-secondaries.toml"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Reading config: "/usr/lib/sota/conf.d/60-polling-interval.toml"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Reading config: "/etc/sota/conf.d/61-custom-polling-interval.toml"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Reading config: "/usr/lib/sota/conf.d/70-reboot.toml"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Reading config: "/etc/sota/conf.d/99-offline-updates.toml"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Use existing SQL storage: "/var/sota/sql.db"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Couldn`t import data: empty path received
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Initializing docker-compose Secondaries...
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Use existing SQL storage: "/var/sota/storage/docker-compose/sql.db"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Adding Secondary with ECU serial: 5ae0967423406824c8156914969028ed6db1309799e6a608ad142eb3926535d1>
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Initializing torizon-generic Secondaries...
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Use existing SQL storage: "/var/sota/storage/bootloader/sql.db"
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: No valid metadata found in storage.
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Adding Secondary with ECU serial: c876936a446d837e4e314a34c47a7bbad887b43e5e6c09d74961876b990fab32>
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Primary ECU serial: 69e2b3b43d09995076b9755077475af174c0ff7446029abb6b4ad35f470a74d9 with hardware>
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Device ID: 346413bd-6b49-4860-8454-9a22f7eef13c
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Device Gateway URL: https://dgw.torizon.io
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Certificate subject: CN=346413bd-6b49-4860-8454-9a22f7eef13c
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Certificate issuer: CN=ota-devices-CA
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Certificate valid from: Apr  5 16:23:38 2024 GMT until: Apr  5 16:23:38 2124 GMT
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: PROXY: initializing...
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: PROXY: using TCP port 8850.
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: Offline Updates are enabled
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: PROXY: starting thread.
Apr 11 13:07:39 verdin-imx8mp-15230140 aktualizr-torizon[921]: PROXY: listening to connections...
Apr 11 13:07:42 verdin-imx8mp-15230140 aktualizr-torizon[921]: Event: SendDeviceDataComplete
Apr 11 13:15:04 verdin-imx8mp-15230140 aktualizr-torizon[921]: fetchMetaOffUpd() called with source_path: "/media/INSTALL/update"
Apr 11 13:15:04 verdin-imx8mp-15230140 aktualizr-torizon[921]: Event: UpdateCheckComplete, Result - No updates available

Here are the contents of USB media path, defined in offline updates.toml file

root@verdin-imx8mp-15230140:~# ls /media/INSTALL/update/
images	metadata

Here is the /etc/sota/conf.d/ offline updates toml:

root@verdin-imx8mp-15230140:~# cat /etc/sota/conf.d/99-offline-updates.toml 
[uptane]
enable_offline_updates = true
enable_online_updates = false
offline_updates_source = /media/INSTALL/update

This is from the logs when I used online updates to downgrade version 6.6.0 devel to 6.5.0 release:

Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Device Gateway URL: https://dgw.torizon.io
Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Certificate subject: CN=346413bd-6b49-4860-8454-9a22f7eef13c
Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Certificate issuer: CN=ota-devices-CA
Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Certificate valid from: Apr  5 16:23:38 2024 GMT until: Apr  5 16:23:38 2124 GMT
Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: PROXY: initializing...
Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: PROXY: using TCP port 8850.
Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Offline Updates are disabled
Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: PROXY: starting thread.
Apr 10 15:11:23 verdin-imx8mp-15230140 aktualizr-torizon[1438]: PROXY: listening to connections...
Apr 10 15:11:29 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: SendDeviceDataComplete
Apr 10 15:11:44 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: UpdateCheckComplete, Result - No updates available
Apr 10 15:12:08 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: UpdateCheckComplete, Result - No updates available
Apr 10 15:12:31 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: UpdateCheckComplete, Result - No updates available
Apr 10 15:13:01 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: UpdateCheckComplete, Result - No updates available
Apr 10 15:13:33 verdin-imx8mp-15230140 aktualizr-torizon[1438]: New updates found in Director metadata. Checking Image repo metadata...
Apr 10 15:13:40 verdin-imx8mp-15230140 aktualizr-torizon[1438]: 1 new update found in both Director and Image repo metadata.
Apr 10 15:13:40 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: UpdateCheckComplete, Result - Updates available
Apr 10 15:13:43 verdin-imx8mp-15230140 aktualizr-torizon[1438]: ostree-pull: Scanning metadata: 198
Apr 10 15:13:43 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: DownloadProgressReport, Progress at 0%
Apr 10 15:13:44 verdin-imx8mp-15230140 aktualizr-torizon[1438]: libostree pull from 'aktualizr-remote' for 0 refs complete
                                                                security: GPG: disabled 
                                                                security: SIGN: disabled http: CA-pinned
                                                                non-delta: meta: 3 content: 0
                                                                transfer: secs: 3 size: 3.3 kB
Apr 10 15:13:44 verdin-imx8mp-15230140 aktualizr-torizon[1438]: ostree-pull: 3 metadata, 0 content objects fetched; 3 KiB transferred in 3 seconds; 0 bytes content written
Apr 10 15:13:44 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: DownloadTargetComplete, Result - Success
Apr 10 15:13:44 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: AllDownloadsComplete, Result - Success
Apr 10 15:13:44 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Event: InstallStarted
Apr 10 15:13:45 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Installing package using ostree package manager
Apr 10 15:13:45 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Commit metadata kargs=quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3
Apr 10 15:13:47 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Copying /etc changes: 4 modified, 2 removed, 24 added
Apr 10 15:13:48 verdin-imx8mp-15230140 aktualizr-torizon[1438]: Transaction complete; bootconfig swap: yes; bootversion: boot.1.1, deployment count change: 0

This is the earliest available log:

Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: Device Gateway URL: https://dgw.torizon.io
Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: Certificate subject: CN=346413bd-6b49-4860-8454-9a22f7eef13c
Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: Certificate issuer: CN=ota-devices-CA
Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: Certificate valid from: Apr  5 16:23:38 2024 GMT until: Apr  5 16:23:38 2124 GMT
Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: PROXY: initializing...
Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: PROXY: using TCP port 8850.
Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: Offline Updates are enabled
Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: PROXY: starting thread.
Apr 10 09:12:57 verdin-imx8mp-15230140 aktualizr-torizon[989]: PROXY: listening to connections...
Apr 10 09:13:03 verdin-imx8mp-15230140 aktualizr-torizon[989]: Event: SendDeviceDataComplete
Apr 10 09:36:01 verdin-imx8mp-15230140 aktualizr-torizon[989]: fetchMetaOffUpd() called with source_path: "/media/INSTALL/update"
Apr 10 09:36:01 verdin-imx8mp-15230140 aktualizr-torizon[989]: Event: UpdateCheckComplete, Result - No updates available

Thank you,
Andrey

5

It does appear that Aktualizr is seeing your Lockbox on your USB but reports no updates. I just did a quick test myself to try and reproduce this. Surprisingly I was able to reproduce this almost right away. I think I know what’s wrong, but I’ll need some time to discuss and confer with the team internally about this.

Just to confirm you and I are having the same issue. Could you share the contents of the following file from your Lockbox. There should be a file in your Lockbox by the name of metadata/director/<NAME OF YOUR LOCKBOX>.json. Please share the contents of this file if you could, it will help me confirm my theory.

Best Regards,
Jeremias

6

Here are the contents of the update12.json lockbox

{"signatures":[{"keyid":"815f1380ef0666d3eb2dbc4a2e8d6001571f3fb6551e5003c6ba9117421b5127","method":"ed25519","sig":"/FS/uWsQr7NeJvmnddkjg8CGiLpYZddkP5Vsxfi7OzrmhVjYtODMwmlFGSXO0xqivgNqUeCDJTl6qxABTMMVDQ=="}],"signed":{"_type":"Offline-Updates","expires":"2025-04-10T15:04:41Z","targets":{"torizontest-4":{"custom":{"hardwareIds":[null]},"hashes":{"sha256":"ac473b0f7e78b69163c74ab08c2ea518aaa59a8a6a4c0427dfa45474fb3c4fb1"},"length":956}},"version":1}}
7

Thank you for sharing so quickly, this seems to support my suspicions. Let me discuss with the team and I’ll get back to you once there’s some news to share here.

Best Regards,
Jeremias

8

Okay the team was able to deploy a fix for this. You’ll need to log out of your account, then clear site data and cookies for torizon.io. Once this is done log back in, then create a new Lockbox or modify your previous one to refresh it. Then try performing an offline update on the device with the new Lockbox.

I just tested this myself and it seems to work fine now.

Best Regards,
Jeremias

9

@jeremias.tx , thanks to you and the team, now it works just fine!

10

Perfect! Glad we were able to help, and thank you for reporting this issue so we could fix it in a timely manner.

Best Regards,
Jeremias