Offline updates - provision is required?

Technical Support
, , ,
1

Hello,

I’m working on offline updates process for our Verdin imx8m-plus boards with slightly modified Torizon OS image with some bundled docker containers.

I define /etc/sota/conf.d/99-offline-updates.toml file with offline updates enabled and online updates disabled flags.

I see that there is a requirement for provisioning the board in case of online updates. For the offline updates it is required to store shared data provided by torizoncore-builder:

torizoncore-builder platform provisioning-data --credentials <path/to/your/credentials.zip> --shared-data shared-data.tar.gz
# and then copy archive to the board and put it into /var/sota/import folder

Is this step obligatory?
I tried to run update without this information and it looks like it worked normally, although with some log errors.
And if it is required, should I follow this documentation approach then: Production Programming in Torizon | Toradex Developer Center

input:
  easy-installer:
    local: <INPUT_IMAGE>
customization:
  <YOUR_CUSTOMIZATIONS>
output:
  easy-installer:
    local: <OUTPUT IMAGE>
    provisioning:
      mode: "online"
      shared-data: <SHARED-DATA TARBALL>
      online-data: <ONLINE-DATA STRING>

Can I skip online-data part? The device will never get updates via internet, always will have offline updates.

2

Hi, I’m the Toradex AI Assistant. I’m in early beta, and I would appreciate your feedback.
The Toradex Support team will follow up with you shortly!

Did you like my answer?

:+1: :-1:

Hello,

For your Verdin iMX8M Plus boards running a slightly modified TorizonCore OS with bundled Docker containers, focusing on offline updates, the provisioning step using torizoncore-builder platform provisioning-data to obtain and apply the shared-data is indeed important, even if your devices are intended for offline updates only.

The shared-data contains essential information for the device to trust the software repository, which is crucial for the integrity and security of the offline update process. While you might have observed that updates appear to work without this step, the log errors you’ve encountered could be indicative of underlying issues with update verification or other processes that rely on this trust relationship.

Since your devices will not receive updates over the internet and will only use offline updates, you can omit the online-data part. The online-data is specifically for devices that will be provisioned for online updates and involves sensitive information that is not applicable to your use case.

For applying the shared-data during production programming, you should indeed follow the approach outlined, with a focus on the shared-data tarball. Here’s a simplified version of the configuration you might use, omitting the online-data since it’s not relevant for your scenario:

input:
  easy-installer:
    local: <INPUT_IMAGE>
customization:
  <YOUR_CUSTOMIZATIONS>
output:
  easy-installer:
    local: <OUTPUT IMAGE>
    provisioning:
      mode: "offline" # Adjusted to reflect offline provisioning
      shared-data: <SHARED-DATA TARBALL>

Ensure that the shared-data tarball is correctly applied to each device, which will allow them to trust the software repository for offline updates. This step is crucial for maintaining the security and integrity of your update process.

Best regards,
Toradex AI Assistant

Please, also check if these links can help you solve your problem:

TorizonCore Builder Tool “build” command | Toradex Developer Center.

3

Greetings @ollamh,

If your device will only ever run offline updates, then you only need the shared-data information to provision your device. The online-data is not required in this circumstance.

I tried to run update without this information and it looks like it worked normally, although with some log errors.

Just to clarify what do you mean here exactly? Do you mean the update ran without provisioning at all? Are you talking about offline or online updates here? It’s not clear to me.

For our update client on the device to work normally, the device needs to be provisioned. As I said the minimal information needed for provisioning depends on what kind of updates you want to do. Offline updates only requires the shared-data, while online updates require both shared-data and online-data information.

Best Regards,
Jeremias

4

Greetings @jeremias.tx , thank you for the quick reply as always!

My bad, I didn’t mention that I had offline update run without provisioning and the update was completed successfully.

That’s why I was wondering if that step is strictly necessary. After the response from AI I tried the suggested changes in tcbuild.yaml file and I saw a message about successful provisioning during the OS build stage.

From the aktualizr-torizon logs perspective I don’t see any significant difference, but I will keep an eye on that and will update the thread in case of useful information found.

For now I am going to mark AI’s response as the solution.

5

My bad, I didn’t mention that I had offline update run without provisioning and the update was completed successfully.

Are you sure this happened? Aktualizr-torizon will not even run properly without at least minimal provisioning of the device. For offline updates it needs at the very least the shared-data information to operate. It shouldn’t be possible to do an update without some kind of provisioning.

Best Regards,
Jeremias

6

I cannot say that there was no provisioning data at all, we are using the Torizon verdin imx8mp Tezi_6.4.0+build.5 image as the base OS image. From the logs I don’t see any complain from aktualizr on missing provision information.

Unfortunately I don’t have enough time now to compare logs, I included provisioning with shared data into tcbuild.yaml file already.

input:
    easy-installer:
        local: torizon-core-docker-verdin-imx8mp-Tezi_6.4.0+build.5.tar
output:
    easy-installer:
        local: core-docker-verdin-imx8mp-easy-installer
        name: "TorizonCore for  imx8mp v XXX"
        description: "TorizonCore customized Toradex Verdin {SOM} on Yavia carrier version XXX"
        bundle:
            dir: bundle/
        provisioning:  # -- this section was missing in the previous version of build
            mode: "offline"
            shared-data: shared-data.tar.gz

bundle/ contains for example a docker-compose.yml

services:
  weston:
    cap_add:
    - CAP_SYS_TTY_CONFIG
    device_cgroup_rules:
    - c 4:0 rmw
    - c 4:1 rmw
    - c 4:7 rmw
    - c 13:* rmw
    - c 199:* rmw
    - c 226:* rmw
    environment:
      ACCEPT_FSL_EULA: '1'
    image: torizon/weston-vivante@sha256:663527c9132ebf6595ae90337f27eac0f7ca1b5c6174e1fcad9dfbf4f4aad4f3
    network_mode: host
    restart: always
    volumes:
    - source: /tmp
      target: /tmp
      type: bind
    - source: /dev
      target: /dev
      type: bind
    - source: /run/udev
      target: /run/udev
      type: bind

I could see the changes in the docker container updates without provisioning. For the OS updates - this I can’t confirm, I was concentrating on the docker part.

7

One more question which I don’t understand from the documentation How to Use Secure Offline Updates with Torizon OS | Toradex Developer Center

$ scp shared-data.tar.gz torizon@192.168.1.125:
shared-data.tar.gz

$ ssh torizon@<your-device-ip-address>

apalis-imx8~$ sudo tar zxvf shared-data.tar.gz --directory /var/sota/import

Do I need to provide both changes in tcbuild.yaml file and do the manual copying and unarchiving the shared credentials data into /var/sota/import folder? After I provisioned the OS with tcbuild.yml file I don’t see /var/sota/import folder on the board filesystem.

8

Do I need to provide both changes in tcbuild.yaml file and do the manual copying and unarchiving the shared credentials data into /var/sota/import folder? After I provisioned the OS with tcbuild.yml file I don’t see /var/sota/import folder on the board filesystem.

No it’s either or. You can copy the shared-data.tar.gz manually to the device and unpack it. Or create an image via tcbuild.yaml that will add this information during image flashing. The former method is good for quick tests, while the latter is better for production programming.

Also are you sure /var/sota/import was empty? I just tried an image provisioned via tcbuild.yaml myself and I can see it’s populated:

sudo ls -l /var/sota/import
total 8
dr-x--x--x 2 root root 4096 May 10 16:28 director
dr-x--x--x 2 root root 4096 May 10 16:28 repo

I cannot say that there was no provisioning data at all

If you notice you can do updates without provisioning at all, please let us know. This would be a very big issue in our system as updates should not be possible without provisioning.

Best Regards,
Jeremias

9

Hi @ollamh,

Just checking since I didn’t hear back from you, did my last reply answer your question? Or do you still have any questions/confusion related to this topic?

Best Regards,
Jeremias

10

Hi @jeremias.tx ,

Yes, it makes sense totally! I also can see the /var/sota/import on the freshly installed system with the provisioned data.

The only odd behavior is that if the system wasn’t originally provisioned and you apply the updates, it seems to be applying those without any problem. I’m not sure if that should be blocked, but the only consequence of /var/sota/import directory missing is that the aktualizr-torizon service fails to start which in my case it is just fine.

Thank you again for checking this, really appreaciate!

11

The only odd behavior is that if the system wasn’t originally provisioned and you apply the updates, it seems to be applying those without any problem. I’m not sure if that should be blocked, but the only consequence of /var/sota/import directory missing is that the aktualizr-torizon service fails to start which in my case it is just fine.

That sounds odd to me. On my side, if my device is not provisioned at all, then the Aktualizr service won’t even start. If I configure the device for offline updates then, manually execute aktualizr-torizon, and finally attach a USB drive with a lockbox. The update appears to be executed as you said. This is quite a problem and shouldn’t happen since the device hasn’t been provisioned.

Thank you for reporting this, I’ll report this to our team.

Best Regards,
Jeremias

12

Just to comment on the unforeseen ability to perform an offline update without provisioning.

We did not catch this in our test cases, because the systemd service will not actually start the aktualizr client unless there is provisioning data present. For this behaviour to occur, you would need to manually start aktualizr-torizon, with the offline updates feature enabled, and then insert a USB stick with a valid lockbox. Since those steps would require root, we aren’t considering this to be a security issue for Torizon OS.

However, we do consider it to be unintended behaviour, and will be changing it in a future release. aktualizr-torizon is open source software, and we want to be mindful of all potential users of the project, not just Torizon OS. We also upstream our contributions to the overall aktualizr project under the Linux Foundation’s Uptane project, although our offline update implementation hasn’t made it upstream just yet. If there were another operating system or distro out there that didn’t have the mitigation of only starting aktualizr if it’s already provisioned, it could create a dangerous situation. If an unsuspecting user were to ship an operating system that was running aktualizr-torizon in the background without provisioning data being present and enabled offline updates by default, it would be a serious issue indeed. We don’t want that to happen, so we will be disabling this trust-on-first-use behaviour before we upstream anything, and also making the change in future releases of Torizon OS.

Best Regards,
Jeremias