Changing device provisioning to a new repository

Technical Support
, , , ,
1

Hi all,

We’ve found ourselves in what I think is a bit of an unexpected corner case in that we need to change the provisioning information on devices and are hoping there is a way to do this without re-flashing them.

Background:
We’ve been doing product development and producing lockboxes/updates on one torizon platform account for a few months. Now that development is winding down and we are nearing release, we would like to switch to a proper company account that is free from all of the development “clutter” (packages, lockboxes, etc).

Of course, the system(s) will not accept lockboxes created on the new account:
Director metadata update failed: A key has an incorrect associated key ID

We tried re-provisioning the devices against the new account using the web interface to generate a provisioning command. This results in a little bit of progress but it still does not work - it appears to accept the OS portion of the lockbox (and it will accept/apply an OS-only lockbox), but not the containers.

Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: NOT-PROGRAMMED is unknown
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: a274d3d83f24ccaf898eca871a1c2b5677eafdc860cae370f37ac69a0bc60822 is unknown
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: cc5f8da17f7b4bf78e6a232899617704981a68a621e4a705f2964114e46a7bbd is unknown
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: New updates found in Director metadata. Checking Image repo metadata...
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: 3 new updates found in both Director and Image repo metadata.
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: UpdateCheckComplete, Result - Updates available
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Update available. Acquiring the update lock...
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: NOT-PROGRAMMED is unknown
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: a274d3d83f24ccaf898eca871a1c2b5677eafdc860cae370f37ac69a0bc60822 is unknown
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: cc5f8da17f7b4bf78e6a232899617704981a68a621e4a705f2964114e46a7bbd is unknown
Aug 28 13:33:35 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Performing a local pull from file:///var/volatile/DEV_UPDATE/update/images/ostree
Aug 28 13:33:36 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: ostree-pull: Scanning metadata: 53
Aug 28 13:33:36 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: DownloadProgressReport, Progress at 0%
Aug 28 13:33:37 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: ostree-pull: Writing objects: 17
Aug 28 13:33:38 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: ostree-pull: Writing objects: 18
Aug 28 13:33:39 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: ostree-pull: Writing objects: 5
Aug 28 13:33:40 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: ostree-pull: 12 metadata, 19 content objects imported; 2.4 MB content written
Aug 28 13:33:40 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: DownloadTargetComplete, Result - Success
Aug 28 13:33:40 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Initiating fetching of file Containers.lock.yml-v0.9.0-alpha11
Aug 28 13:33:40 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: DownloadProgressReport, Progress at 100%
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: DownloadTargetComplete, Result - Success
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Initiating fetching of file TPS25750_Firmware-v4
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: DownloadProgressReport, Progress at 100%
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: DownloadTargetComplete, Result - Success
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: AllDownloadsComplete, Result - Success
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: NOT-PROGRAMMED is unknown
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: a274d3d83f24ccaf898eca871a1c2b5677eafdc860cae370f37ac69a0bc60822 is unknown
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Current version for ECU ID: cc5f8da17f7b4bf78e6a232899617704981a68a621e4a705f2964114e46a7bbd is unknown
Aug 28 13:33:41 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Waiting for Secondaries to connect to start installation...
Aug 28 13:33:42 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Failed to update Director metadata: Could not find any valid offline updates metadata file
Aug 28 13:33:42 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Sending metadata to a274d3d83f24ccaf898eca871a1c2b5677eafdc860cae370f37ac69a0bc60822 failed: "VERIFICATION_FAILED":3 Failed to update Director metadata: Could not find any valid offline updates metadata file
Aug 28 13:33:43 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Event: AllInstallsComplete, Result - docker-compose:VERIFICATION_FAILED
Aug 28 13:33:43 Hub-NOT-PROGRAMMED aktualizr-torizon[21321]: Update install completed. Releasing the update lock...

A similar error is observed in attempting to update the containers after deploying an OS-only lockbox (Which is updated successfully).

I’ve also tried replacing the data in /var/sota with the contents of the “new” provisioning-data.tar.gz, deleting /var/sota/import/gateway.url, and re-running auto-provision.sh (which reports success). However, it still rejects the containers.lock.yml component of the update.

All packages and lockboxes are visible as expected in the “new” account, and the OS image itself has also been provisioned with the correct data. I’m very confused why it is accepting the OS portion of the update package but not the containers (or a firmware blob we are also pushing as an additional secondary)

One of the developers has confirmed that re-flashing a device from scratch with the “new” OS image built against the new account will allow the update to work correctly.

However, as noted previously we’d like to find a less hands-on method of this migration process that can be done in an automated way, e.g. via script or remote SSH. Is there something I am missing that needs to be done to “reset” the provisioning state more fully than the things I’ve already tried?

2

Update: I found the issue. It is also necessary to clear /var/sota/storage/ prior to applying the update.

For others’ information, the total process looks something like this:

  • Stop docker-compose, aktualizr.
  • Erase /var/sota/storage and /var/sota/import
  • Get new provisioning-data.tar.gz from the OS’s tezi-deploy image and extract to /var/sota
  • Run /usr/sbin/auto-provisioning.sh
  • Now your lockboxes built against the new repository/account will be accepted.
3

Glad you were able to resolve your issue here. For future reference we do have a small note on deleting some files in /var/sota for re-provisioning here: Aktualizr - Modifying the Settings of Torizon Update Client | Toradex Developer Center

By the way out of curiosity why were you trying to move devices from one repository to another here?

Best Regards,
Jeremias

4

Just typical development workflow where things started on a personal account in the early R&D phases, and we decided to keep working out of that for a bit to keep the “clutter” in the final company account down while there was a lot of development churn (because you cannot permanently delete a lockbox)