Update using Torizon OTA for newbie

I’ve programmered two .Net Core applications in different containers and came to the next step: Deployment of two application containers and a local UI in Kios mode.

I’ve read the articles Link 1 and link 2

I’ve saved an example from the first article, uploaded in our account and assigned to the device. At first I’ve got an error update failed, then I’ve stopped the containers and tried again. This time there were no errors, but all containers exept three disappiered.

  1. Do I have to include all containers which I want to use all other will be automatically deleted?
  2. Should I stop all containers at first?

Then I did a second step, pushed my containers to docker and added to the docer-compose file additional services analogue to article 2

 immsensorweb:
    container_name: imm-sensor-web
    image: programmierhirsch/imm-sensor-web:preview
    network_mode: host etc.

I’ve applied the new docker-composer file to my device and as a result came upload failed again. I’ve checked the state of portainer, but it didn’t work any more. I had to install it manually from docker.

  1. That I did wrong?

  2. Do you have an examples of multi-container configuration (host network) conneted with REST api, the backend application uses GPIO and CAN controller?

  3. How I can check the reason why an update has failed?

Greetings @programmier-hirsch,

Let me go through your questions one by one and see if I can clear some things up.

Do I have to include all containers
which I want to use all other will be
automatically deleted?

In short yes. The longer answer is that during a container update process the following occurs:

  1. New docker-compose is downloaded as a tmp file.
  2. The container images defined in the new docker-compose are pulled/downloaded to the system.
  3. docker-compose down is ran on the old docker-compose file. Meaning it will stop any running containers as defined by the previous docker-compose. If you have other containers not defined by docker-compose they will be unaffected
  4. docker-compose up is ran on the new docker-compose file.
  5. If everything is successful up to now then the new docker-compose will officially replace the old one. And any container/images not associated with the new docker-compose will be removed from the system.
  6. If at any point this process fails the system will rollback to the previous state with no trace of the new stuff.

Should I stop all containers at first?

As per the process I laid out above you shouldn’t need to as the update process will stop and remove them for you. Unless they’re not defined in the docker-compose file in which case you may need to manually clean this up yourself.

Do you have an examples of multi-container configuration (host network) conneted with REST api, the backend application uses GPIO and CAN controller?

Unfortunately I’m not aware of such an example from us. We have individual examples of each of these use-cases but I don’t believe we have a single example with all 3 together.

Here is a list of all Torizon documentation including examples: https://developer.toradex.com/software/torizon#tab-documentation

Here’s the source repository for all code examples: https://github.com/toradex/torizon-samples

How I can check the reason why an update has failed?

The only real way to know what is happening behind the scenes with an update is to run the following on the device: journalctl -f -u aktualizr*

This will produce the logs for the update client showing what is happening in the background during an update. If an update fails it should also state the reason as to why.

Best Regards,
Jeremias

Thank you @jeremias.tx !

I’ll try to follow this guidlines and then ask again :slight_smile:

Some additional questions:

  1. I’ve defined dockercomposefile for both (container-)applications. How does it correspond with docker-compose in OTA? Do I need this files only for release published directly onto device or I have to define both and they will be somehow used during update process?
  2. Is it possible to implement following scenario during update process: the applications will be ugedated, but some (settings) files will be unchanged? Some devices are configured after initial setup and we want to preserve this settings during possible update.
  3. In this article are used docker-compose.yml and docker-compose.override.yml files. Does it mean, that if I add package in my account, both files will be uploaded automatically?

I’ve updated my docer-compose file, started Update again and started journalctl -f -u aktualizr* on the device.

May 25 13:45:32 colibri-imx8x-06789119 aktualizr[3940405]: Total reclaimed space: 1.455GB
May 25 13:45:33 colibri-imx8x-06789119 aktualizr[867]: Event: InstallTargetComplete
May 25 13:45:33 colibri-imx8x-06789119 aktualizr[867]: Event: AllInstallsComplete
May 25 13:45:33 colibri-imx8x-06789119 aktualizr[867]: Update install completed. Releasing the update lock...
May 25 13:45:33 colibri-imx8x-06789119 aktualizr[867]: **Current versions in storage and reported by OSTree do not match**
May 25 13:45:34 colibri-imx8x-06789119 aktualizr[867]: Event: PutManifestComplete

What does it mean?

alt text
Best Regards,
Dmitry

I’ve defined dockercomposefile for both (container-)applications. How does it correspond with docker-compose in OTA? Do I need this files only for release published directly onto device or I have to define both and they will be somehow used during update process?

Let me clarify something here. So the only docker-compose.yml that OTA is aware about is the docker-compose.yml in this location on the device: /var/sota/storage/docker-compose/docker-compose.yml. If you have a compose file in any other location on the device then it is unknown to the OTA system. The compose file in that location signifies the current OTA-tracked compose file for the system. So referring back the process I explained in my initial comment. During an update this is the “old” compose file that will be replaced by the “new” compose file upon a successful update.

With that being said you don’t really need anything “published” to the device you just need to upload your compose file to the OTA system then trigger an update. Then the update will occur as per the process I described.

Is it possible to implement following scenario during update process: the applications will be ugedated, but some (settings) files will be unchanged? Some devices are configured after initial setup and we want to preserve this settings during possible update.

I’m not sure I understand your question here. If you perform only a container update then the only thing that gets changed is that one docker-compose.yml file. Any other files on the system will not be touched, unless you also do an OS update.

In this article are used docker-compose.yml and docker-compose.override.yml files. Does it mean, that if I add package in my account, both files will be uploaded automatically?

What are you intending here? You upload one docker-compose.yml file at a time and then you can push one update for a single docker-compose.yml file at a time. So you can push both docker-compose files to your account but you can only push one of these as an update at a time. With the latest update being what decides which compose file ends up on the device.

I’ve updated my docer-compose file, started Update again and started journalctl -f -u aktualizr* on the device.

Based off the log it looks like there’s no update/work for the client to do. Bear in mind if you trigger an update in the UI it doesn’t happen immediately. It happens the next time the device checks in with the server which by default happens every ~5mins or so.

Best Regards,
Jeremias

Thank you @jeremias.tx!

based on the output of journalctl -f -u aktualizr* I could build step by step a working docker-compose.yml. I’ve not got a local UI in a kiosk container yet but hopefully I’m not far from my target.

I’m not sure I understand your question here. If you perform only a container update then the only thing that gets changed is that one docker-compose.yml file. Any other files on the system will not be touched, unless you also do an OS update.
I mean, what in my application I’ve stored a settings.json file in app_data directory. If I’ll update something in my application and push a new image to the docker repository, during a next update (pull) in my target device the existent image will be replaced with a new one, including settings.json file.
Am I right? If yes, where I can store my settings so they will not be replaced during update process.


Best Regards,
Dmitry

Hi @jeremias.tx

it doesn’t work properly. I’ve changed only one line for Kiosk in the docker-compose (command start Web-Address) and container weston was removed (untagged despite the same depends_on - weston)

Practically every time I do a small change of the configuration, this container disappears and kiosk doesn’t start.

I load the previous version again, it works, but not always. I have to rename the docker-composer file in oder to get a working configuration.

And now this trick doesn’t work too:
alt text
link text

I mean, what in my application I’ve stored a settings.json file in app_data directory.

To clarify, this app_data directory and settings.json file are in the container yes?

If yes then why not store this on the host system and then access it via bindmount in the container. That way you can update your containers but then the directories/files will be persistent and untouched on the host system. Or am I still misunderstanding?

Describe to me in detail what you’re doing.

it doesn’t work properly.

What doesn’t work properly, the update, upload, the containers?

this container disappears and kiosk doesn’t start.

Disappearing where? from the device? What are your container logs saying? If your docker-compose is being managed by the docker-compose.service. You can see the logs with journalctl like you do for aktualizr.

And now this trick doesn’t work too:

This looks like it’s from the OTA web UI yes? This happens when you try to upload your compose file? How are you naming the package? Are you using the same package name?

Best Regards,
Jeremias

Sorry @jeremias.tx ,

I was a little bit upset, because I have to struggle with different problems in this project may be more than writing code.

  • With aktualizr I could see what weston container was pulled from repository, started, when two other containers stopped and weston was removed and deleted from the device with a message “untagged…”
  • The file name was docker-compose-Imm-1 .yml Now I have found: it was because of space in the file name. Just wrong text in the message box.
  • I have restarted iMX8 with sudo reboot and now it works in a kiosk mode properly (only screen resolution seems to be incorrect, I should possibly check this article)
  • One question more: if I push a new image to the docker registry with the same tag, torizon doesn’t check it? Do I have to apply the OTA package again in oder to pull a new verstion of my application from Docker?

To clarify, this app_data directory
and settings.json file are in the
container yes?

If yes then why not store this on the
host system and then access it via
bindmount in the container. That way
you can update your containers but
then the directories/files will be
persistent and untouched on the host
system. Or am I still
misunderstanding?

Yes, you’re write, there are files in container. I supposed I have to use this way, but I dont’t know yet how to do it. It is my first linux/container/torizon project.
Do I have to use this recomendation and define volumes in docker-composer files as well?

Best Regards,
Dmitry

With aktualizr I could see what weston container was pulled from repository, started, when two other containers stopped and weston was removed and deleted from the device with a message “untagged…”

So were all the containers from the updated compose file correctly started/updated?

The file name was docker-compose-Imm-1 .yml Now I have found: it was because of space in the file name. Just wrong text in the message box.

Ahh yes that will do it. I’ll bring this up to the team to make a more clear error message here. Or maybe just not allow files with spaces in the name.

I have restarted iMX8 with sudo reboot and now it works in a kiosk mode properly (only screen resolution seems to be incorrect, I should possibly check this article)

What display interface are you using?

One question more: if I push a new image to the docker registry with the same tag, torizon doesn’t check it? Do I have to apply the OTA package again in oder to pull a new verstion of my application from Docker?

If you push a new image with the same tag to the registry, but you don’t change the docker-compose.yml then the system will just use whatever version of the image is on the device. You would need to manually pull the image onto the device. Or initiate an update so that the system will auto pull all images specified in the compose file. Otherwise there’s no real way for the system to know that you changed the image in the registry.

Do I have to use this recomendation and define volumes in docker-composer files as well?

Yes you need to define any volumes/bind-mounts in your compose file otherwise your container will not start with a bind-mount present.

Best Regards,
Jeremias

@jeremias.tx ,
we have resistive touch screen 7" Display
The resolution seems to be okey, but touch screen doesn’t react.

Best regards,
Dmitry

If you’re referring to the Toradex 7" resistive touch display then you need to apply certain device tree overlays for full functionality as described here: https://developer.toradex.com/knowledge-base/device-tree-overlays#tab-resistive-touch-display-7-parallel

Best Regards,
Jeremias

Thanks @jeremias.tx,
it works.
Now I have only a problem with internes CAN Bus, but I’ll ask your in my second topic.

Best regards,
Dmitry

Perfect that it works. Thanks for your feedback.