Torizon IDE, pushing containers in differents docker hub repositories using the same project

Hi,

I am currently developing a python software using the Torizon IDE inside VSCode.

I am pushing my containers inside a docker hub repository using differents Tags which are really usefull when i have to do some tests “on the fields”.

I am using the VSCode task : create-production-image

With this method the container generated doesn’t have the Dockerfile.debug inside.

I really like the approach with the Dockerfile & the Dockerfile.debug and I’m thinking about making a dockerhub container with this approach to be able to accelerate the debuging phase “on the field”.

I still need the version without the debugging (generated by the docker-compose.prod.yml) but i was wondering if it’s possible within the same Torizon IDE / VSCode project to publish inside another docker hub repository ?

My idea is to have one repo with the “stable release” so no need to have the debug terminal and another one which would be the “testing release” which include this time the debug terminal.

I already tried to change the field image inside my docker-compose.yml

version: "3.9"
services:
  my-container-debug:
    build:
      context: .
      dockerfile: Dockerfile.debug
    image: ${LOCAL_REGISTRY}:5002/my-container-debug:${TAG}
    ports:
      - 6502:6502
      - 6512:6512
      - 4840:4840

  fbdevice-kilobaser-container:
    build:
      context: .
      dockerfile: Dockerfile
    image: ${DOCKER_LOGIN}/my-container:${TAG}  #here i tried to change the "path"
    ports:
      - 6502:6502
      - 6512:6512
      - 4840:4840

I assume that it’s the file :

createDockerComposeProduction.ps1

which handle the creation of the docker-compose.prod.yml

Do you have any documentation on how to modify it to include the dockerfile.debug inside the docker-compose.prod.yml?

It would be really nice for me to be able to choose between generating the regular docker-compose.prod.yml and maybe the “docker-compose.debug.yml”.

As we use the file settings.json to give the Tag version, maybe we could use in a similar way a variable inside this file to choose with or without debuging?

I have also a question about Torizon.io OTA :

Currently i have 5 Toradex SoM using my container, we are still in a early dev & research phase so I’m still using the command

docker pull mycontainername:tag

to update my SoMs, as I only have 5 boards, It’s still okay to use this method to update my “fleet”.

Still I think that the idea with Torizon.io OTA is really intersting for the future of the project and I’m currently testing it.

I’m not sure to understand really well how it works.

When I give to Torizon.io a new docker-compose.prod.yml (or maybe docker-compose.debug.yml)

Does Torizon.io connect to my dockerhub and search for the container with the same tag as inside my docker-compose file ?

And then accomplish a fleet “docker pull” to update all the SoMs inside the fleet ?

For me it sound kind of crazy to be able to do this whole process with only the docker-compose file.

For me to be able to update my whole fleet with either my docker-compose.prod.yml or docker-compose.debug.yml in one click would be so great !

To be able to switch between a release container or a testing container for my whole fleet in no time would be a truly good thing.

I hope I’ve been clear enough about my intentions regarding these two container versions and my desire to be able to deploy them on all my modules fairly quickly.

Thank you a lot !

Hi, I’m the Toradex AI Assistant. The Toradex Support team will follow up with you shortly!

Did you like my answer?

:+1: :-1:


To address your questions and intentions regarding managing different Docker Hub repositories and utilizing Torizon.io OTA, here’s a structured approach:

  1. Dual Repository Strategy:

    • You can indeed publish containers to two different Docker Hub repositories within the same Torizon IDE/VSCode project. To achieve this, you can modify your docker-compose.yml to specify different image paths for stable and debug versions. You’ve already started this by changing the image field in your docker-compose.yml. Ensure that the image field points to the correct repository and tag for each version.
  2. Modifying createDockerComposeProduction.ps1:

    • To include the Dockerfile.debug in the generated docker-compose.prod.yml, you would need to modify the script to conditionally include the debug configuration. Unfortunately, without the specific content of createDockerComposeProduction.ps1, I can’t provide exact modifications. However, you can add logic to check for a variable (perhaps set in settings.json) that determines whether to include the debug configuration. This could involve duplicating the relevant sections of the script and modifying them to use Dockerfile.debug.
  3. Torizon.io OTA Functionality:

    • Torizon.io can indeed connect to Docker Hub to pull the container images specified in the docker-compose files. When you provide a new docker-compose.prod.yml or docker-compose.debug.yml to Torizon.io, it will use the specified tags to pull the correct images from Docker Hub and update your fleet. This process is automated, allowing for fleet-wide updates based on the docker-compose file you provide.
  4. Single Command Updates:

    • You can implement a system to switch between stable and testing containers for your entire fleet using a single docker-compose file. By maintaining separate docker-compose.prod.yml and docker-compose.debug.yml files, you can choose which one to deploy via Torizon.io. This setup allows you to quickly switch between different container versions across your fleet.

In summary, with some script modifications and proper configuration of your docker-compose files, you can achieve the desired setup for managing stable and testing releases, as well as leveraging Torizon.io for efficient fleet management.

Please, also check if these links can help you solve your problem:

https://www.torizon.io/.

Greetings @autolab,

There’s a lot to dissect in your message so let me try to break it into more manageable pieces.

Do you have any documentation on how to modify it to include the dockerfile.debug inside the docker-compose.prod.yml?

Just to understand what you want is basically a yaml file that has both your production image and your debug image. So for example:

# example docker-compose.yml
version: '3.9'
services
  debug:
    image: my-container-debug:foo

  production:
    image: my-container:foo

Is my understanding of this correct?

As we use the file settings.json to give the Tag version, maybe we could use in a similar way a variable inside this file to choose with or without debuging?

Depends on how you launch your compose file. If you just use docker-compose up it will try to launch every container specified in the yaml file. This probably doesn’t work well if you try to launch both the production and debug containers at the same time. Otherwise if you do docker-compose <service-name> then it wil only launch the container referenced by that service name. So in my example if you do docker-compose up debug it will only launch the debug container.

Does Torizon.io connect to my dockerhub and search for the container with the same tag as inside my docker-compose file ?

It does not. Our Cloud platform only has access to the docker-compose.yml file that you uploaded to us. During an update this file gets downloaded to the device. The device will then pull the container images as specified by the docker-compose.yml file. So it is the device that will access your container image. This also means your devices need network access to whatever container registry your container images are stored at.

And then accomplish a fleet “docker pull” to update all the SoMs inside the fleet ?

If you do a fleet update, all devices in that fleet will get the same docker-compose.yml via the update. Then each device will individually perform a docker-compose pull action.

Best Regards,
Jeremias

Hi,

Just to understand what you want is basically a yaml file that has both your production image and your debug image

yes exactly I would like to be able to have an access to the debug terminal “on the field” to be able for example to understand faster what went wrong with a new feature etc…

That’s why I would like to have 2 different docker-hub repository :

  1. The “stable release” of the container
  2. The “test release” of the container, I want to use this one to include new functionalities, updates, patches etc… that why It would be interesting to have a debug terminal with it.

and to have a faster deployment I would use Torizon.io OTA with 2 different docker-compose.yml files.

  1. docker-compose.prod.yml : to be able to switch my whole fleet in a stable release set-up because for example some other department have to use my modules and I don’t want them to have some issues with new fonctionalities that I’m currently adding / modifying.

  2. docker-compose.test.yml : It would be really interesting for me to switch in no time my whole fleet in a test setup with the new functionnalities.

Actually to manage which version of the container is “stable” or not, I am basically some Tags name like : stable1.xxx or dev1.xxx.

It works fine, but I really think that I can improve this side of the project and the debug terminal would be amazing.

Depends on how you launch your compose file. If you just use docker-compose up it will try to launch every container specified in the yaml file. This probably doesn’t work well if you try to launch both the production and debug containers at the same time. Otherwise if you do docker-compose <service-name> then it wil only launch the container referenced by that service name. So in my example if you do docker-compose up debug it will only launch the debug container.

I need to have my container to start “automatically” when the SoM boot, so I added the docker-compose there :

/var/sota/storage/docker-compose

and the docker-compose service from systemd start It for me.

I used this source :
https://developer.toradex.com/torizon/application-development/working-with-containers/how-to-autorun-an-application-with-torizoncore/

Thank you for the informations about Torizon.io and the way It pull the new containers. I get It now.

Best regards

I see thank you for clarifying. At the moment from what I understand our VSCode extension probably won’t produce the docker-compose files you want in the format you want for your use-case.

As an alternative you can always manually create the compose files in the form you want then upload these to our platform. Could probably script the interaction once you finalize on exactly what you want for how your compose files should be.

Best Regards,
Jeremias

1 Like