Torizoncore-builder support for profiles in docker-compose.yml

Docker Compose supports profiles that can be used for various uses and environments by selectively enabling services (see here).
I need to build torizoncore with some containers that should not be started automatically, and so I thought to profiles.
I tried adding profiles to docker-compose.yml but torizoncore-builder returns error

Creating Docker Container bundle...
Error: Could not load the Docker compose file 'docker-compose.yml'.

Are profiles supported in torizoncore-builder?
If not, which is the way to build containers that are not run at the but (they can be run manually when needed by the applicaion)?

I’m interested in this too as I have a similar use case. Compose on the SoM supports the syntax but my suspicion is that the compose version that is part of torizoncore-builder does not currently.

Greetings @vix,

This seems to be the same type of issue as I helped you with in this thread: Torizoncore-builder maximum version for docker-compose.yml

Basically, TorizonCore Builder uses a python library based on docker-compose version 1.26.2 to parse and handle compose files. I checked the version history of docker-compose and I can see that support for service profiles was added with compose version 1.28.0. Therefore the version of compose in TorizonCore Builder does not know about this syntax/flag.

One work-around would be to rebuild TorizonCore Builder using a newer version of the compose Python library. The source for TorizonCore Builder can be found here: GitHub - toradex/torizoncore-builder: TorizonCore Builder is a tool that allows the customization of TorizonCore images.

Our team is currently aware of our tool using an older version of compose and it’s still under evaluation whether we upgrade this.

Best Regards,

Thanks @jeremias.tx
I hope the team would upgrade this.

As I said the team is still discussing this, if you need this urgently then you can always try the work-around I suggested.

Best Regards,

An intermediate workaround we’re experimenting with that would likely give similar functionality on older versions is some “dummy” services in your compose with the correct dependencies on the services you want to bring up
Then, instead of docker-compose --profile [name] [args] up you just do something like docker-compose [args] up [dummy-service]

Thank you for sharing the alternative workaround.

@jeremias.tx There’s a slight flaw in this plan which is the hard-coded “test” command in the aktualizr-torizon binary. While the docker-compose.service is easily modified to start a specific chain of services, this particular item is not. Would it be possible to make this (aktualizr/ at c5293cf0bc68e52feaa9e49706284556fe91949d · toradex/aktualizr · GitHub) be read from a configuration file instead? Then the necessary changes to support this (or future profiles) can be made without the need to re-compile any system binaries and only necessitates editing a .service file and the proposed configuration file.

Hi @bw908,

Just to make sure I understand your request instead of running a hard-coded docker-compose up -p torizon up --detach --remove-orphans. You want the ability to customize the exact docker-compose up... command that gets ran here, correct?

On initial impression this could be possible, but of course it will need further discussion amongst our development team. I can bring the topic up with them, of course no guarantees on if or when this will be even worked on.

Best Regards,

Yes, that’s correct - the end goal here is to support starting a specific set of services either with the workaround above or in the future to facilitate --profile usage.

For context, our use case for this is we have a software stack that will be almost identical between two slightly different hardware designs (which are fully compatible at the devicetree level) and we’d like to run a slightly different set of services on each, without needing to maintain two separate sets of OS builds and updates

Ok I see, and that definitely makes sense for your particular use-case. In general customizing the exact docker-compose command that gets executed would add some flexibility as well.

in the future to facilitate --profile usage.

Also just to clarify it’s only TorizonCore Builder that has a problem with the --profile flag since the version of docker-compose used here is old. The version of docker-compose on TorizonCore itself is newer and should actually support this --profile flag I believe. Of course there’s still the limiting factor of Aktualizr using a hard-coded command but just wanted to clarify this aspect in case it wasn’t clear.

Best Regards,

Yes, though of course if no --profile is provided the default configuration(s) on the hardware would still attempt to start all services and could cause some of them to report as unhealthy if they were not meant to be started on the hardware. As you note it’s hard-coded in aktualizr, but can be easily modified in /usr/lib/systemd/system/docker-compose.service.

Potentially an /etc/ file containing the profile(s) to start could be read by both of these tools would be a viable option too once they are supported by torizoncore-builder. (and means one would not need to “override” stock OS files in /usr/lib/systemd/ themselves)

We’ve since found that a better workaround for both sticking points (aktualizr and docker-compose.service) is simply a modified /usr/bin/docker-compose python script which can inject a specific service name to bring up as part of the compose up command.

Hopefully this is useful for others looking for an intermediary solution.

Thank you for sharing your workaround for the time-being.

1 Like

Dear @vix,

Thanks for your patience. This issue has been fixed on an early access version of TorizonCore Builder. You may download it here: GitHub - toradex/tcb-env-setup: TorizonCore Builder Environment Setup Script
I will update you when the official version with the bug fix is released.