Torizoncore-builder maximum version for docker-compose.yml

Technical Support
, , ,
1

Torizoncore Builder Tool can be used to customize a TorizonCore image, as described here.
A docker-compose.yml file can be used.
The specifications for this compose file can be found here and it seems that the top-level element names version has been deprecated.
The problem is that it seems that torizoncore-builder expects some specific value for version element in docker-compose.yml. Is this 2.4?

I tried with newer docker-compose.yml (with version: 3.3), and removing the version element, but in both of these cases I get an error
Error: Could not load the Docker compose file 'custom_erxx/docker-compose-erxx.yml'.
without any specific reason.
This is misleading becuase it’s the same message shown when the docker-compose.yml has ome syntax errors.
But syntax errors can be asily verified using
docker-compose -f docker-compose.yml config
and this is not the case (if version: 3.3 or version doesn’t exist).
Is there some specific requirement from torizoncore-builder about version element?
Can this be clarified in the documentation, please?

2

Greetings @vix,

I’m still investigating and reproducing this issue. But on initial look it seems this error is coming from the 3rd party Python library we use to process docker-compose files. So the error isn’t coming from code that we wrote so it’s not immediately obvious why this error is occurring.

But there does seem to be some kind of limitation or strange behavior here with the version field. As I said I’ll investigate further, but let me know if you find anything else related to this.

As a side-note I tested with version: '3.3' and this seems to work for me all the way up to 3.8 (the latest compose version at this time) using a non-existing version like 4.0 results in a similar error. Maybe you didn’t enter the version field right for 3.3?

Best Regards,
Jeremias

3

After some testing It seems like on my Ubuntu laptop I need this version field for my compose file to be considered “valid” by the docker-compose running on my machine.

I did some research and I found this:

The Compose file is a YAML file defining services, networks, and volumes for a Docker application. The latest and recommended version of the Compose file format is defined by the Compose Specification. The Compose spec merges the legacy 2.x and 3.x versions, aggregating properties across these formats and is implemented by Compose 1.27.0+.

It seems like while yes the version field is deprecated this is only valid for the latest version of Compose (1.27.0+) as stated in the quote above. On my Ubuntu machine my compose version is only 1.25.0. Which is why version: is required for me. Furthermore we have various versions of docker-compose on various versions of TorizonCore, so for now it seems to be safe to just require this version: field. Otherwise we might run into issues on older versions.

Best regards,
Jeremias

4

Hello @jeremias.tx
thanks for your answers.

I confirm that using version: '3.3' gives an error when I launch
torizoncore-builder bundle ....

If the folder with the bundled containers already exists and is up to date, and I lunch
torizoncore-builder build
it works as expected.

I don’t know if it can be related, but I use TorizoCore 6.x.y

5

I confirm that using version: '3.3' gives an error when I launch
torizoncore-builder bundle ....

Hmm that sounds strange to me, could you share your full docker-compose file so I can give this a try?

I don’t know if it can be related, but I use TorizoCore 6.x.y

This shouldn’t matter, the bundle command by itself does not rely on the version of TorizonCore you are using.

Best Regards,
Jeremias

6

Here you can find my docker-compose.yml and tcbuild.yaml files attached.
My usage requires two local folders:

  • tezi-images/ with the “base” torizon-core-docker-verdin-imx8mm-Tezi_6.1.0-devel-202212+build.4.tar input image downloaded manually
  • bundled_containers/ which is populated by torizoncore-builder bundle ... command

docker-compose.yml (1.3 KB)
tcbuild.yaml (4.2 KB)

7

With your compose file I was able to finally get to the bottom of this.

First of all I tried using the bundle command. I got the same results as you with this. When using version: '2.4' the command works fine but if I change it to version: '3.3' then it fails like you said.

I then tried validating the compose file with docker-compose config. Now here is where I get different results. On my laptop when I run docker-compose config with version: '3.3' I get the following output:

ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.weston: 'device_cgroup_rules'
Unsupported config option for services.kiosk: 'device_cgroup_rules'
Unsupported config option for services.nextjs-verdin: 'platform'
services.kiosk.depends_on contains an invalid type, it should be an array

With version: '2.4', the file validates fine. Now earlier you stated:

But syntax errors can be asily verified using
docker-compose -f docker-compose.yml config
and this is not the case (if version: 3.3 or version doesn’t exist).

So I’m getting different behavior than you did here. I did some research to figure out why and here’s what I came up with. I found this issue on Github: device_cgroup_rules support removed in 3.x · Issue #8251 · docker/compose · GitHub

Similar validation issues as I got with your compose file. In that thread the issue was resolved by using a later version of docker-compose. On my laptop I have this version of docker-compose:

docker-compose version 1.25.0, build unknown
docker-py version: 4.1.0
CPython version: 3.8.10
OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020

In the TorizonCore Builder container we use a Python package for docker-compose functionality with version “1.26.2”. As seen in the official repo here: https://github.com/toradex/torizoncore-builder/blob/bullseye/requirements_debian.txt

I re-built TorizonCore Builder using the latest version of the docker-compose python package (1.29.2), and this seems to fix things. The bundle command now works with version: 3.3 or without any version specified at all. I guess the older versions of docker-compose have some weird bugs/quirks with them that were fixed in later versions.

I’ll make a request to our team to update the version of docker-compose used for TorizonCore Builder. In the meantime you can manually do this yourself as I did if you need this right away.

Best Regards,
Jeremias

1 Like
8

Thank you @jeremias.tx for your great investigation.
I confirm that on my Ubuntu 22.04 I have

docker-compose version 1.29.2, build unknown

and this explains why I don’t see any validation issue.

Please, update the topic here when a new release of TorizonCore-Builder is released.

9

The request was submitted to our team to bump the version used for TorizonCore-Builder, but I don’t have any timelines at the moment for when this will be done. I’ll try to keep you informed of any updates here.

11

Dear @vix,

Thanks for your patience. This issue has been fixed on an early access version of TorizonCore Builder. You may download it here: GitHub - toradex/tcb-env-setup: TorizonCore Builder Environment Setup Script
I will update you when the official version with the bug fix is released.