Try to make a customized TorizonCore with Pre-provisioned Docker-Images

Technical Support
, , , ,
1
  1. As a first step I build the Docker-Image direct on the target. → Success
  2. As a second step customize TorizonCore with the Builder to include above made Docker-Images. → Build failed

Now step 1 in detail:
Install unchanges Image “docker-apalis-imx8-Tezi_5.7.2+build.20.tar” on apalis-imx8.
Start apalis and open remote-terminal on it.
Build Image according Docker Guideline:

Quickstart: Compose and Django according following link:
(awesome-compose/README.md at master · docker/awesome-compose · GitHub)

Create an empty project directory.
Create a new file called Dockerfile in your project directory and add suggested text.
Create a requirements.txt in your project directory and add suggested text.
Create a file called docker-compose.yml in your project directory and add suggested text.
Create the Django project by running the docker compose run command as suggested.

→ The image is built with success.

Now step 2 in detail:
Open terminal on Ubuntu 18.04.
Install TorizonCore Builder according following link:
(TorizonCore Builder Tool - Customizing TorizonCore Images | Toradex Developer Center)
$ mkdir -p ~/tcbdir/ && cd ~/tcbdir/
$ wget https://raw.githubusercontent.com/toradex/tcb-env-setup/master/tcb-env-setup.sh
$ cd ~/tcbdir/
$ source tcb-env-setup.sh
$ torizoncore-builder build --create-template

add my project files (same as above):
/baseImage/docker-apalis-imx8-Tezi_5.7.2+build.20.tar
/myProject/docker-compose.yml
/myProject/requirements.txt
/myProject/Dockerfile

edit tcbuild.yaml:

input:
  easy-installer:
    local: baseImage/torizon-core-docker-apalis-imx8-Tezi_5.7.2+build.20.tar
output:
  easy-installer:
    local: hums-torizon-core-docker-Tezi_0.0.1.CUSTOM
    bundle:
      platform: linux/arm/v7
      dir: bundle/
      compose-file: myProject/docker-compose.yml

execute TorizonCore Builder with:
$ torizoncore-builder --verbose bundle myProject/docker-compose.yml

Build Output:
2023-04-17 08:33:18,490 - docker.utils.config - DEBUG - Trying paths: [‘/root/.docker/config.json’, ‘/root/.dockercfg’]
2023-04-17 08:33:18,490 - docker.utils.config - DEBUG - No config file found
2023-04-17 08:33:18,490 - docker.utils.config - DEBUG - Trying paths: [‘/root/.docker/config.json’, ‘/root/.dockercfg’]
2023-04-17 08:33:18,490 - docker.utils.config - DEBUG - No config file found
2023-04-17 08:33:18,494 - urllib3.connectionpool - DEBUG - http://localhost:None “GET /version HTTP/1.1” 200 786
2023-04-17 08:33:18,496 - urllib3.connectionpool - DEBUG - http://localhost:None “GET /v1.41/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22name%22%3A+%5B%22tcb_1681720397%22%5D%7D HTTP/1.1” 200 1653
2023-04-17 08:33:18,497 - urllib3.connectionpool - DEBUG - http://localhost:None “GET /v1.41/containers/f4d0d7d4da0f395d6c8a03404a16a3cad05b2a2178ab7fddbb822a0a61b39706/json HTTP/1.1” 200 None
2023-04-17 08:33:18,498 - torizon.tcbuilder.backend.common - DEBUG - Current container ID (found by container name): f4d0d7d4da0f395d6c8a03404a16a3cad05b2a2178ab7fddbb822a0a61b39706
2023-04-17 08:33:18,499 - urllib3.connectionpool - DEBUG - http://localhost:None “GET /v1.41/containers/f4d0d7d4da0f395d6c8a03404a16a3cad05b2a2178ab7fddbb822a0a61b39706/json HTTP/1.1” 200 None
2023-04-17 08:33:18,499 - torizon.tcbuilder.cli.bundle - INFO - Creating Docker Container bundle…
2023-04-17 08:33:18,499 - compose.config.config - DEBUG - Using configuration files: /workdir/myProject/docker-compose.yml
2023-04-17 08:33:18,508 - root - ERROR - Error: Could not load the Docker compose file ‘myProject/docker-compose.yml’.
2023-04-17 08:33:18,509 - root - DEBUG - Traceback (most recent call last):
File “/builder/tcbuilder/backend/bundle.py”, line 535, in download_containers_by_compose_file
config = compose.config.load(details)
File “/usr/local/lib/python3.9/dist-packages/compose/config/config.py”, line 392, in load
processed_files = [
File “/usr/local/lib/python3.9/dist-packages/compose/config/config.py”, line 393, in
process_config_file(config_file, config_details.environment, interpolate=interpolate)
File “/usr/local/lib/python3.9/dist-packages/compose/config/config.py”, line 581, in process_config_file
validate_against_config_schema(config_file)
File “/usr/local/lib/python3.9/dist-packages/compose/config/validation.py”, line 449, in validate_against_config_schema
handle_errors(
File “/usr/local/lib/python3.9/dist-packages/compose/config/validation.py”, line 514, in handle_errors
raise ConfigurationError(
compose.config.errors.ConfigurationError: The Compose file ‘/workdir/myProject/docker-compose.yml’ is invalid because:
Unsupported config option for services: ‘db’

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/builder/torizoncore-builder”, line 221, in
mainargs.func(mainargs)
File “/builder/tcbuilder/cli/bundle.py”, line 97, in do_bundle
bundle(bundle_dir=args.bundle_directory,
File “/builder/tcbuilder/cli/bundle.py”, line 49, in bundle
bundle_be.download_containers_by_compose_file(
File “/builder/tcbuilder/backend/bundle.py”, line 538, in download_containers_by_compose_file
raise OperationFailureError(
tcbuilder.errors.OperationFailureError: Error: Could not load the Docker compose file ‘myProject/docker-compose.yml’.

2

Greetings @ah4711,

What is the contents of your compose file? According to the stack trace, the file is invalid because of:

Unsupported config option for services: ‘db’

It seems like a format or syntax issue in the file, but it’s hard to say without seeing the full file.

Best Regards,
Jeremias

4

Hi Jeremias

You are right, this was also my first thought, but that is not the problem.

If you execute the test step by step according the guideline, you will get the same result.

I will also send you all my files (except your base image), but they are absolutely 1 by 1 copies of the guideline.

Regards

ah4711

myProjectFiles.tar.xz (5.94 KB)

5

Ok I see the issue now, your compose file does not have a version declaration at the top like version: "2" or version: 3. This is apparently needed to parse your compose file properly. Once I added this I no longer got the error anymore.

However, I do see another issue your web service makes use the the build property. This won’t work for our bundle command. The bundle command does not build anything it just fetches the container images specified by your compose file. Therefore this container image needs to be built and pushed to some registry prior to running the bundle command.

Best Regards,
Jeremias

6

Hi Jeremias

Yes I could reproduce your advice.

I see here a problem for us. We don’t want to pull/push to any Docker Hub. Even the “Local Registry Push and Pull” we would like to avoid.

Is it possible to work with the “Docker Save and Load” method in combination with the “TorizonCore Builder” to make my customized TorizonCore with Pre-provisioned Docker-Images?

Regards

ah4711

7

I see here a problem for us. We don’t want to pull/push to any Docker Hub. Even the “Local Registry Push and Pull” we would like to avoid.

Can I ask why you want to avoid this?

Is it possible to work with the “Docker Save and Load” method in combination with the “TorizonCore Builder” to make my customized TorizonCore with Pre-provisioned Docker-Images?

You can manually docker save/load yourself but as I said that’s not how TorizonCore Builder works. TorizonCore Builder expects the container images that it needs to bundle can be found in some container registry.

Now what exactly are you trying to do? Is this just for development, are you preparing these images for production? If it’s just for development why not just transfer the container images to the device?

Best Regards,
Jeremias

8

Why not any Docker Hub?

We don’t store any data outside of our own servers.

Why not “Local Registry” on our own server?

I think it is too much additional equipment and IT-Support only for a intermediate step without any really necessary functionality.

Yes we are preparing these images for production. We intend to install with “Toradex Easy Installer”. Full updates will be very seldom.

Regards

ah4711

12

I think it is too much additional equipment and IT-Support only for a intermediate step without any really necessary functionality.

I think there’s a misconception here. It really requires no special equipment or setup to create a simple local registry. If you know what to do it takes maybe a minute at most. As an example here’s what many of our other customers do:

# This command starts a container that acts as a simple registry on your machine
$ docker run -d -p 5000:5000 --restart=always --name registry registry:2
# Now you can tag and push your image to this regisry like so
$ docker push localhost:5000/my-image
# Now with a compose file like this
$ cat docker-compose.yml 
version: '2.4'
 
services:
  test: 
    image: localhost:5000/my-image
# You can run the bundle command targeting this local registry
$ torizoncore-builder bundle --dind-param="--insecure-registry=localhost:5000" docker-compose.yml 
Creating Docker Container bundle...

Starting DIND container
Using Docker host "tcp://127.0.0.1:22376"
Connecting to Docker Daemon at "tcp://127.0.0.1:22376"
Fetching container image localhost:5000/my-image
  Pulling from my-image
  2db29710123e: Pull complete 
  Digest: sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4
  Status: Downloaded newer image for localhost:5000/my-image:latest
Saving Docker Compose file
Exporting storage
Storing container bundle into "bundle"
Stopping DIND container
Successfully created Docker Container bundle in "bundle"!

It’s as simple as that. Most of the instructions regarding the local registry were adapted from the official Docker documentation here: Registry | Docker Docs

Best Regards,
Jeremias

13

Hello @ah4711,

Do you have any updates on this topic?

14

Not yet, but we will install such a registry server

15

@ah4711,

Is there anything we can do to help you?
do you have any updates?

Best Regards,

Matthias

16

Hi Matthias

Unfortunately I had not yet time to try it. I hope to try it next month.

Regards

Ah4711

~WRD3381.jpg

17

Hi Matthias

Now I was able to build a simple customized Pre-provisioned Docker-Images with a local Registry server.

Regards

Ah4711

~WRD3381.jpg

1 Like
18

Hi @ah4711,

Glad to hear that! Please feel free to open a new thread in case you come across other issues where you might need our support.