Docker-compose.yml

leighjboyd · August 16, 2022, 2:37pm

I have been reading

which states that:

the containers described by a docker-compose.yml file present in /var/sota/storage/docker-compose/ directory of your TorizonCore system.

But I dont see this folder, neither on the regular system nor inside the container.

In the container i see only (no “sota”):

root@colibri-imx6-10934586:/var# ls
backups  cache  lib  local  lock  log  mail  opt  run  spool  tmp
root@colibri-imx6-10934586:/var#

and in the host:

colibri-imx6-10934586:/var$ ls
cache  lib  lock  log  rootdirs  run  spool  tmp  usrlocal  volatile
colibri-imx6-10934586:/var$

My goal was to validate that the docker-compose file is working before building it into the image.

when I do a systemctl status docker-compose I get the following:

 docker-compose.service - Docker Compose service with docker compose
     Loaded: loaded (/usr/lib/systemd/system/docker-compose.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
  Condition: start condition failed at Mon 2022-08-15 21:38:50 UTC; 17h ago
             └─ ConditionPathExists=/var/sota/storage/docker-compose/docker-compose.yml was not met

also, nothing in the logs:

colibri-imx6-10934586:/$ journalctl -f -u docker-compose
-- Logs begin at Tue 2022-08-16 14:36:43 UTC. --

Do I need to build the image with a docker compose file first, in order that the directory gets created, then modify it as needed?

Leigh

jeremias.tx · August 16, 2022, 5:52pm

Greetings @leighjboyd,

That file path only exists if you are using a TorizonCore image that already has a docker-compose.yml built into that location. For example if you flash one of our images that has “evaluation containers” this path should be defined.

For creating a TorizonCore image with your docker-compose.yml file you can use the TorizonCore Builder tool. See this article for more info on this process: Pre-provisioning Docker Containers onto a TorizonCore image | Toradex Developer Center

Best Regards,
Jeremias

leighjboyd · August 16, 2022, 6:23pm

I forget why, but a while back I needed to discontinue using the evaulation containers according to one of the things I was trying to install…

in fact, even in the article you referenced,
Pre-provisioning Docker Containers onto a TorizonCore image | Toradex Developer Center
the section “prerequisites” says I should have an image “without evaluation containers”.

How should I add the path if I have no evaluation containers? is this something I can achieve with a docker COPY command perhaps, or some other docker CMD? from VSCODE and then export docker-compose file?

jeremias.tx · August 16, 2022, 10:57pm

As I said you can create a custom TorizonCore image with your docker-compose.yml file using the article I linked. Otherwise just manually add the file on that path in your running system (outside of a container, on the host system). This has nothing to do with Docker or VSCode, it’s just writing a file to a particular file-path.

Best Regards,
Jeremias

leighjboyd · August 24, 2022, 7:46pm

I see what you are saying now,

But if I need to re-install the OS, it would be better in the long run to do it with docker-compose.yml right?

And if the VSCODE has a command which says “export docker-compose” then I assume I should not be modifying the docker-compose by hand, right?

So then I am thinking the prefered approach is to put it in “target commands” or something, like this:

preinstallcommands CMD mkdir /var/sota/storage/docker-compose/

Then do an “export docker-compose”…
but maybe I am trying to run before I can crawl?

Leigh

jeremias.tx · August 24, 2022, 9:59pm

You still seemed to be confused about this.

But if I need to re-install the OS, it would be better in the long run to do it with docker-compose.yml right?

I’m not sure what you mean by this.

And if the VSCODE has a command which says “export docker-compose” then I assume I should not be modifying the docker-compose by hand, right?

When you run that export command the extension produces a docker-compose file that when executed, runs your containers in a fashion similar to how the extension executes them. You are welcome to modify this file after it’s produced as long as you are aware of what you’re modifying and how it affects things.

So then I am thinking the prefered approach is to put it in “target commands” or something, like this:
preinstallcommands CMD mkdir /var/sota/storage/docker-compose/

This is incorrect. As I said previously:

This has nothing to do with Docker or VSCode, it’s just writing a file to a particular file-path.

The preinstallcommands run commands inside the container. You want to install your docker-compose file outside the container on the host OS. The host OS and the container are separate filesystems. All this would do is create the path /var/sota/storage/docker-compose/, inside the container filesystem. Which does nothing for you because we’re talking about the host OS filesystem.

The recommended workflow is:

Develop in VSCode, debug, deploy, test, etc.
When you are ready to deploy your application. Export your compose file, push your container images to whatever registry
Follow this article: Pre-provisioning Docker Containers onto a Torizon OS image | Toradex Developer Center
This will produce a TorizonCore image with your application pre-bundled. This image can then be flashed and then you will have an image that will auto-run your application on boot.

Best Regards,
Jeremias

leighjboyd · September 1, 2022, 3:08pm

Ok I am starting to understand (i hope) what you are saying about the /var/sota/storage/docker-compose , and as usual understanding things better means there are even more follow-up questions…

I understand there are two steps: first, a sort of “sanity check” step to make sure the docker-compose.yml file even works, and second the more permanent “bundle the containers into the image” approach.

For step 1, (just trying to make sure the docker-compose.yml file is correct), I just:

mkdir /var/sota/storage/docker-compose
copy the “F1 -Torizon export docker-compose” output to a file there called docker-compose.yml
When I reboot, the systemctl docker-compose.service will run this file and start my application.

But later, for step 2, when I see this is all working, I need to pre-provision the containers into the Easy Installer image. I have a ton of questions about this step, but I see I have more reading to do before I even bother you about it

So, I am still on Step 1.

So far I have managed to:

create the /var/sota/storage/docker-compose/ folder
F1 export docker-compose file and copy it to this folder

# docker-compose.yml
services:
  datahandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c:
    depends_on: []
    devices: []
    image: datahandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c
    ports:
    - 8081:8080/tcp
    - 8883:8883/tcp
    volumes:
    - /dev:/dev:rw
    - /home/torizon/logs:/home/torizon/logs:rw
    - /tmp:/tmp:rw
version: '2.4'

reboot
check ‘systemctl status docker-compose’, which reports:

docker-compose.service - Docker Compose service with docker compose
     Loaded: loaded (/usr/lib/systemd/system/docker-compose.service; enabled; vendor preset: enabled)
     Active: active (exited) since Thu 2022-09-01 13:52:03 UTC; 7min ago
    Process: 850 ExecStart=/usr/bin/docker-compose -p torizon up -d --remove-orphans (code=exited, status=0/SUCCESS)
    Process: 851 ExecStartPost=/usr/bin/rm -f /tmp/recovery-attempt.txt (code=exited, status=0/SUCCESS)
   Main PID: 850 (code=exited, status=0/SUCCESS)

Sep 01 13:52:03 colibri-imx6-10866293 systemd[1]: Starting Docker Compose service with docker compose...
Sep 01 13:52:03 colibri-imx6-10866293 systemd[1]: Started Docker Compose service with docker compose.
Sep 01 13:52:09 colibri-imx6-10866293 docker-compose[850]: Creating network "torizon_default" with the default driver
Sep 01 13:52:11 colibri-imx6-10866293 docker-compose[850]: Creating torizon_devicehandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c_1 ...
Sep 01 13:52:14 colibri-imx6-10866293 docker-compose[850]: Creating torizon_devicehandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c_1 ... done
~

which seems to look okay,

I can see that my image is present

colibri-imx6-10866293:~/logs$ docker images
REPOSITORY                                                                                        TAG                 IMAGE ID            CREATED             SIZE
emerablockenergybox_arm32v7-debian-no-ssh_bullseye_release_69ac5093-51e4-4f03-8827-ec9f9390dfe3   latest              56a12ba56d48        About an hour ago   95.3MB
datahandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c          latest              268df50dc558        2 hours ago         118MB
devicehandler_arm32v7-debian-python3_bullseye_debug_fb2145c3-20ed-43f9-b41d-1657ebce567c          latest              db7e1b6a6941        7 days ago          149MB
devicehandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c        latest              b237b6a3160e        7 days ago          120MB
leighjboyd/emera1                                                                                 latest              e90dfb901f0b        2 weeks ago         97.5MB
emerablockenergybox_arm32v7-debian-no-ssh_bullseye_debug_69ac5093-51e4-4f03-8827-ec9f9390dfe3     latest              33f3d7f957f9        3 weeks ago         99.2MB
torizon/weston                                                                                    2                   829f5eb1882b        3 months ago        309MB

but when I check ‘docker ps’, nothing is running.

But if I export the docker run command:

docker run --volume /dev:/dev --volume /home/torizon/logs:/home/torizon/logs --volume /tmp:/tmp --publish 8081:8080/tcp --publish 8883:8883/tcp datahandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c

The application runs fine.

Any ideas?

jeremias.tx · September 1, 2022, 8:04pm

Based on the logs from the docker-compose service it sounds like a container was successfully started. The only reason I can think of for nothing to be seen in docker ps, is that the container started and then shortly after it stopped/exited for some reason.

Seeing as your docker run command here worked. My only assumption is that for some reason, the docker-compose file doesn’t have the same behavior as the docker run command.

Let’s try this, run docker-compose -p torizon up --remove-orphans on your docker-compose file and see what happens. This command is more or less what the docker-compose service does. See if the container stays running or exits shortly after for some reason.

Best Regards,
Jeremias

leighjboyd · September 2, 2022, 2:10pm

Hi again,

Let’s try this, run docker-compose -p torizon up --remove-orphans on your docker-compose file and see what happens.

Not sure where to type this. do you mean like this?

# docker-compose.yml
services:
  devicehandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c:
    depends_on: []
    devices: []
    image: devicehandler_arm32v7-debian-python3_bullseye_release_fb2145c3-20ed-43f9-b41d-1657ebce567c
    ports:
    - 8081:8080/tcp
    - 8883:8883/tcp
    volumes:
    - /dev:/dev:rw
    - /home/torizon/logs:/home/torizon/logs:rw
    - /tmp:/tmp:rw
version: '2.4'

run docker-compose -p torizon up --remove-orphans

or did you mean I should run it from the command line? Please forgive my n00biness…

Oh, I see now. You mean I should type this on the command line:
$ docker-compose -p torizon up --remove-orphans

gotchya… seems like I had some mistakes in the docker-compose. For example the name of the image did not match, AND there was another process running based on the same image etc…

jeremias.tx · September 2, 2022, 6:34pm

I mean to run the command on the command line targeting your docker-compose file. Then compare the behavior to when you use docker run.