How many docker-compose files do I need?

Technical Support
, ,
1

I am a little confused about the role of docker-compose.yml in getting my easy-install project up and running. I have several different docker-compose files and I seem to be breaking the don’t repeat yourself principle.

So far I have a docker-compose.yml in my python application folder which I use for debugging. This is the docker-compose that gets copied to my platform when I use the debug button in VSCode. I also created a second “production” docker-compose.yml, which does not contain the debug image, and which I intend to put in /var/sota/storage/docker-compose/ for auto-running the application. I also use it to install the release containers on my Ivy board when testing them. Now I am reading through the Torizon Core Builder docs and it looks like I need a third docker compose for building these same images into an Easy Installer – this third one lives in my torizoncore builder workspace. I have tried referencing the other docker-compose.yml files from the tcbuild “bundle” utility, but it refuses to find them, and says they do not exist (they do exist):

torizoncore-builder bundle "~/repos/chipotle_dbpt/docker-compose.yml" --bundle-directory bundle
Creating Docker Container bundle...
Error: File does not exist: ~/repos/chipotle_dbpt/docker-compose.yml. Aborting.

So, I have three docker-compose.yml files, which are all basically identical to each other:
docker-compose.yml for debugging
docker-compose.yml for auto-running
docker-compose.yml for torizoncore bundling

All three files just define how to build my application (the app consists of two custom containers/services and nothing else).
Is this the way I am expected to be doing things? It seems wrong. I think I should just have one docker-compose that defines my application, right? Is there a different way to be including my application containers in my OS using the docker-compose that is already part of my Torizon python project?

2

Hi, I’m the Toradex AI Assistant.

I might make mistakes. Our support team will step in if needed. If I answered your question, please tick the “Solution” checkbox. For follow-up questions to me, please use @ToradexAI in your post.

Did you like my answer? :+1: :-1:

Hello! Thanks for your detailed question.

Your confusion is quite common when working with containerized applications across development, testing, and deployment stages, especially using TorizonCore and its tooling. Let’s clarify the roles of the different docker-compose.yml files and how you can streamline your workflow:

Why So Many Compose Files?

Each compose file exists for different operational needs:

  1. Debug Compose file

    • Used by the Torizon IDE Extension for development/debugging.
    • May have features only needed in debug (e.g., mounting source folders, extra ports, debug-only containers).

  2. Production Compose file

    • Intended for deployment and production environments.
    • Minimal setup: excludes debug containers and mounts.
    • Often copy to /var/sota/storage/docker-compose/ to auto-start containers with Torizon, or use for initial manual bring-up/testing.

  3. TorizonCore Builder Compose file (“Bundling”)

    • Used to generate a ready-to-use OS image with pre-provisioned containers (as required for Toradex Easy Installer images).
    • Used by the torizoncore-builder bundle command.
    • Should describe the “production” version of your stack, as this is used to build the static OS bundle.

Reducing Duplication

You can absolutely reuse the same compose file for production and TorizonCore Builder. The debug version is usually different due to VS Code/IDE requirements, but the “production” and “bundle” compose file can (and should) be the same, or at least generated from a shared base.

Tips to Avoid Duplication:

  • Keep a single production docker-compose.yml and use it both for:
    • Direct deployment to /var/sota/storage/docker-compose/
    • TorizonCore Builder’s bundle command (by providing its absolute path)
  • Only maintain a separate debug compose file for IDE-specific features.

Path Issue with tcbuild

Note: The tilde (~) shortcut for a user’s home directory is NOT expanded inside the torizoncore-builder tool. Always provide the full absolute path, like this:

torizoncore-builder bundle "/home/youruser/repos/chipotle_dbpt/docker-compose.yml" --bundle-directory bundle

See documentation for reference:

Typical Development Setup

  • Debugging:
    Use .dev/docker-compose.debug.yml for IDE extension (development only).

  • Production / Bundling:
    Use one docker-compose.yml (e.g., in your project root).

    • Deploy it to /var/sota/storage/docker-compose/ for live testing,
    • Give its absolute path to TorizonCore Builder for image bundling (pre-provisioning).

More details:

In summary:
It’s both expected and best practice to have just:

  • 1 compose file for debugging
  • 1 compose file for both production deployment and TorizonCore image bundling

This way, you minimize duplication and maintenance effort.

Would you like specific guidance on consolidating your files, or have further questions about your setup?

Best regards,
Toradex AI Assistant.

3

When specifying the absolute path to my compose file I get this exception in torizoncore builder:

jzylkin@DZYNE-2J3VZ94:~/tcbdir$ torizoncore-builder bundle "/home/jzylkin/repos/chipotle_dbpt/docker-compose.yml" --bund
le-directory bundle
Creating Docker Container bundle...
An unexpected Exception occurred. Please provide the following stack trace to
the Toradex TorizonCore support team:


Traceback (most recent call last):
  File "/builder/torizoncore-builder", line 231, in <module>
    mainargs.func(mainargs)
  File "/builder/tcbuilder/cli/bundle.py", line 132, in do_bundle
    bundle(bundle_dir=args.bundle_directory,
  File "/builder/tcbuilder/cli/bundle.py", line 83, in bundle
    bundle_be.download_containers_by_compose_file(
  File "/builder/tcbuilder/backend/bundle.py", line 601, in download_containers_by_compose_file
    if not os.path.exists(compose_path):
UnboundLocalError: local variable 'compose_path' referenced before assignment
4

Also, if builder only supports absolute paths, it does not seem very practical. I work on a team, and I am the only one on my team with a /home/jzylkin/repos/docker-compose.yml file.

How do I work around your lack of support for relative paths?

5

You just need one compose file for production purposes.

TorizonCore Builder takes your compose file creates a custom Torizon OS image that installs it in /var/sota/storage/docker-compose/. The resulting image can then be used for production programming. See the following article for more details: Pre-provisioning Docker Containers onto a Torizon OS image | Toradex Developer Center

When specifying the absolute path to my compose file I get this exception in torizoncore builder:

TorizonCore Builder runs in a container, it can’t see the full absolute path on your PC. Therefore it can only see relative paths from the reference point of the directory you are executing it from.

Also, if builder only supports absolute paths, it does not seem very practical. I work on a team, and I am the only one on my team with a /home/jzylkin/repos/docker-compose.yml file.
How do I work around your lack of support for relative paths?

I think you have it backwards, the tool DOES NOT support absolute paths, it supports relative paths.

Best Regards,
Jeremias