TorizonCore Builder on Azure VM results in ConnectionRefusedError

[Casper]

When using the torizoncore builder on my Azure VM I have an odd issue, most of the time it works fine, but sometimes I run into an ConnectionRefusedError when using the bundle command. It is hard to determine exactly what happens as it doesn’t happen very often. It is however an issue for me as the bundle command is part of our build pipeline, and it failing means the whole build fails.
I was able to capture debug logs of the issue occurring but when I ran the exact same command again afterwards it somehow worked.

c.pennings@terberg.com@aztamyocto:~/torizon-core-builder$ source tcb-env-setup.sh 
You may have an outdated version installed. Would you like to check for updates online? [y/n] n
Setting up TorizonCore Builder with version 3.

Setup complete! TorizonCore Builder is now ready to use.
TorizonCore Builder internal status and image customizations will be stored in Docker volume named 'storage'.
********************
Important: When you run TorizonCore Builder, the tool can only access the files inside the current working directory. Files and directories outside of the current working directory, or links to files and directories outside of the current working directory, won't be visible to TorizonCore Builder. So please make sure that, when running TorizonCore Builder, all files and directories passed as parameters are within the current working directory.
Your current working directory is: /home/c.pennings/torizon-core-builder
********************
For more information, run 'torizoncore-builder -h' or go to https://developer.toradex.com/knowledge-base/torizoncore-builder-tool

c.pennings@terberg.com@aztamyocto:~/torizon-core-builder$ torizoncore-builder --log-level DEBUG bundle --platform=linux/arm/v7 docker-compose.yml --bundle-directory bundle --login-to docker
.io casperterberg *****
2024-10-10 10:21:48,398 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:21:48,398 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:21:48,398 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:21:48,398 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:21:48,408 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /version HTTP/11" 200 826
2024-10-10 10:21:48,410 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22name%22%3A+%5B%22tcb_1728555707%22%5D%7D HTTP/11" 200 1741
2024-10-10 10:21:48,411 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/0f02b04e3d1360348b7ebcee9d4e68cbf03147e65ddf202eb32c100d85826445/json HTTP/11" 200 None
2024-10-10 10:21:48,411 - torizon.tcbuilder.backend.common - DEBUG - Current container ID (found by container name): 0f02b04e3d1360348b7ebcee9d4e68cbf03147e65ddf202eb32c100d85826445
2024-10-10 10:21:48,412 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/0f02b04e3d1360348b7ebcee9d4e68cbf03147e65ddf202eb32c100d85826445/json HTTP/11" 200 None
2024-10-10 10:21:48,413 - torizon.tcbuilder.cli.bundle - INFO - Creating Docker Container bundle...
2024-10-10 10:21:48,413 - compose.config.config - DEBUG - Using configuration files: /workdir/docker-compose.yml
2024-10-10 10:21:48,427 - torizon.tcbuilder.backend.bundle - DEBUG - Using DindManager
2024-10-10 10:21:48,427 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:21:48,427 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:21:48,427 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:21:48,427 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:21:48,432 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /version HTTP/11" 200 826
2024-10-10 10:21:48,438 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/info HTTP/11" 200 None
2024-10-10 10:21:48,438 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:21:48,438 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:21:48,438 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:21:48,438 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:21:48,442 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /version HTTP/11" 200 826
2024-10-10 10:21:48,444 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22name%22%3A+%5B%22tcb_1728555707%22%5D%7D HTTP/11" 200 1741
2024-10-10 10:21:48,445 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/0f02b04e3d1360348b7ebcee9d4e68cbf03147e65ddf202eb32c100d85826445/json HTTP/11" 200 None
2024-10-10 10:21:48,445 - torizon.tcbuilder.backend.common - DEBUG - Current container ID (found by container name): 0f02b04e3d1360348b7ebcee9d4e68cbf03147e65ddf202eb32c100d85826445
2024-10-10 10:21:48,446 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/0f02b04e3d1360348b7ebcee9d4e68cbf03147e65ddf202eb32c100d85826445/json HTTP/11" 200 None
2024-10-10 10:21:48,446 - torizon.tcbuilder.backend.bundle - INFO - 
Starting DIND container
2024-10-10 10:21:48,446 - torizon.tcbuilder.backend.bundle - INFO - Using Docker host "tcp://127.0.0.1:22376"
2024-10-10 10:21:48,451 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/volumes/create HTTP/11" 201 182
2024-10-10 10:21:48,451 - torizon.tcbuilder.backend.bundle - DEBUG - Default platform: linux/arm/v7
2024-10-10 10:21:48,451 - torizon.tcbuilder.backend.bundle - DEBUG - Volume mapping for DinD: [{'Target': '/workdir/', 'Source': '/home/c.pennings/torizon-core-builder', 'Type': 'bind', 'ReadOnly': False}, {'Target': '/var/lib/docker/', 'Source': 'dind-volume', 'Type': 'volume', 'ReadOnly': False}]
2024-10-10 10:21:48,451 - torizon.tcbuilder.backend.bundle - DEBUG - Running DinD container: ports=None, network=host
2024-10-10 10:21:48,564 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/create?name=tcb-fetch-dind HTTP/11" 201 88
2024-10-10 10:21:48,565 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/570bc0a78c48632d1ce9afd798226ca8b4d2b376dbd8cec98a505e24ffa4a9ed/json HTTP/11" 200 None
2024-10-10 10:21:48,682 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/570bc0a78c48632d1ce9afd798226ca8b4d2b376dbd8cec98a505e24ffa4a9ed/start HTTP/11" 204 0
2024-10-10 10:21:50,685 - torizon.tcbuilder.backend.bundle - INFO - Connecting to Docker Daemon at "tcp://127.0.0.1:22376"
2024-10-10 10:21:50,685 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:21:50,685 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:21:50,685 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:21:50,685 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:21:50,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): 127.0.0.1:22376
2024-10-10 10:21:50,686 - torizon.tcbuilder.backend.bundle - INFO - Stopping DIND container
2024-10-10 10:21:50,762 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/570bc0a78c48632d1ce9afd798226ca8b4d2b376dbd8cec98a505e24ffa4a9ed/stop HTTP/11" 204 0
2024-10-10 10:21:51,769 - urllib3.connectionpool - DEBUG - http://localhost:None "DELETE /v1.46/volumes/dind-volume HTTP/11" 204 0
2024-10-10 10:21:51,769 - root - CRITICAL - An unexpected Exception occurred. Please provide the following stack trace to
the Toradex TorizonCore support team:


2024-10-10 10:21:51,787 - root - ERROR - Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/urllib3/connection.py", line 196, in _new_conn
    sock = connection.create_connection(
  File "/usr/local/lib/python3.9/dist-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/usr/local/lib/python3.9/dist-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
  File "/usr/local/lib/python3.9/dist-packages/urllib3/connectionpool.py", line 490, in _make_request
    raise new_e
  File "/usr/local/lib/python3.9/dist-packages/urllib3/connectionpool.py", line 466, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.9/dist-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.9/dist-packages/urllib3/connection.py", line 615, in connect
    self.sock = sock = self._new_conn()
  File "/usr/local/lib/python3.9/dist-packages/urllib3/connection.py", line 211, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f9d282c3a60>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/requests/adapters.py", line 589, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.9/dist-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.9/dist-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='127.0.0.1', port=22376): Max retries exceeded with url: /version (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f9d282c3a60>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/docker/api/client.py", line 223, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/usr/local/lib/python3.9/dist-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/usr/local/lib/python3.9/dist-packages/docker/utils/decorators.py", line 44, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/docker/api/client.py", line 246, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.9/dist-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.9/dist-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/requests/adapters.py", line 622, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='127.0.0.1', port=22376): Max retries exceeded with url: /version (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f9d282c3a60>: Failed to establish a new connection: [Errno 111] Connection refused'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/builder/torizoncore-builder", line 222, in <module>
    mainargs.func(mainargs)
  File "/builder/tcbuilder/cli/bundle.py", line 97, in do_bundle
    bundle(bundle_dir=args.bundle_directory,
  File "/builder/tcbuilder/cli/bundle.py", line 49, in bundle
    bundle_be.download_containers_by_compose_file(
  File "/builder/tcbuilder/backend/bundle.py", line 555, in download_containers_by_compose_file
    dind_client = manager.get_client()
  File "/builder/tcbuilder/backend/bundle.py", line 320, in get_client
    dind_client = docker.DockerClient(base_url=self.docker_host, tls=tls_config)
  File "/usr/local/lib/python3.9/dist-packages/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/docker/api/client.py", line 207, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/local/lib/python3.9/dist-packages/docker/api/client.py", line 230, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: HTTPSConnectionPool(host='127.0.0.1', port=22376): Max retries exceeded with url: /version (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f9d282c3a60>: Failed to establish a new connection: [Errno 111] Connection refused'))

And immediately afterwards running the same command again:

c.pennings@terberg.com@aztamyocto:~/torizon-core-builder$ torizoncore-builder --log-level DEBUG bundle --platform=linux/arm/v7 docker-compose.yml --bundle-directory bundle --login-to docker.io casperterberg ***
2024-10-10 10:30:24,470 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:24,470 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:24,470 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:24,470 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:24,476 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /version HTTP/11" 200 826
2024-10-10 10:30:24,477 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22name%22%3A+%5B%22tcb_1728556223%22%5D%7D HTTP/11" 200 1741
2024-10-10 10:30:24,478 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/e062d06acc82769a0393a9cd496bcdc8c22399a76a09079cf18cb3c53de5dd89/json HTTP/11" 200 None
2024-10-10 10:30:24,479 - torizon.tcbuilder.backend.common - DEBUG - Current container ID (found by container name): e062d06acc82769a0393a9cd496bcdc8c22399a76a09079cf18cb3c53de5dd89
2024-10-10 10:30:24,479 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/e062d06acc82769a0393a9cd496bcdc8c22399a76a09079cf18cb3c53de5dd89/json HTTP/11" 200 None
2024-10-10 10:30:24,480 - torizon.tcbuilder.cli.bundle - INFO - Creating Docker Container bundle...
2024-10-10 10:30:24,480 - compose.config.config - DEBUG - Using configuration files: /workdir/docker-compose.yml
2024-10-10 10:30:24,485 - torizon.tcbuilder.backend.bundle - DEBUG - Using DindManager
2024-10-10 10:30:24,485 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:24,486 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:24,486 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:24,486 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:24,491 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /version HTTP/11" 200 826
2024-10-10 10:30:24,496 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/info HTTP/11" 200 None
2024-10-10 10:30:24,496 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:24,496 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:24,496 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:24,496 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:24,501 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /version HTTP/11" 200 826
2024-10-10 10:30:24,502 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/json?limit=-1&all=0&size=0&trunc_cmd=0&filters=%7B%22name%22%3A+%5B%22tcb_1728556223%22%5D%7D HTTP/11" 200 1741
2024-10-10 10:30:24,503 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/e062d06acc82769a0393a9cd496bcdc8c22399a76a09079cf18cb3c53de5dd89/json HTTP/11" 200 None
2024-10-10 10:30:24,503 - torizon.tcbuilder.backend.common - DEBUG - Current container ID (found by container name): e062d06acc82769a0393a9cd496bcdc8c22399a76a09079cf18cb3c53de5dd89
2024-10-10 10:30:24,504 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/e062d06acc82769a0393a9cd496bcdc8c22399a76a09079cf18cb3c53de5dd89/json HTTP/11" 200 None
2024-10-10 10:30:24,504 - torizon.tcbuilder.backend.bundle - INFO - 
Starting DIND container
2024-10-10 10:30:24,504 - torizon.tcbuilder.backend.bundle - INFO - Using Docker host "tcp://127.0.0.1:22376"
2024-10-10 10:30:24,510 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/volumes/create HTTP/11" 201 182
2024-10-10 10:30:24,510 - torizon.tcbuilder.backend.bundle - DEBUG - Default platform: linux/arm/v7
2024-10-10 10:30:24,510 - torizon.tcbuilder.backend.bundle - DEBUG - Volume mapping for DinD: [{'Target': '/workdir/', 'Source': '/home/c.pennings/torizon-core-builder', 'Type': 'bind', 'ReadOnly': False}, {'Target': '/var/lib/docker/', 'Source': 'dind-volume', 'Type': 'volume', 'ReadOnly': False}]
2024-10-10 10:30:24,510 - torizon.tcbuilder.backend.bundle - DEBUG - Running DinD container: ports=None, network=host
2024-10-10 10:30:24,596 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/create?name=tcb-fetch-dind HTTP/11" 201 88
2024-10-10 10:30:24,597 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/29163f294f150b6debfa7eba4aa383c9913b318bf5f002fca7c06c67b2b72b4d/json HTTP/11" 200 None
2024-10-10 10:30:24,700 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/29163f294f150b6debfa7eba4aa383c9913b318bf5f002fca7c06c67b2b72b4d/start HTTP/11" 204 0
2024-10-10 10:30:27,703 - torizon.tcbuilder.backend.bundle - INFO - Connecting to Docker Daemon at "tcp://127.0.0.1:22376"
2024-10-10 10:30:27,703 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:27,703 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:27,703 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:27,703 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:27,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): 127.0.0.1:22376
2024-10-10 10:30:27,769 - urllib3.connectionpool - DEBUG - https://127.0.0.1:22376 "GET /version HTTP/11" 200 861
2024-10-10 10:30:27,769 - torizon.tcbuilder.backend.bundle - INFO - Attempting to log in to registry 'docker.io' with username=casperterberg
2024-10-10 10:30:27,769 - docker.utils.config - DEBUG - Trying paths: ['/root/.docker/config.json', '/root/.dockercfg']
2024-10-10 10:30:27,769 - docker.utils.config - DEBUG - No config file found
2024-10-10 10:30:27,769 - docker.auth - DEBUG - Looking for auth entry for 'docker.io'
2024-10-10 10:30:27,769 - docker.auth - DEBUG - No entry found
2024-10-10 10:30:29,135 - urllib3.connectionpool - DEBUG - https://127.0.0.1:22376 "POST /v1.40/auth HTTP/11" 200 48
2024-10-10 10:30:29,135 - torizon.tcbuilder.backend.bundle - INFO - Fetching container image casperterberg/hello-world:latest
2024-10-10 10:30:29,135 - docker.auth - DEBUG - Looking for auth config
2024-10-10 10:30:29,135 - docker.auth - DEBUG - Looking for auth entry for 'docker.io'
2024-10-10 10:30:29,135 - docker.auth - DEBUG - Found 'docker.io'
2024-10-10 10:30:29,135 - docker.auth - DEBUG - Found auth config
2024-10-10 10:30:30,255 - urllib3.connectionpool - DEBUG - https://127.0.0.1:22376 "POST /v1.40/images/create?tag=latest&fromImage=casperterberg%2Fhello-world&platform=linux%2Farm%2Fv7 HTTP/11" 200 None
  Pulling from casperterberg/hello-world
  2d3d56bba6ab: Pull complete 
  Digest: sha256:d4d5c6c7728f837602dfefe67da7aae5e318b75ac89b045ea38611e9a998d1b3
  Status: Downloaded newer image for casperterberg/hello-world:latest
2024-10-10 10:30:30,722 - urllib3.connectionpool - DEBUG - https://127.0.0.1:22376 "GET /v1.40/images/casperterberg/hello-world:latest/json HTTP/11" 200 1630
2024-10-10 10:30:30,722 - torizon.tcbuilder.backend.bundle - INFO - Saving Docker Compose file
2024-10-10 10:30:30,723 - torizon.tcbuilder.backend.bundle - INFO - Exporting storage
2024-10-10 10:30:30,723 - torizon.tcbuilder.backend.bundle - INFO - Storing container bundle into "bundle"
2024-10-10 10:30:30,723 - torizon.tcbuilder.backend.bundle - DEBUG - Volume mapping for tar container: [{'Target': '/mnt', 'Source': '/home/c.pennings/torizon-core-builder', 'Type': 'bind', 'ReadOnly': False}, {'Target': '/var/lib/docker/', 'Source': 'dind-volume', 'Type': 'volume', 'ReadOnly': True}]
2024-10-10 10:30:30,723 - torizon.tcbuilder.backend.bundle - DEBUG - tar command: ['tar', '--numeric-owner', '--preserve-permissions', '--directory=/var/lib/docker', "--xattrs-include='*'", '--create', '--file', '/mnt/bundle/docker-storage.tar', 'overlay2/', 'image/']
2024-10-10 10:30:30,809 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/create?name=tcb-build-tar HTTP/11" 201 88
2024-10-10 10:30:30,811 - urllib3.connectionpool - DEBUG - http://localhost:None "GET /v1.46/containers/a3442572220f14d341b60e3059544790ed7527160150ccd77e95e6f123e28c46/json HTTP/11" 200 None
2024-10-10 10:30:31,170 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/a3442572220f14d341b60e3059544790ed7527160150ccd77e95e6f123e28c46/start HTTP/11" 204 0
2024-10-10 10:30:31,171 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/a3442572220f14d341b60e3059544790ed7527160150ccd77e95e6f123e28c46/wait HTTP/11" 200 None
2024-10-10 10:30:31,467 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/a3442572220f14d341b60e3059544790ed7527160150ccd77e95e6f123e28c46/stop HTTP/11" 304 0
2024-10-10 10:30:31,479 - urllib3.connectionpool - DEBUG - http://localhost:None "DELETE /v1.46/containers/a3442572220f14d341b60e3059544790ed7527160150ccd77e95e6f123e28c46?v=False&link=False&force=False HTTP/11" 204 0
2024-10-10 10:30:31,479 - torizon.tcbuilder.backend.bundle - DEBUG - compression_command: ['xz', '-3', '-z', 'docker-storage.tar']
2024-10-10 10:30:31,494 - torizon.tcbuilder.backend.bundle - INFO - Stopping DIND container
2024-10-10 10:30:32,570 - urllib3.connectionpool - DEBUG - http://localhost:None "POST /v1.46/containers/29163f294f150b6debfa7eba4aa383c9913b318bf5f002fca7c06c67b2b72b4d/stop HTTP/11" 204 0
2024-10-10 10:30:33,581 - urllib3.connectionpool - DEBUG - http://localhost:None "DELETE /v1.46/volumes/dind-volume HTTP/11" 204 0
2024-10-10 10:30:33,582 - torizon.tcbuilder.cli.bundle - INFO - Successfully created Docker Container bundle in "bundle"!

[ToradexAI]

Hi, I’m the Toradex AI Assistant. The Toradex Support team will follow up with you shortly!

Did you like my answer?

---

The issue you’re experiencing with the ConnectionRefusedError when using the bundle command in TorizonCore Builder on an Azure VM seems to be intermittent and might be related to network or resource availability issues on the VM. Here are a few suggestions to help diagnose and potentially resolve the issue:

1. Resource Availability: Ensure that your Azure VM has sufficient resources (CPU, memory, and disk space) to handle the operations being performed by TorizonCore Builder. Sometimes, resource constraints can lead to intermittent failures.

2. Network Stability: Check the network stability and connectivity of your Azure VM. Intermittent network issues can cause connection errors when the tool tries to communicate with Docker or other services.

3. Docker Daemon: Verify that the Docker daemon is running properly on your VM. You can restart the Docker service to ensure it’s in a good state before running the TorizonCore Builder commands.

4. Docker-in-Docker (DinD) Configuration: Since TorizonCore Builder uses Docker-in-Docker, ensure that the DinD setup is correctly configured and that there are no conflicts with other Docker services running on the VM.

5. Logs and Debugging: Continue capturing debug logs when the issue occurs. Look for any patterns or specific conditions that might trigger the error. This information can be valuable for further troubleshooting.

6. Retry Logic: As a temporary workaround, consider implementing a retry mechanism in your build pipeline. If the bundle command fails due to a connection error, automatically retry the command after a short delay.

7. Contact Support: If the issue persists and you have captured detailed logs, consider reaching out to Toradex support with the logs and a description of the problem. They might be able to provide more specific guidance or identify any known issues.

These steps should help you narrow down the cause of the intermittent ConnectionRefusedError and improve the reliability of your build pipeline on the Azure VM.

Please, also check if these links can help you solve your problem:

TorizonCore-Builder Bundle

I successfully created and pushed (docker push) an image onto a docker registry using VS Code, In my WSL working directory and can successfully pull the image (docker pull). I placed a docker-compose.yml file in my WSL working directory. When I issue the TorizonCore-Builder bundle command I receive the response “Can’t retrieve container information from docker”. I’m not sure why i get this. I 'm also not certain about which docker-compose.yml file should be used. In my VS Code project director…

Run Docker Desktop for Windows in a VM or VDI environment

Issues with private docker registries using torizoncore-builder 3.7.0+early-access.

[jeremias.tx]

Greetings @Casper,

The issue here isn’t very obvious as to what the root cause could be. The error message itself is a fairly generic connection refused error, which could be caused by many things. This gets even more complicated by the fact it only happens “sometimes” as you showed. This suggests there’s something inconsistent/changing between your executions of the bundle command. But what exactly is hard to say.

Maybe some network hiccup/instability, or some kind of strange race condition like issue. Do you have a feel for how often bundle fails in your setup?

For context from a technical aspect the bundle command doesn’t do anything too crazy. It simply starts a docker-in-docker container that runs in parallel to the TorizonCore Builder container. The container images specified in your compose file are then pulled by the docker-in-docker container.

Particularly here where it’s failing for you is when TorizonCore Builder attempts to get a Docker client from the docker-in-docker container. We use the the Docker python library to get this client. But, for some reason the docker-in-docker container does not respond with the expected response and you get this connection refused error.

So I guess, then the issue really is with this parallel docker-in-docker container. Maybe you can try testing docker-in-docker containers in your Azure VM and see if they have occasional issues/errors. You can see from the verbose logs how we launch the container from TorizonCore Builder.

Best Regards,
Jeremias

[bruno.tx]

Hello @Casper,

Were you able to get this issue resolved?
Or do you need further support with this topic?

Best Regards,
Bruno