Pulling from local registry fails

tjallingt · June 5, 2025, 2:47pm

Pulling image from local registry fails

 *  Executing task: DOCKER_HOST=192.168.188.24:2375 LOCAL_REGISTRY=192.168.188.20 TAG=arm64 GPU=-am62 DEBUG_SSH_PORT=2231 DEBUG_PORT1=2232 DEBUG_PORT2= DEBUG_PORT3= docker compose pull slinttest-debug 

WARN[0000] The "DOCKER_LOGIN" variable is not set. Defaulting to a blank string. 
[+] Pulling 1/1
 ! slinttest-debug Warning Get "https://192.168.188.20:5002/v2/": net/http: request canceled while waiting...                         15.1s 
WARNING: Some service image(s) must be built from source by running:
    docker compose build slinttest-debug
1 error occurred:
        * Error response from daemon: Get "https://192.168.188.20:5002/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)



 *  The terminal process "/bin/bash '-c', 'DOCKER_HOST=192.168.188.24:2375 LOCAL_REGISTRY=192.168.188.20 TAG=arm64 GPU=-am62 DEBUG_SSH_PORT=2231 DEBUG_PORT1=2232 DEBUG_PORT2= DEBUG_PORT3= docker compose pull slinttest-debug'" terminated with exit code: 1.

I’m on Windows using the Torizon WSL. I can access the local registry from my windows install by visiting either http://192.168.188.20:5002/v2/ or http://localhost:5002/v2/.

However inside the Torizon WSL install curl http://192.168.188.20:5002/v2/ hangs forever whereas curl http://locahost:5002/v2/ responds correctly.

I tried following along with this issue Docker container registry not running - #11 by nimble-william but killing the process just kills the registry.

sudo tdx-info

Software summary
------------------------------------------------------------
Bootloader:               Unknown
Kernel version:           6.6.87.1-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC Mon Apr 21 17:08:54 UTC 2025
Kernel command line:      initrd=\initrd.img WSL_ROOT_INIT=1 panic=-1 nr_cpus=14 hv_utils.timesync_implicit=1 console=hvc0 debug pty.legacy_count=0 WSL_ENABLE_CRASH_DUMP=1
Distro name:              NAME="TorizonOS"
Distro version:           VERSION_ID=1.0.0
Distro variant:           VARIANT=Docker
Hostname:                 NL-ENS-L-100046
------------------------------------------------------------
cat: /sys/devices/virtual/dmi/id/board_name: No such file or directory
cat: /sys/devices/virtual/dmi/id/board_vendor: No such file or directory
cat: /sys/devices/virtual/dmi/id/board_version: No such file or directory
cat: /sys/devices/virtual/dmi/id/board_serial: No such file or directory

Hardware info
------------------------------------------------------------
HW model:                   
Toradex version:           
Serial number:            -
Processor arch:           x86_64
------------------------------------------------------------

ToradexAI · June 5, 2025, 2:47pm

Hi, I’m the Toradex AI Assistant.

I might make mistakes. Our support team will step in if needed. If I answered your question, please tick the “Solution” checkbox. For follow-up questions to me, please use @ToradexAI in your post.

Did you like my answer?

Hello! Thank you for providing detailed information about your setup and issue.

This appears to be a network connectivity or Docker configuration issue between your Torizon WSL2 environment and your local Docker registry when accessed via the external IP address. Here are step-by-step recommendations to address this:

1. Local Docker Registry Access – Insecure Registry

Docker, by default, blocks communication with registries over HTTP unless they are explicitly marked as insecure.
Since your registry responds on HTTP (not HTTPS) at http://192.168.188.20:5002, you must configure the Docker daemon inside the Torizon WSL2 to treat 192.168.188.20:5002 as an insecure registry.

Docker Insecure Registry Configuration
Create or modify /etc/docker/daemon.json in your WSL2 environment:

{
  "insecure-registries" : ["192.168.188.20:5002"]
}

After editing, restart the Docker daemon within WSL:

sudo service docker restart

More details:

2. WSL2 Networking and Windows Firewall

WSL2 uses a virtual network, and sometimes access via the VM’s external interface (your 192.168.188.20 address) can be blocked by Windows Defender Firewall or not properly routed.
Since localhost works but the IP does not, your registry may be listening only on localhost within Windows, not all interfaces.

Ensure Registry is Listening on All Interfaces:

When running your registry, use -p 5002:5002 and set the registry config or Docker command so it binds to 0.0.0.0 rather than 127.0.0.1.
Example:
```
docker run -d -p 5002:5002 --restart always --name registry -e REGISTRY_HTTP_ADDR=0.0.0.0:5002 registry:2
```
Ensure your Windows Firewall allows inbound connections on port 5002.

3. Docker Engine Version

Only use a single Docker Engine inside WSL2. Do not attempt to use Docker Desktop and the WSL Docker Engine simultaneously, as this can lead to networking conflicts.
See:
Troubleshooting Docker Deployment Timeout Issues on Torizon IDE with WSL2 and Verdin Development Board (i.MX8MP)
and
Configure Build Environment for Torizon Containers | Toradex Developer Center

4. Testing

Test with:
```
curl http://192.168.188.20:5002/v2/
```
from inside WSL2 after making the above adjustments.

Also test pulling an image manually:

docker pull 192.168.188.20:5002/your-image:tag

Summary Checklist

Registry binds to 0.0.0.0 (not 127.0.0.1).
Windows firewall allows port 5002.
Docker’s daemon.json in WSL2 includes your registry as insecure.
Only one Docker Engine instance active (inside WSL2).

Let me know if these changes resolve the issue or if you encounter additional error messages!

Best regards,
Toradex AI Assistant.

bruno.tx · June 11, 2025, 9:22am

Hello @tjallingt,

I apologize for the delay here.

However inside the Torizon WSL install curl http://192.168.188.20:5002/v2/ hangs forever whereas curl http://locahost:5002/v2/ responds correctly.

Is this causing any issue while using the Torizon IDE extension?
If so, what is the problem you see on the extension?

Best Regards,
Bruno

tjallingt · June 12, 2025, 6:53am

Yes it causes the “Torizon arm64” task from launch.json to fail with the following error:

tjallingt:

 *  Executing task: DOCKER_HOST=192.168.188.24:2375 LOCAL_REGISTRY=192.168.188.20 TAG=arm64 GPU=-am62 DEBUG_SSH_PORT=2231 DEBUG_PORT1=2232 DEBUG_PORT2= DEBUG_PORT3= docker compose pull slinttest-debug 

WARN[0000] The "DOCKER_LOGIN" variable is not set. Defaulting to a blank string. 
[+] Pulling 1/1
 ! slinttest-debug Warning Get "https://192.168.188.20:5002/v2/": net/http: request canceled while waiting...                         15.1s 
WARNING: Some service image(s) must be built from source by running:
    docker compose build slinttest-debug
1 error occurred:
        * Error response from daemon: Get "https://192.168.188.20:5002/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)



 *  The terminal process "/bin/bash '-c', 'DOCKER_HOST=192.168.188.24:2375 LOCAL_REGISTRY=192.168.188.20 TAG=arm64 GPU=-am62 DEBUG_SSH_PORT=2231 DEBUG_PORT1=2232 DEBUG_PORT2= DEBUG_PORT3= docker compose pull slinttest-debug'" terminated with exit code: 1.

matheus.tx · June 12, 2025, 2:49pm

Hey @tjallingt

could you please check from the device side? Because the issue is not from inside the Windows, from WSL is normal you do not have the registry under the Windows IP, so we use localhost. From outside the WSL that you should always use the Windows IP to access the local registry. So, from the device side:

curl http://192.168.188.20:5002/v2/

What’s the behavior?
If it hangs, does not respond, or times out, probably you have some firewall issue that is blocking the connection between Windows and the outside, like the device.

In this case, please could you report the return from the follow command:

This should be run in a new WSL terminal

sudo iptables -L -v -n

BR,

tjallingt · June 18, 2025, 8:54am

I logged in on the device via the UART 1 debug port and ran the curl command. This indeed failed to connect after a timeout:

$ curl http://192.168.188.20:5002/v2/

curl: (7) Failed to connect to 192.168.188.20 port 5002 after 3142 ms: Couldn't connect to server

Running sudo iptables -L -v -n from a new Torizon WSL terminal gives me:

$ sudo iptables -L -v -n
Chain INPUT (policy ACCEPT 230K packets, 1301M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 279K  876M DOCKER-USER  0    --  *      *       0.0.0.0/0            0.0.0.0/0
 279K  876M DOCKER-FORWARD  0    --  *      *       0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 201K packets, 1631M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     6    --  !docker0 docker0  0.0.0.0/0            172.17.0.2           tcp dpt:5000
    0     0 DROP       0    --  !docker0 docker0  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-BRIDGE (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-CT (1 references)
 pkts bytes target     prot opt in     out     source               destination
 207K  872M ACCEPT     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED

Chain DOCKER-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
 279K  876M DOCKER-CT  0    --  *      *       0.0.0.0/0            0.0.0.0/0
72830 4035K DOCKER-ISOLATION-STAGE-1  0    --  *      *       0.0.0.0/0            0.0.0.0/0
72830 4035K DOCKER-BRIDGE  0    --  *      *       0.0.0.0/0            0.0.0.0/0
72830 4035K ACCEPT     0    --  docker0 *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
72830 4035K DOCKER-ISOLATION-STAGE-2  0    --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       0    --  *      docker0  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
 279K  876M RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0

I’m not sure how to interpret this myself.
It is likely the firewall rules are setup to block external connections but that’s managed by IT.

matheus.tx · July 10, 2025, 4:22pm

@tjallingt

These rules are applied under your local WSL. Try to run the following in the WSL terminal:

sudo iptables -I FORWARD -i docker0 -j ACCEPT && sudo iptables -I FORWARD -o docker0 -j ACCEPT

After that, please share with us again the output of the sudo iptables -L -v -n.
Also, try to deploy the debug sessions again to see if something changes.

BR,

rudhi.tx · August 12, 2025, 9:25am

Hello @tjallingt,

Is this issue solved for you? Do you need any support from our side?

tjallingt · August 12, 2025, 2:18pm

Apparently this was caused by my network getting marked as a “public network” by Windows. I can work around this by setting the network as “private network” manually.

koverly · September 17, 2025, 2:50pm

I’m experiencing what I believe is the same issue and have been unable to resolve it. I suspect it’s a Windows firewall issue, but would like to confirm it before reaching out to our corporate IT for support.

Here’s the output from vscode when the module attempts to pull the image:

 *  Executing task: DOCKER_HOST=192.168.10.1:2375 LOCAL_REGISTRY=192.168.10.10 DOCKER_LOGIN=192.168.10.10:5002 TAG=v0.0.0 GPU=-am62 DEBUG_SSH_PORT=2230 DEBUG_PORT1= DEBUG_PORT2= DEBUG_PORT3= docker compose pull network_manager_test 

WARN[0000] /home/torizon/networkmanagertest/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
[+] Pulling 1/1
 ! network_manager_test Warning Get "http://192.168.10.10:5002/v2/": net/http: request canceled while waiting for connection (Clien...                              30.1s 
WARNING: Some service image(s) must be built from source by running:
    docker compose build network_manager_test
1 error occurred:
        * Error response from daemon: Get "http://192.168.10.10:5002/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)



 *  The terminal process "/usr/bin/bash '-c', 'DOCKER_HOST=192.168.10.1:2375 LOCAL_REGISTRY=192.168.10.10 DOCKER_LOGIN=192.168.10.10:5002 TAG=v0.0.0 GPU=-am62 DEBUG_SSH_PORT=2230 DEBUG_PORT1= DEBUG_PORT2= DEBUG_PORT3= docker compose pull network_manager_test'" terminated with exit code: 1. 
 *  Terminal will be reused by tasks, press any key to close it.

When I test in wsl with curl using localhost I get a response:

🐧 torizon @STERIS-US-x3Jes~: curl http://localhost:5002/v2
<a href="/v2/">Moved Permanently</a>.

But if I test in wsl with curl using the IP address it times out:

🐧 torizon @STERIS-US-x3Jes~: curl http://192.168.10.10:5002/v2/
curl: (28) Failed to connect to 192.168.10.10 port 5002 after 133517 ms: Couldn't connect to server

It times out when tested from the device side as well:

torizon@verdin-am62-15601305:~$ curl http://192.168.10.10:5002/v2
curl: (28) Failed to connect to 192.168.10.10 port 5002 after 135016 ms: Couldn't connect to server

The output from the iptables command run in a new wsl terminal is:

🐧 torizon @STERIS-US-x3Jes~: sudo iptables -L -v -n
[sudo] password for torizon:
Chain INPUT (policy ACCEPT 17589 packets, 36M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-USER  0    --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-FORWARD  0    --  *      *       0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 19589 packets, 20M bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     6    --  !docker0 docker0  0.0.0.0/0            172.17.0.2           tcp dpt:5002
    0     0 DROP       0    --  !docker0 docker0  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-BRIDGE (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-CT (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     0    --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED

Chain DOCKER-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-CT  0    --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-1  0    --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-BRIDGE  0    --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     0    --  docker0 *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-ISOLATION-STAGE-2  0    --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       0    --  *      docker0  0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination

wsl release info:

🐧 torizon @STERIS-US-x3Jes~: cat /etc/os-release
PRETTY_NAME="Torizon Development Environment for WSL 2"
NAME="TorizonOS"
VERSION=1.0.3
VERSION_CODENAME="Development Environment for WSL 2 (Debian Bookworm)"
ID=torizon
VERSION_ID=1.0.3
VARIANT=Docker
HOME_URL="https://www.torizon.io"
SUPPORT_URL="https://community.toradex.com"
BUG_REPORT_URL="https://github.com/torizon/cookbook-torizon-wsl/issues"
🐧

eric.tx · September 19, 2025, 1:04pm

Hey @koverly,

If you wouldn’t mind making a new ticket, so we can track your solution if it’s separate from this one (make it easier for people searching). You can mention this ticket too so we can have it as a reference.

As for now, It does look like your system is failing to connect, so if you have firewalls, this could be the cause.

-Eric