Pull-container-torizon-debug-arm64 failure with exit code 18

flepron · June 3, 2024, 4:02pm

Using username "torizon".
torizon@verdin-imx8mp-15229850's password:
Last login: Mon Jun  3 15:41:24 2024 from 2a05:6e02:102d:5710:395a:ae1a:aa57:2b4d

** WARNING **
THIS DEVICE IS IN ENGINEERING MODE

Check for more info:
https://developer.toradex.com/torizon/application-development/ide-extension

torizon@verdin-imx8mp-15229850:~$ sudo txd-info
Password:
sudo: txd-info: command not found
torizon@verdin-imx8mp-15229850:~$ sudo tdx-info

Software summary
------------------------------------------------------------
Bootloader:               U-Boot
Kernel version:           5.15.148-6.6.1+git.23a8e831749d #1-TorizonCore SMP PREEMPT Thu Feb 29 20:25:21 UTC 2024
Kernel command line:      root=LABEL=otaroot rootfstype=ext4 quiet logo.nologo vt.global_cursor_default=0 plymouth.ignore-serial-consoles splash fbcon=map:3 ostree=/ostree/boot.1/torizon/f5f0b9e40c1595ab904ce493a792a0b54e17f0dc3ce6832ddb889452bdd13704/0
Distro name:              NAME="TorizonCore"
Distro version:           VERSION_ID=6.6.1-build.14
Distro variant:           VARIANT="Docker"
Hostname:                 verdin-imx8mp-15229850
------------------------------------------------------------

Hardware info
------------------------------------------------------------
HW model:                 Toradex Verdin iMX8M Plus WB on Verdin Development Board
Toradex version:          0058 V1.1B
Serial number:            15229850
Processor arch:           aarch64
------------------------------------------------------------

[Configuration]
Verdin iMX8M Plus Evaluation Kit with Touchscreen
with:
SOM i.MX8M Plus Quad 4GB WB IT v1.1B
Dahlia Carrier Board v1.1D
Verdin DSI to LVDS rev 1.1A
Capacitive Touch Display 10.1" v1.0A

Executing task: sshpass -p 959798 ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no torizon@192.168.10.111 LOCAL_REGISTRY=10.2.84.164 TAG=arm64 docker compose pull i2c-adc-csharp-debug 

Warning: Permanently added '192.168.10.111' (ED25519) to the list of known hosts.
time="2024-06-03T15:49:09Z" level=warning msg="The \"DOCKER_LOGIN\" variable is not set. Defaulting to a blank string."
 i2c-adc-csharp-debug Pulling 
 i2c-adc-csharp-debug Warning 
WARNING: Some service image(s) must be built from source by running:
    docker compose build i2c-adc-csharp-debug
1 error occurred:
        * Error response from daemon: Get "http://10.2.84.164:5002/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

 *  The terminal process "sshpass '-p', '959798', 'ssh', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', 'torizon@192.168.10.111', 'LOCAL_REGISTRY=10.2.84.164 TAG=arm64 docker compose pull i2c-adc-csharp-debug'" terminated with exit code: 18. 
 *  Terminal will be reused by tasks, press any key to close it.

I have this failure. Everything works fine when I start to debug my C# .NET 8.0 console application, but at the step of pulling my container to torizon device in debug arm64, there is this error 18, even if I start a new blank project.

Just an hint which may help you. I use WSL2 with Ubuntu 22.04.4 TLS. Before it worked, but now this fails at this step, and I do not know what has changed.

Could please help me to solve this issue ?

Many thanks.
François.

ToradexAI · June 3, 2024, 4:03pm

Hi, I’m the Toradex AI Assistant. I’m in early beta, and I would appreciate your feedback.
The Toradex Support team will follow up with you shortly!

Did you like my answer?

Hello François,

The error you’re encountering, indicated by exit code 18, typically relates to network connectivity issues, specifically a timeout error when trying to reach your local registry. This problem suggests that the Docker daemon on your Torizon device is unable to communicate with the registry at http://10.2.84.164:5002/v2/.

Here are several steps to troubleshoot and potentially resolve this issue:

Verify Network Connectivity: Ensure that both your Torizon device and the machine hosting your local registry (10.2.84.164) are on the same network and can communicate with each other. You can test this by pinging the registry host from your Torizon device.
Check Docker Registry Running: Ensure that the Docker registry service is running on 10.2.84.164 and listening on port 5002. You can verify this by running sudo netstat -tuln | grep 5002 on the host machine.
Firewall Configuration: Check if there’s a firewall on the host machine that could be blocking incoming connections on port 5002. If so, you may need to configure the firewall to allow traffic on this port.
Docker Daemon Configuration on Torizon Device: If you’re using a proxy on your Torizon device, ensure that the Docker daemon is configured to bypass the proxy for your local registry. This can be done by adding your registry’s IP address and port to the NO_PROXY environment variable in the Docker service configuration.
Registry URL Accessibility: Verify that the registry URL http://10.2.84.164:5002/v2/ is correct and accessible from other machines on the same network. You can test this by using a tool like curl from another machine on the same network.
Use of SSH and SSHpass: The use of sshpass and SSH to execute the Docker compose command remotely may introduce complexities. Ensure that the SSH command is correctly formatted and that there are no issues with SSH key verification or user permissions.

Given the transition from a working state to the current issue without changes in your project, it’s likely that the problem lies in network configuration, Docker registry service status, or firewall rules. Double-checking these areas should help identify and resolve the connectivity issue preventing your container from being pulled to the Torizon device.

Best regards,
Toradex AI Assistant

Please, also check if these links can help you solve your problem:

TorizonCore Builder Tool - Commands Manual | Toradex Developer Center.

jeremias.tx · June 3, 2024, 9:23pm

Greetings @flepron,

There’s not much clues to go on, but here’s what I understand so far. The extension first builds your container image and stores it into a local registry running on your development machine. It then executes a command on your embedded device telling it to go and pull this container image from the local registry on your machine. However this request fails with the following:

Error response from daemon: Get "http://10.2.84.164:5002/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

If this error message is accurate for some reason the http request to the local registry was “cancelled” due to timeout it seems. There’s a number of things that could cause this. Perhaps a firewall or network restriction is blocking the request. Or some network configuration changed since last time.

It’s hard to say exactly what is causing this in your setup. In your logs you can see the exact command that is being sent to your embedded device:

*  The terminal process "sshpass '-p', '959798', 'ssh', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', 'torizon@192.168.10.111', 'LOCAL_REGISTRY=10.2.84.164 TAG=arm64 docker compose pull i2c-adc-csharp-debug'" terminated with exit code: 18.

It’s essentially just docker compose pull targeting the local registry that should be running on your development machine. Perhaps you can debug this connection and see where it breaks down if possible.

Best Regards,
Jeremias

flepron · June 4, 2024, 1:07pm

Greetings @jeremias.tx

I have tried a lot of solutions that I have found searching on internet or on Bing Copilot, but I did not manage to fix the issue after more than 2 hours. I think that there is something which is corrupted somewhere, and I don’t have the time to waste more time and I do not know really how to fix it honestly.

So I reinstall VS Code, IDE Torizon Extension, WSL2, Ubuntu 24.04 and Docker on my Windows 11 Pro PC.

I just wonder if this is better to install docker under my Ubuntu 24.04 distribution which is under WSL2, or if this is better to install it under Windows 11 using Docker Desktop ?

Also, I wonder if the pre-release of Torizon IDE Extension is stable enough to install it, or should I stay on 2.4.2 ?

Thanks you for your advice.

Sincerely,
François.

jeremias.tx · June 4, 2024, 4:03pm

I just wonder if this is better to install docker under my Ubuntu 24.04 distribution which is under WSL2, or if this is better to install it under Windows 11 using Docker Desktop ?

Our documentation recommends to install Docker in your WSL instance. This is the setup that is tested and validated by our team. Now that said, I personally have it installed in Windows via Docker Desktop and that has worked fine for me so far. Though this isn’t the exact setup that is tested by our team so it may be more “risky” in a sense. Also I’ve seen reports of some customers who use Docker Desktop and from time to time it seems like some Docker Desktop versions break compatibility with our extension somehow.

Whichever you choose, what is important is that you only have one of these installed at a time. I’ve seen customers who have had both installed and it resulted in a lot of strange issues with our extension.

Also, I wonder if the pre-release of Torizon IDE Extension is stable enough to install it, or should I stay on 2.4.2 ?

It should be good enough to install. But as a “pre-release” it’s inherently not as stable as the stable release. It may have experimental changes or features that are not fully working yet. Is there a reason you’re interested in the pre-release version?

Best Regards,
Jeremias

flepron · June 5, 2024, 9:58am

Hi @jeremias.tx

I have installed now docker engine in my Ubuntu 22.04.4 distro under WSL2 as advised.

Before I used Docker Desktop on Windows 11 Pro and it worked fine, but I prefer to follow the recommended path.

I was interested in the new Torizon IDE Extension because I have watched the Youtube video on this topic, and there was no warning. So, I have preferred to ask you.

Torizon IDE Installation Wizard: The New Era of “Easy-To-Use”

I keep you updated for my problem after the re-installation of the tool-chain.

Best regards,
François.

jeremias.tx · June 5, 2024, 5:43pm

I see, well as I said before just keep in mind the early access version is expected to have some bugs or issues here and there.

Best Regards,
Jeremias

flepron · June 6, 2024, 5:57am

I understand @jeremias.tx, but nothing is said in this video saying that this an early access version.

The screen does not show the version, but just show how this new pre-release version brings easyness in development.

Honestly, I really appreciate what @matheus.tx is making. He is an outstanding developer because he knows that the road to the development of an application must be shortened by great tool like his new Torizon IDE Extension, but it should be call "Presentation of the coming Torizon IDE extension which brings a new area of “Easy-To-Use”, with a warning saying that is only an early access version to install with caution.

Best regards,
François.

jeremias.tx · June 6, 2024, 4:38pm

but nothing is said in this video saying that this an early access version.

To be fair this is stated right in the description of that video:

To overcome the hurdle of setting up the Torizon development environment, we have released in Early Access a Torizon IDE Installation Wizard, providing a first-class, easy-to-use setup experience.
It’s very simple! All you have to do is:

While this information could have been more visible. It’s unfair to say we did not do anything to inform users of this.

Anyways, this whole discussion is now currently irrelevant as the latest stable release of the extension (Just released yesterday evening), now has this installation wizard as an experimental feature: Changelog | Visual Studio Marketplace

Experimental features can be enabled as documented here: Settings - Enable Experimental Features | Toradex Developer Center

If you have any feedback regarding the new installation wizard feel free to give it.

Best Regards,
Jeremias

matheus.tx · June 7, 2024, 11:54am

Hey @flepron

thanks for the feedback, we added a banner there and released an update with fixes. Unfortunately, there is no way to update without uninstalling the previous version of Torizon for WSL. See the instructions here: Torizon IDE Installation Wizard for Windows - Early Access Feedback Thread - #2 by matheus.tx

If you have more feedback or reports about this feature, you can post them directly in the thread.

BR,

flepron · June 7, 2024, 2:10pm

Hi @jeremias.tx and @matheus.tx

Thanks you both for your answers.

I do appreciate the new version of Torizon IDE Extension.

I can just add, that on my side (Windows 11 Pro, WSL2 Ubuntu 22.04 already installed, and Torizon IDE 2.4.2 installed and working with my account on my existing Ubuntu 22.04, with many test projects already designed), when I have switched from version Torizon IDE extension 2.4.2 to a pre-release version, I had to create a new account on the new WSL2 torizon machine created by the @matheus.tx 's new Torizon IDE extension.

I was a bit lost because, I had to copy my previous projects on the new WSL2 torizon machine.

It has been not to much a problem for me because my projects were saved on GitHub.

I do not know if this comes form my installation and my lack on knowledge in Linux, but I have been confused, because the IP addresses of my Ubuntu and Torizon WSL2 were the same.

Forget that because this comes probably from my side and my bad initial configuration where I have installed Docker Desktop on my Winows 11 PC instead of Ubuntu WSL2.

@jeremias.tx, you can consider this topic solved “Pull-container-torizon-debug-arm64 failure with exit code 18” because I have uninstalled Docker Desktop, I have deleted my Ubuntu machine and deleted WSL2, and after I have reinstalled everything with docker engine in my new Ubuntu 22.04, cloned back my projects repos on my fresh new Ubuntu 22.02, and the pull-container-torizon-debug-arm64 exit code failure 18 disappeared.

Everything works fine back now.

Many thanks,
François.

jeremias.tx · June 7, 2024, 4:20pm

Glad we were able to assist.