Internal server error Error(500) - (errno None) unable to connect to port 22 on 192.168.1.xxx

Technical Support
, , ,
1

Colibri iMX6 512 MB IT v 1.1B
Colibri Evaluation Board
TorizonCore Upstream 5.6.0 devel 202202+build.21

rebooted my system once and reinstalled the OS on the Colibri Module twice, rebuilt the container, yet still can’t seem to connect to my device today…

When I click Torizon Extension, Devices, + and enter my IP address, username and password, I get.

Internal server error :: Error (500) - [Errno None] Unable to connect to port 22 on 192.168.1.123

I am using the ip address which I find over the USB debug port by typing >

# ip addr | grep ethernet0

and I can ping it from my pc which gets replies.

image

username torizon
password torizon1

I know my password is correct because I did this:

image

here is my output

image
image882×317 12.7 KB

yet it DOES work with my other Colibri module (ip address 192.168.1.119) but if I switch back to the other module it stops working again. Tried removing and adding the device…

In the “Terminal Window” in the Torizon IDE backend process, I get these messages:

image
image938×308 10 KB

I expected it would try to create a 2nd device, but it still seems to be trying to open the old one.

has anyone experienced something like this before?
How can I get past this?

2

It seems I had tried to add a new device while inside the remote terminal of VSC.
When I opened a local terminal, then I could add a new device.
Here’s the steps I took to fix it.

Boot PC
wait for docker to be running
image
start VSC
click torizon extension
click the plus beside devices ----- + (Add device)
choose network (bottom right)
type the ip address
type the username torizon
type the password ********
now in the bottom left, click the white on green arrows and choose open folder in container.
find the folder that contains your project and open it!

3

Seems like my fix worked only for a day.

I have recently need to switch between my two Colibri modules a lot. But when I swap them, I get these errors. One of them works, and one of them does not.

The error I get today is:

[02-23 15:44:43.718] No devices registered

[02-23 15:45:04.727] Internal server error

[02-23 15:45:04.728] Error (500) - [Errno None] Unable to connect to port 22 on 192.168.1.123

[02-23 15:48:15.189] Toradex Colibri iMX6DL/S on Colibri Evaluation Board V3(10866289) not reached

[02-23 15:48:15.248] Torizon: unreachable devices

[02-23 15:48:15.248] Unreachable devices:

Toradex Colibri iMX6DL/S on Colibri Evaluation Board V3(10866289)

Make sure the devices are turned on and connected to the network

I switch between my two Colibri modules a lot, and I need to know the best procedure to do this.

4

more info from various windows in VSC

My first device (which works) has
IP address 192.168.1.119 and
serial number 10866289

I can see that when I changed devices, and it asks me for the IP address, I give the info for the new board:
IP address 192.168.1.123
(it has serial number 10866293 although this is never asked for, just username and password are needed when adding a new device.)

However, when it is connecting I get these messages in the terminal window:
First you see it cannot connect to my .119 device (because I disconnected it)

Then it tries to connect to the .123 device, but it mentions the path /api/devices/10866289/image which is the first board…

Also on the first line, it says it processed file C: C:\Users\LeighBoyd.moses\devices\10866289\id_rsa but I don’t see a similar message for 10866293…

So I think I’m on to something.

processed file: C:\Users\LeighBoyd\.moses\devices\10866289\id_rsa
Successfully processed 1 files; Failed processing 0 files
INFO:root:REST <- /api/devices/10866289/privatekey - 200
INFO:root:REST -> /api/devices/10866289/current_ip
INFO:root:REST <- /api/devices/10866289/current_ip - 200
2022-03-25 11:54:54,157| ERROR   | Could not connect to gateway 192.168.1.119:22 : Unable to connect to 192.168.1.119: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
ERROR:sshtunnel.SSHTunnelForwarder:Could not connect to gateway 192.168.1.119:22 : Unable to connect to 192.168.1.119: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
ERROR:root:Error: 539 SSH tunnel error. SSH tunnel error: Could not establish session to SSH gateway
ERROR:root:Exception: SSH tunnel error: Could not establish session to SSH gateway
ERROR:root:Could not establish session to SSH gateway
INFO:root:REST <- /api/devices/10866289/images - 539
INFO:root:REST -> /api/devices/10866289/images
INFO:root:SSH - Creating tunnel to 10866289
2022-03-25 11:55:32,728| ERROR   | Could not connect to gateway 192.168.1.119:22 : Unable to connect to 192.168.1.119: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
ERROR:sshtunnel.SSHTunnelForwarder:Could not connect to gateway 192.168.1.119:22 : Unable to connect to 192.168.1.119: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
ERROR:root:Error: 539 SSH tunnel error. SSH tunnel error: Could not establish session to SSH gateway
ERROR:root:Exception: SSH tunnel error: Could not establish session to SSH gateway
ERROR:root:Could not establish session to SSH gateway
INFO:root:REST <- /api/devices/10866289/images - 539
INFO:root:REST -> /api/devices/10866289/images
INFO:root:SSH - Creating tunnel to 10866289
  File "connexion\decorators\validation.py", line 384, in wrapper  File "connexion\decorators\parameter.py", line 121, in wrapper
  File "api.py", line 189, in devices_network_detect_get  File "targetdevice.py", line 765, in add_network_device
  File "targetdevice.py", line 691, in _setup_device
  File "sshconsole.py", line 109, in login
  File "paramiko\client.py", line 368, in connect
paramiko.ssh_exception.NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.1.123
INFO:root:REST <- /api/devices/network_detect - 500
INFO:root:REST -> /api/devices/10866289/images
INFO:root:REST <- /api/devices/10866289/images - 404INFO:root:REST -> /api/devices/10866289/images
INFO:root:REST <- /api/devices/10866289/images - 404INFO:root:REST -> /api/devices/10866289/images
INFO:root:REST <- /api/devices/10866289/images - 404
INFO:root:REST -> /api/devices/10866289/images
INFO:root:REST <- /api/devices/10866289/images - 404
INFO:root:REST -> /api/devices/10866289/images
INFO:root:REST <- /api/devices/10866289/images - 404
5

Still no luck with this. I can still only program with one module. This slows down our development considerably since others need me to program modules for them to get my lastest code, but I am forced to give them MY module instead (and I cannot code in the meantime)

Please help.

6

I’m not able to reproduce this issue on my side I can switch between devices fairly easily with no issues. When you deploy your application with the extension and you have multiple devices available, there should be a prompt to choose which device you want to use.

Other than that I didn’t do anything special. Just added both devices, create a C/C++ project and it seems to go well.

Best Regards,
Jeremias

7

Is there a specific order one should follow?
example add the devices first, then create the project?

Until now I have been removing and adding devices each time I change, but I suppose I could have multiple devices available from what you are saying.

I will try recreating the project. Haven’t tried this.

Also cannot open a device with serial port. It asks for a port like COM3 or whatever. No matter what I enter here it doesn’t work. I made sure I turned off PuTTY so there is no conflict. Should I say COM3: or COM3 without the colon for example?

Leigh

8

The way I did it was as follows:

  • Add device 1
  • Add device 2
  • Create new C/C++ project
  • Begin to debug & deploy
  • Choose which device I want to deploy to via the prompt that should appear upon deployment

Also cannot open a device with serial port. It asks for a port like COM3 or whatever. No matter what I enter here it doesn’t work. I made sure I turned off PuTTY so there is no conflict. Should I say COM3: or COM3 without the colon for example?

Should just be COM3, or whatever COM device the serial port is on. Though on some setups the extension may not be able to access the ports due to permissions and such. Also it should be noted that opening a device via serial port is only for the initial detection/setup. After the device is added it will then use the same network based connection as if you opened it with network in the first place.

Best Regards,
Jeremias

9

Still no luck.

Here’s what I tried this time:
use SSH from WSL to connect to my board:

ssh -v torizon@192.168.1.119
OpenSSH_7.6p1 Ubuntu-4ubuntu0.6, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.168.1.119 [192.168.1.119] port 22.
debug1: Connection established.
debug1: identity file /home/leighjboyd/.ssh/id_rsa type 0
debug1: key_load_public: No such file or directory
debug1: identity file /home/leighjboyd/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/leighjboyd/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/leighjboyd/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/leighjboyd/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/leighjboyd/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/leighjboyd/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/leighjboyd/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2
debug1: match: OpenSSH_8.2 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.1.119:22 as 'torizon'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:GRayaiN9PyBsST8A5TOOyF3R2kLM7uy2cn17pVUOEKY
debug1: Host '192.168.1.119' is known and matches the ECDSA host key.
debug1: Found key in /home/leighjboyd/.ssh/known_hosts:2
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:QIxdn2O4Gttciab1UjKWnwYcWqwJjnSc24Um/TLFMSE /home/leighjboyd/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/leighjboyd/.ssh/id_dsa
debug1: Trying private key: /home/leighjboyd/.ssh/id_ecdsa
debug1: Trying private key: /home/leighjboyd/.ssh/id_ed25519
debug1: Next authentication method: password
torizon@192.168.1.119's password:

which works just fine

but now I swap boards to my 192.168.1.123 Colibri and try again.

ssh -v torizon@192.168.1.123
OpenSSH_7.6p1 Ubuntu-4ubuntu0.6, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to 192.168.1.123 [192.168.1.123] port 22.
debug1: connect to address 192.168.1.123 port 22: Connection timed out
ssh: connect to host 192.168.1.123 port 22: Connection timed out

Then I re-installed EasyInstaller

Colibri-iMX6_ToradexEasyInstaller_5.6.0-devel-20220217+build.278

Then re-installed my image.

torizon-core-docker-evaluation-colibri-imx6-Tezi_5.6.0-devel-202202+build.21.container

and tried it again.
Same timeout response.

repeated the installation using a different (but exactly the same) image

tried again, same timeout.

it also doesn’t work to do an

ssh-keyscan -t ecdsa -T 240 192.168.1.123

simply times out after 240 seconds…

Do you think the module is damaged? I can still ping it, and log in through the usb port, etc.

10

Ok so this isn’t related to our IDE extension at all then?

SSH on that specific module just doesn’t seem to work?

You said you can still access this module’s serial port. When you access over serial do you notice anything amiss? Is the networking on that module working?

It might possibly be a module/hardware specific issue with that unit. But I don’t believe such an issue would be likely. Can no one in your team SSH into this specific module? Has anyone been able to SSH into this module before?

Best Regards,
Jeremias

11

I dont have the module with me today, but I can say that I managed to SSH to it before, and work with VSC to install my application. We follow strict ESD protocols, but it is possible it got a shock (we have carpet in the front office where I work)

The only thing Im not sure of is if the IDE somehow mixes up the ip address with the hostname by incorrectly remembering what hostname was used last. I will know more when I get my 3rd module, whenever that arrives. I’ll keep you posted.

12

The only thing Im not sure of is if the IDE somehow mixes up the ip address with the hostname by incorrectly remembering what hostname was used last.

Hmm possibly but I haven’t experienced this myself yet. Do let me know how it goes with this 3rd module.

13

got my third module on Friday.
Intially things were going well.

Here is what I tried:

  • downloaded windows-recovery.bat for Easy Installer v 5.3
  • downloaded Easy Installer v 5.6.0 build.9
  • installed Torizon Core with Evaluation Containers v 5.6.0 + myCUSTOMoverlay
  • connect serial port to get ip address, 192.168.1.158
  • changed password to torizon1
  • tried adding a device - didn’t work at first
  • remembered to add the device to the LOCAL VScode Torizon Extension
  • opened VSC as “new window”
  • hard-rebooted the target at random intervals
  • Finally the device was added! (10934586)
    image
    image1039×705 40.4 KB

There’s where the good news stopped.
-Then went back to the remote window and clicked refresh, received error (500) - list index out of range
tried to SSH, this was sluggish but then allowed me to type a password and I could get in, but still could not connect to the device under Torizon Extension Devices.

-tried above procedure again, this time left-click on green >< symbol and choose “open folder in container” and selected my project… I get “trying connection …”

Finally it actually connected ONCE for a short time (5 min) and I though all was going to be good.

But then when I tried to open the SSH terminal, it kicked me off, as if someone clicked “refresh” on the device. I have never been able to get it to connect again, either getting an error message Error (500) - list index out of range or just “trying connection …”

Tried to rebuild the container (while in remote mode, perhaps I should be in local mode for this?) F1 - Remote-containers: rebuild container

Tried torizon-Extension: Detect device via network connection get an Internal server error: Error (500) Operation not permitted ‘/home/torizon/.moses/devices/10934586/id_rsa’
image

I have had a LOT of pressure to get a second module running as others on my team need it to develop, it didnt help that the replacement module was incorrectly shipped to our head office, and delayed us even further. Please help me to understand what I am doing wrong.

Unlike my previous behaviour earlier in this thread, however, I CAN connect via SSH from WSL on my pc. if I run ssh with -v option you can see the keepalive lines:

image
image876×358 13.6 KB

I feel I must be doing something wrong, which may even have damaged my last module, so I am stopping and would very much appreciate a detailed procedure (with expected responses/messages) to go from here, a fresh VSCode window. Should I then click the green >< symbol and open folder in container? You mentioned that you built a new C/C++ project, but should I do this when I already have a project? I would like to be able to switch effortlessly between the two modules.

It seems that the command to reboot the module (from Devices) has no effect, I wonder if a reboot is necessary at each connection attempt and this is why the connection is not happening?

Leigh

14

found this useful troubleshooting article. It didnt solve my problems, but my problems miraculously disappeared after a reboot of my PC.

Also reinstalled the Torizon Extension, which might have helped…

Troubleshooting guide for the IDE extensions (toradex.com)

15

Well that’s all very strange, but glad to hear it sounds like things are resolved now.