As I have had the same problem, at the vs code info section (host name position @drew.tx has mentioned) I have put the hostname in the following format:
hostname.local
This has solved the issue. This is in accordance with the Toradex’s SSH Connection Tutorial.