Have you tying disabling the socket-based activation and enabling the service-based activation of the ssh server?
You can also play with ClientAliveInterval server configuration parameter or ServerAliveInterval on a client side. Please refer to this thread for details
To try that solution is easy and works fine on a flashed device. But how can I make an image with that workaround? Do you plan to change to sshd.service in a future image?
Given that enabling or disabling a systemd service or socket makes changes in /etc/, you can use TorizonCore Builder to get those changes in /etc with the isolate command and then create a custom TorizonCore (now called Torizon OS) image with these changes applied using the build command, as instructed here: