Remote Access Performance Very Poor

When using remote access to remote into a fresh install of a no-container torizon OS: torizon-core-docker-verdin-imx8mp-Tezi_6.4.0+build.5.tar (on a imx8mp)

The performance of remote access is really poor. Latency is almost one second while typing at the terminal, and transfer speeds are 500KB/s when using SCP on a large tar file.

Is this expected?

Greetings @alexkl,

First of all I assume the device you’re remote accessing is just next to you as you test the feature right? Or is it actually remotely located relative to you?

As for the typing latency, when I use remote-access I do certainly have noticeable latency while typing. I don’t know if we have any hard numbers on expected latency since it probably is affected by many factors.

On the SCP point, is this something you plan to do when your system is deployed? For example do you plan to transfer files remotely regularly like this?

Best Regards,
Jeremias

Yes, the device is right next to me. Performance from speedtest.net for external internet shows 465/468 down/up w/ 2ms latency, and plenty of speed when scp’ing to the device directly.

We’re not entirely sure if the cost of remote access is worth it at this stage in development, but if we were to use it we’d likely occasionally want to transfer data logs from our devices which can get large.

Futhermore - I can’t imagine remote desktop or VNC performance would be too usable at these rates.

Yes, the device is right next to me. Performance from speedtest.net for external internet shows 465/468 down/up w/ 2ms latency, and plenty of speed when scp’ing to the device directly.

Well I would expect direct SCP to be a whole lot quicker than through remote access. Since one is a LAN connection while remote access goes through quite a bit more middle men for the connection to be made.

but if we were to use it we’d likely occasionally want to transfer data logs from our devices which can get large.

I don’t know how your logs will look like. But have you considered using our device monitoring feature or even fluent-bit directly as a way to capture and send these logs rather than SCP via remote -access?

Futhermore - I can’t imagine remote desktop or VNC performance would be too usable at these rates.

I don’t believe we’ve done too much testing ourselves on using remote access for these purposes. I’ll see if I can have someone from our Platform team comment on this.

Best Regards,
Jeremias

After discussing this with our Cloud team here’s some points we came to.

First of all as I said earlier “some” delay is expected, especially if you’re in the US. This is because the servers for all this are located in the EU. That said I am also based in the US and did some tests myself. While I do have a noticeable delay when typing via remote-access it’s more on the order of say 100-200ms rather than the 1s you are reporting. I confirmed similar latency from other colleagues in the US as well.

All that to say, while some latency is expected the level of latency you are reporting here seems exceptionally poor. To that point is there anything you could share about your network setup/configuration? For example firewalls, VPNs, or anything of that sort. Unfortunately, there could be a lot of factors here so unless there’s more information it’s hard to point at a specific thing to investigate.

Now on the point about remote-access with VNC. It turns out some of my colleagues did test this use-case. They did a VNC connection from Italy to a module in Brazil. While the performance wasn’t perfect it was certainly usable for debugging purposes.

This is good news, since this means your use-case should be possible. It’s just a matter of figuring out where this poor latency is coming from.

Best Regards,
Jeremias

1 Like