Hi,
I am trying to resolve a Docker networking issue. One of my applications is a .NET/Blazor web server that I’m developing with Visual Studio Code. I can debug it just fine with the default configuration, but if I change the network mode by adding the “extraparms” parameter “network_mode host”, the debug session gets stuck – i.e. it seems to hang up after I see the “Running application: starting new instance” status message at the bottom right, and the app does not appear to start. I also observe the following:
- The TERMINAL window clears, but the view does not swap to the DEBUG CONSOLE window (which is what I normally see)
- The Debug toolbar does not appear.
- At the top left in the Run and Debug panel, the icon shows a permanent blue clock, and there is a continuously scrolling blue indicator at the top. Here’s a screen shot of this:
- I cannot do anything more until I stop the debugger using Shift-F5.
I have another .NET application that is a gRPC Server and I can debug that one just fine even with the same “network_mode host” parameter added.
Please can you tell me what might be causing this and how to fix it?
Thanks for your time,
Peter
Hi @bcpberry ,
That’s strange. I wasn’t able to reproduce this on a quick test with a Hello World .NET project.
Can you reproduce this issue and copy the logs from the extension backend process? They should be found here:
And also copy the output logs found here:
You can send one text file for each result.
Best regards,
Lucas Akira
Hi Lucas,
Thanks for the response.
I tried copying the extension backend process log, but it doesn’t appear in the list of terminals. Here’s what I see:
I can copy the OUTPUT log – please find it attached.
Output Log.txt (10.2 KB)
I have also attached a copy of the entire Visual Studio solution. The project that I am having this issue with is S36072.OperatorWebServer. I’ve given you the whole solution because there are dependencies. I have deleted all the build files.
Databerry.zip (4.0 MB)
Just to re-cap:
- Without the “network_mode host” setting in the S36072.OperatorWebServer project, it works fine, adding that setting causes the debugger to hang.
- Setting “network_mode host” in another application (S36072.ControllerExecutive – included in the solution) works fine.
Thanks for your help,
Peter
I’m not seeing anything obvious between the two projects that would cause this difference in behavior.
What happens if you run the “S36072.OperatorWebServer” project not in debug mode but in release mode with the “network_mode host” parameter? Does the application/container hang then? Or is it only in debug mode where this occurs?
Best Regards,
Jeremias
Well, … today it works (?!).
But, … now the problem is occurring on the other application (??? !!!).
I tried a number of things, but in summary this is what I can reproduce at the moment:
- Open the OperatorWebServer project, with the “network_mode host” setting, and run the debugger. It works.
- Open the ControllerExecutive project, without the “network_mode host” setting, and run the debugger. It works.
- Add the “network_mode host” setting to the ControllerExecutive project, and run the debugger. It hangs up.
Today, I can see the IDE backend process terminal - here it is at the point of the hang-up:
Backend IDE Log.txt (8.8 KB)
And here is the Output log at the same time:
Output Log 2.txt (5.1 KB)
As per your request in your last post, I tried running it using the “Run Without Debugging” option (I assume that’s the same as “release mode”), and it still hung up. So it seems it is not specifically a debugger issue.
Thanks again,
Peter
As per your request in your last post, I tried running it using the “Run Without Debugging” option (I assume that’s the same as “release mode”), and it still hung up. So it seems it is not specifically a debugger issue.
Given this information it sounds like it’s perhaps the application itself that is hanging?
Since you also shared that the issue now happened to your other application, is it possible that running in host networking mode causes issues for your application stack somehow?
Best Regards,
Jeremias
Hi Jeremias,
Thanks again for your response.
I am not sure how to answer your last question – it is absolutely true that “running in host networking mode causes issues for [my] application stack”. The question is why, and, how to fix it.
After some thinking, a lot of reading, and some more testing. I think I may be partially able to answer the “why” question but am still not sure about how to fix it.
What I have found after more testing, is that if I remove the containers for both applications, and then start again to run either one of the applications on its own, it works fine. But, if I run one, and then run the other, the second one fails to start. I.e., it is always the second one that fails, regardless of which one it is. This occurs with no change to any of the application code, or the “network_mode host” setting that is applied to both of the apps.
I suspect that this has something to do with the port allocations used to communicate between the Verdin containers and the Torizon/Visual Studio Code environment on the PC. It seems that once that environment is managing one container running in “host” mode, it cannot then manage another that is also running in “host” mode. I am guessing that this is caused by some sort of clash because they are both using the host network directly – but this is only a guess, I am certainly no expert on Docker (quite the opposite). I have re-read all the networking docs I can find on your website and can find no mention of any possible problems or limitations using this “host” mode.
So, is it possible for you to confirm that this might be the case? I’m pretty sure that if you were to try this yourself with two “Hello World” apps you would be able to re-produce this.
Best regards,
Peter
Well by default the extension uses port “2222” for debug SSH communication. If you have 2 containers running in debug mode then this would be the same port and obviously a conflict.
However, as you previously stated:
As per your request in your last post, I tried running it using the “Run Without Debugging” option (I assume that’s the same as “release mode”), and it still hung up. So it seems it is not specifically a debugger issue.
So are there other conflicts between the two application containers? Is there an issue if there’s 2 containers running gRPC servers on the host networking stack at the same time?
Also if you need both containers up and running at the same time for debugging, then why not start one container in release mode and then the one you’re working on in debugging mode.
Best Regards,
Jeremias
Hi @jeremias.tx ,
Only one of the containers is running a gRPC server, so there are no conflicting ports in the two apps. I don’t know how the Torizon system is working behind the scenes, but isn’t it using a port to communicate with the container even if it’s not debugging? If so, it seems that this will cause this problem when both apps are using host mode (???)
Also if you need both containers up and running at the same time for debugging, then why not start one container in release mode and then the one you’re working on in debugging mode.
As I said in an earlier post, once one is running the other will not run, regardless of the debug/release mode.
Anyway, after doing all the reading around this issue, and given this problem, I have decided that using this “host” mode is probably not a good idea, and I am now trying to get it all to work using a user defined bridge network instead. Unfortunately I have still have some problems with that, but I will post that in another question.
Thanks for your help,
Peter
I don’t know how the Torizon system is working behind the scenes, but isn’t it using a port to communicate with the container even if it’s not debugging?
If you start the containers from the command line rather than from the VSCode extension then the only ports/network communications are whatever the containers themselves are using. This would be how the containers operate in production anyways.
If i run 2 containers via the command line like so:
# Run twice
docker run --rm -it -d --network host torizon/debian-shell:2 /bin/bash
Then I can see that both containers are connected to the host network and stay up and running. Whatever conflict is happening with your containers is internal to them and not to the system.
Anyway, after doing all the reading around this issue, and given this problem, I have decided that using this “host” mode is probably not a good idea, and I am now trying to get it all to work using a user defined bridge network instead.
What exactly did you need host network mode for then?
Best Regards,
Jeremias
