NAT Traversal? #8338
Replies: 3 comments 7 replies
-
|
KubeSpan requires one-way connectivity between all machines, so your homelab can be perfectly behind a NAT/firewall, as long as they can connect to each other and Azure workers (over public IP), you're good. |
Beta Was this translation helpful? Give feedback.
-
|
As a simple demonstration, maybe we can run Talos at a local computer except on different VM platforms, e.g. run one of them in Hyper-V and one of them in VirtualBox (both using NAT network) to simulate. Scale up if you want to |
Beta Was this translation helpful? Give feedback.
-
|
What I said above still stays, if there's no way to connect to other machine's IP at least one way, KubeSpan won't help. E.g. if you deploy Azure VMs with public IP and homelab behind NAT, it's fine. |
Beta Was this translation helpful? Give feedback.
-
|
Well, the point is I do have public IPs, except they are shared between different machines behind the scene. In Azure, I have a NAT gateway with a public IP of 128.0.0.1 And in Azure I have two Talos VMs in 172.35.0.1 and 172.35.0.2, and they both do not have a public IP on their network interface. As such all the Internet traffic will go through the NAT gateway and masqueraded as 128.0.0.1, thus both VMs would appear to have the same KubeSpan endpoint of 128.0.0.1:51820. How does both Azure VMs and home network VMs reach each other? So this is why NAT traversal is needed. At the very least we need to try UDP hole punching. Port forwarding would be difficult though. Of course, if we have IPv6...that's another story. |
Beta Was this translation helpful? Give feedback.
-
|
Talos needs to know how to connect. Say host A has IPs IP_A_1, IP_A_2, etc. Talos will try all of them with the port 51820. If the host has a public IP mapped, Talos will learn it. Talos will not see a hole punched in the NAT. KubeSpan doesn't have a real NAT traversal, nor it has any proxy in the middle atm. |
Beta Was this translation helpful? Give feedback.
-
|
Exactly. We are missing the feature to do proper NAT traversal. |
Beta Was this translation helpful? Give feedback.
-
|
One end of the network connection needs to be directly reachable.
So you can have multiple hosts behind your home NAT router - the outgoing
traffic will have distinct source UDP ports, even if all going to 51820 on
the publicly addressed host.
port 51820 does not need to be reachable on all hosts - just in one
direction.
…On Wed, Feb 21, 2024 at 10:11 AM Steve Fan ***@***.***> wrote:
Well, the point is I do have public IPs, except they are shared between
different machines behind the scene.
In Azure, I have a NAT gateway with a public IP of 128.0.0.1
And my home router has a public IP of 129.0.0.1. In the home network I
have two VMs in 192.168.11.1 and 192.168.11.2. Since most home users would
only have one public IP, and would not have a chance to get an assignment.
So the KubeSpan endpoint for both home VMs would be 129.0.0.1:51820,
right?
And in Azure I have two Talos VMs in 172.35.0.1 and 172.35.0.2, and they
both do not have a public IP on their network interface. As such all the
Internet traffic will go through the NAT gateway and masqueraded as
128.0.0.1, thus both VMs would appear to have the same KubeSpan endpoint of
128.0.0.1:51820.
So this is why NAT traversal is needed. At the very least we need to try
UDP hole punching.
Of course, if we have IPv6...that's another story.
—
Reply to this email directly, view it on GitHub
<#8338 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AQGWG5IDOHL5CWCTVIMNDXTYUY2E5AVCNFSM6AAAAABDTNSSFGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DKNBXGIYDQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
I have just tried out my settings, and as I predicted, the connectivity between each KubeSpan node is very fragile at it its best against NAT'd nodes, that most of the time after rebooting the node, you lost the connection to the NAT node almost guaranteed. |
Beta Was this translation helpful? Give feedback.
-
|
So yeah I think Talos really need to improve on this, I see that after applying some KubeSpan address filters, the specs and status are not immediately visible |
Beta Was this translation helpful? Give feedback.
-
I want to experiment Talos between my home and my Azure network over IPv4, and my home network is obviously having a NAT with a private network range behind a router. More specifically, I want to have my homelab 1 and homelab 2, hidden behind my home network as 192.168.0.1 and 192.168.0.2 respectively, to be able to connect to my Azure machine as 10.10.10.1. So, having a VPN, or rather an overlay network is on the right track and KubeSpan is clearly the choice here.
However, far as I can tell, KubeSpan actually requires public IP addresses for the peers to discover each other (which to be honest, covers 90% of the cases for Kubernetes deployment), and so while it is likely that Wireguard can run as a one-armed bandit if one side has a public IP and the other side doesn't (but that means you would have a unidirectional channel which could slow things down), but if both sides doesn't, no dice.
Adding salt to injury, most home network routers only support symmetric NAT (you can understand it as iptables SNAT/Masquerade if you want to make things easier), so the source port can be rewritten at router level when sending out. Therefore, a proper NAT traversal solution is needed for KubeSpan, similar to the kind of Tailscale or NetBird, and use TURN/ICE as a last resort.
Beta Was this translation helpful? Give feedback.
All reactions