Assign public IPs to workers in multi-cloud cluster

[jakoberpf]

Hey peoples,

I am stuck on expanding a multi-cloud deployment with some worker nodes in the Oracle Cloud. My controlplane is running in Hetzner and I have some Proxmox VMs, Strato VServers and AWS EC2s as workers, all work great. Now I want to add some Oracle VMs. If I run talosctl get --insecure nodeaddresses --nodes 123.123.123.123 (123.123.123.123 is a Oracle VM booted up with TalosOS) I get

NODE   NAMESPACE   TYPE          ID             VERSION   ADDRESSES
       network     NodeAddress   accumulative   2         ["10.0.12.84/24"]
       network     NodeAddress   current        2         ["10.0.12.84/24"]
       network     NodeAddress   default        1         ["10.0.12.84/24"]
       network     NodeAddress   routed         2         ["10.0.12.84/24"]

because the networking on Oracle, the VM does not know it public IP. I can assign the public IP to the certSANs

  certSANs:
    - 10.0.12.84
    - 123.123.123.123

to make the certificate working, but once I apply the machine config talosctl apply-config --nodes 123.123.123.123 --file worker-123.123.123.123.yaml --insecure the API is not reachable anymore because Talos is not configured to listen on the public IP 123.123.123.123 and it does not know about it from the nodeaddresses.

talosctl edit machineconfig --nodes 123.123.123.123
# rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 123.123.123.123:50000: i/o timeout"

How can I provider this public IP in the machine configuration? I tried different variations of (only the custom part of the config)

version: v1alpha1
debug: true
persist: true
machine:
  type: worker
  certSANs:
    - 10.0.12.84
    - 123.123.123.123
  kubelet:
    nodeIP:
      validSubnets:
        - 123.123.123.123/32
  network:
    interfaces:
      - interface: enp0s6
        addresses:
          - 10.0.12.84/24
          - 123.123.123.123/32
        dhcp: true
    kubespan:
      enabled: true

[smira]

because the networking on Oracle, the VM does not know it public IP.

If using oracle image of Talos, it should automatically pull its public IP.

rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 123.123.123.123:50000: i/o timeout"

This depends on Oracle Cloud, not Talos. Usually in the clouds public IPs are not assigned to the machine, but rather mapped 1:1 on the infrastructure side.

The way you assigned public IP is correct for Talos Linux, but not sure how it should work with Oracle Linux.