Kafka operator creates a LoadBalancer service for each kafka replica

[superfly-git]

I have noticed that the Strimzi operator creates a my-cluster-kafka-external-bootstrap service of type load balancer which can be used to access the cluster externally. I tested that and its working.

It also create a separate K8s Service of type LoadBalancer for each kafka pod in the stateful set. If for example i put 3 kafka replicas it will create 3 services of type Load Balancer.

What is the purpose of those additional services/LBs since i have the my-cluster-kafka-external-bootstrap service which load balances among the kafka nodes/pods? This adds additional costs to something that should not be there if its a bug and not a configuration issue i have implemented wrong.

The yaml definition of the cluster is the below

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    replicas: 1
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: external
        port: 9094
        type: loadbalancer
        tls: false
    storage:
      type: ephemeral
    config:
      offsets.topic.replication.factor: 1
      transaction.state.log.replication.factor: 1
      transaction.state.log.min.isr: 1
  zookeeper:
    replicas: 3
    storage:
      type: ephemeral
  entityOperator:
    topicOperator: {}
    userOperator: {}

The service output is the below:

my-cluster-kafka-0                    LoadBalancer   100.71.120.27    {public ip}       9094:31087/TCP               11m
my-cluster-kafka-bootstrap            ClusterIP      100.65.196.121   <none>            9091/TCP,9092/TCP            11m
my-cluster-kafka-brokers              ClusterIP      None             <none>            9091/TCP,9092/TCP            11m
my-cluster-kafka-external-bootstrap   LoadBalancer   100.70.28.16     {public ip}       9094:31725/TCP               11m
my-cluster-zookeeper-client           ClusterIP      100.66.64.160    <none>            2181/TCP                     11m
my-cluster-zookeeper-nodes            ClusterIP      None             <none>            2181/TCP,2888/TCP,3888/TCP   11m

FYI i use DigitalOcean as a cloud provider and the k8s cluster is created using KOPS. I installed the strimzi operator using Helm in a new namespace called kafka.

[scholzj]

The way Kafka is designed, it requires the clients to have direct access to each of the brokers in the cluster. When the client connects, it uses the first connection to get the metadata about the brokers in the cluster. And then opens subsequent connections to the individual brokers where it needs to send / consume messages. So although you connected only over the bootstrap load balancer, the client really used all of them in the background.

[superfly-git]

That clarified things, thanks @scholzj