How to expose Kafka without enabling ssl passthrough on the controller

[gtato]

I am deploying Kafka in a shared K8S cluster where I am not allowed to change the controller configuration.
I was wondering if there is a way handle ssl at the ingress level, without the need to change the controller.

[scholzj]

Not in the Ingress. Kafka is not HTTP connection, so without the TLS passthrough, your Ingress controller will try to decode it and fail. The TLS passthrough allows it to get through the Ingress controller without being decoded and that way you can use it for any TCP+TLS connections. So you either need to have it enabled or you need to use load balancers or node ports for example.

[scholzj]

btw using the --tcp-services-configmap on the controller could also be a solution

Well, you should be able to use it with Strimzi 0.32+ with the type: cluster-ip listener and by creating the Ingress resources manually.

[gtato]

interesting.. sorry for the silly questions (I am relatively new to kafka) but how would you configure that in the config map.
say we have 3 brokers and 4 services all running in 9094:
xxx-kafka-0
xxx-kafka-1
xxx-kafka-2
xxx-kafka-external-bootstrap

would you have only the external bootstrap
like:

data:
  9094: "kafka-strimzi/xxx-kafka-external-bootstrap:9094"

or all of them?
like

data:
  9094: "kafka-strimzi/xxx-kafka-external-bootstrap:9094"
  9094: "kafka-strimzi/xxx-kafka-0:9094"
  9094: "kafka-strimzi/xxx-kafka-1:9094"
  9094: "kafka-strimzi/xxx-kafka-2:9094"

[scholzj]

I don't know, sorry. You need to make sure the clients can access every broker. So you would need to expose each one of them separately and configure in the Kafka CR the addresses in the advertisedHost and advertisedPort fields.

[gtato]

Actually that answered my question, here is my config which is working (in case someone runs into this later)

...
      # the cluster ip listener
      - name: external
        port: 9094
        type: cluster-ip
        tls: true
        authentication:
          type: scram-sha-512
        configuration:
          brokers:
          - broker: 0
            advertisedHost: kafka.example.com
            advertisedPort: 9095
          - broker: 1
            advertisedHost: kafka.example.com
            advertisedPort: 9096
          - broker: 2
            advertisedHost: kafka.example.com
            advertisedPort: 9097

then the config map:

...
data:
  9094: "kafka-strimzi/kafka-cluster-kafka-external-bootstrap:9094"
  9095: "kafka-strimzi/kafka-cluster-kafka-0:9094"
  9096: "kafka-strimzi/kafka-cluster-kafka-1:9094"
  9097: "kafka-strimzi/kafka-cluster-kafka-2:9094"

by pointing kafka.example.com to the controller IP, this should be working.

Thanks Jakub, and great job with this operator btw!

[scholzj]

That is great. Thanks for sharing it back!