From f00f82fca10360b4c8fc4f581bf78c58e5caf2ba Mon Sep 17 00:00:00 2001 From: hpgrahsl Date: Tue, 14 May 2024 08:17:33 +0000 Subject: [PATCH] =?UTF-8?q?[CI]=20Publish=20Documentation=20for=20daea12a5?= =?UTF-8?q?ff3be3a46caeb48875e81e82b5e0d7cb=20-=20daea12a5ff3be3a46caeb488?= =?UTF-8?q?75e81e82b5e0d7cb=20=F0=9F=9A=80?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- kafka-tutorial/01-setup.html | 115 ++++- kafka-tutorial/02-topics-partitions.html | 82 +++- kafka-tutorial/03-consumers-producers.html | 114 ++++- kafka-tutorial/04-java-consumer-producer.html | 315 ++++++++++--- kafka-tutorial/05-kstreams.html | 430 ++++++++++++++--- kafka-tutorial/06-java-kstreams.html | 433 +++++++++++------- kafka-tutorial/07-kubernetes.html | 173 +++---- kafka-tutorial/08-kafka-listeners.html | 58 +-- sitemap.xml | 20 +- 9 files changed, 1267 insertions(+), 473 deletions(-) diff --git a/kafka-tutorial/01-setup.html b/kafka-tutorial/01-setup.html index 708af2d..d7787d9 100644 --- a/kafka-tutorial/01-setup.html +++ b/kafka-tutorial/01-setup.html @@ -470,37 +470,88 @@

Run Kafka

-

Assuming you’ve got Docker & Docker Compose installed.

+

Assuming you’ve got either Docker (Desktop) & Docker Compose installed or Podman (Desktop) & Podman Compose installed

+
+
+
    +
  • +

    Podman

    +
    • +
  • +

    Docker

    +
    • +
+
+
+
cd $TUTORIAL_HOME/apps/it
-docker compose up --remove-orphans
+podman compose up +
+
+
+
+
+
+
cd $TUTORIAL_HOME/apps/it
+docker compose up
+
+
+
...
 
-kafka      | [2022-12-09 08:29:01,867] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
-kafka      | [2022-12-09 08:29:01,878] INFO [MetadataCache brokerId=0] Updated cache from existing <empty> to latest FinalizedFeaturesAndEpoch(features=Map(), epoch=0). (kafka.server.metadata.ZkMetadataCache)
-kafka      | [2022-12-09 08:29:01,893] INFO [ExpirationReaper-0-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
-kafka      | [2022-12-09 08:29:01,929] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
-kafka      | [2022-12-09 08:29:01,946] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Enabling request processing. (kafka.network.SocketServer)
-kafka      | [2022-12-09 08:29:01,974] INFO Kafka version: 3.3.1 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-09 08:29:01,974] INFO Kafka commitId: e23c59d00e687ff5 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-09 08:29:01,974] INFO Kafka startTimeMs: 1670574541958 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-09 08:29:01,975] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
-kafka      | [2022-12-09 08:29:02,093] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
-kafka      | [2022-12-09 08:29:02,113] INFO [BrokerToControllerChannelManager broker=0 name=alterPartition]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
+[2024-05-13 12:46:44,110] INFO [SocketServer listenerType=BROKER, nodeId=1] Enabling request processing. (kafka.network.SocketServer) +[2024-05-13 12:46:44,110] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor) +[2024-05-13 12:46:44,111] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor) +[2024-05-13 12:46:44,112] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer) +[2024-05-13 12:46:44,112] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer) +[2024-05-13 12:46:44,112] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer) +[2024-05-13 12:46:44,112] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer) +[2024-05-13 12:46:44,112] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer) +[2024-05-13 12:46:44,112] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser) +[2024-05-13 12:46:44,112] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser) +[2024-05-13 12:46:44,112] INFO Kafka startTimeMs: 1715604404112 (org.apache.kafka.common.utils.AppInfoParser) +[2024-05-13 12:46:44,113] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)

Open a new terminal to run Kafka commands:

-

Verify that Zookeeper and Apache Kafka containers are up and running:

+

Verify that the Apache Kafka container is up and running:

+
+
+
+
    +
  • +

    Podman

    +
    • +
  • +

    Docker

    +
    • +
+
+
+
+
+
+
podman ps
+
+
+
+
+
CONTAINER ID  IMAGE                                     COMMAND               CREATED             STATUS             PORTS                                             NAMES
+8558952ed132  quay.io/strimzi/kafka:0.40.0-kafka-3.7.0  sh -c ./bin/kafka...  About a minute ago  Up About a minute  0.0.0.0:9092->9092/tcp, 0.0.0.0:29092->29092/tcp  kafka
+
+
+
docker ps
@@ -509,8 +560,10 @@

Run Kafka

CONTAINER ID   IMAGE                                      COMMAND                  CREATED         STATUS         PORTS                                              NAMES
-0d4bc6ec2ddf   quay.io/strimzi/kafka:0.32.0-kafka-3.3.1   "sh -c 'bin/kafka-se…"   2 minutes ago   Up 2 minutes   0.0.0.0:9092->9092/tcp, 0.0.0.0:29092->29092/tcp   kafka
-d8fdbab971c6   quay.io/strimzi/kafka:0.32.0-kafka-3.3.1   "sh -c 'bin/zookeepe…"   2 minutes ago   Up 2 minutes   0.0.0.0:2181->2181/tcp                             zookeeper
+0d4bc6ec2ddf quay.io/strimzi/kafka:0.40.0-kafka-3.7.0 "sh -c ./bin/kafka..." 2 minutes ago Up 2 minutes 0.0.0.0:9092->9092/tcp, 0.0.0.0:29092->29092/tcp kafka +
+
+
@@ -532,15 +585,18 @@

Run Kafka

  • -

    kcat

    +

    kcat

  • -

    kcat in Docker

    +

    kcat in Podman

    +
    • +
  • +

    kcat in Docker

-
+
kcat -b localhost:29092 -L
@@ -555,7 +611,22 @@

Run Kafka

-
+
+
+
+
podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 -b kafka:9092 -L
+
+
+
+
+
Metadata for all topics (from broker 1: kafka:9092/1):
+ 1 brokers:
+  broker 1 at kafka:9092 (controller)
+ 0 topics:
+
+
+
+
docker run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 -b kafka:9092 -L
@@ -563,9 +634,9 @@

Run Kafka

-
Metadata for all topics (from broker 0: kafka:9092/0):
+
Metadata for all topics (from broker 1: kafka:9092/1):
  1 brokers:
-  broker 0 at kafka:9092 (controller)
+  broker 1 at kafka:9092 (controller)
  0 topics:
diff --git a/kafka-tutorial/02-topics-partitions.html b/kafka-tutorial/02-topics-partitions.html index c9ed8ab..dc7d884 100644 --- a/kafka-tutorial/02-topics-partitions.html +++ b/kafka-tutorial/02-topics-partitions.html @@ -458,9 +458,32 @@

Topic Creat

This tool is bundled with the Kafka container image, so let’s exec a bash terminal inside the running Kafka container.

-
+
+
+
    +
  • +

    Podman

    +
    • +
  • +

    Docker

    +
    • +
+
+
+
+
-
docker exec -it $(docker ps -q --filter "label=com.docker.compose.service=kafka") /bin/bash
+
podman exec -it kafka /bin/bash
+
+
+
+
+
+
+
docker exec -it kafka /bin/bash
+
+
+
@@ -468,8 +491,17 @@

Topic Creat

-
./bin/kafka-topics.sh --create --bootstrap-server kafka:9092 --replication-factor 1 --partitions 1 --topic songs
-./bin/kafka-topics.sh --list --bootstrap-server kafka:9092
+
./bin/kafka-topics.sh --create --bootstrap-server kafka:9092 --replication-factor 1 --partitions 1 --topic songs
+
+
+
+
+
Created topic songs.
+
+
+
+
+
./bin/kafka-topics.sh --list --bootstrap-server kafka:9092
@@ -478,7 +510,7 @@

Topic Creat

-

Now that you have valdiated the topic exists you can run the exit to leave the container

+

Now that you have validated the topic exists you can run exit to leave the container

@@ -497,15 +529,18 @@

Get Topic informati
  • -

    kcat

    +

    kcat

    +
    • +
  • +

    kcat in Podman

  • -

    kcat in Docker

    +

    kcat in Docker

-
+
kcat -b localhost:29092 -L
@@ -513,16 +548,33 @@

Get Topic informati

-
Metadata for all topics (from broker 0: localhost:29092/0):
+
Metadata for all topics (from broker 1: localhost:29092/1):
+ 1 brokers:
+  broker 1 at localhost:29092 (controller)
+ 1 topics:
+  topic "songs" with 1 partitions:
+    partition 0, leader 1, replicas: 1, isrs: 1

+
+
+
+
+
+
+
podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -L
+
+
+
+
+
Metadata for all topics (from broker 1: kafka:9092/1):
  1 brokers:
-  broker 0 at localhost:29092 (controller)
+  broker 1 at kafka:9092 (controller)
  1 topics:
   topic "songs" with 1 partitions:
-    partition 0, leader 0, replicas: 0, isrs: 0
+ partition 0, leader 1, replicas: 1, isrs: 1
-
+
docker run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -L
@@ -530,12 +582,12 @@

Get Topic informati

-
Metadata for all topics (from broker 0: kafka:9092/0):
+
Metadata for all topics (from broker 1: kafka:9092/1):
  1 brokers:
-  broker 0 at kafka:9092 (controller)
+  broker 1 at kafka:9092 (controller)
  1 topics:
   topic "songs" with 1 partitions:
-    partition 0, leader 0, replicas: 0, isrs: 0

+    partition 0, leader 1, replicas: 1, isrs: 1
diff --git a/kafka-tutorial/03-consumers-producers.html b/kafka-tutorial/03-consumers-producers.html index d55b206..bf01c87 100644 --- a/kafka-tutorial/03-consumers-producers.html +++ b/kafka-tutorial/03-consumers-producers.html @@ -391,6 +391,9 @@

Consuming messa

kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -403,6 +406,13 @@

    Consuming messa

    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -C -K:
    +
    +
    +
    @@ -440,6 +450,9 @@

    Producing messa

    kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -452,6 +465,13 @@

    Producing messa

    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial -v $(pwd)/documentation/modules/ROOT/examples/initial_songs.json:/home/initial_songs edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -P -l -K: /home/initial_songs
    +
    +
    +
    @@ -495,6 +515,9 @@

    Pla

    kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -507,6 +530,13 @@

    Pla

    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -C -K:
    +
    +
    +
    @@ -539,6 +569,9 @@

    Pla

    kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -551,6 +584,13 @@

    Pla

    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -o 1 -C -K:
    +
    +
    +
    @@ -584,9 +624,32 @@

    Chang

    This tool is bundled with the Kafka container image, so let’s exec a bash terminal inside the running Kafka container.

    -
    +
    +
    +
      +
    • +

      Podman

      +
      • +
    • +

      Docker

      +
      • +
    +
    -
    docker exec -it $(docker ps -q --filter "label=com.docker.compose.service=kafka") /bin/bash
    +
    +
    +
    +
    podman exec -it kafka /bin/bash
    +
    +
    +
    +
    +
    +
    +
    docker exec -it kafka /bin/bash
    +
    +
    +
    @@ -623,22 +686,32 @@

    Chang
    • -

      kcat

      +

      kcat

      +
      • +
    • +

      kcat in Podman

    • -

      kcat in Docker

      +

      kcat in Docker

    -
    +
    kcat -b localhost:29092 -t songs -C -K:
    -
    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -C -K:
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -C -K:
    @@ -659,7 +732,7 @@

    Chang

    Also note, that Kafka brokers have a configuration for log.retention.check.interval.ms and this setting must be lower than/equal to a topic’s retention.ms, otherwise cleaning out offsets will not be effective "immeadiately" but only after the log cleaner ran to do its job. In other words, it may take longer (up to log.retention.check.interval.ms) before you can actually observe the data retention happening in the corresponding Kafka topic.

    -

    Change retention time to the default one by running in the third terminal (the one you did docker exec), the following command:

    +

    Change retention time to the default one by running in the third terminal (the one you did either podman | docker exec …​) the following command:

    @@ -680,9 +753,32 @@

    Del

    This tool is bundled with the Kafka container image, so let’s exec a bash terminal inside the running Kafka container.

    -
    +
    +
    +
      +
    • +

      Podman

      +
      • +
    • +

      Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman exec -it kafka /bin/bash
    +
    +
    +
    +
    +
    -
    docker exec -it $(docker ps -q --filter "label=com.docker.compose.service=kafka") /bin/bash
    +
    docker exec -it kafka /bin/bash
    +
    +
    +
    diff --git a/kafka-tutorial/04-java-consumer-producer.html b/kafka-tutorial/04-java-consumer-producer.html index e2603f2..3b263ec 100644 --- a/kafka-tutorial/04-java-consumer-producer.html +++ b/kafka-tutorial/04-java-consumer-producer.html @@ -323,8 +323,17 @@

    Developing Consumers and Producers in Java

    The first service acts as a Kafka producer to write song related information (e.g. id, author, name) into the songs topic. Then there is a second service acting as a Kafka consumer which reads this data from the songs topic. Usually this consumer would perform some "real business logic" such as taking the songs data and process it somehow, e.g. adding songs in a graph database like Neo4J, or build a fulltext search index based on Elastic. However, for the sake of simplicity, our service is just printing the consumed data to the console and exposes every consumed Kafka record over a server-sent event (SSE) stream for connected HTTP clients.

    -
    -

    Remember you need to have both, ZooKeeper and Apache Kafka correctly configured as well as up and running. If not, you can follow the [Setup Kafka] section to learn how to do it using docker compose.

    +
    + + + + + +
    + + +Remember you need to have Apache Kafka successfully up and running. If not, you can follow the Kafka Setup section to learn how to do it using podman | docker compose. +

    It’s best to have 4 terminal windows open for running this example, 2 terminals for the producer and consumer services, and another 2 terminals for sending and receiving HTTP requests/responses.

    @@ -346,66 +355,114 @@

    Producing mes

    Deploying Producer

    In this case, the Spring Boot service is deployed to produce songs. -You’ve got two options, using Docker or building the service yourself from the sources. -In terminal 1, run one of these two options:

    +You’ve got different options, using either pre-built container images run with Podman / Docker or building the application and image on your local machine from the sources. +In terminal 1, run one of these options:

    • -

      Building & Run

      +

      [Optional] Local Build & Run Podman

    • -

      Docker

      +

      [Optional] Local Build & Run Docker

    -
    +
    -
    cd $TUTORIAL_HOME/apps/song-app/springboot/song-app
-./mvnw clean package -DskipTests
    +
    cd $TUTORIAL_HOME/apps/song-app/springboot/song-app
    -
    docker build -t quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:latest .
    +
    ./mvnw clean package -DskipTests
    -
    docker run --rm --network=kafka-tutorial -p 8080:8080 quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:latest
    +
    podman build -t quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:latest .
    -
    +
    -
    2022-12-19T12:31:01.985Z  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
-2022-12-19T12:31:01.986Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 646 ms
-2022-12-19T12:31:02.334Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
-2022-12-19T12:31:02.349Z  INFO 1 --- [           main] org.acme.song.app.SongApplication        : Started SongApplication in 1.382 seconds (process running for 1.821)
    +
    podman run --rm -it -p 8080:8080 --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:latest
    -
    +
    -
    docker run --rm --network=kafka-tutorial -p 8080:8080 quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:v22.12
    +
    cd $TUTORIAL_HOME/apps/song-app/springboot/song-app
    -
    +
    -
    2022-12-19T12:31:01.985Z  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
-2022-12-19T12:31:01.986Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 646 ms
-2022-12-19T12:31:02.334Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
-2022-12-19T12:31:02.349Z  INFO 1 --- [           main] org.acme.song.app.SongApplication        : Started SongApplication in 1.382 seconds (process running for 1.821)
    +
    ./mvnw clean package -DskipTests
    +
    +
    +
    docker build -t quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:latest .
    +
    +
    +
    +
    +
    docker run --rm -it -p 8080:8080 --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:latest
    +
    +
    +
      +
    • +

      Pre-built image with Podman

      +
      • +
    • +

      Pre-built image with Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman run --rm -it -p 8080:8080 --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:v22.12-xa
    +
    +
    +
    +
    +
    +
    +
    docker run --rm -it -p 8080:8080 --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-song-app-springboot:v22.12-xa
    +
    +
    +
    +
    +
    +
    +

    Console output when running the containerized application looks as follows:

    +
    +
    +
    +
    ...
+2024-05-13T14:57:49.489Z  INFO 1 --- [           main] org.acme.song.app.SongApplication        : Starting SongApplication v0.0.1-SNAPSHOT using Java 17.0.6 with PID 1 (/deployments/song-app.jar started by jboss in /deployments)
+2024-05-13T14:57:49.491Z  INFO 1 --- [           main] org.acme.song.app.SongApplication        : The following 1 profile is active: "prod"
+2024-05-13T14:57:49.955Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
+2024-05-13T14:57:49.960Z  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
+2024-05-13T14:57:49.960Z  INFO 1 --- [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.1]
+2024-05-13T14:57:49.993Z  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
+2024-05-13T14:57:49.994Z  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 475 ms
+2024-05-13T14:57:50.222Z  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
+2024-05-13T14:57:50.231Z  INFO 1 --- [           main] org.acme.song.app.SongApplication        : Started SongApplication in 0.931 seconds (process running for 1.148)
    +
    +
    +
    +

    Consuming messages with Java

    @@ -417,72 +474,115 @@

    Consuming mes

    Deploying Consumer

    In this case, the Spring Boot service is deployed to consume songs. -You’ve got two options, using Docker or building the service yourself from the sources. -In terminal 2, run one of these two options:

    +You’ve got different options, using either pre-built container images run with Podman / Docker or building the application and image on your local machine from the sources. +In terminal 2, run one of these options:

    • -

      Build & Run

      +

      [Optional] Local Build & Run Podman

    • -

      Docker

      +

      [Optional] Local Build & Run Docker

    -
    +
    -
    cd $TUTORIAL_HOME/apps/song-indexer-app/springboot/song-indexer-app
-./mvnw clean package -DskipTests
    +
    cd $TUTORIAL_HOME/apps/song-indexer-app/springboot/song-indexer-app
    -
    docker build -t quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:latest .
    +
    ./mvnw clean package -DskipTests
    -
    docker run --rm --network=kafka-tutorial -p 9090:8080 quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:latest
    +
    podman build -t quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:latest .
    -
    +
    -
    2022-12-15T09:35:14.545+01:00  INFO 43531 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] (Re-)joining group
-2022-12-15T09:35:14.549+01:00  INFO 43531 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Successfully joined group with generation Generation{generationId=35, memberId='consumer-G1-1-678970e3-7deb-4679-a469-10b1b53a200c', protocol='range'}
-2022-12-15T09:35:14.550+01:00  INFO 43531 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Finished assignment for group at generation 35: {consumer-G1-1-678970e3-7deb-4679-a469-10b1b53a200c=Assignment(partitions=[songs-0])}
-2022-12-15T09:35:14.555+01:00  INFO 43531 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Successfully synced group in generation Generation{generationId=35, memberId='consumer-G1-1-678970e3-7deb-4679-a469-10b1b53a200c', protocol='range'}
-2022-12-15T09:35:14.555+01:00  INFO 43531 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Notifying assignor about the new Assignment(partitions=[songs-0])
-2022-12-15T09:35:14.557+01:00  INFO 43531 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Adding newly assigned partitions: songs-0
-2022-12-15T09:35:14.564+01:00  INFO 43531 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Setting offset for partition songs-0 to the committed offset FetchPosition{offset=49, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[localhost:29092 (id: 0 rack: null)], epoch=0}}
-2022-12-15T09:35:14.565+01:00  INFO 43531 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : G1: partitions assigned: [songs-0]
    +
    podman run --rm -it -p 9090:8080 --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:latest
    -
    +
    -
    docker run --rm --network=kafka-tutorial -p 9090:8080 quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:v22.12
    +
    cd $TUTORIAL_HOME/apps/song-app/springboot/song-app
    -
    +
    +
    +
    ./mvnw clean package -DskipTests
    +
    +
    +
    +
    +
    docker build -t quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:latest .
    +
    +
    +
    -
    2022-12-15T09:44:02.594+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Finished assignment for group at generation 1: {consumer-G1-1-95c3358a-e4a1-4d93-bd10-37453f8d21ed=Assignment(partitions=[songs-0])}
-2022-12-15T09:44:02.632+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Successfully synced group in generation Generation{generationId=1, memberId='consumer-G1-1-95c3358a-e4a1-4d93-bd10-37453f8d21ed', protocol='range'}
-2022-12-15T09:44:02.633+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Notifying assignor about the new Assignment(partitions=[songs-0])
-2022-12-15T09:44:02.637+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Adding newly assigned partitions: songs-0
-2022-12-15T09:44:02.646+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Found no committed offset for partition songs-0
-2022-12-15T09:44:02.651+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Found no committed offset for partition songs-0
-2022-12-15T09:44:02.660+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.SubscriptionState    : [Consumer clientId=consumer-G1-1, groupId=G1] Resetting offset for partition songs-0 to position FetchPosition{offset=0, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[localhost:29092 (id: 0 rack: null)], epoch=0}}.
-2022-12-15T09:44:02.682+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : G1: partitions assigned: [songs-0]
    +
    docker run --rm -it -p 9090:8080 --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:latest
    +
    +
    +
      +
    • +

      Pre-built image with Podman

      +
      • +
    • +

      Pre-built image with Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman run --rm -it -p 9090:8080 --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:v22.12-xa
    +
    +
    +
    +
    +
    +
    +
    docker run --rm -it -p 9090:8080 --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-song-indexer-app-springboot:v22.12-xa
    +
    +
    +
    +
    +
    +
    +
    +
    ...
+2024-05-13T15:23:11.572Z  INFO 1 --- [           main] o.a.s.i.app.SongIndexerApplication       : Started SongIndexerApplication in 1.215 seconds (process running for 1.456)
+2024-05-13T15:23:11.722Z  INFO 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.Metadata        : [Consumer clientId=consumer-G1-1, groupId=G1] Resetting the last seen epoch of partition songs-0 to 0 since the associated topicId changed from null to RV2jwIqHTqCXddmgtb1stw
+2024-05-13T15:23:11.724Z  INFO 1 --- [ntainer#0-0-C-1] org.apache.kafka.clients.Metadata        : [Consumer clientId=consumer-G1-1, groupId=G1] Cluster ID: DIAO2SH9SdGuqBkcKYXtiw
+2024-05-13T15:23:11.725Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Discovered group coordinator kafka:9092 (id: 2147483646 rack: null)
+2024-05-13T15:23:11.726Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] (Re-)joining group
+2024-05-13T15:23:11.737Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Request joining group due to: need to re-join with the given member-id: consumer-G1-1-802608ee-06dd-4671-9cc6-1449f30473fa
+2024-05-13T15:23:11.738Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Request joining group due to: rebalance failed due to 'The group member needs to have a valid member id before actually entering a consumer group.' (MemberIdRequiredException)
+2024-05-13T15:23:11.738Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] (Re-)joining group
+2024-05-13T15:23:14.743Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Successfully joined group with generation Generation{generationId=3, memberId='consumer-G1-1-802608ee-06dd-4671-9cc6-1449f30473fa', protocol='range'}
+2024-05-13T15:23:14.746Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Finished assignment for group at generation 3: {consumer-G1-1-802608ee-06dd-4671-9cc6-1449f30473fa=Assignment(partitions=[songs-0])}
+2024-05-13T15:23:14.751Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Successfully synced group in generation Generation{generationId=3, memberId='consumer-G1-1-802608ee-06dd-4671-9cc6-1449f30473fa', protocol='range'}
+2024-05-13T15:23:14.752Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Notifying assignor about the new Assignment(partitions=[songs-0])
+2024-05-13T15:23:14.754Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Adding newly assigned partitions: songs-0
+2024-05-13T15:23:14.761Z  INFO 1 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-G1-1, groupId=G1] Setting offset for partition songs-0 to the committed offset FetchPosition{offset=0, offsetEpoch=Optional.empty, currentLeader=LeaderAndEpoch{leader=Optional[kafka:9092 (id: 1 rack: null)], epoch=0}}
+2024-05-13T15:23:14.762Z  INFO 1 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : G1: partitions assigned: [songs-0]
    +
    +

    Providing Songs

    @@ -509,7 +609,7 @@

    Providing S 
    HTTP/1.1 201
 Connection: keep-alive
 Content-Length: 0
-Date: Thu, 15 Dec 2022 08:46:57 GMT
+Date: Mon, 13 May 2024 15:26:57 GMT
 Keep-Alive: timeout=60

    @@ -518,9 +618,9 @@

    Providing S

    -
    2022-12-15T09:47:08.234+01:00  INFO 45680 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : G1: partitions assigned: [songs-0]
+
    2024-05-13T15:23:14.762Z  INFO 1 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : G1: partitions assigned: [songs-0]
 Song[id=1000, name=Portals, author=Alan Silvestri, op=ADD] indexed.
-2022-12-15T09:49:09.068+01:00  INFO 46855 --- [ntainer#0-0-C-1] reactor.Flux.SinkManyBestEffort.1        : onNext(ServerSentEvent [id = '6fad2b6f-6400-45a1-9dc9-8bcfed7b5399', event='null', retry=null, comment='null', data=Song 1000 processed])
    
+2024-05-13T15:26:57.899Z  INFO 1 --- [ntainer#0-0-C-1] reactor.Flux.SinkManyBestEffort.1        : onNext(ServerSentEvent [id = 'ee19132e-ae48-4c2b-8df9-e42939d19039', event='null', retry=null, comment='null', data=Song 1000 processed])
    @@ -535,7 +635,7 @@

    Providing S Vary: Access-Control-Request-Headers transfer-encoding: chunked -id:6fad2b6f-6400-45a1-9dc9-8bcfed7b5399 +id:ee19132e-ae48-4c2b-8df9-e42939d19039 data:Song 1000 processed

    @@ -546,34 +646,101 @@

    Providing S

    Clean Up

    -

    Stop the processes that are running in the terminal 1 and 2 by typing Ctrl+C.

    +

    Stop the two application containers that are running in the terminal 1 and 2 by typing Ctrl+C.

    -

    To restart Kafka (and ZooKeeper) go to the docker compose terminal and stop the process by typing Ctrl+C.

    +

    To restart Kafka go to the podman | docker compose terminal and stop the process by typing Ctrl+C.

    Then run:

    -
    +
    +
    +
      +
    • +

      Podman

      +
      • +
    • +

      Docker

      +
      • +
    +
    -
    docker compose rm
-
-? Going to remove kafka, zookeeper (y/N) -> y
-[+] Running 2/0
- ⠿ Container zookeeper  Removed
- ⠿ Container kafka      Removed
    +
    +
    +
    +
    podman compose down
    -
    +
    -
    docker compose up --remove-orphans
-
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka version: 3.3.1 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka commitId: e23c59d00e687ff5 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka startTimeMs: 1671094765681 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,689] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
-kafka      | [2022-12-15 08:59:25,750] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
-kafka      | [2022-12-15 08:59:25,751] INFO [BrokerToControllerChannelManager broker=0 name=alterPartition]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
    +
    podman-compose version: 1.0.6
+['podman', '--version', '']
+using podman version: 5.0.1
+** excluding:  set()
+podman stop -t 10 kafka
+kafka
+exit code: 0
+podman rm kafka
+kafka
+exit code: 0
    +
    +
    +
    +
    +
    podman compose up
    +
    +
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    +
    +
    +
    +
    docker compose down
    +
    +
    +
    +
    +
    ...
    +
    +
    +
    +
    +
    docker compose up
    +
    +
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    diff --git a/kafka-tutorial/05-kstreams.html b/kafka-tutorial/05-kstreams.html index 926d2f2..ad8ef40 100644 --- a/kafka-tutorial/05-kstreams.html +++ b/kafka-tutorial/05-kstreams.html @@ -397,8 +397,17 @@

    KStreamscat

    Pre-built releases can be downloaded from the GitHub repository’s release page. In case you want to work with the latest version simply clone the repo and and build the tool by running ./mvnw package. The target directory will contain a runnable JAR file named kstreamscat-<version>-jar-with-dependencies.jar.

    -
    -

    Remember you need to have both, ZooKeeper and Apache Kafka correctly configured as well as up and running. If not, you can follow the [Setup Kafka] section to learn how to do it using docker compose.

    +
    + + + + + +
    + + +Remember you need to have Apache Kafka successfully up and running. If not, you can follow the Kafka Setup section to learn how to do it using podman | docker compose. +
    @@ -428,6 +437,9 @@

    (Gl

    kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -440,6 +452,13 @@

    (Gl

    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial -v $(pwd)/documentation/modules/ROOT/examples/update_songs.json:/home/update_songs edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -P -l -K: /home/update_songs
    +
    +
    +
    @@ -459,6 +478,9 @@

    (Gl

    kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -471,6 +493,13 @@

    (Gl

    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -C -K:
    +
    +
    +
    @@ -534,6 +563,9 @@

    (Gl

    kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -546,6 +578,13 @@

    (Gl

    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial -v $(pwd)/documentation/modules/ROOT/examples/update2_songs.json:/home/update2_songs edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -P -l -K: /home/update2_songs
    +
    +
    +
    @@ -565,6 +604,9 @@

    (Gl

    kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -577,10 +619,17 @@

    (Gl

    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -C -K:
    +
    +
    +
    -
    docker run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:29092 -t songs -C -K:
    +
    docker run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 kcat -b kafka:9092 -t songs -C -K:
    @@ -627,31 +676,98 @@

    (Gl

    Clean Up

    -

    To restart Kafka (and ZooKeeper) go to the docker compose terminal and stop the process by typing Ctrl+C.

    +

    To restart Kafka go to the podman | docker compose terminal and stop the process by typing Ctrl+C.

    Then run:

    -
    +
    +
    +
      +
    • +

      Podman

      +
      • +
    • +

      Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman compose down
    +
    +
    +
    +
    +
    podman-compose version: 1.0.6
+['podman', '--version', '']
+using podman version: 5.0.1
+** excluding:  set()
+podman stop -t 10 kafka
+kafka
+exit code: 0
+podman rm kafka
+kafka
+exit code: 0
    +
    +
    +
    +
    +
    podman compose up
    +
    +
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    +
    +
    +
    +
    docker compose down
    +
    +
    +
    +
    +
    ...
    +
    +
    +
    -
    docker compose rm
-
-? Going to remove kafka, zookeeper (y/N) -> y
-[+] Running 2/0
- ⠿ Container zookeeper  Removed
- ⠿ Container kafka      Removed
    +
    docker compose up
    -
    +
    -
    docker compose up --remove-orphans
-
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka version: 3.3.1 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka commitId: e23c59d00e687ff5 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka startTimeMs: 1671094765681 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,689] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
-kafka      | [2022-12-15 08:59:25,750] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
-kafka      | [2022-12-15 08:59:25,751] INFO [BrokerToControllerChannelManager broker=0 name=alterPartition]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    @@ -753,22 +869,32 @@

    • -

      kcat

      +

      kcat

      +
      • +
    • +

      kcat in Podman

    • -

      kcat in Docker

      +

      kcat in Docker

    -
    +
    kcat -b localhost:29092 -t played -P -l -K: apps/windowing/first-batch.json
    -
    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial -v $(pwd)/apps/windowing/first-batch.json:/home/first-batch edenhill/kcat:1.7.1 kcat -b kafka:9092 -t played -P -l -K: /home/first-batch
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial -v $(pwd)/apps/windowing/first-batch.json:/home/first-batch edenhill/kcat:1.7.1 kcat -b kafka:9092 -t played -P -l -K: /home/first-batch
    @@ -803,22 +929,32 @@

    • -

      kcat

      +

      kcat

    • -

      kcat in Docker

      +

      kcat in Podman

      +
      • +
    • +

      kcat in Docker

    -
    +
    kcat -b localhost:29092 -t played -P -l -K: apps/windowing/first-batch.json
    -
    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial -v $(pwd)/apps/windowing/first-batch.json:/home/first-batch edenhill/kcat:1.7.1 kcat -b kafka:9092 -t played -P -l -K: /home/first-batch
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial -v $(pwd)/apps/windowing/first-batch.json:/home/first-batch edenhill/kcat:1.7.1 kcat -b kafka:9092 -t played -P -l -K: /home/first-batch
    @@ -850,22 +986,32 @@

    • -

      kcat

      +

      kcat

    • -

      kcat in Docker

      +

      kcat in Podman

      +
      • +
    • +

      kcat in Docker

    -
    +
    kcat -b localhost:29092 -t played -P -l -K: apps/windowing/first-batch.json
    -
    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial -v $(pwd)/apps/windowing/first-batch.json:/home/first-batch edenhill/kcat:1.7.1 kcat -b kafka:9092 -t played -P -l -K: /home/first-batch
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial -v $(pwd)/apps/windowing/first-batch.json:/home/first-batch edenhill/kcat:1.7.1 kcat -b kafka:9092 -t played -P -l -K: /home/first-batch
    @@ -896,31 +1042,98 @@
    Clean Up

    Stop kstreamscat process by typing Ctrl+C in the terminal 2.

    -

    To restart Kafka (and ZooKeeper) go to the docker compose terminal and stop the process by typing Ctrl+C.

    +

    To restart Kafka go to the podman | docker compose terminal and stop the process by typing Ctrl+C.

    Then run:

    -
    +
    +
    +
      +
    • +

      Podman

      +
      • +
    • +

      Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman compose down
    +
    +
    +
    +
    +
    podman-compose version: 1.0.6
+['podman', '--version', '']
+using podman version: 5.0.1
+** excluding:  set()
+podman stop -t 10 kafka
+kafka
+exit code: 0
+podman rm kafka
+kafka
+exit code: 0
    +
    +
    +
    +
    +
    podman compose up
    +
    +
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    +
    +
    +
    +
    docker compose down
    +
    +
    +
    -
    docker compose rm
-
-? Going to remove kafka, zookeeper (y/N) -> y
-[+] Running 2/0
- ⠿ Container zookeeper  Removed
- ⠿ Container kafka      Removed
    +
    ...
    -
    +
    -
    docker compose up --remove-orphans
-
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka version: 3.3.1 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka commitId: e23c59d00e687ff5 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka startTimeMs: 1671094765681 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,689] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
-kafka      | [2022-12-15 08:59:25,750] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
-kafka      | [2022-12-15 08:59:25,751] INFO [BrokerToControllerChannelManager broker=0 name=alterPartition]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
    +
    docker compose up
    +
    +
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    @@ -945,22 +1158,32 @@

    • -

      kcat

      +

      kcat

    • -

      kcat in Docker

      +

      kcat in Podman

      +
      • +
    • +

      kcat in Docker

    -
    +
    kcat -b localhost:29092 -t played -P -l -K: apps/windowing/first-batch.json
    -
    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial -v $(pwd)/apps/windowing/first-batch.json:/home/first-batch edenhill/kcat:1.7.1 kcat -b kafka:9092 -t played -P -l -K: /home/first-batch
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial -v $(pwd)/apps/windowing/first-batch.json:/home/first-batch edenhill/kcat:1.7.1 kcat -b kafka:9092 -t played -P -l -K: /home/first-batch
    @@ -1051,31 +1274,98 @@
    Clean Up

    Stop kstreamscat process by typing Ctrl+C in the terminal 2.

    -

    To restart Kafka (and ZooKeeper) go to the docker compose terminal and stop the process by typing Ctrl+C.

    +

    To restart Kafka go to the podman | docker compose terminal and stop the process by typing Ctrl+C.

    Then run:

    -
    +
    +
    +
      +
    • +

      Podman

      +
      • +
    • +

      Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman compose down
    +
    +
    +
    +
    +
    podman-compose version: 1.0.6
+['podman', '--version', '']
+using podman version: 5.0.1
+** excluding:  set()
+podman stop -t 10 kafka
+kafka
+exit code: 0
+podman rm kafka
+kafka
+exit code: 0
    +
    +
    +
    +
    +
    podman compose up
    +
    +
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    +
    +
    +
    +
    docker compose down
    +
    +
    +
    -
    docker compose rm
-
-? Going to remove kafka, zookeeper (y/N) -> y
-[+] Running 2/0
- ⠿ Container zookeeper  Removed
- ⠿ Container kafka      Removed
    +
    ...
    -
    +
    -
    docker compose up --remove-orphans
-
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka version: 3.3.1 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka commitId: e23c59d00e687ff5 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka startTimeMs: 1671094765681 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,689] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
-kafka      | [2022-12-15 08:59:25,750] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
-kafka      | [2022-12-15 08:59:25,751] INFO [BrokerToControllerChannelManager broker=0 name=alterPartition]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
    +
    docker compose up
    +
    +
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    diff --git a/kafka-tutorial/06-java-kstreams.html b/kafka-tutorial/06-java-kstreams.html index 13ab145..61fa5f6 100644 --- a/kafka-tutorial/06-java-kstreams.html +++ b/kafka-tutorial/06-java-kstreams.html @@ -315,8 +315,17 @@

    Developing Kafka Streams in Java

    The second service is a Kafka Stream application that joins the songs and played-songs topics, to get the song name. Also it computes the number of times that each particular song has been played by aggregating the joined records based on the song’s id.

    -
    -

    Remember you need to have both, ZooKeeper and Apache Kafka correctly configured as well as up and running. If not, you can follow the [Setup Kafka] section to learn how to do it using docker compose.

    +
    + + + + + +
    + + +Remember you need to have Apache Kafka successfully up and running. If not, you can follow the Kafka Setup section to learn how to do it using podman | docker compose. +

    You need to open 2 terminal windows to run this example, one for each service.

    @@ -332,121 +341,132 @@

    Playe

    Deploying Player Service

    -

    In this case, the Quarkus service is deployed to produce played songs. -You’ve got two options, using the existing Docker image or building the service and image yourself. Note, that due to the configured Quarkus extension quarkus-container-image-jib the container image will be build automatically when you trigger the local build. -In terminal 1, run one of these two options:

    +

    In this case, the Quarkus service is deployed to produce played songs.

    +
    +
    +

    You’ve got different options, using either pre-built container images run with Podman / Docker or building the application and image on your local machine from the sources.

    +
    +
    + + + + + +
    + + +Due to the configured Quarkus extension quarkus-container-image-jib the container image will be built automatically when you trigger the local build. +
    +
    +
    +

    In terminal 1, run one of these two options:

    • -

      Building & Run

      +

      [Optional] Local Build & Run Podman

    • -

      Docker

      +

      [Optional] Local Build & Run Docker

    -
    +
    -
    cd $TUTORIAL_HOME/apps/player-app/quarkus/player-app
-./mvnw clean package
    +
    cd $TUTORIAL_HOME/apps/player-app/quarkus/player-app
    -
    docker run --rm --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-player-app-quarkus:v23.01
    +
    ./mvnw clean package -DskipTests
    -
    +
    -
    __  ____  __  _____   ___  __ ____  ______
- --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
- -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
---\___\_\____/_/ |_/_/|_/_/|_|\____/___/
-2023-01-09 08:45:18,099 INFO  [io.sma.rea.mes.kafka] (smallrye-kafka-producer-thread-0) SRMSG18258: Kafka producer kafka-producer-songs, connected to Kafka brokers 'kafka:9092', is configured to write records to 'songs'
-2023-01-09 08:45:18,471 INFO  [io.sma.rea.mes.kafka] (smallrye-kafka-producer-thread-1) SRMSG18258: Kafka producer kafka-producer-played-songs, connected to Kafka brokers 'kafka:9092', is configured to write records to 'played-songs'
-2023-01-09 08:45:18,573 INFO  [org.acm.PlaySongsGenerator] (main) creating song reference data
-2023-01-09 08:45:18,691 INFO  [org.acm.PlaySongsGenerator] (main) starting to play random songs every 2500 ms
-2023-01-09 08:45:18,739 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=1, name=The Good The Bad And The Ugly, author=Ennio Morricone]
-2023-01-09 08:45:19,675 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=2, name=Believe, author=Cher]
-2023-01-09 08:45:19,686 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=3, name=Still Loving You, author=Scorpions]
-2023-01-09 08:45:19,689 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=4, name=Bohemian Rhapsody, author=Queen]
-2023-01-09 08:45:19,691 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=5, name=Sometimes, author=James]
-2023-01-09 08:45:19,692 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=6, name=Into The Unknown, author=Frozen II]
-2023-01-09 08:45:19,694 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=7, name=Fox On The Run, author=Sweet]
-2023-01-09 08:45:19,696 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=8, name=Perfect, author=Ed Sheeran]
-2023-01-09 08:45:19,761 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 8 (Perfect) for user Edson
-2023-01-09 08:45:19,785 INFO  [io.quarkus] (main) player-app 1.0.0-SNAPSHOT on JVM (powered by Quarkus 2.15.2.Final) started in 7.027s.
-2023-01-09 08:45:19,787 INFO  [io.quarkus] (main) Profile prod activated.
-2023-01-09 08:45:19,788 INFO  [io.quarkus] (main) Installed features: [cdi, kafka-client, smallrye-context-propagation, smallrye-reactive-messaging, smallrye-reactive-messaging-kafka, vertx]
-2023-01-09 08:45:20,102 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-played-songs) [Producer clientId=kafka-producer-played-songs] Error while fetching metadata with correlation id 1 : {played-songs=LEADER_NOT_AVAILABLE}
-2023-01-09 08:45:20,106 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-songs) [Producer clientId=kafka-producer-songs] Error while fetching metadata with correlation id 1 : {songs=LEADER_NOT_AVAILABLE}
-2023-01-09 08:45:20,213 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-songs) [Producer clientId=kafka-producer-songs] Error while fetching metadata with correlation id 3 : {songs=LEADER_NOT_AVAILABLE}
-2023-01-09 08:45:20,220 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-played-songs) [Producer clientId=kafka-producer-played-songs] Error while fetching metadata with correlation id 3 : {played-songs=LEADER_NOT_AVAILABLE}
-2023-01-09 08:45:22,231 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 5 (Sometimes) for user Edson
-2023-01-09 08:45:24,729 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 7 (Fox On The Run) for user Burr
-2023-01-09 08:45:27,230 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 3 (Still Loving You) for user Edson
-2023-01-09 08:45:29,735 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 2 (Believe) for user Edson
-2023-01-09 08:45:32,238 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 8 (Perfect) for user Alex
-2023-01-09 08:45:34,743 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 7 (Fox On The Run) for user Kamesh
-2023-01-09 08:45:37,230 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 3 (Still Loving You) for user Kamesh
-2023-01-09 08:45:39,728 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 3 (Still Loving You) for user Sebi
-2023-01-09 08:45:42,229 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 4 (Bohemian Rhapsody) for user Edson
-2023-01-09 08:45:44,726 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 2 (Believe) for user Burr
-...
    +
    podman run --rm -it --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-player-app-quarkus:v24.05
    -
    +
    -
    docker run --rm --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-player-app-quarkus:v23.01
    +
    cd $TUTORIAL_HOME/apps/player-app/quarkus/player-app
    -
    +
    -
    __  ____  __  _____   ___  __ ____  ______
- --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
- -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
---\___\_\____/_/ |_/_/|_/_/|_|\____/___/
-2023-01-09 08:45:18,099 INFO  [io.sma.rea.mes.kafka] (smallrye-kafka-producer-thread-0) SRMSG18258: Kafka producer kafka-producer-songs, connected to Kafka brokers 'kafka:9092', is configured to write records to 'songs'
-2023-01-09 08:45:18,471 INFO  [io.sma.rea.mes.kafka] (smallrye-kafka-producer-thread-1) SRMSG18258: Kafka producer kafka-producer-played-songs, connected to Kafka brokers 'kafka:9092', is configured to write records to 'played-songs'
-2023-01-09 08:45:18,573 INFO  [org.acm.PlaySongsGenerator] (main) creating song reference data
-2023-01-09 08:45:18,691 INFO  [org.acm.PlaySongsGenerator] (main) starting to play random songs every 2500 ms
-2023-01-09 08:45:18,739 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=1, name=The Good The Bad And The Ugly, author=Ennio Morricone]
-2023-01-09 08:45:19,675 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=2, name=Believe, author=Cher]
-2023-01-09 08:45:19,686 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=3, name=Still Loving You, author=Scorpions]
-2023-01-09 08:45:19,689 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=4, name=Bohemian Rhapsody, author=Queen]
-2023-01-09 08:45:19,691 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=5, name=Sometimes, author=James]
-2023-01-09 08:45:19,692 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=6, name=Into The Unknown, author=Frozen II]
-2023-01-09 08:45:19,694 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=7, name=Fox On The Run, author=Sweet]
-2023-01-09 08:45:19,696 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=8, name=Perfect, author=Ed Sheeran]
-2023-01-09 08:45:19,761 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 8 (Perfect) for user Edson
-2023-01-09 08:45:19,785 INFO  [io.quarkus] (main) player-app 1.0.0-SNAPSHOT on JVM (powered by Quarkus 2.15.2.Final) started in 7.027s.
-2023-01-09 08:45:19,787 INFO  [io.quarkus] (main) Profile prod activated.
-2023-01-09 08:45:19,788 INFO  [io.quarkus] (main) Installed features: [cdi, kafka-client, smallrye-context-propagation, smallrye-reactive-messaging, smallrye-reactive-messaging-kafka, vertx]
-2023-01-09 08:45:20,102 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-played-songs) [Producer clientId=kafka-producer-played-songs] Error while fetching metadata with correlation id 1 : {played-songs=LEADER_NOT_AVAILABLE}
-2023-01-09 08:45:20,106 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-songs) [Producer clientId=kafka-producer-songs] Error while fetching metadata with correlation id 1 : {songs=LEADER_NOT_AVAILABLE}
-2023-01-09 08:45:20,213 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-songs) [Producer clientId=kafka-producer-songs] Error while fetching metadata with correlation id 3 : {songs=LEADER_NOT_AVAILABLE}
-2023-01-09 08:45:20,220 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-played-songs) [Producer clientId=kafka-producer-played-songs] Error while fetching metadata with correlation id 3 : {played-songs=LEADER_NOT_AVAILABLE}
-2023-01-09 08:45:22,231 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 5 (Sometimes) for user Edson
-2023-01-09 08:45:24,729 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 7 (Fox On The Run) for user Burr
-2023-01-09 08:45:27,230 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 3 (Still Loving You) for user Edson
-2023-01-09 08:45:29,735 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 2 (Believe) for user Edson
-2023-01-09 08:45:32,238 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 8 (Perfect) for user Alex
-2023-01-09 08:45:34,743 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 7 (Fox On The Run) for user Kamesh
-2023-01-09 08:45:37,230 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 3 (Still Loving You) for user Kamesh
-2023-01-09 08:45:39,728 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 3 (Still Loving You) for user Sebi
-2023-01-09 08:45:42,229 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 4 (Bohemian Rhapsody) for user Edson
-2023-01-09 08:45:44,726 INFO  [org.acm.PlaySongsGenerator] (executor-thread-0) Played song id 2 (Believe) for user Burr
-...
    +
    ./mvnw clean package -DskipTests
    +
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-player-app-quarkus:v24.05
    +
    +
    +
    +
      +
    • +

      Pre-built image with Podman

      +
      • +
    • +

      Pre-built image with Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-player-app-quarkus:v24.05-xa
    +
    +
    +
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-player-app-quarkus:v24.05-xa
    +
    +
    +
    +
    +
    +
    +

    Console output when running the containerized application looks as follows:

    +
    +
    +
    +
    ...
+2024-05-14 06:46:09,940 INFO  [io.sma.rea.mes.kafka] (smallrye-kafka-producer-thread-1) SRMSG18258: Kafka producer kafka-producer-played-songs, connected to Kafka brokers 'kafka:9092', is configured to write records to 'played-songs'
+2024-05-14 06:46:10,021 INFO  [org.acm.PlaySongsGenerator] (main) starting to play random songs every 2500 ms
+2024-05-14 06:46:10,050 INFO  [org.acm.PlaySongsGenerator] (main) creating song reference data
+2024-05-14 06:46:10,123 INFO  [org.acm.PlaySongsGenerator] (executor-thread-1) Played song id 7 (Fox On The Run) for user Sebi
+2024-05-14 06:46:10,131 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=1, name=The Good The Bad And The Ugly, author=Ennio Morricone]
+2024-05-14 06:46:10,871 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=2, name=Believe, author=Cher]
+2024-05-14 06:46:10,890 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=3, name=Still Loving You, author=Scorpions]
+2024-05-14 06:46:10,904 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=4, name=Bohemian Rhapsody, author=Queen]
+2024-05-14 06:46:10,906 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=5, name=Sometimes, author=James]
+2024-05-14 06:46:10,908 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=6, name=Into The Unknown, author=Frozen II]
+2024-05-14 06:46:10,909 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=7, name=Fox On The Run, author=Sweet]
+2024-05-14 06:46:10,911 INFO  [org.acm.PlaySongsGenerator] (main) producing -> Song[id=8, name=Perfect, author=Ed Sheeran]
+2024-05-14 06:46:10,970 INFO  [io.quarkus] (main) player-app 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.9.5) started in 7.370s.
+2024-05-14 06:46:10,972 INFO  [io.quarkus] (main) Profile prod activated.
+2024-05-14 06:46:10,973 INFO  [io.quarkus] (main) Installed features: [cdi, kafka-client, messaging, messaging-kafka, smallrye-context-propagation, vertx]
+2024-05-14 06:46:11,481 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-played-songs) [Producer clientId=kafka-producer-played-songs] Error while fetching metadata with correlation id 1 : {played-songs=UNKNOWN_TOPIC_OR_PARTITION}
+2024-05-14 06:46:11,481 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-songs) [Producer clientId=kafka-producer-songs] Error while fetching metadata with correlation id 1 : {songs=UNKNOWN_TOPIC_OR_PARTITION}
+2024-05-14 06:46:12,631 INFO  [org.acm.PlaySongsGenerator] (executor-thread-1) Played song id 1 (The Good The Bad And The Ugly) for user Alex
+2024-05-14 06:46:15,129 INFO  [org.acm.PlaySongsGenerator] (executor-thread-1) Played song id 3 (Still Loving You) for user Burr
+2024-05-14 06:46:17,622 INFO  [org.acm.PlaySongsGenerator] (executor-thread-1) Played song id 2 (Believe) for user Alex
+2024-05-14 06:46:20,123 INFO  [org.acm.PlaySongsGenerator] (executor-thread-1) Played song id 5 (Sometimes) for user Edson
+2024-05-14 06:46:22,625 INFO  [org.acm.PlaySongsGenerator] (executor-thread-1) Played song id 3 (Still Loving You) for user Edson
+...
    +
    @@ -460,95 +480,127 @@

    Kafka S

    Deploying Music Chart Service

    -

    In this case, the Quarkus service is deployed to use Kafka Streams. -You’ve got two options, using the existing Docker image or building the service and image yourself. Note, that due to the configured Quarkus extension quarkus-container-image-jib the container image will be created automatically when you trigger the local build. -In terminal 2, run one of these two options:

    +

    In this case, the Quarkus service makes use of Kafka Streams.

    +
    +
    +

    You’ve got different options, using either pre-built container images run with Podman / Docker or building the application and image on your local machine from the sources.

    +
    +
    + + + + + +
    + + +Due to the configured Quarkus extension quarkus-container-image-jib the container image will be built automatically when you trigger the local build. +
    +
    +
    +

    In terminal 2, run one of these two options:

    • -

      Building & Run

      +

      [Optional] Local Build & Run Podman

    • -

      Docker

      +

      [Optional] Local Build & Run Docker

    -
    +
    -
    cd $TUTORIAL_HOME/apps/music-chart-app/quarkus/music-chart-app
-./mvnw clean package
    +
    cd $TUTORIAL_HOME/apps/music-chart-app/quarkus/music-chart-app
    -
    docker run --rm --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-music-chart-app-quarkus:v23.01
    +
    ./mvnw clean package -DskipTests
    -
    +
    -
    __  ____  __  _____   ___  __ ____  ______
- --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
- -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
---\___\_\____/_/ |_/_/|_/_/|_|\____/___/
-2023-01-09 08:45:54,755 WARN  [org.apa.kaf.cli.adm.AdminClientConfig] (main) These configurations '[ssl.endpoint.identification.algorithm]' were supplied but are not used yet.
-2023-01-09 08:45:55,340 INFO  [org.apa.kaf.str.StreamsConfig] (main) StreamsConfig values:
-        acceptable.recovery.lag = 10000
-        application.id = music-chart
-        application.server = localhost:9090
-        bootstrap.servers = [kafka:9092]
-...
-2023-01-09 08:46:00,952 INFO  [org.apa.kaf.str.pro.int.StreamThread] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) stream-thread [music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1] State transition from PARTITIONS_ASSIGNED to RUNNING
-2023-01-09 08:46:00,958 INFO  [org.apa.kaf.str.KafkaStreams] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) stream-client [music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a] State transition from REBALANCING to RUNNING
-2023-01-09 08:46:02,098 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 5 -> PlayedSong [count=1, songName=Sometimes]
-2023-01-09 08:46:02,128 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 8 -> PlayedSong [count=2, songName=Perfect]
-2023-01-09 08:46:02,133 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 7 -> PlayedSong [count=2, songName=Fox On The Run]
-2023-01-09 08:46:02,135 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 3 -> PlayedSong [count=3, songName=Still Loving You]
-2023-01-09 08:46:02,137 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 4 -> PlayedSong [count=1, songName=Bohemian Rhapsody]
-2023-01-09 08:46:02,140 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 2 -> PlayedSong [count=2, songName=Believe]
    -
    -
    -
    -
    +
    podman run --rm -it --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-music-chart-app-quarkus:v24.05
    +
    +
    +
    +
    -
    docker run --rm --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-music-chart-app-quarkus:v23.01
    +
    cd $TUTORIAL_HOME/apps/music-chart-app/quarkus/music-chart-app
    -
    +
    -
    __  ____  __  _____   ___  __ ____  ______
- --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
- -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \
---\___\_\____/_/ |_/_/|_/_/|_|\____/___/
-2023-01-09 08:45:54,755 WARN  [org.apa.kaf.cli.adm.AdminClientConfig] (main) These configurations '[ssl.endpoint.identification.algorithm]' were supplied but are not used yet.
-2023-01-09 08:45:55,340 INFO  [org.apa.kaf.str.StreamsConfig] (main) StreamsConfig values:
-        acceptable.recovery.lag = 10000
-        application.id = music-chart
-        application.server = localhost:9090
-        bootstrap.servers = [kafka:9092]
-...
-2023-01-09 08:46:00,952 INFO  [org.apa.kaf.str.pro.int.StreamThread] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) stream-thread [music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1] State transition from PARTITIONS_ASSIGNED to RUNNING
-2023-01-09 08:46:00,958 INFO  [org.apa.kaf.str.KafkaStreams] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) stream-client [music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a] State transition from REBALANCING to RUNNING
-2023-01-09 08:46:02,098 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 5 -> PlayedSong [count=1, songName=Sometimes]
-2023-01-09 08:46:02,128 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 8 -> PlayedSong [count=2, songName=Perfect]
-2023-01-09 08:46:02,133 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 7 -> PlayedSong [count=2, songName=Fox On The Run]
-2023-01-09 08:46:02,135 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 3 -> PlayedSong [count=3, songName=Still Loving You]
-2023-01-09 08:46:02,137 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 4 -> PlayedSong [count=1, songName=Bohemian Rhapsody]
-2023-01-09 08:46:02,140 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-08427007-2905-4d91-b260-44bcb8ac5c6a-StreamThread-1) music chart updated for song id 2 -> PlayedSong [count=2, songName=Believe]
    +
    ./mvnw clean package -DskipTests
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-music-chart-app-quarkus:v24.05
    +
    +
    +
    +
      +
    • +

      Pre-built image with Podman

      +
      • +
    • +

      Pre-built image with Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman run --rm -it --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-music-chart-app-quarkus:v24.05-xa
    +
    +
    +
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial quay.io/rhdevelopers/kafka-tutorial-music-chart-app-quarkus:v24.05-xa
    +
    +
    +
    +
    +
    +
    +

    Console output when running the containerized application looks as follows:

    +
    +
    +
    +
    ...
+2024-05-14 06:56:34,609 INFO  [org.apa.kaf.str.pro.int.StreamThread] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) stream-thread [music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1] State transition from PARTITIONS_ASSIGNED to RUNNING
+2024-05-14 06:56:34,610 INFO  [org.apa.kaf.str.KafkaStreams] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) stream-client [music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580] State transition from REBALANCING to RUNNING
+2024-05-14 06:56:35,728 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 7 -> PlayedSong [count=1, songName=Fox On The Run]
+2024-05-14 06:56:35,731 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 8 -> PlayedSong [count=1, songName=Perfect]
+2024-05-14 06:56:35,731 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 1 -> PlayedSong [count=1, songName=The Good The Bad And The Ugly]
+2024-05-14 06:56:35,732 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 5 -> PlayedSong [count=2, songName=Sometimes]
+2024-05-14 06:56:35,732 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 4 -> PlayedSong [count=3, songName=Bohemian Rhapsody]
+2024-05-14 06:56:35,733 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 6 -> PlayedSong [count=3, songName=Into The Unknown]
+2024-05-14 06:56:35,733 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 2 -> PlayedSong [count=3, songName=Believe]
+2024-05-14 06:56:36,849 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 6 -> PlayedSong [count=4, songName=Into The Unknown]
+2024-05-14 06:56:38,968 INFO  [org.acm.MusicChartTopologyProducer] (music-chart-7bc6bbdb-fee1-4d18-89f5-6839bd86c580-StreamThread-1) music chart updated for song id 7 -> PlayedSong [count=2, songName=Fox On The Run]
+...
    +
    +
    +

    Music Chart Result

    -

    If you inspect the output of Music Chart service, you see how many times a concrete song has been played, and every time a new song is played, the counter is updated.

    +

    If you inspect the output of Music Chart service, you see how many times a specific song has been played, and every time a new song is played, the respective song counter is updated.

    @@ -560,31 +612,98 @@

    Clean U

    Stop both services by typing Ctrl+C in both terminals 1 and 2.

    -

    To restart Kafka (and ZooKeeper) go to the docker compose terminal and stop the process by typing Ctrl+C.

    +

    To restart Kafka go to the podman | docker compose terminal and stop the process by typing Ctrl+C.

    Then run:

    -
    +
    +
    +
      +
    • +

      Podman

      +
      • +
    • +

      Docker

      +
      • +
    +
    +
    +
    +
    +
    +
    podman compose down
    +
    +
    +
    +
    +
    podman-compose version: 1.0.6
+['podman', '--version', '']
+using podman version: 5.0.1
+** excluding:  set()
+podman stop -t 10 kafka
+kafka
+exit code: 0
+podman rm kafka
+kafka
+exit code: 0
    +
    +
    +
    -
    docker compose rm
-
-? Going to remove kafka, zookeeper (y/N) -> y
-[+] Running 2/0
- ⠿ Container zookeeper  Removed
- ⠿ Container kafka      Removed
    +
    podman compose up
    -
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    +
    +
    -
    docker compose up --remove-orphans
-
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka version: 3.3.1 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka commitId: e23c59d00e687ff5 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,688] INFO Kafka startTimeMs: 1671094765681 (org.apache.kafka.common.utils.AppInfoParser)
-kafka      | [2022-12-15 08:59:25,689] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
-kafka      | [2022-12-15 08:59:25,750] INFO [BrokerToControllerChannelManager broker=0 name=forwarding]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
-kafka      | [2022-12-15 08:59:25,751] INFO [BrokerToControllerChannelManager broker=0 name=alterPartition]: Recorded new controller, from now on will use broker kafka:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)
    +
    docker compose down
    +
    +
    +
    +
    +
    ...
    +
    +
    +
    +
    +
    docker compose up
    +
    +
    +
    +
    +
    ...
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on kafka:9092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,976] INFO Awaiting socket connections on 0.0.0.0:29092. (kafka.network.DataPlaneAcceptor)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,977] INFO [BrokerServer id=1] Finished waiting for all of the authorizer futures to be completed (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO [BrokerServer id=1] Transition from STARTING to STARTED (kafka.server.BrokerServer)
+[2024-05-13 15:36:17,978] INFO Kafka version: 3.7.0 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO Kafka startTimeMs: 1715614577978 (org.apache.kafka.common.utils.AppInfoParser)
+[2024-05-13 15:36:17,978] INFO [KafkaRaftServer nodeId=1] Kafka Server started (kafka.server.KafkaRaftServer)
    +
    +
    +
    diff --git a/kafka-tutorial/07-kubernetes.html b/kafka-tutorial/07-kubernetes.html index 02ebdff..fee6dd5 100644 --- a/kafka-tutorial/07-kubernetes.html +++ b/kafka-tutorial/07-kubernetes.html @@ -643,26 +643,33 @@

    Installat

    -
    customresourcedefinition.apiextensions.k8s.io/kafkas.kafka.strimzi.io created
-rolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator-entity-operator-delegation created
-clusterrolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator created
-rolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator-topic-operator-delegation created
-customresourcedefinition.apiextensions.k8s.io/kafkausers.kafka.strimzi.io created
-customresourcedefinition.apiextensions.k8s.io/kafkamirrormaker2s.kafka.strimzi.io created
+
    clusterrolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator-kafka-broker-delegation created
+serviceaccount/strimzi-cluster-operator created
+customresourcedefinition.apiextensions.k8s.io/kafkatopics.kafka.strimzi.io created
 clusterrole.rbac.authorization.k8s.io/strimzi-entity-operator created
+clusterrole.rbac.authorization.k8s.io/strimzi-kafka-broker created
+rolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator-watched created
+rolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator-entity-operator-delegation created
+rolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator-leader-election created
 clusterrole.rbac.authorization.k8s.io/strimzi-cluster-operator-global created
-clusterrolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator-kafka-broker-delegation created
-rolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator created
+clusterrole.rbac.authorization.k8s.io/strimzi-cluster-operator-leader-election created
 clusterrole.rbac.authorization.k8s.io/strimzi-cluster-operator-namespaced created
-clusterrole.rbac.authorization.k8s.io/strimzi-topic-operator created
-serviceaccount/strimzi-cluster-operator created
-clusterrole.rbac.authorization.k8s.io/strimzi-kafka-broker created
-customresourcedefinition.apiextensions.k8s.io/kafkatopics.kafka.strimzi.io created
-customresourcedefinition.apiextensions.k8s.io/kafkabridges.kafka.strimzi.io created
+customresourcedefinition.apiextensions.k8s.io/kafkas.kafka.strimzi.io created
 deployment.apps/strimzi-cluster-operator created
 customresourcedefinition.apiextensions.k8s.io/kafkaconnectors.kafka.strimzi.io created
-customresourcedefinition.apiextensions.k8s.io/kafkaconnects2is.kafka.strimzi.io created
+customresourcedefinition.apiextensions.k8s.io/strimzipodsets.core.strimzi.io created
+clusterrolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator created
 customresourcedefinition.apiextensions.k8s.io/kafkaconnects.kafka.strimzi.io created
+customresourcedefinition.apiextensions.k8s.io/kafkarebalances.kafka.strimzi.io created
+customresourcedefinition.apiextensions.k8s.io/kafkamirrormaker2s.kafka.strimzi.io created
+clusterrolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator-kafka-client-delegation created
+customresourcedefinition.apiextensions.k8s.io/kafkanodepools.kafka.strimzi.io created
+customresourcedefinition.apiextensions.k8s.io/kafkausers.kafka.strimzi.io created
+rolebinding.rbac.authorization.k8s.io/strimzi-cluster-operator created
+clusterrole.rbac.authorization.k8s.io/strimzi-cluster-operator-watched created
+clusterrole.rbac.authorization.k8s.io/strimzi-kafka-client created
+customresourcedefinition.apiextensions.k8s.io/kafkabridges.kafka.strimzi.io created
+configmap/strimzi-cluster-operator created
 customresourcedefinition.apiextensions.k8s.io/kafkamirrormakers.kafka.strimzi.io created
    @@ -673,8 +680,17 @@

    Installat

    -
    NAME                                          READY   STATUS    RESTARTS   AGE
-strimzi-cluster-operator-f696c85f7-9fggx      1/1     Running   0          6m53s
    +
    NAME                                        READY   STATUS              RESTARTS   AGE
+strimzi-cluster-operator-79fb75cb8d-7k9rj   0/1     ContainerCreating   0          27s
    +
    +
    +
    +

    After a while the pod becomes ready and is running fine.

    +
    +
    +
    +
    NAME                                        READY   STATUS    RESTARTS   AGE
+strimzi-cluster-operator-79fb75cb8d-7k9rj   1/1     Running   0          97s
    @@ -702,15 +718,16 @@

    Installat 
    NAME                                  CREATED AT
-kafkabridges.kafka.strimzi.io         2022-12-19T07:16:50Z
-kafkaconnectors.kafka.strimzi.io      2022-12-19T07:16:50Z
-kafkaconnects.kafka.strimzi.io        2022-12-19T07:16:50Z
-kafkamirrormaker2s.kafka.strimzi.io   2022-12-19T07:16:50Z
-kafkamirrormakers.kafka.strimzi.io    2022-12-19T07:16:50Z
-kafkarebalances.kafka.strimzi.io      2022-12-19T07:16:50Z
-kafkas.kafka.strimzi.io               2022-12-19T07:16:50Z
-kafkatopics.kafka.strimzi.io          2022-12-19T07:16:50Z
-kafkausers.kafka.strimzi.io           2022-12-19T07:16:50Z
    +kafkabridges.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkaconnectors.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkaconnects.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkamirrormaker2s.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkamirrormakers.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkanodepools.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkarebalances.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkas.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkatopics.kafka.strimzi.io 2024-05-14T07:06:10Z +kafkausers.kafka.strimzi.io 2024-05-14T07:06:10Z

    @@ -799,15 +816,15 @@

    Deploying Kafka

    -
    NAME                                          READY   STATUS    RESTARTS      AGE
-my-cluster-entity-operator-7dfb85ccf9-28dqd   3/3     Running   0             95s
-my-cluster-kafka-0                            1/1     Running   0             118s
-my-cluster-kafka-1                            1/1     Running   0             118s
-my-cluster-kafka-2                            1/1     Running   0             118s
-my-cluster-zookeeper-0                        1/1     Running   0             2m21s
-my-cluster-zookeeper-1                        1/1     Running   0             2m21s
-my-cluster-zookeeper-2                        1/1     Running   0             2m21s
-strimzi-cluster-operator-f696c85f7-9fggx      1/1     Running   0             7m49s
    +
    NAME                                          READY   STATUS    RESTARTS   AGE
+my-cluster-entity-operator-5f6c66f65c-k2vww   2/2     Running   0          29s
+my-cluster-kafka-0                            1/1     Running   0          52s
+my-cluster-kafka-1                            1/1     Running   0          52s
+my-cluster-kafka-2                            1/1     Running   0          52s
+my-cluster-zookeeper-0                        1/1     Running   0          2m53s
+my-cluster-zookeeper-1                        1/1     Running   0          2m53s
+my-cluster-zookeeper-2                        1/1     Running   0          2m53s
+strimzi-cluster-operator-79fb75cb8d-7k9rj     1/1     Running   0          6m12s
    @@ -820,12 +837,12 @@

    Deploying Kafka

    -
    NAME                          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
-kubernetes                    ClusterIP   10.96.0.1       <none>        443/TCP                               10m33s
-my-cluster-kafka-bootstrap    ClusterIP   10.98.94.157    <none>        9091/TCP,9092/TCP,9093/TCP            3m12s
-my-cluster-kafka-brokers      ClusterIP   None            <none>        9090/TCP,9091/TCP,9092/TCP,9093/TCP   3m12s
-my-cluster-zookeeper-client   ClusterIP   10.104.185.52   <none>        2181/TCP                              3m35s
-my-cluster-zookeeper-nodes    ClusterIP   None            <none>        2181/TCP,2888/TCP,3888/TCP            3m35s
    +
    NAME                          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                        AGE
+kubernetes                    ClusterIP   10.96.0.1       <none>        443/TCP                                        12h
+my-cluster-kafka-bootstrap    ClusterIP   10.96.137.128   <none>        9091/TCP,9092/TCP,9093/TCP                     72s
+my-cluster-kafka-brokers      ClusterIP   None            <none>        9090/TCP,9091/TCP,8443/TCP,9092/TCP,9093/TCP   72s
+my-cluster-zookeeper-client   ClusterIP   10.96.88.171    <none>        2181/TCP                                       3m13s
+my-cluster-zookeeper-nodes    ClusterIP   None            <none>        2181/TCP,2888/TCP,3888/TCP                     3m13s
    @@ -835,8 +852,8 @@

    Deploying Kafka

    Deploying Services

    -

    Let’s deploy the example done at [Developing Consumers and Producers in Java], but in Kubernetes instead of Docker. -This time, the Quarkus versions of the song / song-indexer applications are deployed.

    +

    Let’s deploy the example done at Developing Consumers and Producers in Java, but in Kubernetes instead of Docker. +This time, the Quarkus versions of the song-app and song-indexer-app are deployed.

    Deploy Song Service

    @@ -846,7 +863,7 @@

    D 
    cd $TUTORIAL_HOME/apps/song-app/quarkus/song-app
-./mvnw clean package
    +./mvnw clean package -DskipTests
    @@ -859,8 +876,8 @@

    D

    -
    service/kafka-tutorial-song-app-quarkus created
-deployment.apps/kafka-tutorial-song-app-quarkus created
    +
    service/song-app created
+deployment.apps/song-app created
    @@ -873,16 +890,16 @@

    D

    -
    NAME                                                       READY   STATUS    RESTARTS      AGE
-kafka-tutorial-song-app-quarkus-55df95589f-fg5v2           1/1     Running   0             7m23s
-my-cluster-entity-operator-7dfb85ccf9-28dqd                3/3     Running   0             21m
-my-cluster-kafka-0                                         1/1     Running   0             21m
-my-cluster-kafka-1                                         1/1     Running   0             21m
-my-cluster-kafka-2                                         1/1     Running   0             21m
-my-cluster-zookeeper-0                                     1/1     Running   0             22m
-my-cluster-zookeeper-1                                     1/1     Running   0             22m
-my-cluster-zookeeper-2                                     1/1     Running   0             22m
-strimzi-cluster-operator-f696c85f7-9fggx                   1/1     Running   0             37m49s
    +
    NAME                                          READY   STATUS    RESTARTS   AGE
+my-cluster-entity-operator-5f6c66f65c-k2vww   2/2     Running   0          12m
+my-cluster-kafka-0                            1/1     Running   0          13m
+my-cluster-kafka-1                            1/1     Running   0          13m
+my-cluster-kafka-2                            1/1     Running   0          13m
+my-cluster-zookeeper-0                        1/1     Running   0          15m
+my-cluster-zookeeper-1                        1/1     Running   0          15m
+my-cluster-zookeeper-2                        1/1     Running   0          15m
+song-app-566b5f7bf5-pdl4b                     1/1     Running   0          2m12s
+strimzi-cluster-operator-79fb75cb8d-7k9rj     1/1     Running   0          18m

    @@ -894,7 +911,7 @@ 

    cd $TUTORIAL_HOME/apps/song-indexer-app/quarkus/song-indexer-app
-./mvnw clean package
    +./mvnw clean package -DskipTests

    @@ -907,8 +924,8 @@

    -
    service/kafka-tutorial-song-indexer-app-quarkus created
-deployment.apps/kafka-tutorial-song-indexer-app-quarkus created
    +
    service/song-indexer-app created
+deployment.apps/song-indexer-app created

    @@ -921,17 +938,17 @@

    -
    NAME                                                       READY   STATUS    RESTARTS      AGE
-kafka-tutorial-song-app-quarkus-55df95589f-fg5v2           1/1     Running   0             9m21s
-kafka-tutorial-song-indexer-app-quarkus-6c4b56f448-xgshm   1/1     Running   0             1m10s
-my-cluster-entity-operator-7dfb85ccf9-28dqd                3/3     Running   0             23m
-my-cluster-kafka-0                                         1/1     Running   0             23m
-my-cluster-kafka-1                                         1/1     Running   0             23m
-my-cluster-kafka-2                                         1/1     Running   0             23m
-my-cluster-zookeeper-0                                     1/1     Running   0             24m
-my-cluster-zookeeper-1                                     1/1     Running   0             24m
-my-cluster-zookeeper-2                                     1/1     Running   0             24m
-strimzi-cluster-operator-f696c85f7-9fggx                   1/1     Running   0             39m33s
    +
    NAME                                          READY   STATUS    RESTARTS   AGE
+my-cluster-entity-operator-5f6c66f65c-k2vww   2/2     Running   0          15m
+my-cluster-kafka-0                            1/1     Running   0          15m
+my-cluster-kafka-1                            1/1     Running   0          15m
+my-cluster-kafka-2                            1/1     Running   0          15m
+my-cluster-zookeeper-0                        1/1     Running   0          17m
+my-cluster-zookeeper-1                        1/1     Running   0          17m
+my-cluster-zookeeper-2                        1/1     Running   0          17m
+song-app-566b5f7bf5-pdl4b                     1/1     Running   0          4m29s
+song-indexer-app-578bfb4d74-6j8sc             1/1     Running   0          26s
+strimzi-cluster-operator-79fb75cb8d-7k9rj     1/1     Running   0          20m

    @@ -948,7 +965,7 @@

    Tes

    -
    minikube service kafka-tutorial-song-indexer-app-quarkus
    +
    minikube service song-app
    @@ -959,15 +976,15 @@

    Tes 
    |-----------|---------------------------------|-------------|---------------------------|
 | NAMESPACE |              NAME               | TARGET PORT |            URL            |
 |-----------|---------------------------------|-------------|---------------------------|
-| default   | kafka-tutorial-song-app-quarkus | http/80     | http://192.168.49.2:30588 |
+| default   | song-app                        | http/80     | http://192.168.49.2:30588 |
 |-----------|---------------------------------|-------------|---------------------------|
 🏃  Starting tunnel for service kafka-tutorial-song-app-quarkus.
 |-----------|---------------------------------|-------------|------------------------|
 | NAMESPACE |              NAME               | TARGET PORT |          URL           |
 |-----------|---------------------------------|-------------|------------------------|
-| default   | kafka-tutorial-song-app-quarkus |             | http://127.0.0.1:65000 |
+| default   | song-app                        |             | http://127.0.0.1:65000 |
 |-----------|---------------------------------|-------------|------------------------|
-🎉  Opening service default/kafka-tutorial-song-app-quarkus in default browser...
    +🎉 Opening service default/song-app in default browser...

    @@ -978,7 +995,7 @@

    Tes

    -
    minikube service kafka-tutorial-song-indexer-app-quarkus
    +
    minikube service song-indexer-app
    @@ -989,15 +1006,15 @@

    Tes 
    |-----------|-----------------------------------------|-------------|---------------------------|
 | NAMESPACE |                  NAME                   | TARGET PORT |            URL            |
 |-----------|-----------------------------------------|-------------|---------------------------|
-| default   | kafka-tutorial-song-indexer-app-quarkus | http/80     | http://192.168.49.2:32548 |
+| default   | song-indexer-app                        | http/80     | http://192.168.49.2:32548 |
 |-----------|-----------------------------------------|-------------|---------------------------|
-🏃  Starting tunnel for service kafka-tutorial-song-indexer-app-quarkus.
+🏃  Starting tunnel for service song-indexer-app
 |-----------|-----------------------------------------|-------------|------------------------|
 | NAMESPACE |                  NAME                   | TARGET PORT |          URL           |
 |-----------|-----------------------------------------|-------------|------------------------|
-| default   | kafka-tutorial-song-indexer-app-quarkus |             | http://127.0.0.1:64938 |
+| default   | song-indexer-app                        |             | http://127.0.0.1:64938 |
 |-----------|-----------------------------------------|-------------|------------------------|
-🎉  Opening service default/kafka-tutorial-song-indexer-app-quarkus in default browser...
    +🎉 Opening service default/song-indexer-app in default browser...

    diff --git a/kafka-tutorial/08-kafka-listeners.html b/kafka-tutorial/08-kafka-listeners.html index 3c33db1..8d1c6b7 100644 --- a/kafka-tutorial/08-kafka-listeners.html +++ b/kafka-tutorial/08-kafka-listeners.html @@ -360,6 +360,9 @@

    Kafka Listeners

    kcat

  • +

    kcat in Podman

    +
    • +

  • kcat in Docker

    • @@ -371,67 +374,46 @@

    Kafka Listeners

    kcat -b localhost:29092 -L
    -
    -
    -
    -
    -
    docker run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 -b localhost:29092 -L
    -
    -
    -
    -
    -
    -
    Metadata for all topics (from broker 0: localhost:29092/0):
+
    Metadata for all topics (from broker 1: localhost:29092/1):
  1 brokers:
-  broker 0 at localhost:29092 (controller)
+  broker 1 at localhost:29092 (controller)
  0 topics:
    -
    -
    -
      -
    • -

      kcat

      -
      • -
    • -

      kcat in Docker

      -
      • -
    -
    -
    +
    -
    kcat -b localhost:9092 -L
    -
    +
    podman run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 -b kafka:9092 -L
    -
    -
    +
    -
    docker run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 -b localhost:9092 -L
    +
    Metadata for all topics (from broker 1: kafka:9092/1):
+ 1 brokers:
+  broker 1 at kafka:9092 (controller)
+ 0 topics:
    +
    +
    +
    +
    docker run --rm -it --network=kafka-tutorial edenhill/kcat:1.7.1 -b kafka:9092 -L
    -
    Metadata for all topics (from broker -1: localhost:9092/bootstrap):
+
    Metadata for all topics (from broker 1: kafka:9092/1):
  1 brokers:
-  broker 0 at kafka:9092 (controller) (1)
+  broker 1 at kafka:9092 (controller)
  0 topics:
    -
    - - - - - -
    1Notice that when the leader is queried using port 9092, then the hostname returned is kafka:9092.
    +
    +