Basic Cluster and Brokers Environment Configuration for Apache kafka, Kafka Connect, Kafka Schema Registry, ksqlDB-server, Kafka REST Proxy, Conduktor, PostgreSQL, and pgAdmin in Docker containers.
Configuration of Cluster and Brokers in Apache Kafka with connection to kafka Connect, kafka-schema-registry, ksqlDB, kafka REST Proxy, Conduktor, PostgreSql and PgAdmin 4 in Docker containers.
Make sure you have the latest versions of Docker and Docker Compose installed on your machine.
Clone this repository or copy the files from this repository into a new folder. In the docker-compose.yml files, you can change the ports (if you are running multiple containers on your system).
With this project, you can quickly execute the following:
- Apache Kafka
- kafka connect
- kafka-schema-registry
- ksqldb
- kafka-rest-proxy
- Conduktor
- Postgresql
- PgAdmin
Contenido:
Edit the .env file to change the default environment variables.
Open a terminal and run cd to the folder where you can see the files docker-compose-kafka-single-stack.yml, which contains the necessary configuration to deploy a Kafka broker, and the file docker-compose-kafka-multiple-stack.yml, which contains the configuration for multiple Apache Kafka brokers.
Execute as needed:
docker-compose -f docker-compose-kafka-single-stack.yml up -d
This will start the containers and create volumes according to the configuration in your docker-compose.yml files. Make sure to grant write permissions to the directory so it can create the necessary volumes for the containers. Similarly, remember to clean the volumes folder when running different files to avoid data inconsistency.
.env– File containing the environment variable data.volumenes– Folder containing container data.
The containers are already built and running. You should be able to access the installation of each container.
You can visit http:/localhost:8083 to check the status of kafka-connect, or go to http:/localhost:8088 to view the status of ksqldb-server, as well as the other services such as kafka-rest-proxy and kafka-schema-registry.
ou can visit http:/localhost:8080 to access Conduktor. The default username is [email protected], and the password is admin. You can modify these credentials in the environment variables located in the .env file."
The Conduktor tool will allow you to view the status of the cluster as well as visualize the brokers you have deployed in Apache Kafka.
In the Conduktor interface, you will be able to see the status of the connection with your configured and connected services, such as kafka-connect and kafka-schema-registry to Apache Kafka.
Conduktor will be useful for conducting the necessary tests and visual management of Apache Kafka. You can create new topics and send/receive messages as well.
From the Conduktor interface, you can create connectors and schemas, manage them, and test them visually.
