Kafka Brokers

A Kafka cluster consists of one or more servers (Kafka brokers) running Kafka. Producers are processes that push records into Kafka topics within the broker. A consumer pulls records off a Kafka topic.

Running a single Kafka broker is possible but it doesn’t give all the benefits that Kafka in a cluster can give, for example, data replication.

../../_images/brokers1.png

Management of the brokers in the cluster is handled by Zookeeper. In the E2E Kafka environment, we manage Zookeeper and users can add brokers according to their requirements.

At the time of creation, users can select the number of brokers they need. In the future, if they need to increase the number of brokers, they can do so. To add more brokers in E2E Kafka, click on the + button in the Brokers tab.

../../_images/brokers.png

Note

Total number of Brokers should be less than or equal to 50

Note

Adding new broker won’t automatically receive any data partitions, so they won’t be active until partitions are manually moved to them or new topics are created.