Chapters

How to Install Apache Kafka on Mac without Zookeeper? (KRaft mode)

Start and run Apache Kafka on Mac OS X without Zookeeper.


KRaft is not Production Ready

KRaft is in early access mode as of Apache Kafka 2.8 and should be used in development only. It is not suitable for production.

Install Kafka with KRaft (without Zookeeper) on Mac OS X

  1. Install Java JDK version 11

  2. Download Apache Kafka from https://kafka.apache.org/downloads under 'Binary Downloads'

  3. Extract the contents on your Mac

  4. Generate a cluster ID and format the storage using kafka-storage.sh

  5. Start Kafka using the binaries

  6. Setup the $PATH environment variables for easy access to the Kafka binaries

Installing Java JDK 11

To install Apache Kafka on Mac, Java 11 is the only prerequisite.

Screenshot of Amazon Corretto installer that will give you the distribution of OpenJDK 11 needed to install Apache Kafka on Mac

Please follow the instructions here to verify your installation of Amazon Corretto 11. Instructions on how to set the default version of Java on a Mac can be found here.

1 2 3 openjdk version "11.0.10" 2021-01-19 LTS OpenJDK Runtime Environment Corretto-11.0.10.9.1 (build 11.0.10+9-LTS) OpenJDK 64-Bit Server VM Corretto-11.0.10.9.1 (build 11.0.10+9-LTS, mixed mode)shell

Install Apache Kafka

Download the latest version of Apache Kafka from https://kafka.apache.org/downloads under Binary downloads.

The download page for Apache Kafka where you can download and install Kafka.

2. Click on any of the binary downloads (it is preferred to choose the most recent Scala version - example 2.13). For this illustration, we will assume version 2.13-3.0.0.

3. Download and extract the contents (double click in the Finder) to a directory of your choice, for example ~/kafka_2.13-3.0.0 .

4. Navigate to the root of the Apache Kafka folder and open a Terminal. Or Open a Terminal and navigate to the root directory of Apache Kafka. For this example, we will assume that the Kafka download is expanded into the ~/kafka_2.13-3.0.0 directory.

Start Kafka

The first step is to generate a new ID for your cluster

1 ~/kafka_2.13-3.0.0/bin/kafka-storage.sh random-uuid

This returns a UUID, for example 76BLQI7sT_ql1mBfKsOk9Q

Next, format your storage directory (replace <uuid> by your UUID obtained above)

1 ~/kafka_2.13-3.0.0/bin/kafka-storage.sh format -t <uuid> -c ~/kafka_2.13-3.0.0/config/kraft/server.properties

This will format the directory that is in the log.dirs in the config/kraft/server.properties file (by default /tmp/kraft-combined-logs)

Now you can launch the broker itself in daemon mode by running this command.

1 ~/kafka_2.13-3.0.0/bin/kafka-server-start.sh ~/kafka_2.13-3.0.0/config/kraft/server.properties
Screenshot of Apache Kafka setup in KRaft mode on Mac via Terminal

Don’t close this terminal window as it will shutdown the broker.

Congratulations, the broker is now running on its own in KRaft mode!

Setup the $PATH environment variable

In order to easily access the Kafka binaries, you can edit your PATH variable by adding the following line (edit the content to your system) to your system run commands (for example ~/.zshrc if you use zshrc):

PATH="$PATH:/Users/stephanemaarek/kafka_2.13-3.0.0/bin"

This ensures that you can now run the kafka commands without prefixing them.

After reloading your terminal, the following should work from any directory

1 kafka-topics.sh

Read more

You can read and learn more about the KRaft mode in Kafka here.

Was this content helpful?
2
0
PreviousHow to Install Apache Kafka on Mac?
NextHow to install Apache Kafka on Mac with Homebrew

Start Kafka clusters in seconds using Conduktor

Conduktor offers the fastest way to start Kafka ever. In a couple of clicks and a few seconds, you will launch a local Kafka cluster with any version of Apache Kafka and optionally the Schema Registry. Download Conduktor and see the instructions here.

Screenshot show how to start a Kafka cluster quickly with Conduktor
Start a local cluster in a couple of clicks
Live local cluster with Schema Registry in a few seconds
Your cluster is live and ready to go