How to Start Kafka using Docker?
Start Kafka with Docker and run on all systems.
Running Kafka on Docker has the benefit that this method is going to work for all operating systems including Mac, Windows and Linux. It also helps to switch Kafka versions easily by simply updating the underlying docker-compose file.
You must install Docker with the steps mentioned below to work.
Install Docker Desktop on Mac.
Install Docker Desktop on Windows.
Install Docker on Linux (choose your distro on the lefthand side menu).
For Mac & Windows, if you have installed Docker Desktop, then Docker Compose is included as part of those desktop installs.
For Linux, follow the steps here (and do all the steps)
Docker Compose is a tool for defining and running multi-container Docker applications. As we'll start each of the Kafka services, e.g, zookeeper, broker, etc. in a different Docker container, docker-compose is a perfect tool to do that.
To use docker-compose, we have to define the services that make up our Kafka deployment in
docker-compose.yml so they can be run together in an isolated environment.
Go to the GitHub project - https://github.com/conduktor/kafka-stack-docker-compose and clone or download the project. It has docker-compose files for many configurations.
Next, go to the folder
kafka-stack-docker-compose and list the files.
Note the file
zk-single-kafka-single.yml. As the name suggests, we'll use it to launch a Kafka cluster with a single Zookeeper and a single broker.
Now, use this command to launch a Kafka cluster with one Zookeeper and one Kafka broker. Add
-d flag to run it in the background.
Check to make sure both the services are running:
Kafka is going to be exposed on your computer at
To run Kafka commands against Docker, we have two options:
Run commands directly from within the Docker container of Kafka (using
Run commands from our host OS (we must first install the binaries)
Option 1: Running commands from within the Kafka docker container
Then, from within the container, you can start running some Kafka commands (without
Option 2: Running commands from outside your container
For this, you must install the Java and the Kafka Binaries on your system:
instructions for Mac (follow the whole document except starting Kafka and Zookeeper)
instructions for Linux (follow the whole document except starting Kafka and Zookeeper)
instructions for Windows (follow the whole document except starting Kafka and Zookeeper)
One this is installed, the Kafka binaries should be in your path, and you will be good to issue your first Kafka command, for example
Now that we've verified the installation, we can stop the services by stopping the respective containers:
To remove all the resources including the containers altogether, use
down instead of