Kafka Producer CLI Tutorial
How to send data into Kafka?
The Kafka console producer CLI, kafka-console-producer is used to read data from standard input and publish it to Kafka. Make sure you have started Kafka beforehand.
Use CLI commands with appropriate extensions for your platform, e.g.,
kafka-console-producer.bat for windows,
kafka-console-producer.sh for Linux and Mac
To produce to a Kafka topic, we need to provide the mandatory parameters:
Find your Kafka hostname and port e.g.,
If Kafka v2.5+, use the
If older version of Kafka, use the
Provide the mandatory parameters: topic name
kafka-console-producer.shCLI as outlined below
Make sure the topic first_topic is already created with 3 partitions and a replication factor of 1:
Start producing to the topic:
Kafka v2.4 or less:
To exit the Kafka console producer, use the keyboard combination
After the producer is opened, you should see a
> sign. Then any line of text you write afterwards will be sent to the Kafka topic (when pressing
Here are the common mistakes and caveats with the
Messages are sent with the
nullkey by default (see below for more options)
If the topic does not exist, it can be auto-created by Kafka:
A topic with the name provided should exist. If you specify a topic that does not exist yet, a new topic with the name provided will be created with the default number of partitions and replication factor.
These are controlled by the broker-side settings (in your
config/server.properties file), with the following defaults:
To enable message compression, default
gzip, possible values
'gzip', 'snappy', 'lz4', or 'zstd'
To pass in any producer property, such as the
An alternative to set the
acks setting directly
topic-input.txt (make sure each message is on a new line)
Produce messages to the topic from the file (see the end of the command)
By default messages sent to a Kafka topic will result in messages with
We must use the properties
key.separator to send the key alongside messages.
In this example, the separator between the key and the value is:
Do not forget to always include your key/value separator otherwise you will get an exception.
Conduktor & Kafka Producer
Now that you have seen how to perform all these tasks using the Kafka Producer CLI, you may wonder how you can achieve this using a graphical UI tool.
Using Conduktor you can produce data to Kafka topics using a define key and value, specify the target partitions, generate random data, use any data format, and even send data automatically every second