Kafka producer CLI tutorial

kafka-console-producer.sh example: send messages to a Kafka topic from the terminal — key-value pairs, acks configuration and delivery verification.

Learn how to produce messages to Kafka using kafka-console-producer

The kafka-console-producer CLI reads data from standard input and publishes it to Kafka topics. It's essential for testing, debugging, and quick data injection.

What you'll learn:

  • How to send messages to a Kafka topic
  • How to produce messages with keys
  • How to produce messages from a file
  • Important producer options and settings

Use CLI commands with appropriate extensions for your platform: for Windows - kafka-console-producer.bat, for Mac and Linux - kafka-console-producer.sh.

How to produce a message into a Kafka topic using the CLI?

To produce to a Kafka topic, we need to provide the mandatory parameters:

  • Find your Kafka hostname and port e.g., localhost:9092
  • If Kafka v2.5+, use the --bootstrap-server option
  • If older version of Kafka, use the --broker-list option
  • Provide the mandatory parameters: topic name
  • Use the kafka-console-producer.sh CLI as outlined below

Example

Make sure the topic first_topic is already created with 3 partitions and a replication factor of 1:

kafka-topics --bootstrap-server localhost:9092 --topic first_topic --create --partitions 3 --replication-factor 1

Start producing to the topic:

Kafka v2.5+:

kafka-console-producer --bootstrap-server localhost:9092 --topic first_topic

Kafka v2.4 or less:

kafka-console-producer --broker-list localhost:9092 --topic first_topic

To exit the Kafka console producer, use the keyboard combination Ctrl+C.

Command output

>

After the producer is opened, you should see a > sign. Then any line of text you write afterwards will be sent to the Kafka topic (when pressing Enter)

$ kafka-console-producer --bootstrap-server localhost:9092 --topic first_topic
>Hello World
>My name is Conduktor
>I love Kafka
>^C  (<- Ctrl + C is used to exit the producer)

Gotchas

Here are the common mistakes and caveats with the kafka-console-producer.sh command:

  • Messages are sent with the null key by default (see below for more options)
  • If the topic does not exist, it can be auto-created by Kafka:

A topic with the name provided should exist. If you specify a topic that does not exist yet, a new topic with the name provided will be created with the default number of partitions and replication factor.

These are controlled by the broker-side settings (in your config/server.properties file), with the following defaults:

auto.create.topics.enable=true
num.partitions=1
default.replication.factor=1

Extra important options you can set (advanced)

--compression-codec

To enable message compression, default gzip, possible values 'none', 'gzip', 'snappy', 'lz4', or 'zstd'

--producer-property

To pass in any producer property, such as the acks=all setting

--request-required-acks

An alternative to set the acks setting directly

How to produce messages from a file with the Kafka console producer CLI?

Example file topic-input.txt (make sure each message is on a new line)

Hello World
My name is John

Produce messages to the topic from the file (see the end of the command)

kafka-console-producer --bootstrap-server localhost:9092 --topic first_topic < topic-input.txt

How to produce messages with key in the Kafka console producer CLI?

By default messages sent to a Kafka topic will result in messages with null keys.

We have to use the properties parse.key and key.separator to send the key alongside messages.

In this example, the separator between the key and the value is: :

kafka-console-producer --bootstrap-server localhost:9092 --topic first_topic --property parse.key=true --property key.separator=:

Example input:

>example key:example value
>name:John

Do not forget to always include your key/value separator otherwise you will get an exception.

Quick reference

OperationCommand
Basic producekafka-console-producer --bootstrap-server localhost:9092 --topic NAME
Produce with keys... --property parse.key=true --property key.separator=:
With compression... --compression-codec snappy
With acks=all... --producer-property acks=all
From file... < filename.txt

See it in practice with Conduktor

Conduktor Console provides a visual interface for producing messages with full control over headers, keys, and serialization formats.

Next steps