Learn about Apache Kafka and its ecosystem in 20 minutes.

What is Apache Kafka?

Learn how Kafka works, get a strong understanding of the different components

Kafka Fundamentals

Understanding Kafka

Starting Kafka

Learn how to use Kafka Command Line Interface (CLI)

Kafka CLI Tutorials

How to interact with Kafka programmatically?

Kafka Programming Tutorials

Practice using Kafka

Kafka Advanced Concepts

Series of advanced tutorials on important Kafka administration subjects

Kafka Administration

Next Level Kafka

Screenshot of Kafka min.insync.replicas dialog in Conduktor

Kafka Topic Configuration for Min Insync Replicas

Conduktor screenshot of Kafka Topic config

Kafka Topics Configuration in Conduktor

Conduktor & Advanced Topic Configuration

Large Kafka Message Diagram showing how you can send large messages in Apache Kafka using an external store and metadata messages that keep within the 1MB message limit recommended for Apache  Kafka.

Send Large Kafka Messages with An External Store

Confusing setting

kafka-topics.sh --bootstrap-server localhost:9092 --create --topic large-message --partitions 3 --replication-factor 1

kafka-configs.sh --bootstrap-server localhost:9092 \
                --alter --entity-type topics \
                --entity-name configured-topic \
                --add-config max.message.bytes=10485880

kafka-console-consumer.sh --bootstrap-server localhost:9092 \
    --topic large-message \
    --from-beginning \
    --consumer-property max.partition.fetch.bytes=10485880

properties.setProperty(ConsumerConfig.FETCH_MAX_BYTES_CONFIG, "10485880");

kafka-console-producer.sh --bootstrap-server localhost:9092 \
    --topic large-message \
    --producer-property max.request.size=10485880

properties.setProperty(ProducerConfig.MAX_REQUEST_SIZE_CONFIG, "10485880");

The Kafka max message size is 1MB. In this lesson we will look at two approaches for handling larger messages in Kafka.

How to send Large Messages in Apache Kafka?

Learn about the Kafka Max Message Size and How to Send Large Messages in Kafka with the two approaches covered in this advanced Kafka tutorial.

Learn about the Kafka Max MEssage Size and How to Send Large Messages in Kafka with the two approaches covered in this advanced kafka topics tutorial. Messages above the Kafka Max Message Size Limit of 1MB can be handled in Kafka using an external store or modifying configurations.