Learn about Apache Kafka and its ecosystem in 20 minutes.

What is Apache Kafka?

Learn how Kafka works, get a strong understanding of the different components

Kafka Fundamentals

Understanding Kafka

Starting Kafka

Learn how to use Kafka Command Line Interface (CLI)

Kafka CLI Tutorials

How to interact with Kafka programmatically?

Kafka Programming Tutorials

Practice using Kafka

Kafka Advanced Concepts

Series of advanced tutorials on important Kafka administration subjects

Kafka Administration

Next Level Kafka

Cleanup policy for Kafka log compaction screenshot from Conduktor

Kafka topics configuration log compaction

Conduktor & Kafka Log Compaction

Log Compaction & API Calls

Ordering Intact

Windows Users: you can't use log compaction if you're not using WSL2

kafka-topics.sh --bootstrap-server localhost:9092 --create --topic employee-salary \
    --partitions 1 --replication-factor 1 \
    --config cleanup.policy=compact \
    --config min.cleanable.dirty.ratio=0.001 \
    --config segment.ms=5000

kafka-topics.sh --bootstrap-server localhost:9092 --describe \
                --topic employee-salary

kafka-console-consumer.sh --bootstrap-server localhost:9092 \
    --topic employee-salary \
    --from-beginning \
    --property print.key=true \
    --property key.separator=,

kafka-console-producer.sh --bootstrap-server localhost:9092 \
    --topic employee-salary \
    --property parse.key=true \
    --property key.separator=,

Patrick,salary: 10000
Lucy,salary: 20000
Bob,salary: 20000
Patrick,salary: 25000
Lucy,salary: 30000
Patrick,salary: 30000

Bob,salary: 20000
Lucy,salary: 30000
Patrick,salary: 30000
Stephane,salary: 0

What is the data retention policy in Kafka?

Kafka Topic Configuration: Log Compaction

Log Compaction and Kafka Data Retention Tutorial about Kafka log cleanup policies, Kafka log compaction practices, configurations and examples.

Advanced Kafka Topic Tutorial about data retention and log compaction in Apache Kafka Topics