How to change a Kafka topic configuration using the CLI
Change Kafka topic configuration from the CLI: use kafka-configs to describe topic config, override it with alter and remove custom configs with delete-config.
Learn how to view and modify topic configurations using kafka-configs in 10 minutes
While broker-level defaults work for most topics, some require customization. The kafka-configs CLI tool lets you override configurations for specific topics without affecting the cluster defaults.
What you'll learn:
- How to view current topic configurations
- How to add configuration overrides with kafka-configs
- How to remove configuration overrides
- Common configurations that need per-topic tuning
Common topic configuration overrides
| Configuration | Default | Description |
|---|---|---|
min.insync.replicas | 1 | Minimum ISRs for acks=all |
retention.ms | 7 days | Data retention time |
retention.bytes | -1 (unlimited) | Maximum partition size |
cleanup.policy | delete | delete or compact |
max.message.bytes | 1 MB | Maximum message size |
compression.type | producer | Broker-side compression |
Override topic configuration defaults
The kafka-configs CLI tool allows you to set, view, and delete configurations for topics, brokers, and other entities without restarting the cluster. Use CLI commands with appropriate extensions for your platform (e.g., kafka-configs.bat for Windows, kafka-configs.sh for Linux).
Let's change the min.insync.replicas configuration of a topic. The default broker value is 1, but for production with replication factor of 3, a value of 2 is recommended. See min.insync.replicas for details.
Before running Kafka CLIs make sure that you have started Kafka successfully.
First, create a topic named configured-topic with 3 partitions and a replication factor of 1, using Kafka topics CLI, kafka-topics
kafka-topics --bootstrap-server localhost:9092 --create --topic configured-topic --partitions 3 --replication-factor 1 Describe the topic to check if there are any configuration override set for this topic.
kafka-topics --bootstrap-server localhost:9092 --describe --topic configured-topic Topic: configured-topic TopicId: CDU7SBxBQ1mzJGnuH68-cQ PartitionCount: 3 ReplicationFactor: 1 Configs:
Topic: configured-topic Partition: 0 Leader: 2 Replicas: 2 Isr: 2
Topic: configured-topic Partition: 1 Leader: 3 Replicas: 3 Isr: 3
Topic: configured-topic Partition: 2 Leader: 1 Replicas: 1 Isr: 1 There is no configuration override set.
Set the min.insync.replicas value for the topic configured-topic to 2
kafka-configs --bootstrap-server localhost:9092 --alter --entity-type topics --entity-name configured-topic --add-config min.insync.replicas=2 And describe the topic again
kafka-topics --bootstrap-server localhost:9092 --describe --topic configured-topic Topic: configured-topic TopicId: CDU7SBxBQ1mzJGnuH68-cQ PartitionCount: 3 ReplicationFactor: 1 Configs: min.insync.replicas=2
Topic: configured-topic Partition: 0 Leader: 2 Replicas: 2 Isr: 2
Topic: configured-topic Partition: 1 Leader: 3 Replicas: 3 Isr: 3
Topic: configured-topic Partition: 2 Leader: 1 Replicas: 1 Isr: 1 Now, you can see there is a topic configuration override set (at the right side of the output) - min.insync.replicas=2.
You can delete the configuration override by passing --delete-config in place of the --add-config flag.
kafka-configs --bootstrap-server localhost:9092 --alter --entity-type topics --entity-name configured-topic --delete-config min.insync.replicas Describe the topic to make sure the configuration override has been removed.
Other common configuration changes
Set retention to 30 days
kafka-configs --bootstrap-server localhost:9092 --alter \
--entity-type topics --entity-name configured-topic \
--add-config retention.ms=2592000000 Enable log compaction
kafka-configs --bootstrap-server localhost:9092 --alter \
--entity-type topics --entity-name configured-topic \
--add-config cleanup.policy=compact Set multiple configurations at once
kafka-configs --bootstrap-server localhost:9092 --alter \
--entity-type topics --entity-name configured-topic \
--add-config min.insync.replicas=2,retention.ms=604800000 Some configuration changes take effect immediately, while others only apply to new data. Test configuration changes in a non-production environment first.
See it in practice with Conduktor
Conduktor Console provides a visual interface for viewing and modifying topic configurations. See all overrides at a glance and make changes without memorizing CLI syntax.
Next steps
- Send large messages to raise size limits via config
- Configure min.insync.replicas for durability
- Set up log retention for data lifecycle