Consumer Read from Closest Replica
From Apache Kafka v2.4+ consumers can now read from the closest replica
Kafka Consumers read by default from the broker that is the leader for a given partition.
In case you have multiple data centers, you run the risk of a slightly higher latency, as well as high network charges if you're being billing for cross data centers network traffic (which is the case with cloud computing platforms such as AWS).
Since Kafka 2.4, it is possible to configure consumers to read from the closest replica. This may help improve latency, and also decrease network costs if using the cloud.
To setup, you must set the following settings:
Broker setting (must be version Kafka v2.4+):
rack.id
config must be set to the data centre ID (ex: AZ ID in AWS)Example for AWS: AZ ID
rack.id=usw2-az1
replica.selector.class
must be set toorg.apache.kafka.common.replica.RackAwareReplicaSelector
Consumer Client setting (v2.4+):
Set
client.rack
to the data centre ID the consumer is launched onExample for AWS
client.rack=usw2-az1