Consumer Read from Closest Replica
From Apache Kafka v2.4+ consumers can now read from the closest replica
Kafka Consumers read by default from the broker that is the leader for a given partition.
In case you have multiple data centers, you run the risk of a slightly higher latency, as well as high network charges if you're being billing for cross data centers network traffic (which is the case with cloud computing platforms such as AWS).
Since Kafka 2.4, it is possible to configure consumers to read from the closest replica. This may help improve latency, and also decrease network costs if using the cloud.
To setup, you must set the following settings:
Broker setting (must be version Kafka v2.4+):
rack.idconfig must be set to the data centre ID (ex: AZ ID in AWS)
Example for AWS: AZ ID
replica.selector.classmust be set to
Consumer Client setting (v2.4+):
client.rackto the data centre ID the consumer is launched on
Example for AWS