KIP-392 implements a new broker plugin interface, ReplicaSelector, that lets you provide custom logic to determine which replica a consumer should use. You configure the plugin using the replica.selector.class option. Kafka provides two selector implementations. LeaderSelector always selects the leader replica, so messaging works exactly like it did before. LeaderSelector is also the default selector, so by default Kafka 2.4.0’s messaging behavior is the same as in previous versions.
RackAwareReplicaSelector attempts to use the rack ID to select the replica that is closest to the consumer.
Kafka already allows you to balance replicas across racks. You can use the broker.rack configuration to assign each broker a rack ID. Brokers will then try to spread replicas across as many racks as possible. This feature improves resiliency against rack failures. AWS availability zones correspond to racks.
you can configure the new client.rack to assign a rack ID for consumers. When it is enabled, RackAwareReplicaSelector tries to match the consumer’s client.rack with available broker.racks. It then selects the replica that has the same rack ID as the client.
Benefits:
Consuming messages from the closest replica could help reduce the load on your network.
Has potential cost benefits for public cloud environments. data transfers are free between brokers and clients in the same availability zone. Data transfers across different availability zones are billed.
Drawback:
the latency between the producer and the consumer will be higher when fetching from followers rather than a leader.