# Custom Partitioners: When Default Hashing Isn't Enough

Kafka's default partitioner uses murmur2 hashing to distribute keyed messages across partitions. Same key, same partition, guaranteed ordering.

But sometimes you need control the default doesn't give you: routing by geographic region, priority lanes, or spreading hot keys.

I've implemented custom partitioners for teams that needed geographic routing for GDPR compliance, priority lanes for payment processing, and hot key distribution for viral content. The pattern is the same each time.

> *Our default partitioner created a hot partition that handled 60% of traffic. A custom partitioner spreading our top 100 keys reduced p99 latency by 40%.*
>
> *Platform Engineer at a media company*

## How Default Partitioning Works

With a key:
```java
targetPartition = Utils.toPositive(Utils.murmur2(keyBytes)) % numPartitions;
```

Without a key (Kafka 2.4+): sticky partitioner batches to the same partition until the batch is full, then picks another.

## When You Need Custom Logic

**Geographic routing:**
```text
EU users -> partitions 0-1
US users -> partitions 2-3
APAC users -> partitions 4-5
```

**Priority lanes:**
```text
CRITICAL -> partition 0 (dedicated consumer)
NORMAL -> partitions 1-3
BULK -> partitions 4-5 (throttled consumer)
```

**Hot key distribution:** When a small number of keys dominate traffic, spread them across partitions instead of concentrating in one. Use [topic monitoring](https://docs.conduktor.io/guide/manage-kafka/kafka-resources/topics) to identify partition imbalances before they cause issues.

## Basic Implementation

```java
public class RegionPartitioner implements Partitioner {
    private Map<String, Integer> regionToPartition;

    @Override
    public void configure(Map<String, ?> configs) {
        String mapping = (String) configs.get("region.partition.map");
        regionToPartition = parseMapping(mapping);
    }

    @Override
    public int partition(String topic, Object key, byte[] keyBytes,
                         Object value, byte[] valueBytes, Cluster cluster) {
        int numPartitions = cluster.partitionsForTopic(topic).size();

        if (keyBytes == null) {
            // WARNING: Random partition breaks ordering for null keys
            // Consider sticky partitioner for better batching
            return ThreadLocalRandom.current().nextInt(numPartitions);
        }

        String keyStr = new String(keyBytes);
        String region = keyStr.split(":")[0].toUpperCase();

        Integer basePartition = regionToPartition.get(region);
        if (basePartition == null) {
            return Math.abs(Utils.murmur2(keyBytes)) % numPartitions;
        }

        return basePartition % numPartitions;
    }
}
```

Configure:
```properties
partitioner.class=com.example.kafka.RegionPartitioner
region.partition.map=EU:0,US:2,APAC:4
```

## Common Errors

**Negative partition:**
```java
// WRONG: Integer.MIN_VALUE stays negative after Math.abs()
int partition = Math.abs(key.hashCode()) % numPartitions;

// CORRECT: Use bitwise AND
int partition = (key.hashCode() & Integer.MAX_VALUE) % numPartitions;
```

**Partition out of range:** Always validate against cluster metadata:
```java
return calculatedPartition % numPartitions;
```

**Not thread-safe:** The producer calls `partition()` from multiple threads. Use `AtomicInteger` for counters.

## When to Avoid Custom Partitioners

Custom partitioners add operational burden:
- All producers must use the same version
- Partition count changes may break logic
- Bugs cause routing issues that are hard to debug

Consider alternatives first:

| Requirement | Alternative |
|-------------|-------------|
| Geographic routing | Separate topics per region |
| Priority lanes | Separate topics per priority |
| Hot key distribution | Add entropy to keys (`userId-timestamp`) |

Custom partitioners give precise control over message placement. Use that power carefully—separate topics are often simpler.

[Book a demo](https://www.conduktor.io/contact/demo) to see how Conduktor Console shows message rates per partition to help you spot hot partitions before they become incidents.