# Kafka Headers: Metadata Without Polluting Your Payload

Every Kafka message carries context that doesn't belong in the payload: trace IDs, correlation IDs, content types, tenant identifiers. Stuffing this into your JSON bloats payloads and couples producers to consumers.

Headers solve this. Key-value pairs attached to records that travel with the message but don't pollute business data. Yet most teams either ignore them or misuse them.

> *We were adding 200 bytes of metadata to every message payload. Headers cut our storage costs by 15% and eliminated three schema migrations.*
>
> *Platform Engineer at an e-commerce company*

## Basic Usage

```java
ProducerRecord<String, String> record = new ProducerRecord<>("orders", "order-123", orderJson);
record.headers()
    .add("trace-id", traceId.getBytes(UTF_8))
    .add("source", "checkout-service".getBytes(UTF_8))
    .add("content-type", "application/json".getBytes(UTF_8));
producer.send(record);
```

On the consumer:

```java
Header traceHeader = record.headers().lastHeader("trace-id");
if (traceHeader != null) {
    String traceId = new String(traceHeader.value(), UTF_8);
    MDC.put("traceId", traceId);
}
```

[Conduktor Console](https://docs.conduktor.io/guide) displays headers alongside message content, making debugging easier than command-line tools.

## What Headers Are For

**Good uses:**
- Trace and correlation IDs
- Content type, schema version hints
- Routing information (tenant ID)
- Audit metadata (source system, timestamp)

**Bad uses:**
- Business data (belongs in payload)
- Large blobs (headers count toward message size)
- Data you need to query (Kafka doesn't index headers)

## Command-Line Inspection

```bash
# Produce with headers
printf 'trace-id:abc123,source:order-service\torder-123\t{"amount":99.99}' | \
  kafka-console-producer --bootstrap-server localhost:9092 \
  --topic orders --property parse.key=true --property parse.headers=true

# Consume with headers
kcat -b localhost:9092 -t orders -C \
  -f 'Headers: %h\nKey: %k\nValue: %s\n---\n' -e
```

## Common Pitfalls

**Headers in transforms (Kafka Streams 2.0+):**
```java
// map() preserves headers automatically since Kafka 2.0
stream.map((key, value) -> new KeyValue<>(key, transform(value)));

// Use transformValues() or process() when you need to read or modify headers
stream.transformValues(() -> new ValueTransformer<>() { ... });
```

**Case sensitivity:** Header keys are case-sensitive. `Trace-ID` and `trace-id` are different keys.

**Encoding mismatches:** Always specify UTF-8 when converting to/from bytes.

**Null checking:** Header values can be null. Always check before converting.

## Spring Kafka Shortcut

```java
@KafkaListener(topics = "orders")
public void process(
    @Payload String orderJson,
    @Header(name = "trace-id", required = false) String traceId,
    @Header(name = "source", required = false) String source) {
    // Headers extracted automatically
}
```

## Performance Considerations

Headers count toward `max.message.bytes`. Keep them small:

| Header Count | Avg Size | Overhead |
|--------------|----------|----------|
| 5 | 50 bytes | ~250 bytes |
| 10 | 100 bytes | ~1 KB |

Use short keys (`tid` vs `transaction-identifier`) and compact values.

Headers are one of Kafka's most underused features. They solve payload bloat, schema coupling, and cross-cutting concerns that don't fit in business data.

[Book a demo](https://www.conduktor.io/contact/demo) to see how Conduktor Console displays message headers alongside keys and values with filtering and search.