# Other advanced Kafka producer configurations

*Learn advanced producer configuration options in 15 minutes*

Beyond basic producer settings, advanced configurations provide fine-grained control over producer behavior, performance, and reliability for production deployments.

**What you'll learn:**
- Buffer memory and timeout configurations
- Connection and network settings
- Monitoring and metrics configuration
- Environment-specific tuning templates

## Configuration categories

![Advanced producer configuration categories grouped into memory management, timeouts, network, and reliability settings](https://www.conduktor.io/assets/kafka/diagrams/other-advanced-kafka-producer-configurations.svg)

```mermaid
flowchart TD
    subgraph Memory["Memory Management"]
        BM["buffer.memory"]
        MBL["max.block.ms"]
    end

    subgraph Timeouts["Timeouts"]
        RT["request.timeout.ms"]
        DT["delivery.timeout.ms"]
        MA["metadata.max.age.ms"]
    end

    subgraph Network["Network"]
        CI["connections.max.idle.ms"]
        IF["max.in.flight.requests"]
        RB["reconnect.backoff.ms"]
    end

    subgraph Reliability["Reliability"]
        ID["enable.idempotence"]
        TID["transactional.id"]
    end
```

## Buffer memory configuration

### buffer.memory

Total memory allocated for producer buffering:

```properties
# Default: 33554432 (32MB)
buffer.memory=67108864  # 64MB total buffer
```

**Impact:**
- **Too small**: Producer blocks frequently when buffer fills
- **Too large**: Excessive memory usage, potential GC pressure
- **Optimal size**: Balance between memory efficiency and producer throughput

### max.block.ms

Time to block when buffer is full:

```properties
# Default: 60000 (1 minute)
max.block.ms=30000  # Block for 30 seconds maximum
```

### Memory calculation

```
Required buffer = batch.size × active_partitions × safety_factor

Example:
- 100 active partitions
- 32KB batch size
- 2x safety factor
= 32KB × 100 × 2 = 6.4MB minimum
```

## Request timeout settings

| Setting | Default | Description |
|---------|---------|-------------|
| `request.timeout.ms` | 30s | Timeout for individual requests |
| `delivery.timeout.ms` | 2min | Overall timeout including retries |
| `metadata.max.age.ms` | 5min | How long to cache topic metadata |

### Timeout relationship

If `delivery.timeout.ms` is shorter than the worst-case retry sequence, the producer throws a `ConfigException` at startup:

```properties
delivery.timeout.ms >= request.timeout.ms + (retries × retry.backoff.ms)
```

## Connection configurations

### max.in.flight.requests.per.connection

Maximum unacknowledged requests per connection:

```properties
# Default: 5
max.in.flight.requests.per.connection=5
```

**Impact on ordering:**

| Setting | Ordering | Throughput |
|---------|----------|------------|
| 1 | Strict | Lower |
| 5 (with idempotence) | Maintained | Higher |
| > 5 (without idempotence) | May reorder | Highest |

## Configuration templates

### High-throughput configuration

```properties
# Optimize for maximum throughput
batch.size=65536
linger.ms=20
buffer.memory=134217728
compression.type=lz4
max.in.flight.requests.per.connection=5
request.timeout.ms=30000
delivery.timeout.ms=120000
enable.idempotence=true
```

### Low-latency configuration

```properties
# Optimize for minimum latency
batch.size=16384
linger.ms=0
buffer.memory=33554432
compression.type=none
max.in.flight.requests.per.connection=1
request.timeout.ms=10000
delivery.timeout.ms=30000
```

### High-reliability configuration

```properties
# Optimize for maximum reliability
acks=all
retries=2147483647
enable.idempotence=true
max.in.flight.requests.per.connection=5
delivery.timeout.ms=600000
request.timeout.ms=30000
buffer.memory=67108864
batch.size=32768
```

### Memory-constrained configuration

```properties
# Minimize memory usage
buffer.memory=16777216      # 16MB
batch.size=8192             # 8KB
max.request.size=1048576    # 1MB
connections.max.idle.ms=60000
```

## Monitor your configuration

### Essential metrics to track

| Metric | Description | Alert threshold |
|--------|-------------|-----------------|
| `buffer-available-bytes` | Available buffer space | < 10% |
| `request-latency-avg` | Average request latency | > baseline |
| `record-error-rate` | Rate of failed records | > 0 |
| `record-retry-rate` | Rate of retried records | > baseline |

### Client identification

```properties
# Always set for easier troubleshooting
client.id=order-service-producer-01
```

## Common configuration mistakes

| Mistake | Problem | Solution |
|---------|---------|----------|
| Insufficient `buffer.memory` | Frequent blocking | Increase buffer or reduce batch size |
| Too high `linger.ms` | Unnecessary latency | Reduce to 0-20ms |
| Mismatched timeouts | `delivery.timeout` too small | Ensure delivery > request + retries |
| Missing `client.id` | Hard to troubleshoot | Always set meaningful client ID |
| Wrong `max.in.flight` | Ordering issues | Use 5 with idempotence |

Some configurations are interdependent. Enabling `enable.idempotence=true` automatically sets `retries=Integer.MAX_VALUE`, `max.in.flight.requests.per.connection=5`, and `acks=all`. Begin with default configurations and adjust one parameter at a time based on monitoring data—most applications work well with minimal configuration changes.

> **See it in practice with Conduktor**
> [Conduktor Console](https://docs.conduktor.io/guide/manage-kafka/kafka-resources/topics) helps you test different producer configurations and observe their effects. Monitor producer metrics to validate your configuration changes work as expected.

## Next steps

- [Configure producer batching](https://www.conduktor.io/kafka/kafka-producer-batching) for throughput optimization
- [Understand producer acknowledgments](https://www.conduktor.io/kafka/kafka-producer-acks-deep-dive) for durability
- [Enable compression](https://www.conduktor.io/kafka/kafka-message-compression) for network efficiency