# Avro Schema Evolution: Compatibility Guide

I've debugged enough 3 AM deserialization failures to know: schema evolution isn't optional. Your schemas will change. The question is whether those changes break consumers or not.

Schema Registry enforces compatibility rules. Understanding them is the difference between smooth deployments and production incidents. [Visual schema management](https://docs.conduktor.io/guide/manage-kafka/kafka-resources/schema-registry) makes tracking versions and compatibility across all your clusters simpler.

> *We added a field without a default value. Deploy passed. Then 47 consumers started throwing deserialization errors. One line of JSON would have prevented it.*
>
> *Data Engineer at an e-commerce company*

## The Four Compatibility Modes

| Mode | Who Reads What | Upgrade Order |
|------|----------------|---------------|
| **BACKWARD** (default) | New consumers read old data | Consumers first |
| **FORWARD** | Old consumers read new data | Producers first |
| **FULL** | Both directions work | Any order |
| **NONE** | No checking | Careful coordination |

BACKWARD is the default because you can always rewind consumers to replay historical data.

## The Compatibility Matrix

Memorize this table:

| Change | BACKWARD | FORWARD | FULL |
|--------|----------|---------|------|
| Add field with default | ✓ | ✓ | ✓ |
| Add field without default | ✗ | ✓ | ✗ |
| Remove field with default | ✓ | ✗ | ✗ |
| Remove field without default | ✓ | ✗ | ✗ |
| Rename field | ✗ | ✗ | ✗ |
| Change field type | ✗ | ✗ | ✗ |

**Key insight:** Adding a field without a default breaks BACKWARD compatibility. Old data doesn't have that field, and there's no default to use.

## Safe Schema Evolution

Start with this schema:

```json
{
  "type": "record",
  "name": "User",
  "fields": [
    {"name": "id", "type": "string"},
    {"name": "email", "type": "string"}
  ]
}
```

**Adding a field (FULL compatible):**

```json
{"name": "country", "type": "string", "default": "US"}
```

New consumers use "US" for old data. Old consumers ignore the new field. Both directions work.

**Adding a nullable field:**

```json
{"name": "phone", "type": ["null", "string"], "default": null}
```

The union type with `default: null` makes this explicitly optional.

**Type changes never work.** Changing `int` to `string` breaks everything. Use a new topic or migration strategy.

## Test Before Deploying

This curl command prevents production incidents:

```bash
curl -X POST -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"schema": "{...your new schema...}"}' \
  "http://localhost:8081/compatibility/subjects/users-value/versions/latest?verbose=true"
# {"is_compatible":true} or {"is_compatible":false,"messages":["...reason..."]}
```

Add `?verbose=true` to see why a schema fails. Run this in CI before every deployment.

## Handling Breaking Changes

Sometimes you need incompatible changes. Options:

**New topic:** Create `users-v2` with the new schema. Migrate producers and consumers. Cleanest approach.

**Disable compatibility temporarily:**

```bash
# Set to NONE
curl -X PUT -H "Content-Type: application/vnd.schemaregistry.v1+json" \
  --data '{"compatibility": "NONE"}' \
  http://localhost:8081/config/users-value

# Register breaking schema, then restore
curl -X PUT ... --data '{"compatibility": "BACKWARD"}'
```

This risks deserialization failures for any consumer not upgraded simultaneously.

## Transitive Modes

Each mode has a transitive variant: `BACKWARD_TRANSITIVE`, `FORWARD_TRANSITIVE`, `FULL_TRANSITIVE`.

Non-transitive checks against the previous version only. Transitive checks against all versions.

**Use transitive when:** Consumers might replay from the beginning of the topic, or during disaster recovery.

## Common Errors

**"Schema being registered is incompatible"** — Your change violates the compatibility mode. Add default values or change the mode.

**"Reader missing default value"** — You added a field without a default in BACKWARD mode. Add `"default": "..."`.

**"Writer field missing from reader"** — You removed a required field in FORWARD mode. Add a default to the field first, then remove in the next version.

## Best Practices

1. **Use FULL compatibility** when possible—safest mode
2. **Always add defaults** to new fields, even if you think you'll always have a value
3. **Test in CI** with the compatibility API
4. **Avoid enums** for frequently changing values—use strings instead
5. **Use unions** for optional fields: `["null", "string"]`

Schema evolution is a contract between producers and consumers. Schema Registry enforces that contract. Get the compatibility mode right and you can evolve schemas without breaking production.

[Book a demo](https://www.conduktor.io/contact/demo) to see how Conduktor Console provides visual schema management and compatibility testing across all your clusters.