# Kafka Streams vs ksqlDB: Choosing Right

Both process data from Kafka in real-time. Choosing wrong wastes engineering time and creates operational headaches.

ksqlDB is built on Kafka Streams. Every query compiles to a Streams topology. The question is whether SQL abstraction helps or limits you.

> *We started with ksqlDB because the team knew SQL. When we needed external API calls, we switched to Kafka Streams for that pipeline. Now we use both.*
>
> *Data Engineer at a retail company*

## The Core Difference

Kafka Streams is a Java library you embed in your application. ksqlDB is a standalone server with SQL interface.

**Kafka Streams:**

```java
KTable<Windowed<String>, Double> hourlyRevenue = orders
    .groupBy((key, order) -> order.getRegion())
    .windowedBy(TimeWindows.ofSizeWithNoGrace(Duration.ofHours(1)))
    .aggregate(() -> 0.0, (region, order, total) -> total + order.getAmount());
```

**ksqlDB:**

```sql
CREATE TABLE hourly_revenue AS
  SELECT region, SUM(amount) AS total
  FROM orders
  WINDOW TUMBLING (SIZE 1 HOUR)
  GROUP BY region;
```

Same result. Different tradeoffs.

## When to Use Kafka Streams

**Complex logic:** ksqlDB handles standard SQL. When you need conditional routing with external validation, Kafka Streams wins.

```java
transactions
    .filter((key, tx) -> tx.getAmount() > 10000)
    .mapValues(tx -> {
        FraudScore score = fraudService.evaluate(tx);  // External call
        tx.setFraudScore(score.getValue());
        return tx;
    })
    .split()
    .branch((key, tx) -> tx.getFraudScore() > 0.8, Branched.withConsumer(s -> s.to("fraud-review")))
    .defaultBranch(Branched.withConsumer(s -> s.to("approved")));
```

ksqlDB cannot call external services. HTTP calls, database lookups, ML inference—use Kafka Streams.

**Custom state stores:** Direct access to RocksDB, custom serializers, TTL policies.

**Embedded in microservices:** No additional infrastructure. Deploy as standard JAR. Scale by running more instances.

**Processor API:** When DSL isn't enough, raw access to stream processor lifecycle.

## When to Use ksqlDB

**Rapid prototyping:** Explore data without writing code.

```sql
SELECT * FROM orders EMIT CHANGES LIMIT 10;
```

**SQL-native teams:** If your team knows SQL but not Java, ksqlDB removes the learning curve.

**Connect integration:** Manage connectors from SQL.

```sql
CREATE SOURCE CONNECTOR postgres_source WITH (
  'connector.class' = 'io.debezium.connector.postgresql.PostgresConnector',
  'database.hostname' = 'postgres'
);
```

**Simple aggregations:** Straightforward windowed operations without business logic.

## Decision Matrix

| Criteria | Kafka Streams | ksqlDB |
|----------|---------------|--------|
| Team skills | Java developers | SQL analysts |
| External API calls | Supported | Not supported |
| Testing | Standard unit/integration | Limited |
| Deployment | JAR in your app | Dedicated cluster |
| Debugging | Full stack traces | Query analysis |

## Operational Differences

**Deployment:** Kafka Streams is a library—no cluster to manage. ksqlDB requires dedicated server instances.

**Scaling:** Both limited by partition count. Maximum parallelism = number of partitions. A [unified console](https://docs.conduktor.io/guide) helps track consumer lag across both Kafka Streams and ksqlDB applications.

**Performance:** ksqlDB has SQL parsing overhead. For high-volume, latency-sensitive workloads, measure before committing.

**State restoration:** Both maintain local state stores backed by changelog topics. After crashes:

| State Size | Recovery Time |
|------------|---------------|
| 1 GB | ~30 seconds |
| 10 GB | 2-5 minutes |
| 100 GB+ | 30-60 minutes |

During recovery, the instance can't process new records. Use `num.standby.replicas=1` for faster failover.

## Hybrid Approach

Use both. ksqlDB for quick transformations. Kafka Streams for complex business logic.

```
[Source] → [ksqlDB] → [Intermediate Topics] → [Kafka Streams] → [Output]
           filtering    simple enrichment       external calls    complex logic
```

Common in mature organizations. Use ksqlDB for the 80% that fits SQL. Use Kafka Streams for the 20% that requires code.

The best choice depends on your team and constraints. Neither is universally better.

[Book a demo](https://www.conduktor.io/contact/demo) to see how Conduktor Console shows Kafka Streams and ksqlDB consumer lag side-by-side, with state store metrics and topology visualization.