Not Everything Belongs in Kafka
Vinzenz Halhammer is a data engineer at Steadforce, a Munich-based IT consultancy, where he builds streaming applications for energy providers and grid operators. His definition of good Kafka architecture starts before any design work: evaluate what you actually want to achieve.
In this conversation
- →Building streaming applications for energy providers and grid operators
- →Why good architecture starts with evaluating what you want to achieve
- →The freshness question: millisecond latency vs hourly or daily workloads
- →How matching data freshness to the use case improves efficiency and cost
"Maybe I have some application where I need sub-second or millisecond latency. But maybe I also have a lot of applications, often in BI, where daily is enough, or hourly. So that I don't waste my precious Kafka resources."