How a Leading German Automaker Unified Three Kafka Platforms for 700+ Users

Executive Summary

Apache Kafka is critical infrastructure at this German automaker. The event-driven architecture behind its vehicle sales process runs on it, which means every car the company sells, at a dealership or online, flows through Kafka. But Kafka had grown without coordination: each division (R&D, sales, production) had its own IT department, its own budget, and its own Kafka platform, each staffed by roughly ten people all doing more or less the same work.

Today, over 200 applications and 700+ active users run on a single multi-tenant Kafka platform operated by one team of ten, with Confluent Cloud underneath and Conduktor Console as the governance layer and API for the entire organization. Nobody touches the Confluent portal.

Why not just use Confluent's built-in tools? Confluent Cloud provides the broker and the ecosystem. As this team found, that governance alone doesn't cover 700+ users spread across divisions. They needed resource-level Kafka RBAC, self-service provisioning with automated guardrails, topic and application catalogs, and cross-team approval workflows. See how Console extends Confluent.

Challenges

Kafka is easy to plug into, so it spread across divisions faster than anyone set standards for it.

One IT project lead had spent four years running the platform that started in R&D and quietly became the company-wide default. He had neither the mandate nor the tooling to support it, but he knew the platform, and he had a plan for what it should become.

• Three platforms, three teams, no coordination: at least three separate Kafka platforms operated independently, one per division, each with roughly ten people duplicating the same work.
• 200,000 runaway partitions: users routinely created topics with 100 partitions for workloads that didn't need them, because, as the project lead put it, "they just like the number 100. One and two zeros." With no guardrails to stop them, production ballooned to 100,000 partitions and staging reached 200,000. When a broker went down, recovery took ages, with "everyone holding hands and fingers crossed."
• Shared superusers: application teams shared a single superuser per namespace, a model the project lead called "a huge security risk as well as compliance risk."
• No way to discover data: there was no public topic catalog, so teams could only see their own data. Finding out what other teams had was, in his words, "just not present."
• Onboarding measured in days: requests sat in what the project lead called a "horrible" ServiceNow queue for up to a week, and cross-team access required email chains and manual approval. "Super time consuming."

The stakes were rising at the same time. The company was rebuilding its entire vehicle sales process on event-driven architecture, every division was moving onto the same infrastructure, and the platform had to work.

Solution

When central IT raised the question of consolidating Kafka company-wide, the project lead was ready. He pulled out the plan he had been sitting on and became the owner of the initiative: one Kafka platform, operated by one team, serving every division.

None of the existing platforms was fit to become the central service, so the company moved to Confluent Cloud for the underlying infrastructure. A German IT consultancy introduced the team to Conduktor Console, which became the single interface for the entire organization. The project lead chose it in part because he didn't want to lock into any single Kafka provider: Conduktor gave him the flexibility to switch, with all resources managed in one place.

The architecture works on two levels:

• Platform team (~10 people): uses the Conduktor CLI and API to automate provisioning, enforce policies, and manage infrastructure through GitOps pipelines.
• Developers (700+): use the Conduktor UI as their day-to-day interface for browsing the topic catalog, creating resources, configuring alerts, monitoring connectors, and requesting access to other teams' data.

Confluent Cloud runs underneath, but no one interacts with it. Everything goes through Conduktor.

From shared superusers to individual identity

With Console's SSO and RBAC, every user authenticates via the company's identity provider and lands in Conduktor with the right group membership automatically. During onboarding, a GitOps pipeline creates owner groups in Conduktor and maps them to identity provider groups via external group mapping, which syncs users from the company directory and assigns the appropriate roles. From there, team members extend access to colleagues themselves, without involving the platform team.

Role-based access control gives the platform team control over what each user and team can do without becoming a bottleneck for every request. Users pushed back at first, because they weren't used to being controlled, but that control is exactly what made it safe to put every division on the same cluster. With thousands of IT staff and each division running its own identity infrastructure, SSO and RBAC were the first things that had to work.

Developers discover, create, and collaborate without waiting

Through Console's federated ownership model, every application is registered with a defined ownership group, so topics, connectors, and schemas all map to accountable teams. No more guessing who owns what.

Developers open the Conduktor UI and browse data across the entire cluster: what topics exist, who owns them, what schemas are in use. They create topics, schemas, and connectors themselves, and resource policies enforce the boundaries automatically. A cluster-wide policy caps topics at a default maximum of six partitions.

When a team needs another team's data, they request it in the Conduktor UI and the owning team approves or rejects it directly. The platform team is no longer in the middle.

Teams go live in 30 minutes, not five days

The flow from request to a team being live on the platform:

1. A team submits a request through Jira Service Management with their application name, ID, and technical details.
2. A platform engineer validates the OAuth client ID (the technical user credential for Kafka authentication). Because that credential is provisioned in a separate system, this is the one step that couldn't be automated.
3. A ticket status change triggers a GitOps pipeline that runs sanity checks, then uses the Conduktor CLI to create application instances, application groups, owner groups, and external group mappings.
4. The team logs in via SSO and manages their use case independently.

Once onboarded, developers use the Conduktor UI for their day-to-day work. The platform team also rolled out Console's built-in alerting and connector auto-restart, which teams configure directly in the UI. Before, every team was expected to build its own monitoring for Kafka Connect failures, and most never did. It was a blind spot.

Results

During the migration, the platform team forced every team to recreate its topics from scratch. No replicator, no cluster linking, no shortcuts. Unused and oversized topics turned up everywhere, because, as the project lead put it, "people tend not to know Kafka in and out." Combined with the six-partition guardrail, that drove a 10x reduction in partitions and a direct cut in Confluent Cloud infrastructure cost.

Before	After
100,000 production partitions	10,000
200,000 staging partitions	26,000
Onboarding queues of 5 to 7 days	Under an hour, typically 30 minutes
3 separate platforms, 3 separate teams	1 platform, 1 team
No guardrails on topic creation	Default max 6 partitions per topic
Cluster instability from ungoverned growth	Zero major incidents since adoption

- Three platforms consolidated into one: what previously required three separate teams now requires one, cutting the total cost of ownership for Kafka infrastructure significantly. In the project lead's words, "a huge cost saving."

• 10x fewer partitions: 100,000 to 10,000 in production and 200,000 to 26,000 in staging, with a policy in place to keep it that way.
• 700+ users self-sufficient: developers discover data, provision resources, configure alerting, and grant each other access without a ticket.
• Platform team off the firefighting line: with less time spent on manual provisioning and incidents, the team now focuses on new features and onboarding the next wave of platforms.

Conclusion

The consolidation isn't finished. One more platform was migrated this year and another with 500 users is in progress. The project lead who started as a firefighter and a misfit now owns the initiative reshaping how the entire company moves data, and his goal goes beyond consolidation: he wants to spread Kafka adoption across the company, replacing inefficient, time-delayed batch jobs used for moving data from A to B with smart, low-maintenance, real-time pipelines.

Frequently Asked Questions