If you need a real boundary between two things on Kafka in 2026, you build another cluster. Two environments, two business units, an acquired company: each one is its own brokers, its own controller quorum, its own bill. Multi-tenancy inside a cluster is a naming convention.
Every other layer of the infrastructure stack has already been through this. A resource starts out shared by convention, someone builds a layer that separates the logical boundary from the physical thing, boundaries stop costing hardware, and nobody goes back. Kafka is the last layer where that hasn't happened, and what's changing it is the virtual cluster, arriving from outside the broker.
Every Other Layer Made Boundaries Free
| Layer | Shared by convention | The layer that changed it | A boundary became |
|---|---|---|---|
| Servers | One machine, many logins | Hypervisor | A VM, created in seconds |
| Networks | One switch, firewall rules by agreement | VLAN tagging | A config entry, not a cable run |
| Compute | One Kubernetes cluster, everyone in default | Namespaces | A YAML object |
| Databases | One instance, everyone in the same tables | Schemas and views | A grant |
| Kafka | One cluster, topic prefixes by agreement | Another cluster |
Why Kafka Can't Add Namespaces in the Broker
Namespaces were proposed a decade ago: KIP-37, "Add Namespaces to Kafka", has been "Under Discussion" since October 2015. The multi-tenancy guidance that grew up in its place says what you get instead:
"Creating User Spaces (Namespaces) For Tenants With Topic Naming." — Apache Kafka documentation, Multi-Tenancy
That means topic prefixes, prefixed ACLs, and quotas: everyone sees the global name table, permissions deny after the fact, and the convention lives in every application's code. It works, with discipline, at moderate scale, the same way multi-user Unix worked. The hypervisor still won, because sharing one machine politely and having your own machine are different products, and the second one is what security teams, auditors and acquisitions ask for.
The layer never landed in the broker for a structural reason. Kafka's protocol tells clients exactly what's there: a client's first request is for metadata, and the answer is broker addresses, partition leaders, and the full topic list. Anything that wants to show a client a different picture has to sit between the client and the broker and speak the protocol itself.
That means a proxy, and for Kafka's first decade a protocol-aware proxy didn't exist.
Everyone Building It Puts It in Front of the Broker
It exists now, and more than one vendor has built the same thing. Confluent's own Kora engine paper describes how their cloud works:
"The unit of provisioning, however, is known as a logical cluster which represents an isolated tenant within that cluster." — Confluent, on the Kora engine
The largest Kafka vendor virtualizes Kafka at scale, in production, with strong isolation, so the model is proven. It's built into their cloud rather than offered in front of yours, though, so when you need a boundary between two of your own things on Confluent Cloud, you still buy another cluster or use RBAC inside one.
WarpStream reached the same design from a different direction, with a virtual cluster as the unit its agents serve. Kroxylicious, the open-source Kafka protocol proxy, lists multi-tenancy as a headline use case, and Conduktor Gateway ships it as Virtual Clusters against any Kafka vendor. The first two built the layer inside their own platforms, for their own tenants. The second two are proxies you run in front of clusters you already have. Four teams arrived at the same place: the boundary layer goes in front of the broker.
What a Virtual Cluster Isolates, and What It Doesn't
A hypervisor virtualizes compute: it schedules CPU, partitions memory, and guarantees one guest can't starve another. A Kafka proxy doesn't do that. Tenants on virtual clusters share the same brokers, the same disks, and the same page cache. What the proxy does isolate is everything a tenant can see, name, or be identified by, and the table below puts that next to the two layers people compare it with.
| Isolated per tenant | Kubernetes namespace | Hypervisor | Virtual Cluster |
|---|---|---|---|
Names: two tenants can both have orders | Yes | Yes | Yes, by prefix translation and aliases |
| Identity: own service accounts, own credentials | Yes | Yes | Yes, own authentication layer |
| Access control: own ACL universe | Yes | Yes | Yes |
| Visibility: other tenants' resources are absent, not denied | Yes | Yes | Yes, in the metadata the client receives |
| Policy at the boundary: what may enter, in what shape | Yes, admission controllers | No | Yes, scoped to the tenant |
| Data at the boundary: encrypt or mask per tenant | No | No | Yes |
| Audit of every crossing, attributed to the tenant | Partial | No | Yes |
| Metering per tenant | Partial | Yes | Yes |
| CPU, disk, page cache | No | Yes | No, that stays with Kafka's per-broker quotas |
Ask why a cluster-per-boundary deployment exists and the answer is almost never page cache. It's that two teams can't both have a topic called orders, that the acquired company's credentials must not work against yours, or that the auditor wants to see the partner's access as a separate thing, and all of those are rows the layer owns.
🚫 "We need a separate cluster for the partner integration. Isolation."
Isolation of what? If the answer is names, credentials, visibility and blast radius, a namespace does it and the cluster is paying for a guarantee nobody asked for. If the answer is that the partner's replay of a week of cold data must never touch your p99, that's compute, and you still need the hardware. The second case is rarer than the number of clusters suggests.
Which Boundaries Need a Wall
VMs made sense per application because applications don't share memory. Internal teams share data constantly; that's the point of a streaming platform. Giving every team its own virtual cluster rebuilds the silo the platform was meant to remove.
Walls are for tenants who should never meet, and the teams inside one company sit together under them:
- Environments. Dev and QA on one non-production cluster, each with its own namespace, and production untouched. Topics keep the same name across environments because the prefix lives in the proxy rather than in the code.
- Subsidiaries and acquisitions. The acquired company keeps its topic names inside its own namespace. Integration happens through explicit aliases instead of a renaming campaign.
- Partners. An external consumer sees a small, business-named surface, with its own credentials and its own audit trail.
For teams inside one company, the right model is a shared namespace with ownership and governed access. There's a practical guide to choosing between the two, and a walkthrough of the non-production case specifically.
Kafka Is Where the Other Layers Were
The other layers all went through the same sequence: a cost workaround, then the standard for environments, then physical-per-boundary became the thing you justified in an architecture review. Kafka is early in that sequence. The protocol surface is virtualized, the compute is still shared, and the proposal to do it inside the broker has been under discussion for a decade.
Everywhere else, boundaries stopped costing hardware. Kafka is the last place they still do, and the layer that ends that sits in front of the broker, not inside it.
Related: How to Approach Multi-Tenancy in Kafka → · What a Kafka Proxy Can Do → · Virtual Clusters docs →