SNI Routing vs Port Routing in a Kafka Proxy

You put a proxy in front of Kafka. You expect every client that connects to reach every broker through it. This is how the Kafka protocol works: a client bootstraps on any broker and asks for the topology, the list of all the other brokers. Each broker comes back with its own advertised.listeners address, and the client then talks directly to those addresses. A Kafka proxy has to rewrite them, or it gets bypassed.

You have N gateways deployed. You have M brokers. How does the mapping work? An application still opens one connection per partition-leader it produces to or consumes from, so every one of those brokers needs an address the client can actually reach.

On the proxy, a listener is simply an entry point a client connects to (Kafka brokers use the same word for their own listeners, but here we always mean the proxy's). It's typical to run more than one to segregate different sources of traffic. For instance, you want Console to reach Gateway on an internal cluster DNS name, but external clients to use a different name, on a load balancer, over TLS. Same proxy, two populations, two ways of being addressed. You can't do that with one listener.

Port mapping and routing strategy

A proxy listener exposes two independent settings:

• Port mapping. What port does the proxy advertise for a broker, versus the port it actually binds locally? These can differ.
• Routing strategy. How does the proxy tell which broker a connection is for: by the port it landed on, or by the hostname in the TLS handshake?

The address remap: advertised vs local

A proxy binds a socket locally, but the address it advertises to clients doesn't have to match. This is exactly what NAT, Docker port mapping, and load balancers do. In Conduktor Gateway, a port spec uses the same advertised:local form:

9092                    # advertised = local = 9092
443:9092                # bind 9092 locally, advertise 443 (an LB fronts :443)
29092-29095:9092-9095   # bind 9092-9095, advertise 29092-29095 (Docker/NAT)

The proxy resolves an inbound connection by its local port, and rewrites the Metadata response using the advertised port.

Why any of this is necessary. Kafka puts network addresses inside its responses. Compare this to a reverse proxy, where the client never needs or learns the real backend's address. Kafka clients connect straight to the broker leading each partition, so the cluster has to tell them where every broker lives. Put a proxy in the middle and you inherit the job of translating those addresses.

A port per broker

The simplest routing strategy gives every broker its own port on the proxy: a connection to port 9092 is for broker 0, 9093 for broker 1, and so on. The proxy knows the target broker purely from the local port it landed on.

Gateway lays the ports out in order: a range like 9092-9095 maps to brokers in sequence, and with multiple clusters it assigns a fixed port to every broker across all of them.

This falls apart in production for two reasons:

• Firewalls. A port for every broker, in every cluster you route, adds up fast. Ask your security team to open a range of broker ports to a client network, and they will push back.
• Topology changes. Adding or removing a cluster shifts the ports of every cluster after it, and scaling a cluster past its reserved range does the same. Either way, you now have a dependency on the order and size of your clusters, and that isn't stable.

Port routing is fine for development, test, and a stable single cluster. It doesn't scale to multi-cluster production.

SNI routing: one port, a hostname per broker

The production way is to keep a single port on the proxy and use SNI routing. During the TLS handshake the client sends the server name it's connecting to (the SNI field of the ClientHello). The proxy reads that name and resolves the target broker from it.

The broker's advertised address becomes a hostname built from a template:

broker-{{physicalCluster}}-{{nodeId}}.{{advertisedHost}}

• {{advertisedHost}} is the stable suffix every address shares, something like kafka.company.com that points at the Gateway or the load balancer in front of it.
• {{physicalCluster}} identifies the cluster and {{nodeId}} the broker; both are filled in dynamically. When a client connects to broker-orders-2.kafka.company.com, the proxy reads cluster orders and node 2 from the name and routes there. Such a configuration looks like this:

GATEWAY_LISTENER_EXTERNAL_SECURITY_PROTOCOL=SASL_SSL
GATEWAY_LISTENER_EXTERNAL_ROUTING=sni
GATEWAY_LISTENER_EXTERNAL_PORTS=9092
GATEWAY_LISTENER_EXTERNAL_ADVERTISED_HOST=kafka.company.com
GATEWAY_LISTENER_EXTERNAL_ADVERTISED_HOST_PATTERN=broker-{{physicalCluster}}-{{nodeId}}.{{advertisedHost}}

One port to open, and broker identities resolve dynamically, so adding clusters and brokers doesn't touch your load balancer or firewall rules. The one thing you still line up is DNS, and a wildcard makes it a one-time setup (more on that below).

Gateway enforces the shape of a valid SNI listener at startup:

• it must use TLS (SNI lives in the TLS handshake; there's no plaintext path),
• its advertisedHostPattern must contain {{nodeId}} (otherwise there's nothing to route per broker),
• it must bind a single port (the whole point).

What's the cost of using SNI?

• TLS on the SNI listener. The routing signal is in the TLS handshake, so an SNI listener can't be PLAINTEXT (you shouldn't expose PLAINTEXT externally anyway).
• DNS for every broker name. The advertised names (broker-orders-0.kafka.company.com and the bootstrap) all have to resolve, on the client side, to your load balancer. A wildcard record (.kafka.company.com → the LB) makes this a one-time setup and lets brokers scale with no DNS change. Without it, because your org bans wildcard DNS or you don't own the zone, you hand-create an A or CNAME per broker (or Terraform them) and add one for every new broker. It's a separate check from the cert (next): the name has to resolve (DNS) and sit in the cert's SAN* (TLS), and a wildcard on one side doesn't cover the other.
• A wildcard cert. Using *.kafka.company.com means a single wildcard or SAN cert covering every broker hostname. Gateway can hold several certs in the keystore and present the right one by SNI, but the common setup is one multi-SAN cert. Monitoring expiry and rotating on time are part of the deal, though Gateway reloads the keystore on a timer (every five minutes by default), so a renewed cert is picked up without a restart.

On Kubernetes, cert-manager has limitations. It manages a single certificate per keystore, so the "one cert per broker hostname in the keystore" setup some security teams ask for isn't possible. On K8s, plan for one multi-SAN cert.

• The load balancer has to preserve SNI. The proxy needs to read the SNI name, so the load balancer must forward the TLS connection untouched and let Gateway terminate it. If the load balancer terminates TLS itself, it has to re-originate the connection to Gateway with the right SNI.
• SNI is a routing hint, not authentication. It's a client-supplied string, sent in cleartext (pre-ECH), spoofable. Routing to a broker is not the same as being allowed to use it. Authentication still has to happen a layer up, via mTLS or SASL.

Real deployments run several listeners

As mentioned earlier, you often deploy the proxy with several listeners at once, each addressing a different population.

We had this example of a bank using Kubernetes: Console talks to Gateway over the internal cluster DNS and external clients come in through a load balancer. Often, traffic is not allowed to leave the K8s network and hairpin back in. Two listeners solve it: an internal one on port routing and plaintext for Console, an external one on SNI and SASL_SSL for clients.

# Different advertised hosts per listener → internal load balancing must be off
GATEWAY_FEATURE_FLAGS_INTERNAL_LOAD_BALANCING=false

# Internal: Console, inside the cluster
GATEWAY_LISTENER_CONSOLE_SECURITY_PROTOCOL=PLAINTEXT
GATEWAY_LISTENER_CONSOLE_ROUTING=port
GATEWAY_LISTENER_CONSOLE_PORTS=9092-9095
GATEWAY_LISTENER_CONSOLE_ADVERTISED_HOST=gateway.kafka.svc.cluster.local

# External: clients, over the load balancer (local port distinct from the internal listener)
GATEWAY_LISTENER_EXTERNAL_SECURITY_PROTOCOL=SASL_SSL
GATEWAY_LISTENER_EXTERNAL_ROUTING=sni
GATEWAY_LISTENER_EXTERNAL_PORTS=443:19092
GATEWAY_LISTENER_EXTERNAL_ADVERTISED_HOST_PATTERN=broker-{{physicalCluster}}-{{nodeId}}.kafka.company.com

Gateway's internal load balancing must be disabled here, since it otherwise requires a single shared advertised host across all listeners.

Workloads across clouds. If you have apps across AWS, GCP, Azure, and on-prem, each network has its own DNS. Run a listener per entry point, each rewriting broker addresses into names that resolve from that network, with SNI carrying the cluster and broker inside the name.

So which one?

	Port per broker	SNI routing
Ports exposed	One per broker, per cluster	One, total
TLS	Optional	Mandatory
DNS	One A record	Wildcard (*.kafka.company.com)
Certificate	Optional	Usually one wildcard/SAN cert
Load balancer	One TCP listener per port	One, L4 TLS passthrough
Add/remove a cluster	Shifts ports, can break clients	Other clients unaffected
Best for	Dev, test, single cluster, stable topology	Production, multi-cluster, scale

The routing strategy is really a production networking and security decision: it wires into your firewall, your DNS, and your certs.

For the mechanics of how the proxy rewrites addresses and why it's a control point and not a basic load balancer, see what a Kafka proxy can do. For the exact configuration, check the Gateway reference docs. Happy to discuss your specific topology, talk to us.