How to Build Zero Trust Around Apache Kafka

"We have a zero trust policy from the cyber security."

Kafka engineer at a payments company

That's how zero trust usually reaches the team that runs Kafka: one sentence in a security policy, with nothing in it specific to a streaming platform. The team does what a zero trust checklist asks. Every connection is encrypted, every application gets its own credentials, every topic gets access rules, and the review passes.

The same cluster can still hand the card numbers in a payments topic to every service allowed to read it. And when security deletes an application's password, that application keeps reading and writing for as long as it stays connected (yes). AI agents now want the same data, and the same gaps apply to them.

It takes many controls to validate data access:

What Kafka Covers in Each Zero Trust Pillar

NIST SP 800-207 defines zero trust as a set of tenets, starting with "Network location alone does not imply trust." CISA's Zero Trust Maturity Model groups the work into five pillars and three capabilities that cut across them. Applied to a Kafka cluster:

Pillar	What Kafka does	What Kafka doesn't do
Identity	SASL and mTLS, one principal per client	Re-check a client once it's connected (off by default)
Devices	Nothing	Know which device or pod a client runs on
Networks	TLS on every listener	Separate clients from each other inside one cluster
Applications and workloads	ACLs on topics, groups and transactional IDs	Record who owns a topic
Data	Nothing at record level	Hide or encrypt a field for some readers only
Visibility and analytics	The authorizer log	Log allowed reads and writes (only refusals, by default)
Automation and orchestration	Admin API, Strimzi, Terraform	Revoke access on its own when a risk signal fires
Governance	kafka-acls	Track who requested an access and who approved it

Building zero trust around Kafka comes down to these steps:

1. Give applications and people their own principals. Deny by default, no wildcard ACLs, a short super.users list.
2. Force re-authentication with connections.max.reauth.ms. A deleted password then closes the connections it already opened.
3. Encrypt or mask sensitive fields per consumer. Each consumer gets only the fields it needs.
4. Log reads and ACL approvals. Every grant gets an owner and an approver.
5. Add topic ownership and a Kafka proxy. Steps 3 and 4 need them, because Kafka doesn't do either on its own.

Give Applications and People Their Own Principals

Most production clusters already encrypt connections, authenticate clients and enforce ACLs. Zero trust asks more of that baseline:

• One principal per application. A service account shared by several applications can't be scoped or audited per application.
• People and their tools use their own credentials. A developer who connects a Kafka UI, a CLI or an MCP server with an application's service account reads with that application's rights, and the logs show the application.
• Deny by default. allow.everyone.if.no.acl.found stays false, so a topic without ACLs is closed to everyone except super users.
• No wildcard ACLs for applications. A * or a broad prefix also grants every matching topic created later.
• A short super.users list. Super users skip ACL checks entirely.

Kafka checks ACLs on every request, so removing an ACL stops the client's next produce or fetch once the change reaches the broker.

Force Re-Authentication With connections.max.reauth.ms

Authentication happens once, when the client opens a connection, and producers and consumers keep their connections for as long as they run. KIP-368, which added re-authentication, describes the problem:

"once a client is connected to a broker the client is never challenged again and the connection may remain intact beyond the token expiration time"

KIP-368, Apache Kafka

Say alice is a service account writing to payments once a second. On Kafka 4.4.0 with SCRAM and default settings, we deleted its credential while the producer ran, opened a new connection as alice, then removed its write ACL:

20:12:52  delete alice's SCRAM credential
20:12:56  new connection as alice
          SaslAuthenticationException: Authentication failed during authentication
          due to invalid credentials with SASL mechanism SCRAM-SHA-512
20:13:13  last record written on the open connection: payment-32
20:13:14  remove alice's WRITE ACL on payments
          TopicAuthorizationException: Not authorized to access topics: [payments]

The open connection kept writing for 21 seconds after its credential was gone, and stopped only when the ACL went. The fix ships with Kafka and is off by default: connections.max.reauth.ms (default 0) caps a SASL session's lifetime, and a client that can't re-authenticate is disconnected.

Set connections.max.reauth.ms to the longest window a revoked identity may keep its access. With OAUTHBEARER, any positive value also ends each session when its token expires, if that comes first.

Encrypt or Mask Sensitive Fields per Consumer

Kafka authorizes per resource: a topic, a consumer group, a transactional ID, the cluster. Say the fraud service and the analytics service both hold Read on payments. Both get every field of every record, the card number included, because the broker stores the value as bytes it never opens.

CISA's model asks for "just-in-time and just-enough data access controls" at its most mature stage. On Kafka, just enough means per field and per reader: the fraud service sees the card number, the analytics service sees it masked or encrypted.

🚫 "Our Kafka is encrypted at rest and uses TLS everywhere, so the data is protected."

Encryption at rest and in transit protect the disk and the network, and both end at the broker. Every authorized reader gets plaintext, and so does anyone with a reader's credentials or access to the brokers. That last case is what security teams raise when someone else runs the brokers:

"The CSO concern was primarily on what if somebody get an access to [the managed Kafka], where we don't maintain that environment"

Engineering lead at a bank

Encrypting sensitive fields before they reach the broker, and decrypting them only for the readers that need them, makes those fields unreadable to anyone else who gets to the data. It also decides what an attacker gets with leaked credentials.

Log Reads and ACL Approvals

NIST's seventh tenet asks the enterprise to collect "as much information as possible", access requests included. Kafka's kafka-authorizer.log writes one line per refused request at its default INFO level:

INFO Principal = User:alice is Denied operation = WRITE from host = 127.0.0.1
on resource = Topic:LITERAL:payments for request = Produce with resourceRefCount = 1
based on rule DefaultDeny (kafka.authorizer.logger)

The writes alice was allowed to make left nothing: allowed operations log at DEBUG. Confluent and most managed services ship richer audit logs that record what each principal did. Who owns a topic and who approved a grant come from a catalog or a ticket, because Kafka doesn't store those facts.

kafka-acls --list shows that User:analytics may read payments, never which team owns payments, who approved the grant or whether it's still needed. In the conversations we have with platform teams, the part of zero trust they describe as manual is least privilege per identity: every grant needs someone who knows whether it should exist. When nobody does, ACLs pile up, and the platform team approves access to data it doesn't own.

Conduktor stores those facts. With federated ownership, each topic belongs to an application, access requests go to the team that owns it, and Console's audit log records who approved each grant and when. Gateway can also log the produce and fetch requests that pass through it.

Auditors ask for those same facts. PCI DSS restricts access to cardholder data by business need to know (Requirement 7) and requires logging every access to it (Requirement 10). DORA asks financial entities to limit access to "what is required for legitimate and approved functions and activities only" (Article 9). On Kafka, the evidence is a record per sensitive topic: who can read it, who approved each grant and when, which fields each reader gets in clear, and what was read. The full mapping is in One Kafka Control Layer for DORA, GDPR and IT-Grundschutz.

Add Topic Ownership and a Kafka Proxy

NIST splits zero trust into two components. A policy decision point decides whether a subject gets a resource. A policy enforcement point sits on the path and applies the decision, for example a "gateway component in front of resource that controls access." The two talk over a control plane, while "application data is communicated on a data plane."

On Kafka, the broker is the enforcement point for ACLs, and no component plays the decision point: an ACL is a decision stored without its reason. Field protection, read logs and approvals need both pieces:

• A decision layer. Owners, access requests, approvals and roles. It decides who may read which topic and which fields, and records who decided.
• An enforcement layer on the Kafka protocol. A proxy between clients and brokers that reads every produce and fetch, so it can encrypt or mask fields per reader, reject records that break a rule, re-check identities and log reads.

How Conduktor Fits

Conduktor splits the work the way NIST does. Console decides: with federated ownership, applications own their topics, other teams request access, the owning team approves, Console applies the Kafka ACLs, and every grant is logged with its approver. Gateway enforces on the Kafka protocol, in front of the Kafka clusters you already run, managed or not. AI agents follow the same rules: one connected through Console's MCP server acts with the permissions of the token it's given, and nothing more.

Pillar	Conduktor
Identity	Gateway authenticates each client (mTLS, OAUTHBEARER, SASL) and can force re-authentication; Console uses your OIDC or LDAP groups
Networks	Virtual Clusters and Partner Zones expose only the topics shared with each client
Applications and workloads	Federated ownership; traffic policies on acks, naming and rate limits
Data	Field-level encryption with your KMS, decrypted per user or group; masking on read; data quality rules before records reach Kafka
Visibility and analytics	Console and Gateway audit logs: resource changes, approvals, authentications, policy actions
Governance	RBAC per topic prefix, resource policies, per-subject schema access

Give AI Agents Their Own Principals

Few Kafka platforms meet the zero trust bar today. Most have relied for years on their readers being known teams and applications. AI agents change who reads: they read data like people and act like applications, and governing what they can see is becoming part of the security team's job.

That job lands on Kafka, where agents go for the latest data. Each agent should get what every reader on the platform should already have: its own identity, access its data owner approved, only the fields it needs, and a record of what it read.

See how Gateway encrypts Kafka fields per reader →

---

Related: If Kafka Credentials Leak, Does the Attacker Get Plaintext or Ciphertext? → · Where Should Your Kafka Encryption Keys Live? → · Kafka Schema Registry: Open by Default →