What a Kafka ACL actually is

An ACL (Access Control List) in Kafka is an allow rule. Each entry maps a principal (like User:alice) to an operation (Read, Write, Create, Describe, Alter) on a specific resource (Topic, ConsumerGroup, Cluster, TransactionalId). If a request doesn't match at least one allow rule, the broker denies it.

ACLs live inside the Kafka cluster itself. They're stored in KRaft (or ZooKeeper on older deployments) and enforced by the broker on every request.

Turning ACLs on

ACLs are off by default. Without them, every authenticated principal can do anything. Turn them on in broker.properties before connecting any clients you actually care about:

# broker.properties
authorizer.class.name=org.apache.kafka.metadata.authorizer.StandardAuthorizer
super.users=User:admin
allow.everyone.if.no.acl.found=false

The most common mistake here is leaving allow.everyone.if.no.acl.found=true. That flag makes Kafka fall back to "allow" when no ACL exists, which defeats the whole point. Set it to false so the broker denies any request that isn't explicitly permitted.

Adding ACLs with the CLI

Kafka ships with kafka-acls.sh for adding, listing, and removing rules:

# Grant alice read access to a topic
bin/kafka-acls.sh --bootstrap-server localhost:9092 \
  --add --allow-principal User:alice \
  --operation Read --topic customer-events

# List all ACLs on a cluster
bin/kafka-acls.sh --bootstrap-server localhost:9092 --list

# Remove an ACL
bin/kafka-acls.sh --bootstrap-server localhost:9092 \
  --remove --allow-principal User:alice \
  --operation Read --topic customer-events

Each grant is per-principal, per-operation, per-resource. No groups, no role inheritance, no bulk patterns beyond wildcard topic matching.

Where ACLs stop working

For small clusters with a handful of users, ACLs are fine. Past that, four things start to hurt:

• No groups. Every ACL is per-principal. Onboarding ten new engineers means adding ten sets of ACLs one at a time.
• No role inheritance. A "developer" role doesn't exist natively, so you reproduce the same permissions for every developer principal.
• No audit trail. The broker doesn't log who added which ACL when, so when an auditor asks about change history, you don't have one.
• No cross-cluster view. Listing ACLs across five clusters means running kafka-acls.sh --list five times and diffing the output.

Most teams start with raw ACLs, scale past the point where they're manageable, and end up layering RBAC on top. The ACLs stay. They're still the enforcement layer at the broker. Something else just manages them.

Managing Kafka ACLs at scale

The four limits above all start to matter at roughly the same scale. Most teams don't notice how fast the entries multiply: 20 teams, 500 topics, three permissions per topic per team is 30,000 ACL entries before you count service accounts, consumer groups, or non-prod environments. Each one is a separate ACL binding to create, review, and remove. Each cluster gets its own copy.

In practice, teams hit three failure modes:

1. The CLI script that becomes a load-bearing artefact. Someone writes a bash wrapper around kafka-acls.sh that iterates over a YAML file of grants. It works. Then it lives in a Git repo nobody owns, runs from one engineer's laptop, and drifts from production whenever a manual ACL is added during an incident.
2. The audit question with no answer. When a compliance reviewer asks "who has produced to payments-events in the last 90 days, and who granted them access?", kafka-acls.sh --list answers half the first part and none of the second. Broker logs are not an audit trail.
3. Off-boarding, principal by principal. A developer leaves the team. Revoking their access means finding every principal they own, across every cluster, and running a --remove for each. Until someone finishes that sweep, their credentials still authorize requests at the broker.

What scale looks like

The fix has the same shape regardless of which platform you use to deliver it. Grants are expressed against groups or roles, not individual principals. Changes go through a reviewable, audited interface, not ad-hoc CLI on a laptop. And the same definition applies across every cluster, not one config per environment.

Conduktor delivers this as a layer on top of native Kafka ACLs — the broker still enforces ACLs the way it always has; Conduktor manages them:

# payments-developers.yaml — grants the "payments-developers" group
# read on any topic with the "payments-" prefix, across every connected
# cluster, with an audit entry.
apiVersion: v2
kind: Group
metadata:
  name: payments-developers
spec:
  displayName: Payments Developers
  externalGroups:
    - payments-developers
  permissions:
    - resourceType: TOPIC
      cluster: "*"
      name: payments-
      patternType: PREFIXED
      permissions:
        - topicConsume

Applied through the Conduktor CLI:

conduktor apply -f payments-developers.yaml

A single declarative resource replaces dozens of kafka-acls.sh bindings. The same definition lives in Git, runs in CI, and is also available as a Terraform provider.

kafka-acls.sh vs Conduktor vs Confluent RBAC

Concern	kafka-acls.sh	Conduktor	Confluent RBAC
Granularity	Per-principal, per-resource	Group / role with pattern matching	Role-based, Confluent-only
Multi-cluster	One invocation per cluster	Single command across every connected cluster	Confluent Platform / Cloud only
Change audit	None natively (Log4j only)	Audit history of direct changes; request/approval metadata for self-service access	Confluent audit logs
Off-boarding	Manual --remove per principal	Revoke at the group / IdP level, all permissions follow	Tied to Confluent identity
Works against any Kafka	Yes	Yes (MSK, Confluent, Redpanda, Aiven, self-managed)	No — Confluent only
Self-service	None	Topic / access requests with approval workflow	Limited

Conduktor does not replace the broker's ACL enforcement; it replaces the spreadsheet and the bash wrapper around it. Underneath, the broker still sees standard Kafka ACLs.

ACLs vs RBAC

ACLs and RBAC aren't competing technologies; they stack. RBAC handles the who and what role at the identity layer (users, groups, roles), then translates those assignments into the ACLs the broker actually enforces. You get the usability of roles and groups, and the broker still enforces the low-level rules it's always enforced.

For the full picture (how ACLs fit alongside encryption, authentication, and auditing), read the Kafka Security Guide or the deeper Apache Kafka Security Playbook. For the RBAC side specifically, see Kafka RBAC.

Frequently asked questions