Know who depends on a topic before you change it

This release shows platform teams what depends on what, and puts their Kafka know-how where application teams create topics and connectors.

See every application that can read or write a topic

Before a migration or a topic change, no single place showed who produces to a topic and who reads from it. Stream Lineage now maps it from the access rules Console already reads:

• applications and service accounts, linked through the topics they can read or write
• edges from Kafka ACLs, Self-service permissions or Confluent role bindings
• a drill-down on one application or service account, hop by hop

The platform team knows who to warn before a change. It needs no Gateway or client change, and shows granted access, not live traffic. Explore Stream Lineage →

Hand application teams vetted settings for new topics and connectors

Not every application team has a Kafka expert, and creating a topic or connector in Console started from a blank form. Platform teams can now publish templates that pre-fill those forms:

• partitions, replication factor, configuration and labels for a topic
• a connector class and its configuration, never its secrets
• placeholders, such as a team name in your naming convention, that the user fills in

Application teams start from settings the platform team trusts. Templates only suggest: pair them with a resource policy to enforce limits. Admins publish them with the CLI. Write a topic template →

Charge Kafka Connect costs to the teams that run the connectors

Kafka Connect can be a large share of the Kafka bill, but Chargeback covered storage, partitions and traffic, not connectors. It now charges connector task hours to the applications that own them:

• one task-hour rate per cluster, with overrides per connector class
• Connect costs by cluster, application or label, down to each connector
• the same figures in the CSV export and the API

Platform leads can show each team what its connectors cost. Connect costs appear once the Connect cluster is configured in Console, and Chargeback is now on by default. Set your Connect rates →

Give a team part of a topic without copying it, now ready for production

Giving a team only part of a topic meant a Flink or Kafka Streams job writing a filtered copy to a new topic. Topic Views, in preview since May, are ready for production. Through Gateway, a consumer reads only:

• the records that match a SQL filter on the value, key, headers or timestamp
• the fields it needs

The platform team has no extra job to run and no copy to store. Views read JSON records, and each consumer names its view in its group ID. Set up a topic view →

Serve internal and partner clients from one Gateway, now ready for production

Internal services and partners need different network and security settings, which often meant running a second Gateway. Multiple listeners on one Gateway, in preview since May, are ready for production:

• one listener per group of clients, each with its own address and routing
• its own security protocol, with mTLS required on one listener and not another
• all of them in front of the same Kafka clusters

Platform teams run one Gateway for both. Listeners are now the recommended way to configure Gateway networking. Set up multiple listeners →

Also in this release

• Connector and consumer group fixes, scripted: stop a connector and set where it resumes, or reset a consumer group's offsets, from the API or CLI.
• Mask every field by default: one masking policy can cover every value in a message, with exceptions granted through RBAC.
• ACLs visible to topic owners: owners can be allowed to see the ACLs on their own topics, without a cluster-wide permission.
• The new consumer protocol, at a glance: consumer group descriptions say whether each group uses the classic protocol or KIP-848.

For a full list of changes, read the complete release notes.

Enjoying the new features? Share your experience on G2