Kafka: Your Last FinOps Bastion

At Conduktor, we work with large enterprises, and this feedback is common:

"Kafka is the last unresolved cost area"

— A Conduktor customer, discussing their FinOps program

Most teams we speak with already import their provider's Kafka bill into a central FinOps tool but can't assign the spend to applications across environments. Kafka remains the small village holding out: the infrastructure bill is available, but who owns what? Who should pay for what?

During our Customer Advisory Board (CAB) sessions, executives want to encourage Kafka adoption internally. They also need visibility and accountability by team and use case, internal billing they can defend, and ways to influence how teams use Kafka.

Kafka cost allocation has to give application teams a reason to act.

Why Kafka cost allocation is hard

A cloud tag can assign a cluster to the streaming platform team. But Kafka is a shared resource, so you need to go one level deeper. Application teams decide what to produce, how long to retain it, and how often to read topics (some use cases are far from typical). Each decision creates costs that a cluster tag alone can't allocate:

• A topic can serve several teams. The team producing the events and the teams consuming them may belong to different business units. Topic ownership alone doesn't explain all the traffic.
• Topic count doesn't explain consumption. Topic retention, partitions, writes, reads, and connectors contribute differently to the cost.
• Some costs are shared. Brokers, disaster recovery, support, and platform tooling (like Conduktor) need an agreed allocation policy.
• Technical identities need business context. Service accounts must map to applications and cost centers. Renaming a generic producer-1 helps investigations, but client.id is a client-supplied label, not an authenticated identity. Shared credentials still combine several applications' traffic in service-account metering.

Idle topics can still consume capacity and affect cluster sizing. For instance, on Confluent Cloud Dedicated, a Confluent Unit for Kafka (CKU) bundles throughput, partition, and connection limits. A cluster can need more capacity because of its partition count even when throughput is low.

Partition-hours capture that footprint over time. A topic with 12 partitions that exists for 30 days represents 12 × 24 × 30 = 8,640 partition-hours, even with no writes. The same topic kept for one day represents 288. This measures how long the partitions exist, regardless of the topic's retention setting. Topic count and traffic alone can't capture that difference. See the cost of Kafka partition waste for the impact on infrastructure costs.

Showback and chargeback change priorities

When doing FinOps, you separate reporting consumption from financial responsibility:

Approach	What the team receives	What changes
Showback	A report of its attributed Kafka costs and the usage behind them	The team can review its footprint and question the allocation before money moves
Chargeback	An allocated cost applied to its budget or cost center	The team becomes financially accountable through the organization's finance process

Showback and chargeback make usage visible and give teams a reason to take responsibility for cleanup.

Cleanup competes with other priorities and takes engineering time. If an application team is measured on shipping features while the platform team carries the cost, leaving an old topic alone can be the easier decision. The application's budget owner needs to give cleanup a place on the roadmap.

Showback gives teams time to validate ownership and usage before costs are charged to their budgets. It can give cleanup priority without formal chargeback. That's why the platform team can't fix Kafka costs alone.

Who pays when several teams read the same topic?

For example, an orders application produces events, fraud detection consumes them, and analytics ingests them too. One topic, different demands on the platform. You can allocate those costs as follows:

• Ingress to the producing application.
• Egress to consuming applications.
• Storage and partitions to the topic owner, or a shared allocation when several teams require specific retention.
• Shared platform costs to a central budget or a fixed distribution across teams.

Charging the topic owner for every downstream read makes them pay for decisions they don't control. Charging consumers for their reads gives them an incentive to avoid unnecessary full-history replays.

Allocate fixed costs explicitly

What if you want to distribute a fixed cost by usage? Finance and the platform team first agree on the shared costs to allocate for the period: the amount to distribute across teams after any centrally funded share. You can split that amount by partition-hours:

Team allocation = shared costs to allocate × team's partition-hours / total in-scope partition-hours.

Console exports usage metrics so finance can apply this allocation policy. Its built-in cost reports use configured unit rates.

Define which costs to include, such as broker capacity, support, platform tooling, cross-cluster replication, and other shared infrastructure. The amount can change between billing periods. Storage and network can use separate allocation keys.

Cost proxies to support adoption

Some of our customers base internal billing on proxies such as team size, keeping the infrastructure bill separate. The goal is to encourage adoption without letting infrastructure costs discourage teams from using Kafka.

Measure	What it tells you
Kafka usage	The storage, partition-hours, traffic, and connector activity associated with a team or use case
Actual platform cost	What the organization spends on Kafka infrastructure and the shared services
Internal charge	What a team is asked to fund under the organization's pricing policy

If billing depends only on team size, retiring an unused topic won't reduce the team's charge. Cleanup then needs another mechanism: a usage review with the budget owner, or provisioning rules that limit waste. Conduktor's reporting and policies support that process.

How Conduktor brings Kafka into FinOps

Conduktor Chargeback connects Kafka consumption to applications and business ownership, with resource detail to investigate each team's costs.

Measure the resources behind the cost

We chose these five dimensions to track the resources Kafka workloads consume over time. These usage trends also inform capacity planning, regardless of your provider's billing model:

Cost dimension	Measured usage	Configured rate
Storage	Data retained over time	Per GB-hour
Partitions	Partition count over time	Per partition-hour
Ingress	Bytes produced	Per GB
Egress	Bytes consumed	Per GB
Kafka Connect	Running connector task hours	Per task-hour, with optional rates by connector class

Console collects Kafka usage metrics; it doesn't import cloud invoices. You set rates per cluster, choosing whether to mirror provider pricing or apply your own internal prices. Update them when pricing or your allocation policy changes.

Use the metrics your deployment provides

Storage and partition metrics are available across cluster types. For Confluent Cloud, Console collects throughput through the Confluent Cloud Metrics API (no Gateway needed). For other Kafka deployments, Gateway's observability interceptor meters ingress and egress for traffic passing through it.

Confluent Cloud provides topic and service-account throughput separately. Gateway provides the service-account-to-topic breakdown, so you can trace a service account's traffic to individual topics. The Chargeback documentation details metric coverage by deployment.

Give teams a report they can use

An application owner needs costs by topic and environment. Finance may need several applications grouped under one budget code.

Console's application view lets you drill into application instances, topics, service accounts, and connectors. You can also group by labels such as business unit or cost center. You don't need to encode every budget code in a topic name: Conduktor stores that mapping in metadata, so a reorganization doesn't leave your cost model tied to obsolete names or prefixes.

Console's role-based access control (RBAC) lets application owners see their own costs. Daily and monthly views help them investigate changes and understand their share of the platform's costs.

Connect Kafka to your existing FinOps reporting

You want to combine Kafka costs with cloud, virtual machine, Kubernetes, and shared-service costs in your existing FinOps reporting platform. Apply the same budget codes to Kafka resources through Conduktor, then export the data so finance can group costs using familiar codes without needing to understand each Kafka metric.

Allocation labels are accounting inputs. Resources missing a selected label are excluded from that grouping, even though they still generate costs. With metadata governance enabled, Console provides a dedicated permission to control topic-label changes. Restrict who can edit them, and monitor label coverage over time.

Console applies rates when you query the data, so changing a rate also changes historical calculations. Keep period exports and the rates used in your finance records.

Help owners clean up and prevent recurring waste

Console's Insights section helps application owners understand their Kafka footprint by surfacing empty, stale, and tiny topics for investigation. Owners can inspect them and decide whether they are still needed.

A lack of consumer activity alone doesn't mean you can delete a topic. Some consumers run infrequently (weekly, for example), and recovery requirements may depend on the data. Use federated ownership to involve the team with that business context.

Self-service policies can require ownership metadata and constrain new topic settings, such as partition counts and retention. Gateway policies can also enforce producer compression. See our Kafka cost optimization page for the available controls.

Bring FinOps to your first Kafka cluster

Pick one cluster with a few application owners, then produce a showback report:

1. Set up FinOps rates. Define the cost per partition-hour, GB-hour of storage, and other applicable units.
2. Map the owners. Connect topics and service accounts to applications and finance's budget codes.
3. Review. Trace each team's total to its resources. Check for missing metrics, missing labels, and differences from the in-scope bill.
4. Investigate with the team. Review retention, unused topics, and unexpected reads. Give the responsible team time to act and a date to review the result.
5. Review later. Check changes in usage, actual platform spend, and internal charges. Decide with finance whether the pricing policy encourages the intended behavior before extending it to more teams.