ArrowBack to blog

Enhance Apache Kafka with Conduktor Gateway

We're excited to launch Conduktor Gateway on our Marketplace to provide central platform teams responsible for Kafka with enhanced functionality. Enforce best practices, improve your governance, and reduce your cost.

Author's avatarShashank GollaMay 16th, 2023
NaN min read
Blog's image

Apache Kafka is a highly available, and scalable data streaming platform that thousands of companies use to support their mission-critical applications. While it provides immense power to teams delivering real-time applications, there are limitations in Kafka that lead to time sinks for central platform and development teams. We're excited to launch Conduktor Gateway to provide central platform teams responsible for Kafka with enhanced functionality to help govern, secure, and enable Kafka self-service to teams across an organization.

At a fundamental level, Conduktor Gateway is the transport layer between your Kafka client applications and your Kafka clusters. This transport layer is enhanced with Conduktor Interceptors or plugins that you can install on Gateway.

Announcing Conduktor Marketplace

To ensure people can have easy access to both the Gateway and Interceptors we're announcing the release of Conduktor Marketplace! Our marketplace contains an open-source version of Conduktor Gateway along with pre-built Interceptors or plugins that help you enhance Kafka for free. You'll also be able to build Interceptors to introduce any functionality you feel is missing in your Kafka.

Download open-source Gateway

Apache Kafka Limitations and Gateway Interceptors

Conduktor has been used by developers and central platform teams to explore and troubleshoot issues in Kafka data. Our customers operating Kafka at scale have increasingly had pain points surrounding governance, testing, and cost. With the launch of Conduktor Marketplace, we're releasing our first batch of pre-built Interceptors that are going to help solve the most critical issues for teams using Kafka.

Enforcing Kafka best practices across the organization

Central platform teams often have the most Kafka expertise in an organization. When Kafka starts to spread to individual project teams these central platform teams are often responsible for enabling these teams with Kafka best practices.

Introducing Safeguard

Conduktor Safeguard protects your data and infrastructure from Kafka errors by enforcing rules that are centrally defined. Rules can be defined by the central platform team when they should be followed by the whole organization or defined by the individual domain teams when that expertise is important.

Safeguard Interceptors:

Increasing the reliability of Kafka

Kafka is a resilient system so it is difficult to simulate and test application problems that could occur in production in your dev and testing environments.

Introducing Chaos

With Conduktor Chaos you can simulate incidents in your Kafka application's development or testing environments.

Chaos Interceptors:

  • Ensure that your Kafka applications can handle messages which don't arrive in the format that is expected. For example, how would your Kafka application react if it's expecting a message similar to transferAmount:10, currency: pounds but actually receives transferAmount: 10, currency: poundsABC?

  • Validate that your application behaves correctly when duplicate records are written to Kafka.

Gateway Enterprise

Our pre-built Interceptors in Conduktor Marketplace give you a preview of the kind of functionality that you can build into Kafka. Gateway Enterprise has Interceptors that are more in line with the needs of companies that are operating Kafka at scale.

Ensuring security and compliance in Kafka

Encrypting the data flowing through Kafka either requires you to implement encryption at the application layer before the data is sent or after it is received in Kafka. This either requires you to spend cycles on your development team to build encryption into all of your applications or time from your central platform teams to implement encryption in storage.

Introducing Encryption

Conduktor Encryption works after data is sent from your application before it reaches your Kafka brokers. You can encrypt messages down to the field level using KMS solutions from AWS, Azure, or GCP.

Reducing the cost of Kafka

If you're in a company where you have a large number of clusters for historical, technical, or other reasons, these clusters cost you money, whether they're physical or in the cloud.

Introducing Multi-tenancy

Similar to virtual machines, Conduktor Multi-tenancy enables you to create as many virtual clusters as you need on a single Kafka cluster, allowing you to mathematically reduce the number of clusters you need to pay and maintain.


This is just the start of what we're building with Conduktor Gateway. We'll continue to release more Interceptors based on feedback from our customers. If you have any ideas, please let us know.

Today, users have the choice of downloading the open-source version of Gateway from our Marketplace to start enhancing Kafka with our free Interceptors or contacting us to experience Gateway Enterprise.

We'd love to hear from you!
If you want to accelerate your project delivery, fortify your security, and federate your Kafka ecosystem, you know where to find us.