Amazon MSK + Conduktor MSK handles infrastructure. Conduktor handles the enterprise layer.
Whether you're using MSK today or migrating to it, Conduktor adds Kafka governance, encryption, and resilience at the protocol level — no application changes required.
Trusted by platform engineers at
MSK solves infrastructure.
These challenges live above it.
MSK encrypts data at rest and in transit, but any service with topic access reads the full payload, including PII and financial records.
MSK provisions brokers, but has no guardrails for topic creation, client configurations, or data quality enforcement.
MSK provides single-region high availability, but multi-region failover and multi-tenancy require solutions above the broker.
MSK provides EBS-level encryption and TLS, protecting data from infrastructure-level access. But at the application level:
- Any consumer with topic access reads every field in the message
- PII, financial records, and health data are fully exposed to authorized consumers
- Teams solve encryption independently, each building custom crypto per project
- Security can't verify which topics are actually protected
Regulated workloads stall because security teams can't guarantee field-level data protection on managed brokers.
MSK handles broker provisioning, patching, and scaling. But above the infrastructure layer:
- No validation of producer data quality before messages enter brokers
- No guardrails on client configurations like acks, compression, or commit frequency
- Platform teams either gatekeep every change manually or grant open access with no guardrails
At 5 teams, this is manageable. At 50+, it breaks down.
MSK Replicator syncs data across clusters, but switching hundreds of applications to a backup is the hard part:
- DNS-based failover is slow, unpredictable, and can create split-brain scenarios
- Most DR plans exist on paper but are never validated against real traffic
- Each new team or environment needs dedicated MSK clusters, creating infrastructure sprawl
- Decommissioning unused clusters is equally painful
Multi-region DR is theoretically possible but operationally impractical without a proxy layer.
How Conduktor Gateway Complements MSK
Gateway sits between your applications and MSK brokers, speaking the Kafka protocol natively. Every capability below is enforced centrally with no application code changes.
Encrypt, tokenize, and crypto shred sensitive fields before data reaches MSK brokers, with per-consumer decryption controls and native AWS KMS integration.
Validate every message before it enters MSK, enforcing schema compliance and business rules with the ability to block, route, or flag bad data.
Block unsafe client configurations, detect connection storms, and give operators immediate feedback at the protocol level.
Kafka-protocol-aware routing that rewrites broker metadata, giving clients in any topology access to MSK through a single entry point.
Single-command failover that redirects all client traffic in seconds, plus built-in chaos testing to validate resilience without production risk.
Logically isolated virtual clusters with independent namespaces and access controls, plus S3 payload offloading and caching for broker efficiency.
MSK encrypts data at rest and in transit. Gateway adds protection within messages:
- Field-level encryption: Encrypt
customer.ssnwhile keepingcustomer.regionreadable - Crypto shredding: Dynamic keys per entity. Delete the key and all their data becomes permanently unreadable without touching a single message
- Tokenization: Replace sensitive values with consistent tokens for analytics without exposing raw data
- Per-consumer decryption: Partner A decrypts; Partner B sees ciphertext. Same topic, zero duplication
- Native AWS KMS: Key management through KMS with full CloudTrail audit logging
One policy at the gateway replaces N implementations across N projects.
A single malformed message can crash consumers, break pipelines, and spread incorrect results before anyone notices:
- Schema validation: Enforce compliance with or without Schema Registry
- Business rules: CEL expressions for cross-field validation beyond what schemas express
- Dead-letter routing: Route violations with full error context instead of silent failure
- Gradual adoption: Start monitoring, then blocking. No big-bang enforcement required
Kafka's native quotas cover throughput but can't enforce the configurations that cause the most damage at scale:
- Require compression (GZIP, LZ4, ZSTD) and block uncompressed traffic
- Enforce acks=-1 for durability guarantees
- Limit offset commits to prevent commit storms
- Connection rate limiting to protect brokers from runaway clients
- Naming conventions enforced at the protocol level
AWS networking handles IP routing but doesn't understand Kafka's broker discovery protocol. Clients need every broker individually routable:
- Single entry point: Clients connect to Gateway; Gateway handles broker routing
- Hybrid cloud: On-prem applications access MSK without complex networking
- Multi-VPC: Cross-VPC access without individual broker routing
- SNI-based routing: Single port, hostname-based routing. No port-range exposure
MSK Replicator keeps data in sync. Gateway handles the hard part, switching applications over:
- Single-command failover: Redirect all client traffic in seconds with zero client-side changes
- Cross-region DR: Failover between MSK clusters in different AWS regions
- Built-in chaos testing: Inject specific error codes, targeted latency, and field corruption
- MSK stays untouched: Fault injection happens at the protocol level, not on your brokers
Up to 95% faster recovery compared to manual coordination.
Stop dedicating MSK clusters for every team and environment:
- Virtual clusters: Independent namespaces, topic aliasing, and access controls per tenant on shared infrastructure
- S3 offloading: Large payloads automatically offloaded to Amazon S3 via the claim check pattern
- Caching: In-memory caching for high-read scenarios to reduce broker load
- SQL filtering: Topic views without duplicating data or adding stream processing
Reduce infrastructure costs 20-40% through consolidation.
Operate MSK at Scale with Console
Console complements your existing AWS tooling to enable operational efficiency for Kafka teams.
Built for the AWS Ecosystem
Conduktor integrates natively with AWS services for authentication, encryption, storage, and deployment, so it fits into your existing infrastructure without additional tooling.
MSK IAM Authentication
Native support for AWS_MSK_IAM SASL mechanism. Conduktor Gateway inherits IAM roles from ECS, EKS, and EC2.
AWS KMS
Field-level encryption keys managed in KMS. IAM policies control which consumers decrypt which fields. CloudTrail logs every operation.
Amazon S3
Large payload offloading via the claim check pattern. Payloads never consume broker storage or network bandwidth.
ECS / EKS Deployment
Deploy Conduktor Gateway as ECS tasks or EKS pods within your VPC. Native container orchestration.
AWS Marketplace
Available on AWS Marketplace for simplified procurement. Streamlines vendor onboarding and purchasing.
Glue Schema Registry
Native support for AWS Glue Schema Registry for validation and evolution.
Results with Amazon MSK + Conduktor
Based on results reported by Conduktor customers.
Consolidation, faster migration, and reduced operational overhead.
Virtual clusters and consolidation eliminate cluster sprawl.
Single-command failover vs. manual coordination across teams.
Read more customer stories
Frequently Asked Questions
Does Conduktor work with all MSK cluster types?
Yes. Conduktor connects to MSK Serverless, MSK Provisioned (Standard and Express brokers), and self-managed Kafka on EC2. You can manage all cluster types from a single interface.
How does Conduktor Gateway work at the protocol level?
Conduktor Gateway speaks the Kafka protocol natively. Applications connect to Conduktor Gateway instead of directly to MSK brokers using the same client libraries, same code, just a different address. Conduktor Gateway intercepts, transforms, and governs traffic without any application code changes.
How does Conduktor integrate with AWS IAM?
Conduktor Gateway natively supports the AWS_MSK_IAM SASL mechanism and inherits IAM roles from ECS, EKS, and EC2 through the AWS Default Credentials Provider Chain. No separate credential management required.
Can Conduktor help with migration to MSK?
Yes. Conduktor Gateway enables zero-downtime migration by sitting between applications and brokers. Switch from one cluster to another with a single command while applications continue running unchanged. This works for migrations from self-managed Kafka or other platforms to MSK.
Can I use AWS Glue Schema Registry with Conduktor?
Yes. Conduktor integrates natively with AWS Glue Schema Registry for schema validation, evolution, and compatibility checks across Avro, JSON, and Protobuf data formats.
Is Conduktor available on AWS Marketplace?
Yes. Conduktor is available on AWS Marketplace for simplified procurement. Marketplace availability streamlines vendor onboarding and lets you leverage your existing AWS private purchasing agreements.
How do I deploy Conduktor on AWS?
Conduktor runs as a Docker container. Deploy Conduktor Gateway and Conduktor Console within your VPC on ECS, EKS, Fargate, or EC2 depending on your infrastructure preferences. See Get Started for setup guides.
What does Conduktor Console add beyond CloudWatch monitoring?
CloudWatch and Prometheus provide infrastructure-level metrics. Conduktor Console adds the operational layer above: which teams own which topics, how resources relate across clusters, self-service provisioning with policy guardrails, approval workflows for cross-team access, and per-team cost attribution. It gives platform teams a single operational layer that scales with the organization.
How is Conduktor different from Confluent?
Conduktor is infrastructure-agnostic. It works across MSK, Confluent, Redpanda, and open-source Kafka. Rather than replacing your Kafka infrastructure, Conduktor adds an enterprise governance layer on top. If you're evaluating MSK as an alternative to Confluent, Conduktor fills the governance gap that MSK wasn't designed to address.
Can I use Conduktor across multiple Kafka platforms or clouds?
Yes. Conduktor is designed for multi-cloud and multi-Kafka environments. You can manage MSK alongside Confluent Cloud, Redpanda, Aiven, or self-managed Kafka clusters from a single Conduktor Console instance. Conduktor Gateway can front any Kafka-compatible cluster, so your governance policies, encryption rules, and data quality controls apply consistently regardless of which Kafka platform or cloud provider sits underneath. Teams get one consistent interface and one set of policies, even if your infrastructure spans AWS, GCP, and on-prem.
Running Kafka on AWS?
Whether you're using MSK today or migrating to it, our team can help you design the right governance architecture for your workloads.