How CDC Informatique Scaled Kafka 4x Under Banking Regulation Without Growing the Team

Executive Summary

CDC Informatique runs the IT for Caisse des Dépôts, one of France's largest public financial institutions, under ACPR banking supervision: HDS and ISO 27001 certified, sensitive health, disability, and pension data in flight, a few hours of tolerated downtime a year.

Kafka arrived in 2019 bundled inside a Cloudera Big Data distribution. Developers loved it, adoption took off, and by 2021 the platform underneath, never built for that scale, hit a wall: no naming standard, no ACL convention, no ownership model. CDC Informatique rebuilt Kafka as a governed, self-service product: an internal GitOps "resources as code" layer, plus Conduktor on the human-access path and the Gateway on the data path. In three years it scaled 4x, to ~300 internal users, on a platform team that barely grew.

The full account is in the webinar Julien recorded with us and its written companion, Good Kafka Is Boring Kafka.

Challenges

Kafka is async, durable, and easy to plug into, so it spread from a single ingestion pipeline into cross-application messaging faster than anyone set standards for it.

• A regulated, high-stakes environment: online banking under ACPR supervision, HDS and ISO 27001 certified, handling health, disability, and pension data. Full on-premise, dedicated VLANs per cluster, no crossing of environments, integrity absolute and traceability non-negotiable.
• Governance that couldn't keep up: resource creation was centralized and scattered across teams at once, with no naming convention, no ACL standard, no ownership model, no shared patterns.
• No usable interface: most people who touch Kafka build Spring apps, write Python, or run QA. The bundled UI failed on security and performance, so every non-developer who needed to inspect a topic or test a message had to find one.
• A cluster at the mercy of its clients: with no control point, any client could ship uncompressed data, set acks=0 and silently drop messages, or over-partition a topic, and you only found out from a dashboard, after it hurt.

Solution

CDC Informatique paused, secured sponsorship across production, engineering, and leadership, and re-laid the foundations in a six-month replatforming: out went the Cloudera bundle, in came dedicated clusters owned by a single team.

Govern first, then delegate

CDC Informatique built an internal GitOps tool on a "resources as code" model. Each project team declares the desired state of its Kafka world in one Git file: topics, ACLs, schemas, connectors, Kafka Streams apps, service accounts. The tooling reconciles it, applies controls such as partition caps, and enforces logical isolation per application and environment through naming rules and ACLs. About 99% of requests are standard; the other 1% get a conversation with the platform team.

Conduktor provides the same shape under its self-service framework: a team declares an application and what it owns, and the platform derives the ACLs and ownership boundaries. The key move was pushing access decisions left, onto the project teams, who alone know whether one domain is business-authorized to read another's data. Any team can now stand up a PostgreSQL CDC connector and an outbox pattern in about half an hour, with no central team in the path.

A console for the people who aren't Kafka people

Conduktor Console gave the non-experts a way in: search and filter across topics, produce a test message without code, diff two Avro schema versions, replay events, all behind SSO and an audit trail, mirroring the isolation the platform already enforces. First-level troubleshooting moved to the project teams; the central team only takes the genuinely hard cases.

A control point on the data path

Conduktor Gateway sits in front of the clusters as a Kafka-native proxy. The only change for clients is their bootstrap.servers, so it rolled into integration and pre-production first. From that control point the team surfaces bad client configs (a producer with no compression, say) and requires a fix before production, and runs chaos testing, duplicating messages on the consumer side to confirm a replayed payment event isn't charged twice.

Compliance that holds up to an auditor

Kafka usage is classified on the DICP scale (availability, integrity, confidentiality, traceability). SSO ties access to the enterprise identity system, audit logs record who did what, and the GitOps isolation is reproduced in the Console. Because ownership sits with the project teams, access decisions are made by the people accountable for them, inside guardrails the central team can show an auditor.

Results

• 4x application growth on a flat team: 40 to 160 production applications in three years, with ~50 more in flight. The team went from three to five people and has been flat for eighteen months.
• ~300 internal users enabled: developers, QA, and analysts work independently within the guardrails instead of queuing for the platform team.
• Compliance maintained at scale: logical isolation, SSO, audit, and DICP classification held steady through 4x growth under ACPR supervision.
• Lower support load: first-level troubleshooting shifted to the project teams, freeing the central team for higher-value work.

Conclusion

Designed to be delegated and industrialized, governance is what lets a Kafka platform scale faster than the team that runs it. Good Kafka, it turns out, is boring Kafka, and in a regulated bank that is the highest compliment.

Frequently Asked Questions

How did CDC Informatique scale Kafka without growing the platform team? | CDC Informatique encoded governance (naming, ACLs, ownership, partition caps) into an internal GitOps frame and Conduktor's self-service model, so project teams provision and troubleshoot their own Kafka resources within guardrails. The platform absorbed 4x application growth (40 to 160 apps) while the team stayed flat at five people. What makes Kafka self-service safe in a regulated bank? | Self-service is safe when the guardrails are built in, not bolted on. CDC Informatique enforces logical isolation per application and environment through naming rules and ACLs, ties access to SSO, captures everything in audit logs, and classifies Kafka usage on the DICP scale, so teams move autonomously while the platform stays compliant under ACPR supervision. How does Conduktor Gateway enforce Kafka best practices? | Conduktor Gateway is a Kafka-native proxy that sits between clients and clusters; clients only change their bootstrap.servers. From that control point, CDC Informatique surfaces and blocks risky client configurations (missing compression, unsafe acks) in lower environments before they reach production, and runs chaos tests like message duplication to verify consumer idempotency.