What we learned from SOC2 Type II? Write what you do. Do what you write.

We are excited to announce that Conduktor has successfully completed a System and Organization Controls (SOC) 2 Type II audit [...] the SOC 2 information security audit provides a report on the examination of controls relevant to the trust services criteria categories covering security, availability, processing integrity, confidentiality, and privacy. [...] Conduktor’s SOC 2 Type II report did not have any noted exceptions and was therefore issued with a “clean” audit opinion from Sensiba.

Good news! This is what we received a few days ago from our auditors. As we are quite proud of this, we wanted to share what we learned from all of this.

Conduktor provides a Data Security Platform for Apache Kafka. We improve Kafka security and data governance, while simplifying complex architectures. By leveraging Data Mesh principles, we equip Platform teams to help them enforce organizational standards and rules. Achieving SOC 2 compliance was a logical step for us, further solidifying our commitment to providing the highest data security and reliability standards.

SOC2: TLDR# 

You wanna implement SOC2? Here are our main takeaways:

• SOC2 is about people, processes, risks, not only about technology.
• SOC2 is about writing down what you do (Type I), and doing what you write down (Type II, over months).
• We worked with Drata and Sensiba to achieve this. Tell them we sent you!
• It's all about security and risks: technical, physical, and administrative.
• It took us one year to get SOC2 Type II compliant.
• There is no best time to start SOC2, except if you feel pressure from your customers. It's just about time, money, and commitment! (surprise)
• Align everyone about the importance of SOC2, it's a team effort. Communicate a lot.

We were SOC2 Type I already, Type II is the proof we actually do what we wrote down over time.

Here, we won't cover SOC2 pillars themselves but everything we learned by implementing it: the good, the bad, and the ugly.

Goodbye large Excel files?# 

We had people in Conduktor who were already familiar with SOC2 from their previous experience. At the same time, we would regularly labor over massive Excel files from our customers, asking tons of questions about everything security, SDLC (Software Development Life Cycle), backups, vulnerabilities, disaster recovery plans, etc. We knew SOC2 was a way to answer all of these questions once and for all.

We were already doing many things right, but not everything was documented correctly or consistently. As a B2B business, SOC2 is a must-have to accelerate our sales cycle and build more trust with our customers. It's also crucial to build a solid foundation for our own business growth and long-term success.

SOC2 impact in our day-to-day# 

Let me give you a few examples of things we've written down and are now enforced:

• All laptops are encrypted, automatic updates enabled, screen locked, and password protected after a few minutes of inactivity.
• Everyone has to complete security awareness training regularly.
• Everyone had to accept a Code of Conduct to ensure a safe, trustworthy, and inclusive workplace.
• We adopted an organization password manager like 1Password, no more Google Chrome isolated password manager.
• SSO, MFA, RBAC on everything we're using if possible! Always rely on the least privilege principle.
• We develop clear procedures for incident response and tracking their resolution.
• Github Pull Requests are reviewed by at least 2 members, main branches are protected, CODEOWNERS are defined.
• We use Teleport to access internal infrastructure.
• Many automated Security Scans for static code analysis and dependency vulnerabilities.
• We review our vendors' security practices regularly.
• We conduct regular penetration testing on our products.
• ... and many more!

Our SOC 2 Type I/Type II reports contain more detailed explanations. To request them, contact us. Note that both existing and prospective customers must sign an NDA to access the reports (it's part of SOC2!).

How long? A year# 

We started our SOC2 journey in Nov 2022 by joining the Drata platform to help us in this process. We were not constrained by time, so we wanted to learn and do it right. We were also a small team, so we had to do it in parallel with our day-to-day work.

Drata is a compliance automation platform that helps companies to get SOC2 compliant (and other certifications) by relying on automation and integrations with many existing tools. From the start, they inspired us with trust and confidence. Our first call was direct and transparent, really clear without any sales pitch – thanks to Italo for leaving us with such a positive impression and being incredibly helpful. This definitely checked a box for us.

This article is not sponsored by Drata. We just really liked working with them, and we think they deserve a shoutout. We also discussed with Vanta but preferred the Drata approach and price. Also, Compleye seems competitive now!

How much? A few Quid# 

It depends. Long story short, the price for such a compliance automation platform depends on:

• How many employees: all must install an agent on their laptop for continuous monitoring (encryption, password policy, etc.)
• The compliance framework to implement: SOC2 in our case, ISO27001, HIPAA, etc.

On top of it, you have to pay the auditors. We were helped by Sensiba, a firm in the Bay Area, and we were really happy with them. They were professional, helpful, and friendly. They know how to use Drata to get evidence and were efficient and responsive (even at the end of December!).

It will also cost a lot of time from your team (HR, Finance, Engineering, Founders...). They will discuss and write down policies, review them, and implement them. Engineers will assess the infrastructure, how secrets are managed, how vulnerabilities are detected and managed, and much much more. Auditors will ask many questions and will need evidence. It's a lot of work, difficult to estimate upfront.

Screenshots to the rescue!# 

When automation is not enough, you take screenshots! Similar to our friends at fly.io in their blog SOC2: The Screenshots Will Continue Until Security Improves, screenshots could be considered as the hidden cornerstone of SOC2 compliance! They are the proof you give to your auditors to show them that you do what you write down.

It's useful when the compliance automation platform you're using (if any) does not have access to everything, or when you have to prove something that is not automated. You can take screenshots of some audit logs, some dashboards, some Trello boards, to prove that the controls are effectively implemented.

Without such automation, be ready to extract JSON from your Cloud Providers and take a lot of screenshots from laptops and all the SaaS you're using!

Sign 20+ policies engaging your responsibility# 

Writing internal policies was the most time-consuming part at the beginning of the process. We had to document everything, from our hiring process to our security policies. We had to be thorough, clear, and consistent across all of them.

Compliance platform generally provides good templates to start with, that must be adapted to your own business. It helps get from Zero to One! You can find many online, and versioning each modification is mandatory.

It was the effort of many hands and VPs, but it was worth it. We now have a comprehensive set of policies that we can use to onboard new employees and partners. It was a great opportunity to question and review our processes, revealing areas for improvement.

All of them are reviewed and signed by all our employees regularly (mandatory).

Fun legal fact: even if we are a US business, we also have many people in France. We had to write most of them in French, as signing these policies engages the responsibility of the employees. It's required to have legal papers in the language of Molière.

Install this intrusive agent on your laptop# 

I'm kidding; it was not intrusive at all. But as any engineer, nobody likes to install a monitoring agent on their laptop because it's going to slow it down or do crappy things, right? Well, no.

Agents are a great way to monitor good practices on a laptop and ensure security policies are respected. It is simply polling the OS on a regular basis to fetch system metrics/flags related to SOC2 controls. It's exactly like osquery.io.

We used agents to monitor our laptops but NOT our infrastructure (where we rely on Cloud solutions). It was a great way to ensure everyone was on the same page and following the same security standards. If a laptop were not compliant (screenlock not configured etc.), we would receive an alert to take action.

Due to preconceived notions, we had to overcome some resistance from engineers. We had to explain why it was important, how it would help them to have a safe laptop, and the non-intrusive nature of the agent (privacy, performance). We also had contractors with their own laptops, so we had to find a way to include or exclude them with reasons (all the aspects of SOC2 are challengeable if you can explain why).

• 1password: We handle all our passwords and secrets in a better way now. We're now using 1Password to keep all our passwords, share them with other members, do MFA, create dedicated vaults for teams to share common passwords or keys, etc. It's more clear and secure for everyone and the organization.
• We're using Google Workspace (G-Suite), so we had to deploy a SCIM Bridge to provision accounts (read more here https://blog.1password.com/1password-google-workspace/)
• With SOC2, you need a clear organizational chart. Everyone has a role to play. Everyone must report to someone, which must be clear to everyone. As we have been using BambooHR for a while, it was quite trivial to provide such evidence. Again, BambooHR has a deep integration with Google to provision accounts making everything consistent and auditable.
• We had to introduce formal background checks using checkr as we hire people in the USA. We don't do it in Europe as it's either barely legal or illegal! Definitely something to push back with auditors if asked.

We realized early that compliance is not just about systems; it’s about people. Employees are at the center of any company; they own it. They own its security. They own its principles and its culture. By giving them the tools and knowledge about security, everyone is more aware and responsible.

Pentesting: Your Secure Code VS Reality# 

Pentesting, penetration testing, white hat hacking, or ethical hacking, is optional in the SOC2 scope but highly recommended if you are serious. It's a method to identify security vulnerabilities in products, limit data breaches (from inside or outside), or discover a criminal conspiracy with plans to use a computer virus that will capsize five oil tankers.

It comes in three flavors:

• White Box: Full access to internals. Think of it as testing from an insider's perspective.
• Gray Box: Limited info, a mix of insider and outsider testing.
• Black Box: No prior knowledge, simulating an external hacker's approach.

A pentesting is not just to try acting as an external hacker. It's also to test if someone from inside the company could do something bad with massive consequences or increase their privileges.

The pentester (a human!) needs to learn about your product, how it works (architecture, use-cases), what are the most critical parts (RBAC, tokens, URLs, encryption, SQL!), what are the most critical data, etc. To find vulnerabilities, you need to be intimate with the product. It's not about running a tool (script kiddies one would say) and being done. You typically run pentesting every 6 months because your product is evolving, and so are its vulnerabilities.

We had large customers doing their own internal pentesting of our products. They found only minor issues, hurray! We also did our own pentesting with an official 3rd party. This is necessary for SOC2 compliance to assess the security of what you build. It's also very helpful as it helped us find minor vulnerabilities like visible tokens in our UI, URL parameters giving too much info, JWT security, HTTP headers, OWASP, etc.

In the end, we were quite happy, I will share with you a little excerpt of a report from a black box testing, just to brag:

During the audit, many good practices have been observed. The auditors rate the application's level of security as high. [...] the Console product and Gateway product have a lot of good practices in terms of security development. The security level of the application is above the average of other audits previously performed on different clients.

Thanks to our engineering team for their hard work here! From Frontend to Backend, we're all responsible for our customers' data security.

Pentesting: How much? For what?# 

Pentesting takes between 1-N weeks, depending on the size of your product. It can lead to a lot of back and forth with the pentester, so the team need to be available. Critical vulnerabilities are reported immediately; this way you can fix them ASAP. In the end, the pentester will provide you a report with all the vulnerabilities found, their severity, and a lot of good practices (that they find or recommend you to implement).

It's difficult to clearly define the scope of a pentesting at the beginning, as typically you want "everything" to be tested! (spoiler: this is impossible) After the first audit, you can work with the pentester to define a narrower scope for the next pentesting. (6 months later)

We did not particularly appreciate vendors asking for ultra specifics (how many API routes, how many frontend screens, how many IPs, etc.) to establish pricing. It's not always that simple and removes the value of the pentesting. A pentester is a HUMAN you interact with. We preferred to work with a pentester that was flexible and open to discussing the scope and pricing.

One tricky aspect is that we don't do only HTTP APIs, we also manage critical aspects through the Kafka protocol that we enhance with various security features like encryption, masking, authentication, authorization, RBAC, etc. Finding the right pentester with some knowledge here was appreciated! Pentesting is not just HTTP.

SOC2 is a marathon, not a sprint# 

We knew it was going to take some time, but as the process unfolded, the initial timelines seemed optimistic. We started OKR style "X policies per months", but failed on them. In the beginning, everyone involved had a spike in productivity, we discovered our new shiny compliance platform, so cool, etc. But, it's a marathon, not a sprint.

As your day-to-day job takes over, policies take more time to write, review, and implement. Writing 2-dozens policies is not easy, especially when you must review them with your team and ensure everything is clear for most. You help others by reviewing their policies, and they help you by reviewing yours. It's a great exercise, and a great way to learn from each other, but it can be a never-ending process. You really have to know when to stop and timebox your efforts.

Infrastructure controls needed to be built and deployed. Synchronization issues occurred. You need more monitoring. You need to integrate systems together for consistency and remove humans from the loop. You need to check control validity on a weekly basis. It's a constant adaptation and SOC2 becomes part of your daily job, not something exceptional.

A chain is only as strong as its weakest link# 

I forgot where I read that but a typical SaaS company uses between 50-100 other SaaS to focus on its core business. It means that managing third-party is a complex dance, requiring us to have clarity over our partners:

• Are they critical? If they were to fail, what would be the impact on us?
• Are they keeping our data safe? What kind of data do they have? Who has access to this? Do they provide any audit mechanism? If someone leaves the company, what happens to their account, accesses, and resources? (this is a funny one)

Definitely interesting questions! (that you need to re-evaluate yearly because you keep adding other systems)

We had to review their security policies, their SLAs, and their security awareness protocols. If they don't have any SOC2 certifications, you hope they have some documentation about their security methodologies and things in place.

In our journey to SOC2 compliance, we meticulously mapped our entire architecture, detailing every internal component and its interconnections. Achieving SOC2 status compelled us to thoroughly understand how our third-party partners manage security, especially regarding their SOC compliance status, to expedite our assessment process. As a last resort, you call them to know. You need evidence.

Github, Segment, Mixpanel, Sentry, Hubspot, Bamboo, Auth0, Checkly, Datadog, Algolia, Linear, etc. sound familiar? Hopefully, they are all SOC2 compliant :)

The magic of CVEs (Code Vulnerabilities)# 

Vulnerability management is an ongoing challenge. As CTO, I (Stéphane) frequently reiterated the need to keep our dependencies updated and free of vulnerabilities to the teams. Letting one vulnerability slip through the cracks can have a domino effect, compromising the entire system.

CVE means Common Vulnerabilities and Exposures. They are categorized as critical, high, or medium/low, based on their severity and potential impact.

Existing dependencies may unexpectedly develop new vulnerabilities due to updates. Vulnerability scanners, which scan the code to find them, are not infallible; we encountered various false positives that necessitated further investigation, as they were NOT actual vulnerabilities, and sometimes provide clear explanations to fussy customers (see all these "false positive" on the Grype Github project).

Constant vigilance and proactive workflow practices are crucial: aiming to prevent adding vulnerabilities rather than just reacting when it's too late and it's already merged. To achieve this, we utilize a suite of tools including:

• Github Dependabot
• Snyk
• Docker Scout
• Harbor
• Grype

Additionally, we've created a custom GitHub bot, affectionately named Grumpy (not Open-source for some reason?), which alerts us if a new dependency with a vulnerability is introduced in a PR. Why so many tools, you might ask? Because they don't always find the same vulnerabilities, customers often use their own tools too! Better be safe than sorry.

Critical vulnerabilities are those that require immediate attention and resolution. It is usually stipulated in your contract with customers, legally binding you to fix the issue in a timely manner. Taking these issues lightly is not an option if you don't want to lose the trust of your customers.

We made tremendous progress in this area. Engineers are fully aware and vulnerability management is a normal part of their job. Looking at old reports, we had something like 50 vulnerabilities including 5 critical and many more High. (shh) Today, we are super clean, 0 Crit, 0 High! Yeah!

Open-Source does not mean Free# 

Not related to SOC2, but equally important: license management. We are dealing with large customers and their lawyers, so for us it's part of the game! In today's ecosystem, it's not uncommon to have hundreds/thousands of dependencies in a single project (hello left-pad: How one programmer broke the internet by deleting a tiny piece of code). Are you confident you are allowed to use them or distribute them in your software?

I don't know how realistic this is, but libraries.io claims that:

• npm: 4.08M Packages (!)
• Maven: 588K Packages
• PyPI: 500K Packages
• Go: 485K Packages
• Rubygems: 183K Packages
• Cargo: 136K Packages

Do you think all of them are "free" to use? (Open-Source does not mean Free). Do you all know how many types of licenses exist out there? Let me give you a non-exhaustive list:

• MIT License
• GNU General Public License (GPL)
• Apache License 2.0
• BSD Licenses (including 2-Clause and 3-Clause versions)
• Creative Commons Licenses
• GNU Lesser General Public License (LGPL)
• Mozilla Public License (MPL)
• Eclipse Public License (EPL)
• ...

Without even counting their variations and the proprietary licenses (Microsoft, Apple, etc.). Open Source Initiative (OSI), approving Open-Source licenses, lists more than 80.

Anyway, it's a mess and we use FOSSA to help us. FOSSA meticulously analyzes our dependencies, ensuring compliance with their licenses in a recursive manner. FOSSA integrates with GitHub and our CI/CD pipelines across multiple programming languages, Scala, Java, Rust, JavaScript, Python, Go, and more. This way, we can ensure we do not use any dependencies whose licenses conflict with our business model or distribution policies (as we distribute on-premise software).

Working with a diverse range of companies —particularly Financial institutions— necessitates meticulous attention to the licenses of our dependencies. It's not uncommon for customers to request a complete list of all the dependencies we're using and their respective licenses. FOSSA allows us to generate and share comprehensive reports easily.

Conclusion: Wait for your customers# 

Our journey to achieving SOC 2 Type 2 compliance was as much about introspection and growth as it was about meeting standards. SOC2 becomes interesting once you reach a scale where it is truly useful. Wait for your customers to request it.

We're proud of this achievement and hope it brings you, our users and partners, a sense of confidence and satisfaction in our solutions. We recognize that in the realm of security, complacency is not an option. This audit is not the finish line but a checkpoint in our ongoing commitment to excellence.

In true Conduktor spirit, we're Kafkaesque in the best way possible – weaving through the intricate world of data streams with agility and precision. Just like Kafka efficiently processes massive streams of information, we pledge to keep our processes sleek and smooth, ensuring they flow as seamlessly as a well-tuned data pipeline. And when it comes to security, we're not just building walls; we're architecting a fortress, fortifying every byte and bit with relentless dedication.

Thank you for joining us on this journey and for placing your trust in Conduktor.