ArrowBack to blog

Conduktor Testing - Deep Dive

Conduktor Testing provides a simple, visual interface for testing applications that incorporate Apache Kafka. Find out more in our deep dive

Author's avatar
James
September 28th, 2022
Blog's image

With the launch of the Conduktor Platform, we will be releasing regular deep dives into each of the new solutions offered, giving you a full understanding of what our solutions can do and how they can bring benefits. First up is a look at Conduktor Testing.

Some of you may already be familiar with Conduktor Testing, as it has been in a small public beta for several months. If not, the premise is simple: Conduktor Testing provides a simple, visual interface for testing applications that incorporate Apache Kafka. It also enables you to easily integrate these tests into your CI/CD environment.

Why Do You Need Testing?

The bigger question of why you should test your software has been very well covered and we won't need to go over it here. The question is perhaps better phrased as to why you need a testing solution for Apache Kafka specifically. After all, if testing is so widely understood, surely there are sufficient solutions for Apache Kafka?

However, Kafka can be a merciless master. It is not simple to design effective tests without already having a lot of Kafka knowledge. And that is fine for Kafka experts, but with its relative youth and high complexity, experts tend to be few and far between. What is necessary is not merely another way to design and build tests in Apache Kafka, but a method that can be both fast & accessible to QAs, Business analysts, and less senior developers also.

Perhaps you could get away with a substandard solution or even ignore testing entirely, but Kafka is increasingly used to support vital infrastructure in organizations. In a microservices world, even small changes can ripple out to cause big impacts, and you simply cannot afford to miss problems too often.

What Conduktor Testing Can Bring

As a visual test editor, you can quite literally see how simple it is to design a test and understand the flow of business data. Let's use an example and compare and contrast with a test via Testcontainers, a commonly used Java library. In this example, we will not be doing anything too complicated. We merely create a test that consists of a consumer & a producer, and involves checking business data.

Below is an example test, taken from the Testcontainers repository:

1protected void testKafkaFunctionality(
2    String bootstrapServers, int partitions, int rf) throws Exception {
3  try (AdminClient adminClient = AdminClient.create(ImmutableMap.of(
4           AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers));
5       KafkaProducer producer = new KafkaProducer<>(
6           ImmutableMap.of(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,
7               bootstrapServers, ProducerConfig.CLIENT_ID_CONFIG,
8               UUID.randomUUID().toString()),
9           new StringSerializer(), new StringSerializer());
10       KafkaConsumer consumer = new KafkaConsumer<>(
11           ImmutableMap.of(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
12               bootstrapServers, ConsumerConfig.GROUP_ID_CONFIG,
13               "tc-" + UUID.randomUUID(),
14               ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"),
15           new StringDeserializer(), new StringDeserializer());) {
16    String topicName = "messages-" + UUID.randomUUID();
17    Collection topics = Collections.singletonList(
18        new NewTopic(topicName, partitions, (short) rf));
19    adminClient.createTopics(topics).all().get(30, TimeUnit.SECONDS);
20    consumer.subscribe(Collections.singletonList(topicName));
21    producer.send(new ProducerRecord<>(topicName, "testcontainers", "rulezzz"))
22        .get();
23    Unreliables.retryUntilTrue(10, TimeUnit.SECONDS, () -> {
24      ConsumerRecords records =
25          consumer.poll(Duration.ofMillis(100));
26      if (records.isEmpty()) {
27        return false;
28      }
29      assertThat(records)
30          .hasSize(1)
31          .extracting(
32              ConsumerRecord::topic, ConsumerRecord::key, ConsumerRecord::value)
33          .containsExactly(tuple(topicName, "testcontainers", "rulezzz"));
34      return true;
35    });
36    consumer.unsubscribe();
37  }
38}

This is not a massive amount of code by any means, but it's complicated and not something that anyone could understand without prior Kafka knowledge.

Now let's see the same thing created with Conduktor Testing:

Pretty seamless, right? That ease of use doesn't just make the lives of Kafka experts easier, but it also extends testing and the understanding of tests to the wider team. Product managers, data analysts, and QA engineers can all create business tests and acknowledge the results.

Every step of the test is visible. This is particularly important for pinpointing failures, as it shows you where in an end-to-end test that a problem actually occurred. Arming team members with this knowledge make tests far more actionable. You'll know where in a streaming pipeline something has failed, and spend less time in a war room trying to decipher the issue.

But it isn't just about having a simple way to make a test. The bigger advantage comes once you're expanding testing to cover more and more of your Kafka applications. Chances are, a lot of the tasks that these applications perform will be similar, with similar testing needs. Deploying more tests with Conduktor is easy, because your test scenarios are saved and reproducible. And since test scenarios are built step-by-step, it is easy to edit for the specific needs of each situation, saving hours of work for every additional test you need to perform.

Conduktor's automation capabilities can lighten the load even further. Conduktor Testing supports automated execution of Test Scenarios via the CI Agent. This process enables you to run collections of test scenarios from within your build pipeline. With this feature you will be able to capture regressions when making changes to your applications, minimizing the risk of introducing failures.

Other Key Features

Moving on from the core visual test editor, let's take a look at some of the other features you might want to know about

  • Test Suites and Scenarios

    • Tests themselves are organized in two ways. Each individual test is termed a "Test Scenario".

    • Scenarios can visually represent all the different tasks that go into a single test, and can be reproduced and edited endlessly to suit the needs of different users.

    • A collection of Test Scenarios is called a Test Suite, enabling you to create logical groups of different scenarios.

    • For example, you may group all tests relating to a specific feature or domain in one suite.

  • Test Environments

    • You can create environments for different tests that enable you to customize certain parameters and configurations.

    • Environment variables can be used in your test scenarios allowing you to rapidly alter your test parameters.

    • You could create a cluster configuration variable, enabling you to execute the same test across development, staging, and production clusters.

  • Workspaces

    • Workspaces are designed for collaboration. A workspace is a collection of Kafka clusters, environments, and associated tests. Workspaces make it easy to organize your personal work, or divide tests by teams, or other categories.

  • Domain-Specific Language

    • The Conduktor Domain-Specific Language (DSL) is a specialized language enabling rapid test declaration and version control of your test scenarios.

    • When authoring test scenarios, you can switch between the standard visual interface and a code representation.

  • Metric Insights

    • Whenever you execute a test, Conduktor tracks certain metrics for you to enable you to continually optimize what and how you test.

    • These are illustrated for you under the Insights tab within the Testing interface, giving you graphical representations of statistics like your total pass/fail rate.

  • HTTP Support

    • It's common practice for systems to integrate both Kafka and HTTP. Picture this, a message is posted to a REST API and is subsequently triaged into a Kafka topic.

    • There are multiple stages here at which we must test the flow of business data. To test the sequence of events in its entirety, you need a consolidated interface that supports both protocols.

    • Postman won't be enough here, but we can definitely help! Testing provides the possibility of combining your Kafka tests alongside HTTP(REST API) testing, providing a unique methodology for testing business scenarios.

  • Random Data Generation

    • To get the most out of testing you need to be able to throw all kinds of different records at your system. Conduktor Testing can do this for you, automatically generating different data for your scenarios.

    • Create entire, randomized messages, or replace individual keys/values within JSON messages.

  • Random, Contextual Data Generation

    • What is more, we even support contextual randomness. This allows you to specify fields, for example, you could generate randomIP, randomCountryCode, or randomUUID.

  • Loading CSV Data

    • If you prepare real or mock data for your tests, then you won't need to rely on our random generators. Instead, you will be able to load from external files.

  • Schema Registry Integration

    • You can integrate Schema Registry into your testing scenarios, with support for

      Avro, Protobuf, and JSON schema registry types.

    • Filter messages and create checks based on the schema IDs.

  • Integrable in any CI/CD Environment

    • As mentioned above, all of your test scenarios can be automated through your chosen CI/CD tool.

    • Continuous testing supports regression testing and ensures that you do not introduce errors in your systems.

  • Data Redaction Controls

    • Restrict the level of data that's stored pertaining to your tests. For example, Kafka message data, or data associated with checks.

  • Formats, Serializers & Deserializers

    • Support for all primitive data types and serializer/deserializer formats (Avro, Protobuf, JSON, MessagePack)

How Do I Get Started?

Conduktor Testing is available to users via the Conduktor Platform. It's accessible through Docker or as a web app. Get started today and execute up to 500 tests/month at no cost at all. Testing works with all major Kafka flavors including local clusters.

Summary

Testing was one of the most demanded features from Conduktor users, for good reason as demonstrated in this post. We have been very carefully testing the new product over the past few months with a small, public beta, but we're delighted to finally make it available for everyone with the launch of the Conduktor Platform.

We aim to accelerate Kafka projects delivery by making developers and organizations more efficient with Kafka.