# Kafka Webhooks: How to Ingest Webhooks Without Losing Events

**Kafka webhook ingestion** is the path that turns an HTTP callback from an outside provider, such as Stripe, GitHub or Shopify, into a record on a Kafka topic. A receiver service checks the provider's signature, writes the event to Kafka and answers the HTTP request. The status code it returns decides whether the provider tries again.

## How a webhook reaches a Kafka topic

For each request, the receiver works in this order:

1. Read the body as bytes and verify the signature over those exact bytes.
2. Parse the JSON, only now, to find the event ID and the object the event describes.
3. Build a record keyed by that object's ID, with the provider's event ID in a header and the raw body as the value.
4. Send it with `acks=all` and wait for the acknowledgement.
5. Answer 2xx if the write succeeded, 5xx if it didn't.

![Webhook receiver between Stripe, GitHub, Shopify and Kafka: verify the signature, key by object ID, event ID in a header, write with acks=all, answer 2xx after the ack; a reconciliation job writes missing events](https://www.conduktor.io/assets/images/glossary/kafka-webhooks-0.webp)

## Verify the signature on the raw bytes

Each provider signs the request with HMAC-SHA256 and a secret it shares with the endpoint. The receiver recomputes the HMAC, compares it in constant time, and rejects the request before anything in the payload is trusted.

| Provider | Signature header | What is signed | Encoding |
|---|---|---|---|
| Stripe | `Stripe-Signature` (`t=` timestamp, one or more `v1=` signatures) | `<timestamp>.<raw body>`, keyed with the endpoint's `whsec_` secret | hex |
| GitHub | `X-Hub-Signature-256` | raw body, keyed with the webhook secret | hex, prefixed `sha256=` |
| Shopify | `X-Shopify-Hmac-Sha256` | raw body, keyed with the app's client secret | base64 |

The HMAC covers bytes, not JSON. A framework that parses the body before the handler runs and serializes it again changes whitespace or key order, and the signature no longer matches. Stripe's documentation is explicit: "Any manipulation to the raw body of the request causes the verification to fail." Its official Python library rejects an event that was re-serialized without its original indentation, with the same signature header that it accepts on the original bytes.

Stripe also signs a timestamp to block replays. Its libraries reject signatures older than five minutes by default, and each retry carries a new timestamp and signature, so a legitimate retry still passes. GitHub's documentation adds that the payload should be treated as UTF-8.

Keeping the raw body as the Kafka record's value has a second benefit: consumers read exactly what the provider sent, including fields a typed model in the receiver would have dropped.

## Answer 2xx only after Kafka acknowledges

A 2xx makes the provider stop retrying, so a receiver that answers before the write is durable loses the event whenever the write fails afterwards, through a broker failover, a full producer buffer or a crash, and no retry brings it back. The receiver answers only after the producer's send completes with `acks=all`, meaning every in-sync replica has the record. With `min.insync.replicas=2` on a topic replicated three times, a write that only one broker could take fails with `NotEnoughReplicas` instead of being acknowledged.

The provider's deadline is the second constraint. GitHub expects a 2xx within 10 seconds, and Shopify allows one second to connect and five seconds for the whole request. Stripe's webhook guide asks for a quick 2xx without giving a number. Kafka's producer defaults are much longer: `delivery.timeout.ms` is 120 seconds and `max.block.ms` is 60 seconds. With those defaults, a receiver can still be waiting on Kafka after the provider has given up on the request, so the producer timeouts have to fit inside the shortest deadline:

```properties
# acks=all and idempotence are the defaults since Kafka 3.0; set them explicitly anyway.
acks=all
enable.idempotence=true
linger.ms=5
request.timeout.ms=2500
delivery.timeout.ms=3000   # must be >= request.timeout.ms + linger.ms
max.block.ms=1000          # caps the wait for metadata or buffer space
```

This handler for Stripe snapshot events was run against Apache Kafka 4.4.0 and stripe-java 34.1:

```java
int handle(byte[] body, String signatureHeader) throws IOException {
    // Verify the raw bytes before any parsing.
    try {
        Webhook.constructEvent(new String(body, UTF_8), signatureHeader, signingSecret, 300);
    } catch (SignatureVerificationException e) {
        return 400;                                     // never reaches Kafka
    }
    JsonNode event = mapper.readTree(body);
    String objectId = event.at("/data/object/id").asText();  // pi_..., in_..., cus_...

    // Key by the object, raw bytes as value, provider ID in a header.
    var record = new ProducerRecord<String, byte[]>("stripe.events", objectId, body);
    record.headers()
          .add("event-id", event.get("id").asText().getBytes(UTF_8))
          .add("event-type", event.get("type").asText().getBytes(UTF_8));

    // 2xx only once acks=all has returned.
    try {
        producer.send(record).get(4, TimeUnit.SECONDS);
        return 200;
    } catch (Exception e) {
        return 503;                                     // Stripe retries a non-2xx answer
    }
}
```

A 503 doesn't prove the record is missing from Kafka. In the same test, pausing the broker during a request made the handler answer 503 after three seconds, and the record appeared in the topic anyway once the broker resumed. The provider's retry then wrote it a second time, on the same partition, with the same `event-id`.

Size limits fail the same way. GitHub caps webhook payloads at 25 MB, while a Kafka producer rejects records over `max.request.size`, 1 MiB by default. A large GitHub delivery fails at the producer, the receiver answers 5xx, and GitHub does not retry it on its own.

## Duplicates: carry the provider's ID in a header

All three providers deliver at least once. Stripe warns that an endpoint "might occasionally receive the same event more than once". GitHub keeps the same `X-GitHub-Delivery` value when a delivery is redelivered, and Shopify tells apps to ignore duplicates using `X-Shopify-Webhook-Id`. Kafka's [idempotent producer](https://www.conduktor.io/kafka/idempotent-kafka-producer) doesn't help here: it removes copies caused by its own retries within one producer session, not a second HTTP request that carries the same event.

The fix is to put the provider's identifier in a [record header](https://www.conduktor.io/glossary/using-kafka-headers-effectively) under the same name for every provider (`event-id` in the handler above), so consumers deduplicate without parsing three payload formats:

| Provider | Deduplication ID | Detail |
|---|---|---|
| Stripe | event `id` (`evt_…`) | Stripe sometimes generates two separate Event objects for one change; its docs suggest the `data.object` ID plus `event.type` to catch those |
| GitHub | `X-GitHub-Delivery` header | A GUID, unchanged when the delivery is redelivered |
| Shopify | `X-Shopify-Webhook-Id` header | Unique per delivery; `X-Shopify-Event-Id` links the deliveries of one merchant action across several subscriptions |

Consumers then apply the [idempotent consumer pattern](https://www.conduktor.io/glossary/idempotent-consumer-pattern) and keep IDs longer than the slowest redelivery path. If a reconciliation job can replay Stripe events up to 30 days old, the deduplication window has to cover 30 days.

## Ordering: key by the object, then read current state

None of the three providers promises order. Stripe "doesn't guarantee the delivery of events in the order that they're generated" and warns against sorting on the `created` timestamp. Shopify "doesn't guarantee ordering within a topic" and points to the `X-Shopify-Triggered-At` header or the payload's `updated_at`. GitHub's webhook documentation doesn't describe delivery order at all.

Kafka keeps order within a partition, so keying each record by the object the event describes puts every event for one payment, order or repository on one partition, in the order the receiver wrote them. That's arrival order, not creation order: a retried `invoice.created` can land after `invoice.paid`. Keys that work:

- **Stripe**: `data.object.id` for snapshot events (`pi_…`, `in_…`, `cus_…`), or `related_object.id` for thin events.
- **Shopify**: the resource's `id` in the payload, such as the order ID.
- **GitHub**: `repository.id` for repository events, so one repository's events share a partition. A very busy repository becomes a hot key; see [partitioning strategies](https://www.conduktor.io/glossary/kafka-partitioning-strategies-and-best-practices).

Keying alone doesn't undo reordering, so consumers don't act on the snapshot in the event. Before acting, a consumer fetches the object's current state from the provider's API, which is what Stripe recommends, or compares `updated_at` with the version it already stored and drops older ones. Stripe's thin events carry only the object's ID for this reason.

## Missing events: reconcile against the provider

Retries stop, and after that the push channel won't deliver the event again:

- **Stripe** retries for up to three days with exponential backoff in live mode, and three times over a few hours in a sandbox.
- **Shopify** retries 8 times over 4 hours. After 8 consecutive failures, it deletes the subscription if it was created through the Admin API.
- **GitHub** does not redeliver failed deliveries automatically. Its REST API lists a hook's deliveries and redelivers those from the past 3 days.

After an outage longer than those windows, or a failed GitHub delivery, a scheduled reconciliation job asks the provider what it sent and writes what's missing to the same topic, with the same ID header, so consumer deduplication absorbs the overlap.

```bash
# Stripe: events from the last 24 hours that haven't reached every endpoint
# (still pending or failed). The Events API goes back 30 days.
curl -G https://api.stripe.com/v1/events \
  -u "$STRIPE_SECRET_KEY:" \
  -d delivery_success=false \
  -d "created[gte]=$(( $(date +%s) - 86400 ))" \
  -d limit=100

# GitHub: list a repository hook's deliveries, then redeliver a failed one.
gh api repos/acme-payments/checkout-api/hooks/482913/deliveries
gh api -X POST repos/acme-payments/checkout-api/hooks/482913/deliveries/9128834117/attempts
```

`delivery_success=false` only finds events that Stripe saw fail. A stricter job lists every event in the window and checks each ID against what consumers processed, which also catches an event the receiver acknowledged and then lost. Shopify's documentation recommends reconciliation jobs that periodically fetch data from Shopify, for example orders updated since the last run. Because Shopify deletes Admin API subscriptions after 8 consecutive failures, the same job can check that the subscription still exists.

## What existing tools cover

- **Conduktor Gateway** (coming soon). [Gateway](https://www.conduktor.io/gateway) accepts Kafka records over HTTPS: an application posts them as itself, and Gateway writes them under that application's Kafka identity, so the topic's ACLs and policies apply as they do to a Kafka client. A webhook receiver still checks the provider's signature, then writes through Gateway like any other application.
- **Provider-native destinations.** Stripe can send events straight to Amazon EventBridge or Azure Event Grid, and Shopify to Amazon EventBridge or Google Cloud Pub/Sub. That removes the public HTTPS endpoint, but the events land on a cloud event bus, and a second hop still has to move them into Kafka.
- **Kafka Connect.** Apache Camel's webhook source connector exposes an HTTP endpoint that writes to Kafka; its documentation doesn't describe signature verification. HTTP source connectors such as Confluent's HTTP Source V2 call an API at a set interval: they poll the provider's API rather than receive its webhooks.
- **HTTP produce APIs.** Confluent REST Proxy and the Strimzi Kafka Bridge accept `POST /topics/{topic}`, but in their own request format, a `records` array with an `application/vnd.kafka.json.v2+json` content type. A provider can't post its webhook to them as is, and neither checks a provider's signature.
- **Webhook services.** Svix Ingest and Hookdeck receive webhooks on a team's behalf, can verify provider signatures, and forward events to HTTP endpoints the team runs. The endpoint that writes to Kafka is still the team's own.
- **Partners on Kafka.** When the sender is a business partner rather than a SaaS platform, it can produce to a dedicated topic with a Kafka client and skip HTTP altogether.

**How do I send webhooks to Kafka?**

Run a small HTTP receiver in front of Kafka. It verifies the provider's signature on the raw request bytes, produces a record keyed by the object the event describes with the provider's event ID in a header, waits for the acks=all acknowledgement, and only then answers 2xx. Kafka has no built-in endpoint that accepts a provider's webhook as is.

**Is there a Kafka webhook connector?**

Apache Camel's webhook source connector for Kafka Connect exposes an HTTP endpoint that writes to Kafka, though its documentation doesn't describe signature verification. Most HTTP source connectors, such as Confluent's HTTP Source V2, poll an API on an interval instead of receiving webhooks.

**Should the endpoint return 200 before writing to Kafka?**

No. A 2xx stops the provider's retries, so an event whose Kafka write fails afterwards is lost. Answering after the acks=all acknowledgement turns a failure into a retry, and the worst case becomes a duplicate that consumers can drop by event ID.

**How do I deduplicate webhook events in Kafka?**

Put the provider's identifier in a record header: the Stripe event id, GitHub's X-GitHub-Delivery or Shopify's X-Shopify-Webhook-Id. Consumers record each ID in the same transaction as their side effect and keep IDs at least as long as the provider's retry window plus any reconciliation look-back.

**Do webhooks arrive in order?**

No. Stripe and Shopify say explicitly that they don't guarantee ordering, and GitHub's webhook documentation doesn't describe delivery order. Keying records by object ID keeps one object's events on one partition in arrival order, and consumers fetch the object's current state or compare updated_at before acting.

## Related Pages

- [Idempotent Consumer Pattern](https://www.conduktor.io/glossary/idempotent-consumer-pattern): how consumers drop the duplicates that webhook retries create.
- [Using Kafka Headers Effectively](https://www.conduktor.io/glossary/using-kafka-headers-effectively): where the provider's event ID and type travel.
- [Kafka Partitioning Strategies and Best Practices](https://www.conduktor.io/glossary/kafka-partitioning-strategies-and-best-practices): choosing object keys and avoiding hot partitions.
- [Kafka Producer Acks Deep Dive](https://www.conduktor.io/kafka/kafka-producer-acks-deep-dive): what `acks=all` and `min.insync.replicas` guarantee before the receiver answers.
- [Idempotent Kafka Producer](https://www.conduktor.io/kafka/idempotent-kafka-producer): the producer-side deduplication that covers Kafka retries, not provider retries.
- [Outbox Pattern for Reliable Event Publishing](https://www.conduktor.io/glossary/outbox-pattern-for-reliable-event-publishing): the same "durable before acknowledged" problem when a database is the source.
- [Kafka Connect: Building Data Integration Pipelines](https://www.conduktor.io/glossary/kafka-connect-building-data-integration-pipelines): the connector framework behind webhook and HTTP source connectors.

## Sources and References

- [Stripe: Receive Stripe events in your webhook endpoint](https://docs.stripe.com/webhooks)
- [Stripe: How events work (destinations, thin and snapshot events, retention)](https://docs.stripe.com/event-destinations)
- [Stripe API: List all events](https://docs.stripe.com/api/events/list)
- [GitHub Docs: Validating webhook deliveries](https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries)
- [GitHub Docs: Best practices for using webhooks](https://docs.github.com/en/webhooks/using-webhooks/best-practices-for-using-webhooks)
- [GitHub Docs: Handling failed webhook deliveries](https://docs.github.com/en/webhooks/using-webhooks/handling-failed-webhook-deliveries)
- [GitHub Docs: Redelivering webhooks](https://docs.github.com/en/webhooks/testing-and-troubleshooting-webhooks/redelivering-webhooks)
- [GitHub Docs: Webhook events and payloads (delivery headers, payload cap)](https://docs.github.com/en/webhooks/webhook-events-and-payloads)
- [GitHub REST API: Repository webhooks](https://docs.github.com/en/rest/repos/webhooks)
- [Shopify: Deliver webhooks through HTTPS](https://shopify.dev/docs/apps/build/webhooks/subscribe/https)
- [Shopify: Webhook delivery structure](https://shopify.dev/docs/apps/build/webhooks/delivery-structure)
- [Shopify: Webhook best practices](https://shopify.dev/docs/apps/build/webhooks/best-practices)
- [Apache Kafka: Producer Configs](https://kafka.apache.org/43/configuration/producer-configs/)
- [Apache Kafka: Topic Configs (`min.insync.replicas`)](https://kafka.apache.org/43/configuration/topic-configs/)
- [Confluent: REST Proxy API reference](https://docs.confluent.io/platform/current/kafka-rest/api.html)
- [Confluent: HTTP Source V2 connector](https://docs.confluent.io/cloud/current/connectors/cc-http-source-v2.html)
- [Strimzi: Kafka Bridge documentation](https://strimzi.io/docs/bridge/latest/)
- [Apache Camel: Webhook source Kafka connector](https://camel.apache.org/camel-kafka-connector/next/reference/connectors/camel-webhook-source-kafka-source-connector.html)
- [Svix: Ingest](https://docs.svix.com/ingest)
- [Hookdeck: Sources](https://hookdeck.com/docs/sources)