Writing events: producers
Every record in Kafka starts in a producer, and most data problems start there too: events in the wrong order, records lost in a broker restart, duplicates after a retry, or a client that floods the brokers with tiny requests. This module covers the three decisions every producer makes: where a record goes, when it counts as stored, and how records travel together.
By the end of this module you can:
- choose a record key and a partition count, and explain what order Kafka does and doesn't keep;
- configure a producer that neither loses nor duplicates data when brokers fail, and read its delivery reports;
- explain acks, min.insync.replicas, the high watermark and the idempotent producer;
- tune batching and compression, and measure throughput, latency and records per request.
Lessons
- Keys decide order
- Durable writes: acks, replicas and idempotence
- Batching, compression and throughput
Then practise with the exercises and check yourself with the module quiz.