Reading events: consumers
Producers write events once; consumers decide what happens to them. Most production incidents on the reading side come down to four questions: who reads which partition, where a restarted reader continues, what a membership change costs, and what to do when the work isn't a stream at all but a pile of jobs. This module answers them with billing workers you start, crash and restart.
By the end of this module you can:
- run several consumers as one group, read its assignment and lag, and size the group for the load;
- choose a commit strategy, predict what a crash repeats or skips, and make processing idempotent;
- explain what triggers a rebalance, and cut its cost with the KIP-848 protocol or static membership;
- tell a stuck consumer from a dead one, and set max.poll.interval.ms and session.timeout.ms;
- use a share group for independent jobs, with acknowledgements, locks and a delivery limit.
Lessons
- Consumer groups share the work
- Committing offsets: where processing resumes
- Rebalances and the new group protocol
- Work queues with share groups
Then practise with the exercises and check yourself with the module quiz.