Module 4 exercises
Start the lab first (docker compose up -d --wait in the code/ folder). Each exercise has a
start notebook to work in and a solution notebook to compare with afterwards.
Exercise 1: Gone within 90 seconds
Goal: configure retention for a deletion deadline and a minimum history. · Time: ~20 min · Starter: code/module-04/exercise-1/start/
The privacy rule says no meter reading may be kept longer than 72 hours, and the grid team needs at
least 36 hours of history for replays. In the lab, time runs 2,880 times faster: no reading may be
readable after 90 seconds, and at least 45 seconds of history must always be there. The
start notebook creates m04e1-telemetry like the topic the audit found, with only retention.ms
set, writes 5 readings a second, and measures the oldest readable reading every 5 seconds for 2
minutes, after a 100-second warm-up.
Choose MY_SETTINGS, run the check, and adjust until it passes. Then, in a markdown cell, translate
your settings to the real rule (72 hours, 36 hours, a 5-minute retention check), and list what could
still hold a reading after Kafka has deleted it.
You're done when: the check prints gone within 90 s: True and at least 45 s of history: True,
and your note gives production values for retention.ms and segment.ms with the worst case they
allow.
Hints
Hint 1
The worst case is `retention.ms` + `segment.ms` + the check interval (10 seconds in the lab). The best case, right after a deletion, is about `retention.ms`.Hint 2
Leave a few seconds of margin: a segment only closes when the next reading arrives, and the check runs on its own clock.Exercise 2: The live charger map
Goal: rebuild state from a compacted topic correctly, then keep it up to date. · Time: ~20 min · Starter: code/module-04/exercise-2/start/
The live map service must load m04e2-charger-status completely before it reports "ready", and then
apply new status changes as they arrive. The start notebook writes yesterday's 3,000 changes and
tombstones for 5 retired chargers. After your code has loaded the map, it starts the status feed and
gives your code 30 seconds of live changes. Then it stops the feed, catches up, and compares your
map with the expected one.
Write two functions:
apply(status, msg)updates the map{charger_id: status}with one record or tombstone;load_map(consumer)assigns all 3 partitions from the beginning, reads up to the end offsets it saw when it started, and returns the map and the number of records it read.
You're done when: the notebook prints ready map is right: True, live map is right: True and
retired chargers in the map: [].
Hints