Start the lab's connect profile first (docker compose --profile connect up -d --wait in the
code/ folder). Each exercise has a start notebook to work in and a solution notebook to compare
with afterwards.
Exercise 1: Add the sites table to CDC, with routing
Goal: configure a Debezium connector with transforms that produce exactly the topics consumers need. · Time: ~15 min · Starter:code/module-06/exercise-1/start/
The live map needs site names, and billing needs tariffs. Both want one plain row per key in a
compacted topic: m06e1.sites (key site_id) and m06e1.tariffs (key tariff_id). The start
notebook creates those two topics and gives you a connector that only captures tariffs and writes
full change events to Debezium's default topic name.
Fill in CHANGES, the settings to add to or replace in the connector's configuration. The check
registers the connector, renames site-N03 in the database, and looks at every m06e1 topic.
You're done when: the check prints only the two topics: True, plain rows: True and
rename arrived: True.
Hints
Hint 1
`table.include.list` takes a comma-separated list of `schema.table` names.
Hint 2
Debezium names topics `..
`, here `m06e1_pricing.public.sites`. A
`RegexRouter` transform with a capture group `(.*)` can keep the table name and replace the rest.
Exercise 2: Break and recover the sink, no data lost
Goal: make a sink connector safe to replay after a crash. · Time: ~15 min · Starter:code/module-06/exercise-2/start/
Finance's reporting table must always end up exactly like the pricing table: same tariffs, same
prices, no extra rows. The start notebook registers the CDC source and a JDBC sink with settings you
choose in MY_SINK. Then it makes 200 price changes on random tariffs and deletes 3 tariffs. It
kills the Kafka Connect container halfway through the changes (docker compose kill connect),
starts it again, and compares the two tables.
Choose the sink settings. Then, in a markdown cell, explain where the extra rows came from with the
start settings, and why your settings survive the crash.
You're done when: the notebook prints reporting matches pricing: True with the same number of
rows on both sides, and your note explains the extra rows.
Hints
Hint 1
Compare the row counts with the start settings: how many change events were there, and how many
rows ended up in reporting?
Hint 2
Lesson 6.2's sink used three settings that the start notebook sets differently.