Module 6 exercises

Start the lab's connect profile first (docker compose --profile connect up -d --wait in the code/ folder). Each exercise has a start notebook to work in and a solution notebook to compare with afterwards.

Exercise 1: Add the sites table to CDC, with routing

Goal: configure a Debezium connector with transforms that produce exactly the topics consumers need. · Time: ~15 min · Starter: code/module-06/exercise-1/start/

The live map needs site names, and billing needs tariffs. Both want one plain row per key in a compacted topic: m06e1.sites (key site_id) and m06e1.tariffs (key tariff_id). The start notebook creates those two topics and gives you a connector that only captures tariffs and writes full change events to Debezium's default topic name.

Fill in CHANGES, the settings to add to or replace in the connector's configuration. The check registers the connector, renames site-N03 in the database, and looks at every m06e1 topic.

You're done when: the check prints only the two topics: True, plain rows: True and rename arrived: True.

Hints

Hint 1 `table.include.list` takes a comma-separated list of `schema.table` names.
Hint 2 Debezium names topics `..`, here `m06e1_pricing.public.sites`. A `RegexRouter` transform with a capture group `(.*)` can keep the table name and replace the rest.

Exercise 2: Break and recover the sink, no data lost

Goal: make a sink connector safe to replay after a crash. · Time: ~15 min · Starter: code/module-06/exercise-2/start/

Finance's reporting table must always end up exactly like the pricing table: same tariffs, same prices, no extra rows. The start notebook registers the CDC source and a JDBC sink with settings you choose in MY_SINK. Then it makes 200 price changes on random tariffs and deletes 3 tariffs. It kills the Kafka Connect container halfway through the changes (docker compose kill connect), starts it again, and compares the two tables.

Choose the sink settings. Then, in a markdown cell, explain where the extra rows came from with the start settings, and why your settings survive the crash.

You're done when: the notebook prints reporting matches pricing: True with the same number of rows on both sides, and your note explains the extra rows.

Hints

Hint 1 Compare the row counts with the start settings: how many change events were there, and how many rows ended up in reporting?
Hint 2 Lesson 6.2's sink used three settings that the start notebook sets differently.
Content licensed CC BY-NC 4.0 · Code licensed MIT · Updated 2026-10-06