Notebook · code/module-01/02-brokers-controllers-and-partitions.ipynb

Download 02-brokers-controllers-and-partitions.ipynb

Open it in JupyterLab or VS Code from the lab folder to run it. This page is a read-only view.

Brokers, controllers and partitions — practical

You look at the lab cluster the way a client and an operator see it: its brokers, its controller quorum, and how a 6-partition topic is spread over three brokers. Then you stop a broker and see which clients can still connect.

Lab services: kafka-1, kafka-2, kafka-3 · Lesson: modules/01-first-events/02-brokers-controllers-and-partitions.md

# Setup: find the lab's shared helpers and check the lab is running
import sys, pathlib
lab_root = next(p for p in [pathlib.Path.cwd(), *pathlib.Path.cwd().parents] if (p / "common").is_dir())
sys.path.insert(0, str(lab_root))

from common import settings, lab

lab.require_kafka()
lab.print_versions()

Step 1: Ask the cluster who is in it

describe_cluster returns the cluster id and the brokers. Each broker is listed with the address it advertises to clients: the one your Mac can reach.

from confluent_kafka.admin import AdminClient

admin = AdminClient({"bootstrap.servers": settings.KAFKA_BOOTSTRAP})
cluster = admin.describe_cluster(request_timeout=10).result()

print("cluster id:", cluster.cluster_id)
lab.table([(n.id, f"{n.host}:{n.port}") for n in sorted(cluster.nodes, key=lambda n: n.id)],
          ["broker", "advertised address"])

You should see cluster id ZpMZEQqGS768XpROns7_Ug and brokers 1, 2 and 3 at 127.0.0.1:9092, :9093 and :9094.

Step 2: Find the active controller

The controllers keep the cluster's metadata in a replicated log of their own. One of them is the active controller (the quorum's leader). The KRaft tool shows it, and how far each controller is behind.

print(lab.kafka_cli("kafka-metadata-quorum.sh", "--bootstrap-server", "localhost:19092",
                    "describe", "--replication"))

You should see three rows, one Leader and two Followers, with Lag 0 or close to it. Which node leads varies between runs.

Step 3: Create a topic with 6 partitions and 3 replicas

Each partition gets one leader and two followers on different brokers. isr lists the replicas that are fully caught up.

from confluent_kafka import TopicCollection
from confluent_kafka.admin import NewTopic

TOPIC = "m01l02-sessions"
lab.delete_topics(admin, [TOPIC])
admin.create_topics([NewTopic(TOPIC, num_partitions=6, replication_factor=3)])[TOPIC].result()

def show_partitions():
    desc = admin.describe_topics(TopicCollection([TOPIC]))[TOPIC].result()
    lab.table([(p.id, p.leader.id if p.leader else "none",
                [r.id for r in p.replicas], [r.id for r in p.isr]) for p in desc.partitions],
              ["partition", "leader", "replicas", "isr"])
    return desc

desc = show_partitions()

You should see partitions 0–5, each with three replicas and all three in isr. The exact brokers vary between runs.

Step 4: Count the leaders per broker

Kafka spreads the leaders so each broker does a fair share of the reading and writing.

from collections import Counter

leaders = Counter(p.leader.id for p in desc.partitions)
print({f"broker {b}": n for b, n in sorted(leaders.items())})

You should see {'broker 1': 2, 'broker 2': 2, 'broker 3': 2}.

Break it: a client that only knows one broker

Predict first: you stop kafka-2. Client A was configured with only 127.0.0.1:9093 (kafka-2). Client B has all three addresses. Which one can still start?

import logging
silent = logging.getLogger("silent"); silent.addHandler(logging.NullHandler()); silent.propagate = False

lab.compose("stop", "kafka-2")

for name, bootstrap in [("A", "127.0.0.1:9093"), ("B", settings.KAFKA_BOOTSTRAP)]:
    client = AdminClient({"bootstrap.servers": bootstrap, "logger": silent})
    try:
        brokers = sorted(client.list_topics(timeout=8).brokers)
        print(f"client {name}: connected, sees brokers {brokers}")
    except Exception as e:
        print(f"client {name}: failed to start: {e.args[0].str()}")

Client A fails: its only bootstrap address is down, so it never learns about the others. Client B connects through kafka-1 or kafka-3 and sees brokers [1, 3].

Now look at the topic again. The partitions kafka-2 led have new leaders, and broker 2 has dropped out of every isr list.

desc = show_partitions()

Clean up

Start kafka-2 again and wait until it's healthy, then delete the topic. If anything gets stuck, docker compose down -v in the code/ folder resets the whole lab.

lab.compose("up", "-d", "--wait", "kafka-2")
lab.delete_topics(admin, [TOPIC])
print("cleaned up")