Module 8 exercises
Each exercise has a start notebook to work in and a solution notebook to compare with afterwards. The notebooks start the lab services they need.
Exercise 1: Least privilege for grid reporting
Goal: write a service's ACLs from what its job does. · Time: ~15 min · Starter: code/module-08/exercise-1/start/
After the incident of lesson 8.2, svc-grid has no ACLs, and the grid report can't run. The job
reads charging.telemetry as a member of the consumer group grid-reporting, and writes one record
per site and 5-minute window to grid.site-load-5m. Fill grid_acls in the start notebook with the
permissions this job needs, and nothing more. The notebook then runs the job as svc-grid, and tries
six things grid reporting must not do: write telemetry, read sessions, join billing's group, delete
telemetry, create a topic, and change telemetry's retention.
You're done when: the notebook prints least privilege: True. The job writes its site-load
records, all six forbidden actions are denied, and no ACL uses ALL or the name *.
Hints
Hint 1
List what the job does, one resource at a time: which topic it reads, which group it joins, which topic it writes. Run the notebook with an empty list first; the job's error names the first missing permission.Hint 2
Three ACLs are enough. `READ` and `WRITE` include `DESCRIBE`, so the job can see the topics it uses.Exercise 2: A lag-in-seconds alert with a runbook link
Goal: express consumer lag as time, and write an alert the on-call person can act on. · Time: ~20 min · Starter: code/module-08/exercise-2/start/
Billing's alert counts records: more than 500 behind. After the growth plan that's half a second of
sessions at peak, and minutes at night. In the start notebook, change seconds_behind into a PromQL
expression for how many seconds billing (group m08e2-billing on topic m08e2-sessions) is behind,
and set runbook_url. The notebook loads the rule BillingBehind (more than 30 seconds, held for
15 seconds) into Prometheus, checks it stays quiet while billing keeps up, then stops billing and
waits for it to fire. Last, replace the runbook cell with the first steps of your runbook.
You're done when: the notebook prints all checks passed: True: quiet while billing keeps up,
firing after billing stops with a value in seconds, and a runbook link. Your runbook cell says what
to check first and what each answer means.
Hints